I have to do modeling to find the correlation between my quantitative and qualitative variables. The qualitative variables cannot be classified as categorical. Is PCA or PCoA the correct choice?
Your specific research question and the quantification of each of the variables both guide the choice of analytic method.
If your outcome variable is continuous and metric (interval or ratio strength scale), and your independent variable is nominal (or categorical), then you can evaluate the relationship by recasting the nominal variable into k - 1 dummy variates (where k is the number of levels of the nominal variable) and use multiple linear regression. That will allow you to answer the question, "How much variance in the DV can be explained by differences in the IV?" by using R^2 as the statistic of interest.
Harshita Singh With a qualitative independent variable and a quantitative dependent variable you should be able to apply an analysis of variance. If you have the qualitative variable as "factor", R will do the work of constructing the dummy variables, as David Morse suggests.
Possibly this manuscript could be helpful in your case
Zhang, Y., Tao, S., Chen, W., & Apley, D. W. (2020). A latent variable approach to Gaussian process modeling with qualitative and quantitative factors. Technometrics, 62(3), 291-302.