If some of the predictor variables are correlated then they first must be decorrelated using techniques such as principal component decomposition analysis (PCA).

PCA is formally a dimension reduction technique, its primary purpose is removing multicollinearity by producing totally uncorrelated (orthogonal) linear combination of the original variables, i.e principal components (when all other assumptions are met). The orthogonal variables are needed instead of original ones because only orthogonal variables produce truly independent regression coefficients.

You can add or remove orthogonal regression variables without recalculation all other coefficients, thus allowing estimation of the true relative contribution of independent variables into the dependent one and finding out which variables are truly significant. Instead of 28 predictor variables you will use only a few uncorrelated linear combinations. You then could proceed with regression with principal components.

See, for example, the book "Healthcare Management Engineering. What does this fancy term really mean? Springer NY, 2012", chapter 5-

In this example, instead 40 original variables for which ordinary regression was totally meaningless, it was found that only 9 principle components account for all data. Performing then regression with principal components allowed determining the primary contributors among all 40 original variables.

If some of the predictor variables are correlated then they first must be decorrelated using techniques such as principal component decomposition analysis (PCA).

PCA is formally a dimension reduction technique, its primary purpose is removing multicollinearity by producing totally uncorrelated (orthogonal) linear combination of the original variables, i.e principal components (when all other assumptions are met). The orthogonal variables are needed instead of original ones because only orthogonal variables produce truly independent regression coefficients.

You can add or remove orthogonal regression variables without recalculation all other coefficients, thus allowing estimation of the true relative contribution of independent variables into the dependent one and finding out which variables are truly significant. Instead of 28 predictor variables you will use only a few uncorrelated linear combinations. You then could proceed with regression with principal components.

See, for example, the book "Healthcare Management Engineering. What does this fancy term really mean? Springer NY, 2012", chapter 5-

In this example, instead 40 original variables for which ordinary regression was totally meaningless, it was found that only 9 principle components account for all data. Performing then regression with principal components allowed determining the primary contributors among all 40 original variables.

Hi Samiksha

You can use PCA as answered by Alexander to make sure the variables don't have a linear correlation.

After that, you can support vector regression is one of the machine learning techniques. This is the reference:

https://papers.nips.cc/paper/1238-support-vector-regression-machines.pdf

https://www.analyticsvidhya.com/blog/2020/03/support-vector-regression-tutorial-for-machine-learning/

Regards

Toto

After PCA, I would try out random forest regression because it deals quite well with continuous "wide-type" data which you have, and because it provides feature importance scores of the chosen metric. These scores can be evaluated to further investigate the relations of predictors to the predicted value (most contributing predictors), something that nonlinear SVMs do not provide. Be careful though, if your limited dataset is noisy then there is a good chance of overfitting, regardless of the method.

Samiksha Pantha You did not provide details of your problem and available data to recommend a specific regression technique. But, as I mentioned in my comment, an example of linear regression with principal components is provided in chapter 5 of the book

https://www.amazon.com/gp/product/1461420679?pf_rd_r=PQ3JX8XHJ7SEM1T7V54M&pf_rd_p=edaba0ee-c2fe-4124-9f5d-b31d6b1bfbee