If you are wanting just non-OLS loss functions, just put these in. The robust regression procedures have lots of these. But, this depends on what you mean by robust. As Kelvyn Jones points out it may be worth using additive models (or loess or spline), and it also might be worth looking at interactions. Personally that is a fairly long list for interpreting the coefficients (I have difficulty figuring out the meaning of a coefficient after it is conditioned on 8 other variables ... or even on one other variable ... that's worth another thread), so I would be interested in how you came up with this model and if you used shrinkage/selection methods (like the lasso) to help or whether this was all based on theory (though I would guess theory might predict more interactions).

Hi dear,

Please check it up:

https://www.sciencedirect.com/science/article/pii/S0166111608711099

Good Luck.

Look also here:

Article Linear Regression without computing pseudo-inverse matrix

or here:

Article Linear Regression without computing pseudo-inverse matrix

Least absolute value methods existed before OLS, but were too computationally demanding. Now it is much more common in situations where OLS is often used for people to use more robust loss functions (google robust regression, or M estimators). And for things like logistic regression OLS is not used. BUT, I would think in your situation you are more interested in the variability of the solution depending on which variables are including in the model (or on aspects of those). What is the reason for your use of this?

Thank you all for your valuable answers.

Professor Ajit kumar Roy

Professor Daniel Wright , As I mentioned earlier, I want to do robustness check for the OLS model. So I am looking for any other alternative model which can use for cross-sectional data. Following many previous studies in same topic, I choosed the OLS model and it is attached below.

Yes of course there's. Principal Components Method is an alternative to OLS.

You could use a General Additive Model to see whether the assumption of linearity between Y and each X is supported.

Article “Moving Out of the Linear Rut: The Possibilities of Generali...

https://www.researchgate.net/publication/225303308_Generalized_Additive_Models_Graphical_Diagnostics_and_Logistic_Regression

Minitab's XLOF feature is also useful

Testing regression lack of fit without replication: a tutorial around Minitab's XLOF procedures

Article Testing regression lack of fit without replication: A tutori...

"The tutorial is concerned with two types of test for the general lack of fit of a linear regression model, as found in the Minitab software package, and which do not require replicated observations. They aim to identify non-specified curvature and interaction in predictors, by comparing fits over the predictor region divided into two parts. "

They give macro commands and this should not be difficult to do in other software.

If you are wanting just non-OLS loss functions, just put these in. The robust regression procedures have lots of these. But, this depends on what you mean by robust. As Kelvyn Jones points out it may be worth using additive models (or loess or spline), and it also might be worth looking at interactions. Personally that is a fairly long list for interpreting the coefficients (I have difficulty figuring out the meaning of a coefficient after it is conditioned on 8 other variables ... or even on one other variable ... that's worth another thread), so I would be interested in how you came up with this model and if you used shrinkage/selection methods (like the lasso) to help or whether this was all based on theory (though I would guess theory might predict more interactions).

Professor Daniel Wright ,The selection of explanatory variables are purely based on the previous literature and several theories including proxies for the ex-ante uncertainty of the firm, the impresario hypothesis and the windows of opportunity hypothesis.

I could have tried GLS or WLS models if the OLS results are consisting with autocorrelation or heteroscedasticity errors. I think (not sure) robust regression methods also concern about above mentioned errors.

My objective is to conduct “robustness check" to examine how far the OLS regression results are consistent with another appropriate regression model (both regression results will generate same significant variables or not).

You could try wrapping a cross-validation analysis around your OLS: see https://en.wikipedia.org/wiki/Cross-validation_(statistics) for the idea and references. Also, this https://www.analyticsvidhya.com/blog/2018/05/improve-model-performance-cross-validation-in-python-r/ has some description and ideas for code.

On a similar theme, you could follow-up the idea of "leverage"tolook for inflential points: see https://newonlinecourses.science.psu.edu/stat501/node/336/ , although others references can be found with simple searches.

This book https://www.springer.com/gb/book/9780387950174 (Robust Diagnostic Regression Analysis by Anthony Atkinson & Marco Riani) may also be helpful if you can find it.

Thank you Professor, @David A. Jones

Dilesha -

I'd expect that you'd want to use weighted least squares (WLS) regression rather than OLS, as there would generally be some level of heteroscedasticity.  (See https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, but also please note https://www.researchgate.net/publication/324706010_Nonessential_Heteroscedasticity.)  The variance of the prediction error would then be different for each predicted y. 

The estimated variance of the prediction error depends upon the estimated sigma, which is impacted by bias, and therefore can be a good overall measure of accuracy for predicted y. 

[Note that an unbiased but less efficient OLS regression coefficient, though unbiased, will not generally be the same as the WLS one.] 

Graphical residual analyses can help you check the fit in a meaningful way, and test data (not used in model selection or to estimate regression coefficients) can help you avoid overfitting.  Cross-validation was mentioned above.

The following link might be of some use to you:

 https://www.researchgate.net/publication/313925533_Comparing_Model_Performances_Graphically

Cheers - Jim

Thanks a lot Professor James R Knaub

You can try Least Squares Ratio Regression (LSRR)Article A Novel Regression Approach: Least Squares Ratio

LSRR has a closed formula of regression coefficients minimizing error rate while OLS aims to minimize Euclidean distance between the actual and estimated values of Y. It is more robust than OLS when outliers/extreme values exist.

Have you tried the GLS method? LASSO ( Least Absolute Shrinkage and Selection Operator ) method?

I can suggest you three other algorithms excluding the OLS. Reduced Major Axis (RMA), Major Axis (MA), and Robust. I do not know your research design or objectives but you can evaluate these three which is more suited to your experimental objectives.

RMA and MA try to minimize both the x and the y errors. RMA/MA fitting, standard error estimation and slope comparison are according to Warton et al. (2006).

The “Robust” method is an advanced Model I (fixed x values) regression which is robust to outliers. It sometimes gives strange results, but can be very successful in the case of “almost” normally distributed errors but with some far-off values. The algorithm is “Least Trimmed Squares” based on the “FastLTS” code of Rousseeuw & Driessen (1999).

I attached the tutorial of Past version 3.18 PAleontological STatistics, you can find other very good tests that can serve you in multilevel studies such as:

Principal components, Principal coordinates, Non-metric MDS and Correspondence analysis.

Thanks for your valuable suggestions oguz akbilgic Jacques Beauty Kazige and Enrique Reynaldo de la Cruz .

Dilesha -

Even though an hypothesis test may have led you to believe your data are homoscedastic, as references I included above indicate, one would usually expect heteroscedasticity, and to check robustness, you could compare results with varying levels of heteroscedasticity.  In the following, I provide ways to consider the level of heteroscedasticity, to obtain those alternative results: 

Estimating the Coefficient of Heteroscedasticity -

https://www.researchgate.net/publication/333642828_Estimating_the_Coefficient_of_Heteroscedasticity

https://www.researchgate.net/publication/333659087_Tool_for_estimating_coefficient_of_heteroscedasticityxlsx

Part of

 https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression

Stratification can be important.  That would leave you with fewer cases for estimating the coefficient of heteroscedasticity, but might be necessary. 

Best wishes - Jim

Maximum likelihood optimization has been shown to more robust to deviations from model assumptions than OLS so you could try that. I never use OLS, always ML to estimate point parameters and variance for regression.

Also generally lowest variance and convergence properties. See any text on ML regression.