1. In the normal lingo, very little. In robust regression, the standard errors are calculated differently, to make them "robust" against heteroscedasticity (or clustering).

2. You interpret the regression coefficient in the same manner: the average change in y given a one-unit increase in x.

3. I have written three textbooks on regression, but I have never heard the term "proper correlation" before. You do regression to find out if there is a (linear) association between x and y and, if so, to find out how large this assocoation is.

4. I seldom use R, but I know for a fact that both plain vanilla and robust regression is are straightforward to estimate in R.

Good luck :-)

This attached teaching unit should introduce you to these concepts. Please ask if have questions. Best wishes, David Booth

Sudheeraka -

Robust regression puts less emphasis on y-values with larger estimated residuals, and thus a potential outlier would have less impact. It might be better for "dirty" data. But first I suggest you see if the larger estimated residuals aren't just associated with larger predicted-y-values. If you are using OLS regression, that is just a special case of weighted least squares (WLS) regression, with coefficient of heteroscedasticity equal to zero, which is often not a good choice. See https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression, and various updates. -

I suggest you try WLS regression. It puts less emphasis on cases with larger predicted-y.

Graphical residual analyses are good to check fit (including heteroscedasticity), and cross-validations help check that you did not fit too closely to one particular sample.

- Cheers, Jim

1-3. See James R Knaub , David Eugene Booth , and @Christer answers.

4. lm versus rlm in the MASS package, but there are tons of other functions for these.

,@James weighted least squares was an early approach to solve this problem. There are more efficient methods today. My program for example is pre-horse and I agree with Daniel that the MASS package in R is a reasonable way to go for most common problems. See the MASS book by Brian Ripley. Other modern approaches are in the works of Peter Rousseau at University of Leuveen in Belgium. All of this as well as comprehensive Robust algorithms in R by Rand R. Wilcox of USC can be found in the z-library. I'm attaching one of my favorite papers on Robust logistic regression which we found useful. Our work can easily be found in Google scholar. Best wishes to all, David Booth

No. That is not what I said. Weighted least squares is not particularly for robust regression. I don't know that anyone ever said that it was. It should probably be used most places people use OLS, as that is a special case of WLS that is overused.

What I'm saying is first see if you really need robust regression. You may not have an outlier problem at all. You may just think you do when the problem really is choosing your coefficient of heteroscedasticity equal to zero (OLS), when that was not a good choice. See https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, and again, https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression.

Sorry if I was not clear. Hope this explanation was better.

Cheers - Jim

PS - Oh. I guess David was referring to using some kind of ad hoc weights for WLS. Nope. Not referring to that.

Maybe you can consider the recursive least squares algorithm (RLS) with forgetting factor (RLS-FF). RLS is the recursive application of the least squares (LS) regression algorithm, so that each new data point is taken in account to modify (correct) a previous estimate of the parameters from some linear (or linearized) correlation thought to model the observed system. The method allows for the dynamical application of LS to time series acquired in real-time. As with LS, there may be several correlation equations with the corresponding set of dependent (observed) variables. For the RLS-FF algorithm, acquired data is weighted according to its age, with increased weight given to the most recent data. The correlation parameters are updated gradually.

Application example ― I have applied the RLS-FF algorithm to estimate the parameters from the KLa correlation, used to predict the O2 gas-liquid mass-transfer, hence giving increased weight to most recent data:

Thesis Controlo do Oxigénio Dissolvido em Fermentadores para Minimi...

Robust regression may be used to handle noisy data, and especially if there are some real outliers, but as I said, I suggested WLS first because your real problem may just be that you are not accounting for natural heteroscedasticity. If you use robust regression when not needed, that is like using a nonparametric method such as looking at ranks instead of continuous data. It may solve a noisy data problem, but if not needed, it throws away good information!

**So I suggest you not throw away good information when you do not need to do so.**

The following may be of interest:

"When Would Heteroscedasticity in Regression Occur?" Preprint, June 2021, J. Knaub, https://www.researchgate.net/publication/352134279_When_Would_Heteroscedasticity_in_Regression_Occur

Please note that by WLS regression, I mean the weights are determined by an appropriate coefficient of heteroscedasticity, gamma, inherently present, NOT ad hoc weights used to handle individual suspected "outliers."

I think this gets back to a problem with the original question. The questioner asks folks to compare:

1. a particular estimation used for regression.

with

2. an adjective about a regression.

Because of the popularity of MASS, when I hear regression in the context of R, I assume the person means a particular set of estimation procedures (those available in rlm). But this is of course just an assumption, so I really don't know what procedure the person here is meaning, so as it stands this question is meaningless unless the person clarifies it.