I am working on a regression problem that holds almost 5000 datapoints (the number of datapoints have been mentioned here to understand the range of MSE , RMSE and MAE values).
My model converges and gives me the following results:
MSE = 2.6980
RMSE = 0.995
Scatter Index = 1.7
MAE = 1.310
I understand that the MSE and MAE values are based on the dataset, its distribution, and variance that further dictates the model outcome. I also understand that there are no universal MSE or MAE values as they are highly dependent on the data and the model. However, in every study, people report MSE/RMSE in conjunction with Model Accuracy (in percentage).
Is there a way/formula to convert MSE into percentage, that could directly report the model accuracy in terms of percentage?
Thank you
Omer -
I am confused by a couple of things above. Your RMSE should be the square root of your MSE, and it is not. Also, MSE/RMSE would just be RMSE.
MAE can be used if you suspect "outliers." Do you? Did you do regression that way?
Actually, the largest predicted values should be associated with larger estimated residuals, and this may take some folks by surprise. Generally, we should use weighted least squares (WLS) regression rather than OLS regression, which is just a special case of WLS regression, and OLS should not actually occur. (See https://www.researchgate.net/publication/320853387_Essential_Heteroscedasticity, as determined by Ken Brewer, and https://www.researchgate.net/project/OLS-Regression-Should-Not-Be-a-Default-for-WLS-Regression, and the various updates there, in reverse chronological order.
Thus, instead of MSE, you may better use the regression weighted MSEw. See Kutner, M.H., Nachtsheim, C.J., and Neter, J.(2004), Applied Linear Regression Models, 4th ed, McGraw-Hill/Irwin, page 424. There
MSEw = SUM(wi ei^2)/(n-p) where p is the number of predictors plus one for an intercept term IF appropriate.
The best way to find the regression weight associated with each predicted-y-value, _i_, is wi = zi^(-2 gamma),
where z is the size measure, best represented by the best predicted-y, and gamma is the coefficient of heteroscedasticity, noted in above ResearchGate references.
But any one statistic, such as MSEw, can only tell you so much. I suggest that you research the terms "graphical residual analysis," and "cross-validation." The former can show you a great deal about model fit, including heteroscedasticity, and the latter is to study whether you may have fit a model too closely to one particular sample, rather than in general for the population or subpopulation to which you wish to apply only one model.
Cheers - Jim
- Badges
- Science topic
- Similar topics
- Mathematics
- Algebra
- Matrix