Dear all, I'm testing 5 different models and I'm using the Absolute Relative Error (ARE) which is (predicted-observed)/observed, the Absolute Error (AE) and the Root Mean Squared Error (RMSE) as estimators. My results are below (in bold the best), please click on the attache (awful) picture.
Anyway I'm puzzled about the disconnection between ARE and AE or RMSE. Actually ARE is a sort of weighting method which tells me also where an error occur. I'm pretty sure ANOVA says that no differences are detectable between models (except exponential) but I'm not sure whether I should use ARE or AE or RMSE.
Any suggestions, cheers
Dear Maurizio,
the resulting model depends on which criterion you have used for fitting. Usually, one uses Least Squares, but also absolute (or normalized absolute) error could be used - they lead to different models, anyway. If you perform a Least Squares fit, then you are guaranteed to obtain the lowest RMS error, which does not imply that this fit is optimal wrt other criteria.
Probably the term “goodness of fit” is not adequate. I meant “better result”. Actually ARE, AE and RMSE are coming from a cross-validation procedure
A look at your result shows a measure of unanimity amongst the three criteria ARE, AE and RMSE. Rather than recommend the exclusive use of one of them I recommend the use of all of them and even more criteria. No single criterion is globally optimal. Therefore it is advisable to base one's choice of model on a number of criteria and not on just one criterion.
Thank you all.
Ette Etuk do you suggest a specific test for what you are saying? A MANOVA for example?
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA