hello all ,
I need to know what is the criteria we can use to see if the validation set of data are doing well/bad in the model. can i use RMSE equation to check the accuracy in the validation data set ?
If I am interpreting correctly your question, you have splitted a dataset in two subsets, one for fitting your model and another one for validating the model you fitted. am I right? Without taking into account the data splitting technique you have used (which is a whole topic); for seeing how well your model is performing on the validation dataset, you need to estimate how good is your model in two aspects 1) bias 2) Precision/variability.
Although RMSE is very commonly used, it can have some drawbacks depending on the characteristics of your data. I prefer to examine several statitistics to compare model performance, for example: Mean absolute error for bias and the fit index (analogous to r2 but varying from -inf to 1) for precision.
In addition to the interesting papers provided by Safa you can also review:
Mayer DG, Butler DG (1993) Statistical validation. Ecological Modelling 68:21-32
Best regards
See the cross validation material in the attached. It is very helpful. Best, David Booth
I would like to thank you for sharing knowledge and it has been very useful comments .
I have used RMSE to validate my regression model. Is there a need to implement data splitting techniques to my regression model for better validation?
- Badges
- Science topic