Hi dears
As you know one of the parameter must be calculated in QSA/PR study or modelling its error value in order of RMSE or MSE. Absolutely its clear that model with minimum error or RMSE its favorable but the upper limit of this parameters its questionable for me. may be its better to ask this simple question: "what is the acceptable max level of RMSE in QSAR model? "
The Best
Mohamad
Indeed, the error based metrics like RMSE, MAE, PRESS etc., provides the true picture of prediction quality. But, deciding a suitable threshold value for these metrics are really problematic. For instance, high values of RMSE can be due to presence of small number of high error predictions (as seen for outliers). Considering the same problem, recently a Mean absolute error (MAE)-based criteria has been reported in the literature (link below) to determine the prediction quality of the model based on the prediction errors. Also, you can use the freely available tool “Xternal Validation Plus” (link below) to compute the same, which categories the prediction quality of the developed model into “Good”, “Moderate” and “Bad” based on the same MAE-based criteria.
http://www.sciencedirect.com/science/article/pii/S0169743916000095
http://teqip.jdvu.ac.in/QSAR_Tools/
Dear Mohammed,
The threshold for RMSE and related error measures are difficult to establish. Normally a RMSE > 0.5 is related to a bad predictive model.
I think the acceptable value of RMSE is dependant on the area under study. Of course a model with low RMSE value is a good preditive model.
There is no absolute value of Good or Bad RMSE as its' unit is the unit of Dependent Variable, I think. Of course the lower the better.
Depends, whether RMSE value is acceptable or not must depend on the actual Range of the target variable, the above answers citing values less than 1 are not a necessity. For eg if your target variable lies within 0-1, obviously RMSE for pair of predicted and target must be much much less than 1, but say in a regression problem your target varies from -500-500, there expecting RMSE lower than 1 is hardly possible in most cases
In case of melting point prediction model, it is impossible to achieve RMSE lower than 0.5 due to large range of numerical values. Below article suggested to measure ratio of RMSE over entire range of observed values since RMSE depends on the range of observed values in dataset. If the ratio is lower than 10%, prediction model is fairly precise.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4530125/