Dear all,
I wanted to evaluate the accuracy of a model using observation data. My problem is the correlation of the model with observed data is really good (bigger than 0.7) but RMSE is very high too (like bigger than 100 mm in a month for monthly rainfall data). How can I explain it? the model also has low bias.
How to explain this case?
Thank you all
You can get a high correlation with a high RMSE if you have a model that is either biased, or have a significant scale error (i.e. the slope of a regression between the observed and modelled values is significantly different from 1). Given that you say the model has low bias, I suspect the later is the case. The correlation coefficient is only a useful measure if it is accompanied by the regressed parameters: intercept (the bias if you subtracted the mean from the x values before regressing) and slope.
You should also note the RMSE can be high if the quantity you are modelling is high. Using a normalised RMSE would help (e.g. ratio of variance of residuals to variance of observations).
Well first I'll assume you are building a predictive model with OLS regression but you haven't told us about your variables and what you wish to predict. Given that it is extremely difficult to evaluate your situation. First I would at least skim this book to make sure you have chosen a good modeling approach for your problem. Then come back with your question and try us again. Book: : https://b-ok.cc/book/5243562/0e8c4b
Be sure to tell us about your variables and methods. Best, David Booth
David gives a different perspective -I was thinking of a more general modelling approach (e.g. time series modelling) rather than OLS (I suspect Jin was approaching this from a similar perspective).
Following on from Jin's answer, it should be noted that there are a number of formulations for R^2. The R^2 for a linear regression (more correctly r^2, though the use of R rather than r is very common) is a poor indicator of performance for general modelling - it is really intended for regression, and even then only gives a partial view of the performance of a model. Other forms of R^2 are intended for general modelling, including what hydrologists refer to as the Nash Sutcliffe efficiency.
It would be helpful to give us a more detailed understanding of what you are doing. This would help us give answers that are more relevant to what you are trying to do.
- Badges
- Science topic