I have been corrected some climate outputs. What are possible indicators other than RMSE (or other biased errors) ? Are there any qualitative indicators ?
i use in my papers (in press) ,
missing data imputation methods performance indicators,
RMS
Absolute mean error,
Coefficient of determination
correlation
My data were air temperature.
Hope it will be best for you too
Usman
Dear Muhamad,
Muhammad Usman's suggestions can satisfy the reviewers. You can also create Taylor Diagrams.
Regards,
@Ram,
Can you please explain your phrase
"But if you used full coupled climate model outputs, these output are not correlated (neither before nor after correction) with the observations. This means that you cannot use the correlation, RMS, Nash-coeff "?
@anyone
Anyone knows any reasonable RMSE for annual, monthly and daily climate (precipitation and temperature?
It depends on what you want to assess. There're a lot of different metrics.
Also, note that RMSE can be decomposed in terms of correlation, variance and bias between observations and model output [Murphy, AH, Monthly Weather Review, 116:2417-2424. (1988)]. This means that there're metrics that are not independent.
With respect to "if you used full coupled climate model outputs, these.. are not correlated with observations", I don't think I totally understand that. That's the case if it's a "free run", but not if the coupled model is running with nudging (e.g., SST nudging).
Dear Bahri,
I completely agree with Ángel G Muñoz. It depends on what is the purpose of your work.For minimization of errors after correction, we use Pearson correlation coefficient R (or coefficient of determination R2), Root Mean Square Error (RMSE), Mean Absolute error (MAE), the F-test (for checking the reliability of R2), Daily root-mean square (DRMS). All these methods are based on the concept of ordinary least square regression models.
There are some measures which are used most widely now-a-days in hydro-climatic model validations e.g. Index of agreement, Nash-Sutcliffe Efficiency (NSE), Percentage Bias (PBIAS), ratio of the root mean square error to the standard deviation of measured data (RSR), Performance virtue statistic (PVk). Many validation methods have also been developed based on these methods so as to be suitable for some particular cases.
There are also graphical methods like percent exceedance probability curves are generally used to have a visual comparison. In hydrology, a hydrograph is a time series plot of predicted and measured flow throughout the calibration and validation periods.
You can refer to the link for evaluation of watershed models.
http://swat.tamu.edu/media/90109/moriasimodeleval.pdf
I hope my answer will be helpful for you.
@anyone : can anyone explain advantages and disadvantages of POD and other correction methods ? some say POD does not see bias while other correction methods look at bias... POD (perturbation observed data)
How about an empirical approach? Some groups are now using the Koppen-Trewartha or Köppen–Geiger climate classification system to assess bias in model outputs.
You might be interested in the paper by Belda, et al. There is considerably more info on their web site.
One point, the CRU at UEA have updated their datasets since Belda. In particular, wet bias was removed, which probably reduces the difference between observations near the beginning and end od the 20th century.
Climate classification revisited from Köppen to Trewartha, Belda, M. et al, Climate Research, 2014
http://www.int-res.com/articles/cr_oa/c059p001.pdf