Dears,
I am working with gridded (EU domain) tri-hourly temperature data ranging from year 2009 to year 2019. In particular, I would like to test whether there are any significant difference between a simulated (obtained from RCM) and observed temperature data.
My idea would be to subset some points from the grid and then compare the monthly averages obtained from the simulated and the observed dataset. Please, could you suggest me some statistical test that might work in this case?
Thank you.
Best,
Giorgio
Dear Giorgio Sperandio
I think that the coefficient of determination is ideal for solving your problem. The coefficient of determination, denoted as R^2, and pronounced “R squared”, is a type goodness-of-fit summary statistics, a tool that determines the quality of a model to replicate results and the proportion of variation of the proposed results (your simulated data) versus the original ones (your observed data). The accuracy of the prediction depends on the relationship between the variables to be analyzed. The coefficient of determination is defined as the square of the Pearson correlation coefficient; if the result is null, it is said that the predicted variable does not have predictive capacity on the model; as this coefficient increases, the prediction of the model turns out to be more accurate. The R^2 values must be between 0 and 1.
Attached you can find a paper where I used this metric and is better described. I hope it will be helpful.
Hi Giogio
I think Pearson correlation analysis or mann-kendall can use to compare the simulated against observed temperature data
I hope this can helping you https://journals.ametsoc.org/jcli/article/28/5/2063/35204/Comparison-between-Observed-and-Model-Simulated
I think the simplest one is the t-test statistics.
Besides this, few other tests such as Kullback- Leibler information measures or Kullback- Leibler based information criteria statistics such as the Akaike information criteria (AIC), Schwartz’s Bayesian information criteria (BIC), or either error indices statistics such as the Mean square error (MSE) and Root Mean Square Error (RMSE).
I think that the best option is to do compare with several statistics, one option is always the Taylor plots, which compare at the same time the error, variance and correlation. Also, a kolmogorov-smirnov test will be useful to compare the PDF. In terms of test, you may want to test variance (fisher), mean (t-test or mann-whitney), correlation (t-test or mann-kendal), or make your own test with bootstrap or monte-carlo simulations.
You can use repeated measures ANOVA to compare the monthly averages . Where exactly the average values differ(posthoc) can be studied as the time progressess. If you are comparing only averages at two time points the either Paired t test(for normal data) or wilcoxn rank sum test (non normal data) can be used.
Sure you used a model to find the simulated data as theoretical part, this model came from the original data (observed data) so the high accuracy between simulated and observed data comes from the best model which formed by the original data as big as possible. If you used model with this case it is OK, if not you can formed a new model via more original data because the accuracy depend totally on the used model. After that you can find the simulated data by the new model, then use on of the mentioned methods above like R^2 for finding the comparison.
@José Luis Echenausía-Monroy Probably this is coming a bit late. My view is that it also depends on how you want to use the data. Like for example, if you are going to use the probabilty distribution functions like in the case of detection and attribution of extreme events, here the kolmogorov-smirnov test will be an ideal choice. Because here you will be interested at the tails of the distributions. If its about a climate change study, where you are working with the mean state of climate, test of different means can be ok like the ttest or mann-whitney. If its about the trends, the mann-kendal works. SO i would say your choice should be dependent on how you are going to use the data for. Also, you can are always also use simple bias correction should you wish to correct the model bias.
You can do a simple correlation between observed and simulated datasets...
You can use
Article Long-Term Estimation of Wave Climate Variability in the West...
All the required equations are stated in the above paper.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning