I am trying to perform ordinary Kriging using gstat package on a hourly particulate matter(PM) concentration dataset. my dataset contains PM concentration for 1 hour of 250 sites.
After performing ordinary kriging, I did cross validation. but I got the correlation coefficient between observed values and predicted values is low (around 0.5). I checked many combination of model (Gau,Exp,Sph), width, cutoff, intial parameter value but still correlation coefficient between observed values and predicted values is low. Am I doing any mistake in my code? Is there anything to do, to get high correlation coefficient?
I am attaching my dataset (urban030101) and R code. Please take look. What I miss?
I did not run your code, but I checked the data. The first thing I noticed is that there is not much of variation in PM. Now, a correlation between predicted and observed values is essentially a measure of how well you can predict the PM by knowing the location. If the PM is spatially diffused, then you should not expect the location to be a strong predictor and thus a lower correlation. In your case the correlation works like the R-square in ordinary regression.
And of course if there is modest variation in the response variable, it will be harder to predict.
Thank you very much for your response. Could you please suggest me what can I do now? My goal is to predict spatially by Kriging. But here I am totally struck for 1 month.
should I do anything with my dataset before kriging analysis? I didn't model the anisotropy, is that might be a reason? should I check any other type of Kriging? Is there any wrong in my code (I am not totally sure that my R code is okay)?
Xiaoyun helped me in re-running your code as I did not have all the necessary libraries installed. First, I did not see a reason to transform the particel matters as it was symmetrically distributed. We re-ran your code and it seems fine. Xiaoyun's first plot shows a scatterplot of predicted vs actual PM-measurements. As expected, the low and high actual values are not well-predicted. This will naturally happen if you have insufficient information in the prediction. Clearly, location is not the only factor that determines PM (in fact, we found it to explain 30% of the PM-variation).
In Xiaoyun's second plot, you can see the observed values (green) and predicted (red) as spatially distributed. If you check carefully the same pattern re-appears - predictions tend to be closer to average PM10 than the actual values. Again, the same explanation, not only location determines the PM10.
Professor Carling and Xiaoyun Zhao ,Thanks you vary much for your kind response.
What I understand from your reply that is only location can not predict well my data. I need any other predictor. Is my understanding right?
Actually, I have dataset contains hourly PM concentration of 250 site of 90 days. Here I have attached only data for 1 hour (march 01,1.00 am). My dataset contains only these (lat, lon, PM concentration) information. My goal is to check the potential of Kriging interpolation for my dataset.
You said, only location can not predict well for my data. But I have no other extra information. Is that mean, Its impossible to get good prediction for my dataset by kriging? And, what could be the other possible predictors? as I have no additional information then could you please suggest, What can I do now?
As for other predictors, the first that comes into mind is weather data. I suppose stronger wind conditions would diffuse the particles in a wider area leading to a lower level of concentration compared to a wind-free day. Landscape is another variabel, is the area sheltered by a valley or high buildings keeping the partciles contained for a long time? A third is the surrounding emitters, do the traffic and industrial plants co-produce PM:s?
Actually the documentation I found about kriging from internet based on that I wrote these R code which can predict PM concentration based on location. I have totally no idea about how to include other conditions (wind condition, landscape etc) in kriging. Could anyone please give me some suggestions about this?
Intuitively one would expect a good interpolation method to produce predicted values that are close to the actual data values and hence that the correlation between the two should be close to one. However there are a number of reasons why this may not be true for a given data set including the following:
1. There is a difference between Simple and Ordinary Kriging, in the case of the former the theoretical correlation is one but in the case of Ordinary Kriging it is less than one (because of the Lagrange Multiplier(s), note that the values of the Lagrange multiplier are not constant, i.e. they change with each prediction location. There is an old paper by A Journel in Mathematical Geology (about 1980) that points out the difference between projection onto a subspace and projecting onto a hyperplane
2. Both Simple Kriging and Ordinary Kriging equations are derived by making certain statistical assumptions, e.g. second order stationarity (of the random function) in the case of Simple Kriging and Intrinsic stationarity in the case of Ordinary Kriging. To the extent that these assumptions are not satisfied affects how well the interpolation method performs. Unfortunately there are no simple statistical tests for stationarity.
3. Given a single finite data set it is not possible to determine "the" variogram but the variogram model used in the cross validation algorithm is not the only user choice that can affect cross validation statistics, e.g the neighborhood search parameters such as the minimum of number of data locations and the maximum number of data locations. The spatial pattern of the data locations can also have an effect. The correlation of predicted vs actual data value is just one of the cross validation statistics, you should also look at the mean square normalized errors, the proportion of normalized errors exceeding 2.5 (use Chebyshev's inequality) and of course the mean error and mean normalized error.
It is useful to make a plot of the data locations where each data location is coded with the prediction error or the normalized error.
It is also useful to look at the correlation between the normalized error and the data values (here the correlation should be close to zero but again there is a distinction between Simple Kriging and Ordinary Kriging.
If you do some experimenting with the variogram parameters and the search neighborhood parameters you can see how sensitive the cross validation statistics are to changes in these parameters (or how insensitive).
Finally, think about what you know about the phenomenon that generated the data and whether your results make sense and answer the question you are wanting to answer in the study.