I usually use ArcGIS's Geo-statistical analyst extension for conducting spatial interpolation of for prediction maps. However, I have seen scenerios where the error (RMSE, R2 etc.) seems to be low for different interpolation techniques (e.g Spline, IDW, Kriging) but, if a change the data set by randomly taking out some data the results change significantly for different methods. ArcGIS's geostatistical analyst is already having a cross validation option where it is going through an iterative process by removing one point and estimating the results based on interpolation of other points. Accordingly, it is calculating errors based on that prediction. How can I carry out K-fold or 10 - fold cross validation for validating the model performance and get the best solution for prediction maps? Is it possible in ArcGIS or R? It would be helpful if anyone can share some codes for these type of problems.
Dear Islam,
In R, you can use function krige.cv() from package 'gstat' for leave-one-out cross-validation and n-fold cross-validation (depending on the nfold argument of the function).
HTH,
Ákos
Different models perform similar operations /analyses, but with some varying unique features. For example, Spatial Analyst tool on ARCGIS performs spatial interpolation, autocorrelation, etc ..and it's widely used for spatial data visualisation and points representation. However, similar topls are also found in some of the open source software packages ..such as QGIS, R, SAGA GIS etc.
R-climdex (a functional package in R) is a very good model that performs cross validation. Therefore, the choice of use matters.
In QGIS there is a plugin for Bayesian Maximum Entropy interpolation. This is a novel geostatistical tool that in my case outperformed the traditional kriging algorithms even with much less data points! However the BME requires high computational power.
Please have a look at my previous study : Poster Geostatistical and multivariate modelling for large scale qu...
You are trying to use tools that simply do not apply to interpolation methods (k-fold, RSME Rsquared). You are mixing up interpolation and regression. There is no such thing as a "best" model, especially since you have not given a criterion for "best".
Dear Islam,
although I agree with Donald Myers that several alternative criteria for 'best' exist, but I'm sure that cross-validation and RMSE-type error measures can be applied for spatial interpolations. Furthermore, if you have independent test data for the interpolated surface, not only coarse-resolution cross-validation errors but fine-resolution independent errors can be calculated.
HTH,
Ákos
It may seem natural to ask this question but it does not have an answer, the question reflects a serious misunderstanding and mis-representation. There is no "best model" and no "methods for reliability assessment". Interpolation is not the same thing as regression so the model evaluation statistics for regression are not relevant.
You probably mean to optimize the performance of the model achieving the minimum skill scores, let's say RMSE, bias, or others, associated with a cross-validation scheme. But optimizing the performance in kriging starts with careful exploratory evaluation of the input, i.e. you may want to exclude data that are obviously wrong (although excluding measurements that are used for input is a process that needs care so you do not exclude interesting extremes that have actually happened). Then you should invest time to the estimation of the parameter values of the variogram. Try different models, try with and without nugget etc. Finally during the estimation there are parameter values that you may want to play with, for example the maximum number of points or distance from a point to be used.
- Badges
- Science topic
- Similar topics
- Linguistics
- Composition Studies
- Publication