Dear Simone,

If possible, I suggest a Leave-One-Out-Cross-Validation approach, in a sense that you calculate the interpolated grid with all data first, then remove single data points and interpolate again. The difference will provide a simple measure of the effect for individual data points. Please keep in mind, that the interpolation algorithm and the parameters used will have a strong influence on the results.

I hope this helps a bit.

Regards,

David

I would suggest David's approach but omit the estimated variables. They likely used interpolation or a geostatistical approach to obtain these values so essentially you may be interpolating on interpolated data... Further I would suggest examining the function they used to predict their unknown data as this may prove useful to your analysis. Indeed, if you use the same or similar function as they utilized then you may be able to omit their data altogether and just re-estimate it.

In general, for meteorological data, I would NOT interpolate or estimate the data. Accept that there will be missing data, and proceed. Any attempt to interpolate data from missing values, especially with regard to sparsely collected and highly variable datasets such as precipitation, will lead to increased errors. Other datasets (temperature, humidity) tend to be more smooth, and interpolation is reasonable, but recall that changing patterns of weather, e.g., significant frontal activity, can lead to rapid, and mis-handled changes. Otherwise, the suggestions from Stuart and David have significant merit.

I fully agree with Gerry, to interpolate or not depends on the (spatially) interpolated parameters you aim for and the (temporal) distribution of the missing data. Do you want to calculate GIS layer of what (percipitation, temperature) and for which time scale (daily temps, mean annual air temp)?

If you decide to interpolate parameter from one station with those from an other, your should calculate the root mean square error RMSE of your interpolation. Therefore, I would chose an interpolation time span, for which you have measurement for both stations and calibrate your interpolation parameter. Then calculate the climate parameter for the station with the missing values (but for a time span, which is covered by both stations). Now, use the interpolated and measured values to calculate the RMSE. Then calculate the climate parameter for the missing station at the time of the missing values. If the RMSE is stable, you can assume that the uncertainties of the temporal interpolation are given by the RMSE.

Cheers

Thomas

In general, if you can identify the data points that have been interpolated, and if they constitute a fairly small part of the overall dataset, I'd suggest you simply eliminate those points from the analysis -- they introduce an unknown amount of variation into your results. If eliminating the points is not feasible, I'd suggest you do a sensitivity analysis to estimate the robustness of your results relative to errors in the interpolated data points. In general, you could accomplish this by running your model using the interpolated data, then randomly perturbing the interpolated data points (e.g., multiply each point by a Gaussian variable scaled to keep the values relatively close to 1) and recomputing your results. If your results don't change much when you perturb the data, that's a good indication that interpolation errors aren't likely to be very consequential. On the other hand, if your results change significantly when you perturb your data, your model is not robust and you have to question its utility in the presence of uncertain data.

Dear Simone,

I would like to comment the problem without touching the question is it sinful or not to interpolate some kind data (because meteorology is not my professional expertise), but just from the technical aspect of interpolation itself. I agree with my colleagues that the leave-one-out-cross-validation ( LOOCV ) is indeed a powerful technique. To my knowledge, it is commonly used to overcome the lack of data (if you have not enough data to separate them into training and test dataset) for validation purposes. However, I think here the issue is different. Here, we have estimated data, which we would like to integrate together with measured data, and, as I understood, to get an idea about their influence and quantify the possible errors (on-top of the interpolation errors). Using the proposed LOOCV technique it will hardly be possible to quantify these effects, since the technique is focused to get an idea of the interpolation goodness/accuracy on measured data, which are assumed accurate and are the models reference. Thus, if the data have initially a great uncertainty after running LOOCV it remains impossible to distinguish where the uncertainties comes from and to which amount because the real reference (true value) will still be missing.

What is needed to know is the initial error of estimation for the not measured but somehow estimated data bearing an uncertainty. First what should be cleared is which methods were used to generate synthetic/estimated data and what is the possible error of this estimation? If the data have been estimated with mathematical/statistical methods accounting for uncertainties, some interpolation scenarios could be produced varying the estimated value for maximum, minimum and mean. So we would have a kind of envelope for the uncertainty in these areas caused by data. Additionally, using statistical interpolation methods (such as Kriging) we could account the uncertainties caused by interpolation on-top of it. But, if the values are estimated by methods, which do not provide quantification of uncertainty, I’m afraid there is no way to solve the error propagation issue and we will have to cope with the fact that an undiscovered error is there.

Therefore, if you’ll ask me, I would suggest that you remove the estimated data from your dataset and run the interpolation just with measured data. The gaps will be closed by the estimates in a specific error range corresponding to the particular interpolation method you use, which would be known to you (maximum control!).

Moreover, I think that integration of estimated data into the spatial analysis (such as interpolation) can be only justified assuming that the estimation method (which was used to generate the synthetic data) is much more accurate than the interpolation method you would like to use finally.

Best regards,

Jewgenij

You are raising a very important question but one which does not have any easy answers. First of all there are a number of different interpolation methods (inverse distance weighting, kriging, just to mention two common choices) However both of these are not a single uniquely determined method but rather a class of methods (in the case of IDW you have to choose the exponent and the maximum distance for data locations, in the case of kriging you must choose between simple, ordinary, universal, a variogram model, parameters for the variogram model and the search neighborhood. In any case there is the question of the spatial arrangement of the data locations and relative to that the spatial arrangement of the interpolation locations. If you don't have access to this information it is hard to assess the uncertainty of the interpolated values and that uncertainty will vary from one interpolation location to another, the actual interpolation errors are unknown and not knowable. Some form of cross validation seems reasonable but that will depend on a number of untestable assumptions, e.g. if you split the data set into a control set and a test set you are still assuming that the two subsets are the "same" in one or more statistical senses. Looking at the robustness of the ultimate use to the different choices, e.g. not using the interpolated data at all, using the interpolated data but more than one interpolation method.

You refer to "GIS interpolation", does this mean simply that you are using GIS software or something else? That terminology does not refer to a common interpolation method.

What is the intended ultimate use? I suggest that you at least need to consider the consequences of producing erroneous results or grossly uncertain results. E.g., many times and in many places an erroneous weather forecast (rain vs no rain) will not have serious adverse consequences but other times an erroneous weather forecast could have catastrophic consequences.

W

If using geostatistics there are methods specifically designed to study uncertainty such as: sequential gaussian simulation and direct sequential simulation. Instead of having a single model you would have "n" (simulations) that have an equal probability of happening.

If you calculate the mean of those simulations you'll see that you'll get something similar to the kriging estimate (kriging mean). On the other hand you can also calculate the percentiles, variance, and so on.., images of simulations detailing the uncertainty to a distribution by node.

An important point, however, is that you do not need to use the estimated data with the same weight that of the hard-data. It is feasible to apply a weight to the estimated data that reflects the amount of uncertainty that particular estimate has. The difficulty here would be how to calculate the weight for each of those samples.

Using the before mentioned cross-validation would be a good start. But it was also mentioned before, that those estimates were probably done with kriging meaning that unless you change drastically the anisotropy model you'll get near full correlogram (or near zero variogram) from those samples, or very little uncertainty.

The uncertainty here is mathematical, so if it is true that the estimates are kriging related maybe you should retrieve not only the cross-validation estimation of those points but also the kriging variance. Those points with higher variance would have (more uncertainty) have less weight than those with little or none variance (I mean kriging variance).

It is an interpretation and there is no obvious solution but by combining sequential simulation with weights study you may get a much more robust model(s).