I advice you to look at this very interesting paper :

Schneider, T. (2001). Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values. Journal of Climate, 14(5), 853–871. doi:10.1175/1520-0442(2001)0142.0.CO;2

This paper comes along a matlab toolbox implementing the EM algorithm that can be used efficiently for data completion.

Ana use some stastistical interpolation to fill in the data gaps if your bioclimatic model is not accepting blank values. That is the best you could do in this case. If that is also difficult take a 4 to 5 data points on either side of the missing value and use their average.

I advice you to look at this very interesting paper :

Schneider, T. (2001). Analysis of incomplete climate data: Estimation of mean values and covariance matrices and imputation of missing values. Journal of Climate, 14(5), 853–871. doi:10.1175/1520-0442(2001)0142.0.CO;2

This paper comes along a matlab toolbox implementing the EM algorithm that can be used efficiently for data completion.

I agree with Philippe Blanc: you must first try to work with data with their missing values before trying to fill gaps. There are very few cases where you really need to fill them.

Looks like you have missing data in your monthly time series. Depending on how much effort you want to put on this, you may try the followings: 1. Use moving averages of say 3 to 5 steps. 2. Use regression with any neighboring stations for the month of the missing data, if it yields acceptable correlation. Correlation with climate GCM data may also be tested if you are familiar with such data. 3. Along the same line, you may gather data of a number of adjacent stations and try to interpolate (draw a map of) the values of the month with missing data. There are numerous interpolation techniques. 4. For temperature, a linear regression among stations may give you acceptable results. 5. Data-driven methods, such ANN, may be applied to your observed time series in order to predict the missing data. But this requires more intensive effort than those above.

You can prepare a mathematical simulation model utilizing with the available data excluding the missing years. calibrate and after a possible validation you can predict the missing data

I have uploaded my PhD thesis in RG this also will help you in doing this exercise

Saravanamuttu good job (Ph.D.) thesis is interesting. However for a simplistic model I guess averaging will not be bad. Mathematical simulation is too sophisticated.

Dear,

I suggest the imputation method. But for precipitation you can use Inverse Distance approach.

Wishes,

Some meteorological data can be calculated from others,

1- Actual vapor pressure can be calculated from minimum temperature,

2- Solar radiation can be calculated from the extraterrestrial radiation which depends on geographic information and the Julian day,

3- wind speed data can be approximated according to the dominant winds in the zone.

More information and equations are in the FAO web http://bit.ly/1hFJj7m

Other meteorological data (like temperature) could be found by interpolation; there are several methods to do that; for example please see this paper: http://bit.ly/1cigyq6

or you can see the attached paper.

Ana, be careful to lower your degrees of freedom equal to the number of data points estimated... this is a common error made in statistics. This will raise the bar for any hypothesis tests made and ensure you are not claiming significance when in fact you shouldn't. Consult Ostle's, "Statistics in Research" for further information.

As for methods for filling in or generating missing data... Ostle gives a method, but there methods abound; virilly, it really doesn't matter what method is used, for all methods create a false data point, hence, the reason for lowering the degrees of freedom. In other words, it is often needed to have the missing data in order to apply a specific analytic method; but, providing or generating data is a form of bias, even if the estimates are based upon existing data.

Thanks for all the responses. Are all very very useful for my diseration paper.

The imputation or other methods assume that statistical properties of available data being fixed. But for physical assumption, one may run a limited-area model to fill missing values. For example the impuatation method is an black-box approach which have an assumption of normal-distribution similar to a logistic-regression. Filling missing values should not be an interpolation, it must have a phsical-based assimilation technique; without phsical features the best linear method could be Kriging. For getting proxy data there is no a true best method.

Dear

I do not know if your interest is particualrly discuss the method or simply fill your climatological series.

If the target is 2nd I suggest you visit the KLIMAHOM page performed by Pter Stepanek and the Anclim software.

There are many possible interpolation methods. I would suggest these papers:

https://www.researchgate.net/publication/257431720_Evaluation_of_efficiency_of_different_estimation_methods_for_missing_climatological_data

Article Evaluation of efficiency of different estimation methods for...

José, it seems the kilmahom.com domain has been sold...

If you have adequate numbers of complete data you can use machine learning approach to predict missing fields !

Article Machine Learning Based Missing Value Imputation Method for C...

Sorry for not having answered earlier. I normally find necessary to fill in

the missing data, e.g., for computation of normals when the reference period

is lacking part of the data. To cope with this problem (and with that of the

non-climatic inhomogeneities) I developed the R package Climatol

(freely available at http://www.climatol.eu/), that performs these tasks

automatically. (It allows the attribution of data for a standard reference

period even for automatic weather stations deployed in recent years, which is

very convenient for current climate monitoring).

When using BSRN data (of solar radiation in particular), this most interesting study compared different techniques to derive unbiased averages when dealing with the missing data issue:

www.atmos-meas-tech.net/4/339/2011/

doi:10.5194/amt-4-339-2011

www.atmos-meas-tech.net/4/973/2011/

doi:10.5194/amt-4-973-2011

I have problem like you, Ms.Ana. Can I fill the missing data of daily rainfall data with linear interpolation ? I used data for 4 months (108 row) consist of 25% missing value.

i hope you answer this. Thank you

Ms. Ana, "filling missing data" is a very important research area. Data assimilation method is not only a interpolation process it is also to resolve how representing the real world. For example, "multiple imputation" has stochastic characteristics should be applied when there is no available spatial data. The other unbiased method "kriging" has advantages over other methods like IDW, Shepherd etc. I have just applied for daily wind data that the "least square directional kriging" has better performance over the other methods (with correction altitude).  I suggest you firstly read the book of Kalnay on this topic...   

Dear Ana, after greetings, Sorry for not having answered earlier.  I have encountered this problem before, and I've solve them using the forecasting tool as in the attached picture, it has been relying on a suitable record for at least 30 years old to predict about five years, and you must find a relationship between the station is not the missing data  and the other is forecasting data.

Dear Ana,

There 3 approaches on this issue: 1) Objective or deterministic, i.e. Inv Dist, Cressman, Surface Splines etc. 2) Statistical and stochastically, ie. Kriging, Imputation etc. 3) Data assimilation based (see Kalnay et al. al), Kalman filters etc. 4) to me, I suggest firstly clustering the data, and then applying the simple kriging or inv. dist. Added the correcting the elevation effects, as in the temperature and humidity values…

If you ask which method is right. The answer to this question is not yet. A few years ago, a general manager of a major scientific institution said, “…I believed that the most important way to understand the world was through the solution of the differential equations, I worked on the improvement of the solution of the differential equations for 30 years, we might have the ability to solve very complicated mathematical problems, but how close we are to the actual truth. I learned this after 30 years ..” :-))   

Good luck,

you can visit my "GitHub"

https://github.com/valizcanos/IDW_PARA_EL_LLENADO_DE_DATOS_FALTANTES