I am working with weather forecasting application of artificial neural network. My data set is based on weather values recorded every 15 minutes. However, it contains 9% of missing values. The problem is that there are cases where there are consecutive days with missing values. Is it possible to train the neural network using these data? Or, is there any way to deal with the missing values?
Red this article: Artificial Neural Network Model in Prediction of Meteorological Parameters during Premonsoon Thunderstorms.
http://www.hindawi.com/journals/ijas/2013/525383/
The following text may help you:
Little RJA, Rubin DB (2002) Statistical analysis with missing data, 2nd edn. Wiley, New Jersey
Most of the approaches in the literature can be grouped into three different types
depending on how the problem is solved:
• deletion of incomplete cases using only the complete data portion;
• imputation or estimation of missing data using the edited set, i.e.complete data portion with imputed values;
• use of model-based procedures, where the data distribution is modeled by means of some procedures. In this type of approache you may model the probability density function (PDF) of the input data (complete and incomplete cases). If you have also the need to classify the data, the PDF is also used to classify using the Bayes decision theory.
Regards,
Raffaele
In my opinion it depends on the problem. In general, NNs have no problem dealing with incomplete or missing data. The NN can fill in the gaps providing solutions even for points where it has not been trained. But that is true if the missing data is "randomly" distributed within the design space. Small random gaps are able to be "corrected" by the NN.
In your case you state that the missing data are consecutive, for example, 6 consecutive days of missing data recorded per 15 minutes. I think that is "too much" missing data. An important part of the design space is lost and no prediction can be made for this part of the design space.
So I am not sure if NN can work for this problem. It will have difficulties for sure, compared to the case where all data were in place... But you can certainly give it a try and see what prediction the NN can produce!
I used to use NNs to predict global solar radiation in UAE (United Arab Emirates). Missing Data were handled by taking (manually) the average of the same period for the year before of after. The more the data you have (number of years) the more NN can be successful in the prediction. Usually, we used to consider large period of data ... more than 10 years.
I think that is not much of an issue. The missing data is only 9% of your data instances, so you should still have enough data for the prediction. How you design your input features is important here. If your input is not using any antecedent reading then, you can simply cut away the missing set and go ahead with your prediction. It is important to note that if your missing data is the only data covering a particular scenario of wheather change, then your model is unlikely to make good prediction for such change.
- Badges
- Science topic