I have hourly temperature data from 2 Automatic Weather Stations located less than 500 m distance. I would like to fill the gaps in both series using the data of the other AWS. Which methodology do you recomend me? How should I proceed?
In some periods, gaps are present in both AWS, can I fill these gaps in some way?
It sounds like your question involves what is known as the "imputation of missing data" (or "missing values"). There is a rather large literature on this topic, and wikipedia entry on imputation provides a basic starting point.
Hello Marti,
Yes, as David Morgan notes, you're looking at a data imputation problem. Ideally, there should be no discernible, systematic relationship between the likelihood of a temperature value being missing and the actual value of the temperature itself.
I might suggest you start by plotting comparable time temperatures from stations A and B against one another. Ideally, these would form a pretty straight line with strong correspondence. If so, a simple regression equation might be your best bet for imputing a missing value for one station, given the observed temperature from the other station. Note that there will be two equations: One to estimate station B values from station A values; and another for the other way around.
Good luck with your work.
Hello Marti,
in R, you can identify and diagnose missing values with the tsibble package
https://pkg.earo.me/tsibble/articles/implicit-na.html
Missing values can imputed then with the fable-package by some sort of forecasting based interpolation. This video by Mitchel O'Hara-Wild and Rob Hyndman explains that at time 32:20:
https://www.youtube.com/watch?v=6v3_AsbhqrE
The book by Rob Hyndmann has a short section on missing values.
Hyndman, R. J., & Athanasopoulos, G. (2018). Forecasting: principles and practice: OTexts.
https://otexts.com/fpp2/
HTH
Holger
In addition to Holger's recommendation, the R package mice also provides helpful functions for missing value treatment.
Best regards,
Florian
Thank you all for your answers.
Dear David Morse, I plot both stations and the correspondence is high. Should I calculate a lineal regression equation for the total shared values? or you suggest one equation for each hour (00,01,02, ...23 h)?
Thank you Florian and Holger for your R recomendations. I'll try to use it! ;)
Hello again, Marti,
If you have sufficient data points to evaluate correspondence of station values by hour, give it a try. You can assess equality of relationships across hours as well (it may be the case that hour makes little difference, but it's certainly an empirical question.)
Any imputation method is a guess as to what the missing value/s might have been, and so the results of which should be interpreted with suitable caution. As well, the degree of data missing is a factor that influences how much one imputation scheme's results might differ from those of others.
Good luck with your work.
Hi Martí Bonshoms
just three questions:
-Do you want a sufficient quality of the gap-filling time series or do you want the best possible?
-How long are the two time series?
-furthermore: does the AWS have other data like humidity or radiation in addition to the temperature data? If so, are these values missing in the same time steps as the temperature data?
My answer to your second questions: Yes, it is possible to fill the gaps even if temperature values are missing in both places. The recommended method depends on the answers to the previous questions. But there are chances, that I can help you calculating it using a framework besed on this paper:Article Introducing Gradient Boosting as a universal gap filling too...
- Badges
- Science method
- Similar topics
- Methodology