Dears,
I would like to interpolate hourly gridded data (e.g. spatial grid 10 x 10 km) to a non regular set of points. The dataset covers around 100,000 points and, within each point, 1 year of data (i.e. 8760 hourly time-steps). I have tried using QGIS but the system crashes when attempting to load a .txt file of 8762 coloumns (e.g. lon, lat, T1, T2, T3, ..., T8760).
Probably Climate Data Operators (CDO) commands could be more useful/efficient in managing such heavy datasets.
Do you have any advice or experience to share on how to load and interpolate heavy datasets?
Thank you in advance for your kind reply.
Best,
Giorgio
I wonder whether an R (e.g. interp, interpp, ...) or Python (e.g. scipy interpolate, ...) package may be useful here.
Just like Babak Mahdavi Ardestani said, I think you might need a scripting/programming solution. Since you have a lot of data, even if you are using good software you might not have enough ram to load all that data at once into your computer so processing data in smaller chunks so that you don't run out of your computer's resources is the way to go.
You should be able to do that with QGIS too, but using QGIS libraries in a combination with python but I haven't faced that problem so I can't tell you the exact steps. But in general, what you need to do is divide the data into smaller chunks and then interpolate it in those chunks and combine them.
Thank you Gaurav Atreya Babak Mahdavi Ardestani. As you said, I would be more prone to use some programming language in order to manage this type of datasets. Probably it would be better than divide files in smaller chunks in order to be processed in QGIS.
- Badges
- Science topic