I have downloaded daily climatological data from the Global Historical Climatology Network (GHCN), but it is provided in a not very common format such as .dly. I have found some R packages related to this, but apparently they are aimed at retrieving temperature measures. Any suggestions?
In R you can use the function "read.fwf". This answer is based on the structure of the dly file as reported here: http://www1.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt
read.fwf('ACW00011604.dly',widths = c(11, 4, 2, 4, rep(c(5, 1, 1, 1),31)))
This gives you a data.frame with 128 columns. You can do some fancier fixing to get the variable of interest, but this works as a first order solution in R.
Could you provide a link to the data or attach an example .dly so we can check the formatting?
If .dly is just a text file with specific delimiters, MATLAB's "importdata" function would probably be the simplest solution. See the attached link. There's also an "Import Data" button under the "Home" tab that provides an intuitive GUI for importing data of virtually any type.
Hope this helps.
http://www.mathworks.com/help/matlab/ref/importdata.html
Mmm I'm afraid that doesn't work, it's not the typical delimited format.
I'm attaching an example.
Thanks Delyle.
The most straightforward method is to use FSCANF. The .DLY format is a simple variable length file with the elements blocked into fixed positions (e.g., variable is given in spaces 18-21). See http://www1.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt for the list of elements and their positions on the line. Once read in using FSCANF, do a FIND command on the column for variable and look for PRCP. Use that index to subset the data matrix for just precip data. Good luck.
I'm trying to do the exact same thing but have limited experience in Matlab - Daniel, have you had any success with this?
In R you can use the function "read.fwf". This answer is based on the structure of the dly file as reported here: http://www1.ncdc.noaa.gov/pub/data/ghcn/daily/readme.txt
read.fwf('ACW00011604.dly',widths = c(11, 4, 2, 4, rep(c(5, 1, 1, 1),31)))
This gives you a data.frame with 128 columns. You can do some fancier fixing to get the variable of interest, but this works as a first order solution in R.
For any future confused people who stumble across this thread, I also found a toolbox here:
http://gce-lter.marsci.uga.edu/public/im/tools/toolbox_download.htm
It is open access and runs in matlab, letting you import your variables without actually writing any of your own script. Good user interface and would highly recommend!
@Simon Goring - Can you explain how this works, which is does:
read.fwf('ACW00011604.dly',widths = c(11, 4, 2, 4, rep(c(5, 1, 1, 1),31)))
Thanks!
its been 3 years now, since the query was made! but here is the link one can check out to access GHCN data directly into R
https://ropensci.org/blog/2014/03/13/rnoaa/
I have come across this post when I first started looking into using the GHCN-Daily database, as I also encountered challenges when first trying to access the data contained in the .dly files. I ended up writing my own script in the MATLAB programming language (compatible with GNU Octave) and have published the toolbox (ghcnd_access) to facilitate data retrieval by other novice users. The toolbox can be accessed from GitHub: https://github.com/jjaffres/ghcnd_access
You can find a great answer here:
http://spatialreasoning.com/wp/20170307_1244_r-reading-filtering-weather-data-from-the-global-historical-climatology-network-ghcn
It may help new data users.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Statistical Software
- R