Could some colleagues please suggest an effective way to read compound variables in (hierarchical) netCDF files, (one of the capabilities of the new netCDF4 format), in the R system.
I have a file of this type (using ‘ncdump', where the 'group: tracks-000*' repeats several times):
group: tracks-0006 {
types:
compound track_dtype {
int CycloneNumber ;
double Datetime ;
float TimeElapsed ;
double Longitude ;
double Latitude ;
double Speed ;
double Bearing ;
double CentralPressure ;
double EnvPressure ;
double rMax ;
}; // track_dtype
dimensions:
time = UNLIMITED ; // (13 currently)
variables:
double time(time) ;
time:units = "hours since 1900-01-01 00:00" ;
time:calendar = "julian" ;
track_dtype track(time) ;
track:long_name = "Tropical cyclone track data" ;
track:time_units = "hours since 1900-01-01 00:00" ;
track:calendar = "julian" ;
track:lon_units = "degrees east" ;
track:lat_units = "degrees north" ;
track:pressure_units = "hPa" ;
track:speed_units = "m/s" ;
track:length_units = "km" ;
track:trackId = "(6, 33)" ;
data:
time = 43116601, 43116602, 43116603, 43116604, 43116605, 43116606,
43116607, 43116608 ;
track =
{6, 43116601, 0, 258.600006103516, 10.8000001907349, 31.6361141204834, 278, 1002, 1005.63439941406, 51.3120727539062},
{6, 43116602, 1, 258.312957763672, 10.8394889831543, 35.2432289123535, 272.674530029297, 1002.35272216797, 1006.88269042969, 74.1966857910156},
{6, 43116603, 2, 257.990386962891, 10.8541202545166, 22.5748672485352, 275.178649902344, 1002.25671386719, 1006.89263916016, 74.0333557128906},
{6, 43116604, 3, 257.784362792969, 10.8723878860474, 15.8934631347656, 261.810882568359, 1002.39923095703, 1006.91198730469, 74.2270736694336},
{6, 43116605, 4, 257.640228271484, 10.8519821166992, 5.5399956703186, 262.884246826172, 1002.57611083984, 1006.93743896484, 74.4559326171875},
{6, 43116606, 5, 257.58984375, 10.8458023071289, 9.16022682189941, 271.360168457031, 1003.10247802734, 1006.98248291016, 75.1571273803711},
{6, 43116607, 6, 257.505920410156, 10.8477478027344, 4.37964916229248, 272.071868896484, 1003.06762695312, 1006.97888183594, 75.1133499145508},
{6, 43116608, 7, 257.4658203125, 10.8491697311401, 7.56261444091797, 285.567169189453, 1003.12829589844, 1006.98022460938, 75.1968688964844}
I am trying to read it using 'library(ncdf4)' and I got an error message:
>library("ncdf4")
>sesa_wnd Error in ncvar_type_to_string(rc$precint) :
Error, unrecognized type code of variable supplied: -1
Is there an R package (or an NCO tool) to read this type of netCDF files?
Thanks for any help you can provide.
Rgrs,
Augusto
Hi Augusto,
I don't know what do you mean by saying "R system". Is it an operating system? or a programming language?
Usually, I read the compound variables in (hierarchical) netCDF files by MATLAB. For example, if the file name is "test", and I want to extract the variable "Latitude". The MATLAB code for thsi task is the followng:
filename = 'test';
ncid = netcdf.open(filename, 'NC_NOWRITE');
varname = 'Latitude';
varid = netcdf.inqVarID(ncid, varname);
Lat = netcdf.getVar(ncid,varid);
netcdf.close(ncid);
Hope it helps!
Yours,
Yongbo
Thank you Yongbo:
The R system is a package for statistical analysis and modelling of data widely used in many areas of sciences and technology, see: (https://www.r-project.org/).
R has similar functions to read conventional netCDF files as the matlab code you sent. However those functions do not read the new hierarchical variable netCDF4 files, that is the problem! In my original question i included the structure of one of those hierarchical files. Thanks anyway.
Rgrds,
Augusto
Augusto,
Hi! Did you ever figure this out? I'm running into the same issue.
Best regards,
Allison
Hi Allison,
The solution I found (not the most elegant one) is to dump the netCDF file into a plain text file (using ncdump), then you can read the text file using any text editor, get rid of the headers and extract the data (for a large number of files it is better to write a little python or R script to do that programatically).
Hope it helps,
Augusto
Thanks! I had gotten a similar error when uploading MODIS data from NASA ocean color, but when I started using mapped instead of binned data, it worked. I worked off of this code: https://oceancolor.gsfc.nasa.gov/forum/oceancolor/topic_show.pl?tid=5954, altering the for loop a bit.
Thanks guys! You saved me. However, I'm not able to download data from different years in the same .csv. I have used the same script of you Allison, maybe you know to change the code.
Thanks a million in advance :)
Allison Bailey, I had gotten the same error also using MODIS data from NASA Ocean Color. The code worked for me. Thank you very much.
For those who are still incurring in that error message.
I had the same problem trying to read MODIS-terra KD490 data from NASA Ocean Color. I wasn't able to solve the problem by using the code suggested by Allison Bailey . I also tried using the R package "satin" which include a specific function for netCDF files downloaded from the NASA Ocean Color website (read.nasaoc), but I had gotten the same error.
I solved this problem by setting the download option "Type" in "Mapped" instead of "Binned".
Background: The NASA binned data format has some user-defined data types that are tricky to map to native data types in R, Python, etc. The NASA OCSSW Processing System (for linux and macOS) has `l3bindump`, which extracts binned data to ASCII. The HDFview program from the HDF Group can export data from binned files to ASCII.
Using the R ncdf4 library with NASA binned data, I get
```
Error in ncvar_type_to_string(rv$precint) : Error, unrecognized type code of variable supplied: -1
4.stop(paste("Error, unrecognized type code of variable supplied:", precint)) 3.ncvar_type_to_string(rv$precint) 2.ncvar_inq(groups[[ig]]$id, ivar - 1) 1.nc_open(fnm) ```
The RNetCDF extends ncdf4 and is able to show the structure of a binned file:
```
library(ncdf4)
fnm
- Badges
- Science topic
- Similar topics
- Engineering
- Control Theory
- Control Systems
- Control Systems Engineering