I am interested in any big data projects that are focused on ocean data and information. "Big data" is loosely defined as data so complex that they can not be processed or managed using traditional tools and techniques. The complexity of big data is often described as emerging from one or a combination of three sources: The sheer volume of the data, the variety of the data, and the "velocity" of the data in terms of its frequency of measurement. Can anyone point me to particular projects or activities that are addressing big data issues for the ocean?
As a follow-up question... did anyone out there notice the new Big Earth Data Initiative (BEDI) in the President's Proposed FY2014 budget? $9M to each of NOAA, NASA, and USGS of new funding to tackle some of the problems with Big Data for the environment. Should be interesting to see how it makes it through the Congressional budget process.
You can get in touch with Jean-Francois Piolle or Fred Paul at Ifremer, they can tell you many stories about satellite data processing (high resolution SST over the globe, SMOS data ...)
Thanks Fabrice! I work with Jean-Francois in the Group for High Resolution SST (GHRSST, http://ghrsst.org) and am familiar with some of his work. It is a small world ;-) He reported last year at our annual meeting an interesting project they conducted using cloud resources... I think it was using Amazon Web Services, but it might have been a private cloud run by IFREMER... they downloaded most of our archive of GHRSST data (tens of terabytes, from http://ghrsst.nodc.noaa.gov) and reformatted it from netCDF-3 to netCDF-4 with internal compression and chunking turned on to improve access performance when serving the data via a THREDDS Data Server (TDS). Great work!
Thanks Ana! I'm looking at their web page now... I appreciate the information.
As a follow-up question... did anyone out there notice the new Big Earth Data Initiative (BEDI) in the President's Proposed FY2014 budget? $9M to each of NOAA, NASA, and USGS of new funding to tackle some of the problems with Big Data for the environment. Should be interesting to see how it makes it through the Congressional budget process.
- Badges
- Science topic