I realize this is not an answer to your question, but stil, I would be interested to know how you would judge the potential of local distributed data (pre-)processing for the field of application you have in mind. Could it be feasible, what would be the major obstacles, which approaches are maybe already being used, and what benefit would you expect in terms of amount of data?

Dear Lehmhus,

Thank you for your observation.

Could you please explain in more detail what you mention by: "local distributed data (pre-)processing".

Many thanks,

Helder

Dear Dr. Sousa,

I'll try - if I understand your question correctly, the system you have in mind gathers data over an extended period of time, which is than collected and evaluated externally, thus creating the extensive databases you mention. I was wondering whether there might not be a possibility to extract information from the data e. g. at the sensor network node using locally installed computing power, and work afterwards with the information only rather than the raw data?

Kind regards,

Dirk

Dear Lehmhus,

Of course! That is a valid possibility. There are already some attempts to do that, mainly for dynamic analysis by computing frequency/mode shapes locally, and then send that 'summarized' information remotely to the central.

However, the SHM of long-span bridges, and even mid-span bridges, stills a big challenge. Many issues remain to be tackled, indeed. But the issue of big-data remains disregarded, in my personal opinion. A lot of focus has been given to sensors' development, architecture of sensor networks, energy harvesting, FE analysis based on monitoring data, but the question about what to do with the data that is increasing exponentially stills not fully recognized as a problem. And indeed, it is a problem. A big problem. My personal experience tells it.

Therefore, what kind of tools could be used for this? I know that the issue of big-data is already a subject of research in other fields such as social networks for example.

Many thanks for your feedback.

BR,

Helder

This may be an option:

http://www.paradigm4.com/scidb-py/

I have not used this, yet, but am looking for options for these same types of problems.

Reduction of data at the source is attractive from a data management point of view, but limits what you can do later when you discover wonderful new analysis methods, or unanticipated failures, etc.

Regards, Dave.