I have a problem with data analysis.
I have several very long time series taken from an Oscilloscope in different situations.
I want to see if there are repeated behavior in time of the chaotic situation that I see normally. The experiment is simple. A multimode laser going in a multimode fiber and connected to Fast Detector connected as well to the scope.
Let's say I have a time trace of 10.000.000 point and long 20 s.
Is there any way easy to find patterns inside the time trace or things that repeat in time range with Matlab or any program? The problem is also that I don' t know exactly what repetition behavior to find and how long is the repetition rate but I see sometimes on the scope that particular behavior repeats in time!
I know is a bit of mess but I think should be an elegant way to see pattern
in long time series.
I think the keyword here is motif discovery. Time series motif discovery is an active research area. An example publication is:
http://siam.org/proceedings/datamining/2009/dm09_045_mueena.pdf
It can be a good starting point.
For sure you can use wavelets. I aswered a question giving an example of this. See https://www.researchgate.net/post/Analysis_with_time_series_historical_data?ch=reg&cp=re68_x_p2&pli=1&login=amores.maimo@gmail.com#view=4fd1f946e39d5e3c2e000000
If you think this could be useful for you, just let me know and I will try to help you as much as I can.
Hi Alessio, I would say that the tool may depend on the type of repeating pattern that you see/expect in the data. For instance, do you expect the pattern to always be the same, or to change with time? What could be a likely type of recurring pattern? (spikes, waves, what else?)
In general, continuous Wavelet analysis could a nice tool to start with. Basically, with that type of analysis you can see how the frequential components of your signal change with time. Matlab has an entire toolbox implementing it (look for the function cwt). The idea could be to focus your examination on those time instants showing similar frequential contents.
I think the keyword here is motif discovery. Time series motif discovery is an active research area. An example publication is:
http://siam.org/proceedings/datamining/2009/dm09_045_mueena.pdf
It can be a good starting point.
Hi Alessio,
One approach is to use the Dynamic Harmonic regression (DHR) routine in the CAPTAIN Toolbox for Matlab (download via my website at http://captaintoolbox.co.uk/Captain_Toolbox.html). When you have put the CAPTAIN folder in your Matlab pathe, type 'help dhr' in the command line and it will inform you how the routine is called; and type 'captdemo' to access the CAPTAIN demos. In the window that opens in the latter case, select the 'TVP command line demos' button and then the 'DHR -Air passengers' button in the new window that opens. This will then take you through an example. printing out the instructions and results in the command line window. This will provide you effectively with a 'template' for analyzing your own data. In this demo example, the peaks in the spectrum are provided but you will want to look at the AutoRegressive (AR) spectrum for your own data: you should use the ARSPEC routine to do this.
Contact me if you want to discuss this further.
Peter
HOw about using auto-correlation? See that the data is correlated with itself with a phase difference, for this auto-correlation analysis is a good one.
Hi Alessio, wavelets analysis is a linear transformation, not meant for detecting recurrent patterns. Of course, you can use wavelet decomposition to preprocess the time series in order to focus on some characteristic features and then run some kind of recurrence analysis. There are various ways of addressing recurrent behaviour.
1) One obvious reference is recurrence analysis as the very name suggests. Check Donner et al, Chaos 21 pp1019-1046 for a recent overview of the technique with some references to network analysis - potentially useful for your purpose due to substantial data length. There is software which implements recurrence analysis for Matlab, R, etc. If you have Matlab start here http://www.recurrence-plot.tk/rp-tutorial.php
2) If you prefer to use other methods, you can try to analyse two-point correlations, these you compute from raw wavelet coefficients (CWT) or from wavelet magnitude.
What do you think about simple time series analysis? in R for example there are several packages that are able to detect "seasonality", a recurrent pattern in your data and then try to analyse it with the autocorrelative methods (AR for example to understand which is the maximum autocorrelative lag)...I'm thinking about Okkam's Razor....try simple things before...and then, it they don't work try with more complicated ones.
I got the same thought.. why don't try ARIMA filters and see if you found some AR components.. in R is pretty straightforward
Dear all
Thank you very much for all the
answer on the topics.
Actually mainly due to the different
background of people, different
technique are proposed.
Maybe would be nice to apply
these different technique on the
same data set.
I can propose to share a working
folder on 'dropbox' with the raw data
and a file who explain the data and starting
data analysis in matlab.
Anyone that want to have a try
could access, just please let me know and
I will share the access.
I think something nice could come
out for all this variety of technique proposed.
Kind Regards.
If you want I can try to do a wavelet analysis as i'm used to do it and I have a computer which can deal with a huge amount of data.
What do you think?
Hi Angel, tahnks I just create a folder on dropbox with the files and matlab file. If you sent me your e-mail address i share with you.
Cheers
Did you try to calculate the Lyapunov exponent between the different time series? it will give an indication of the presence of chaos and its degree. good luck.
- Badges
- Science topic