Time series reveal hidden behaviour in the form of low frequency cycles that is named long term memory or persistence. Short term memory is usually described with the autocorrelation function and it is possible to model stochastic processes using, for example, a Cholevsky decomposition, nevertheless I did not see any work preserving the long term memory, yet. A property of persistance is that it remains the same when using diferrent time resolutions for the studied process. A classical example are discharge flow time series in rivers, rainfall, solar activity index etc. To see the hidden cycles of persistance a transformation of data is performed, please see the attached file. Attached you can see a transformation of the Magdelena river discharges. The transformation performs as follows:
Let's say Qi are Magdalena river discharges at time "i" then ki=Qi/Qo where Qo is the Q expected value. Let Cv be the coefficient of variation for Q time series then we define CDIi = sum((ki-1)/Cv) as de cummulative sum of standarized ki.
In the attachment the red line and the blue one are the same charts but using Qi at a different time resolution (red is monthly data, blue is daily).
I don't know about your specific application, but to model the long-term trends in your data typically requires a parametric model of the trends. For example, if you have periodic behavior in your data, you could fit a sinusoidal function to it, and then use an autoregressive model on the residuals. Thus, the AR model tells you about the short-term variation in your data, while the sinusoid tells you about the long-term variation.
If you seek for a parametric model, consider ARFIMA(p,d,q) which is a popular class of time series allowing for long memory. Parameter d directly controls the rate of decay of the autocovariance function, see, e.g., Hosking (1981) "Fractional Differencing".
There are recent publications that are very useful on this topic of long-term memory (best known to hydrologists as the Hurst phenomenon). There seems to be a resurgence of interest in the phenomenon, associated with hydrologic extremes and their return intervals. I found a few excellent book chapters. These chapters reference relevant journal papers, but not all the information that's in the chapters is in the journals, so I recommend you purchase the books if you can. Some of the chapters explain how to use ARFIMA (which Grigoriy pointed you to, above) for this purpose.
1) Kropp JP and Schellnhuber HJ (Eds) (2011) In Extremis; Disruptive Events and Trends in Climate and Hydrology.
This book includes presentations of EXCELLENT quality on emerging topics that are difficult to find elsewhere. I HIGHLY recommend it. Many chapters will be very useful to you, but don't miss chapter 3 in particular, which describes the use of ARFIMA.
2) Sharma AS, Bunde A, Dimri VP, Baker DN (Eds) (2012) Extreme Events and Natural Hazards; The Complexity Perspective. (AGU book)
See these 3 chapters: by Bunde et al; Schumann et al; and Santhanam.
I hope these will be informative to you.
I would also suggest you go through the list of publications by Demetris Koutsoyiannis - see the list of topics on the right-hand side of the web page:
http://itia.ntua.gr/dk/
D.K. has been studying the "Hurst-Kolmogorov" effect all along, before it became such a popular topic as it appears to be these days.
I would suggest discrete Fourier transforms... that would reveal any periodic trends hidden within the data. The trick with Fourier analysis is, after identifying some periodic behavior, accounting for that behavior! That can be the most difficult thing to do.
I would suggest fractional or multifractional processes. With these processes you can take into account long term memory and the pointwise regularity with a unique parameter.
Thank you everybody for following my question. Just to clarify, as Mariza said Hurst pointed out the same behavior on his seminal paper, nevertheless he did not show how to model time series preserving this pattern. What I want to do is to model the time series of Q in such way that the main features of the CDI graph show up when building this graph with the simulated Q time series. Mariza is ARFIMA capable to do this? I have read D.K. papers and did not realize such kind of modelling but I will check again. In our experience we are able to model Q time series preserving some given Hurst coefficient, nevertheless from obtained simulated Q time series does not emerge the same CDI graph pattern. Please realize the graph I attached is not the series I want to model, it should emerge from modelled Q time series.
Mariza thanks for recommended literature will take a look for sure.
- Badges
- Science topic
- Similar topics
- Biological Science
- Bioecology
- Systems Ecology
- Landscape Ecology
- Connectivity