Im currently reading the foundation of modern time series analysis by Terrence C. Mills
In the fourth chapter he discusses the concept of difference data to remove secular trend and the history behind it.
On page 36, section 4.7 it says:
"Cave and Pearson then computed correlation coefficients for all pairs of pairs of indicies at each level of differencing. we shall content ourselves with reporting correlations ( ± probable errors) between tobacco and savings for d=0,1,....6, the d=0 correlation being the focus of concern...
d Correlation
0 0.984 ± 0.005
1 0.766 ± 0.065
2 -0.044± 0.182
3 -0.327 ± 0.181
4 -0.380± 0.188
5 -0.402 ± 0.188
6 -0.432 ± 0.204
It is clear that the large positive correlation between tobacco and savings at d=0 does appear to be spurious: by d=3 the correlation is negative and by d=6 significantly so."
From this research quoted in the book, a true relationship was uncovered.
This is very interesting for me as a quantitative analyst and amateur social scientist in applying these methods to regression analysis.
However no limit for the "required" amount differences that need to be taken of both a the variables is given to uncover a true relationship.
Is there a limit for differencing? can you over difference data?
Autocorrelation is a concept associated with a stationary time series. For such a series it may be shown that it is a function of the time lag between the time series items. If a time series is non-stationary it is not meaningful to talk of its autocorrelation. Such a series may have to be differenced to make it stationary before considering its autocorrelation. Often it is enough to difference a nonstationary at most twice to render it stationary.
Since Dr. Dimpfl has already and nicely explained what I wanted to tell you, let me just point out the useful notes of Prof. John Cochrane (click the link below).
https://faculty.chicagobooth.edu/john.cochrane/research/papers/overdifferencing.pdf#search=%27Brief+parable+of+overdifferencing%27
However, let me (modestly) tell you the points: differencing has pros (you can get the stationary series) and cons (some of important information disappear). For instance, if you difference three times, then you may get the stationary series. But then, your series is very different from your original time series data...
Summing up, in my opinion, you can difference only once or twice. No more than three times. I've never seen the paper with more than three differencing.
Hope this helps,
Mizuki
- Badges
- Science method
- Similar topics
- Methodology
- Research Methodology
- Qualitative Research