This is a trend detection kind of thing. I have time series data. I have to analyze it after every 5 seconds. I am taking an absolute sum of data values of every 5 seconds of data, dividing it with 100(number of data points in every 5 seconds). I got 83 average values. What do these average values mean? Please explain.
If you are taking a percentage of the absolute sum of amplitude values, I think that the values you obtained are somehow related with the dynamic amplitude at each one of the 84 segments. I would expect that highest average values correspond to segments where the amplitude of the signal values is high in comparison with other segments where the amplitude is low.
I do not understand why you take the "absolute" sum, but expect that, what you are doing is filtering (or smoothing) your data. Instead of taking the exact sample at every 5 sec, you are using a window of 100 observations (= 5 sec), and use this filtered observation (observations with equal weights of 1/100) as if it is actually sampled at the end (? or maybe in the middle of) 5 sec. If you are going to use a time series model with sampling time 5 s on a set of data sampled at a higher frequency (which seems to be your case), pre-filtering (either in time or frequency domain) is a standard method of reducing the noise in your data before modeling it. You asked for the "meaning" of these filtered observations, so I may say that the resulting observations are the hypothetical values that would be closer to the noise-free observations, if you were able to run your process in noise-free fashion.
Dear Burak
I am processing my data after applying low pass filter (in time-domain) in my case to capture desired frequency data. My data contains some negative values so i took the absolute sum. My data is sampled at 20 Hz frequency. It is recorded data of 8 Kilometers distance. I am dividing data in 100 meters of distance and analyzing each window. Now, tell me what exactly average of each window of 100 meters means? My each 100 meters may contain unequal data points. Please provide your view on it.
Thanks
Rajiv
OK, let me try to elaborate on the issue:
Take every 100th observation, and show it with y_100. In this case,
y_100 = E{y (t=100)} + e (t=100)
where the terms on the RHS are the expected value of y (i.e. noise free y) at t = 100 and random error at t = 100. Assuming random error is iid, variance of random error (and y_100) would be var(e).
Now, if you are not interested in modeling your process at (100+1)st time, but every 100 samples (as you do), you may decrease the variance of random error term by averaging over 100 samples. In this case, defining average of 100 observations as :
= mean(y_i) = mean(E{y(t=i)} + e{t=i}) = mean(E{y(t=i)}) + mean(e{t=i})
The expected value of both sides would not change, but the variance will change. Apply variance operator to both sides, you see that var() has dropped from var(e) to var(e)/100! That is, you have decreased noise level in your process. This would be desirable both for visual inspection and for further modeling of the data.
As I said above, let me rephrase it. All measurements you make are clouded by noise, hence the "actual output" (deterministic output of the process) is not equal to what you measure. The values you calculated by this way will be closer to the noise-free hypothesized process outputs.
One last point. In the original post, you talked about dividing your data into equal time bins, whereas you said in your latter post that the equal bins are actually distances. When you model these two types of data, differences are likely to occur in interpretations. However, if you only ask about the meaning of the averaging, they both have the same meaning I've tried to explain above. If you divide your data into equal bins of 5 sec, then is a better estimate for the constant quantity (whatever you are measuring) during that instant. If you are dividing your data into unequal number of data points for every 100 m, then you assume the measured quantity to be constant for 100 m intervals, you get a better estimate of that using averaging over 100 m. The key point is (referring to the equations above), the deterministic signal is constant (or negligibly changing) during that interval (time or distance), so that your averaging ONLY decreases the variance of error, but does not affect your deterministic signal.
Hello Burak
It is little hard to understand what you suggested. Actually my number of data samples are not equal for every 100 meters of data. So i think, i should work on data for a fixed time interval. But here i can loss some important features i think.
- Badges
- Science topic
- Similar topics
- Medicine
- Physiology
- Biosignals
- Biomedical Signal Processing
- Statistical Analysis