I have heterogeneous data and i like to know that is there any method to quantify the similarity and dissimilarity level between these two data. Two data graphically look like the picture i have attached.
What kind of information we can extract from these data?
Is there any machine learning approach to predict or understand the data?
Hi Sarpas,
It would really help to know more about your datasets. What were you measuring, over what time frame? Time-series analysis gets a bit stickier than some other more straightforward types of analysis... As a start, I would look up cross correlation analysis.
Good luck,
Dan
Hi Sarpas! I also think that if you can provide more information on your datasets, it would help us (me) understand the research problem at hand: comparing the datasets. Right now, I would suggest conducting a Kolmorogov-Smirnov test. This test is a goodness-of-fit test that compares whether a given set of data fit a theoretical distribution. But again, not sure if this apply to your research question.
The whole analysis will depend if you consider the oscillation relevant/important. You can focus on the similarity in frequencies, in amplitudes, and in trends (of the average signal over time).
I am not sure if the y-axis label ("Amplitude") is well chosen. To my understanding I would name this axis "signal" and see the difference over the oscillations as the amplitudes (the upper wave having a higher Ssignal but a lower amplitude than the lower wave!).
You could plot the signals of both series against each other to see how they are correlated.
You could do a Fourier transform to have a look at the frequency space of the waves.
There are surely many more thing you can look at - but it depends on the purpose what of this may make sense.
HI
Depend on what you need, if you need discriminate between them a non parametric test can be useful, the aspect of your time series suggest that there are high level of nonlinearity. So, frequency content could not be good tool to characterize. If you need a model able to predict future state from previous, you can see my of paper list, in particular:
Modelling time series using information theory
Divergence measure between chaotic attractors
good luck Luis
There are two aspects to consider: the means of the two data sets, are they significantly different? And then the levels of oscillation. What are you measuring? Now, perhaps this paper will help.
http://wwweb.eecs.umich.edu/db/files/sigmod07timeseries.pdf
@Nati,
"the wave frequency and amplitude seems to be the best parameters to separate the data" is not what I said. These are two possible things that may be relevant, but I don't know. It is also obvious that the two series have different "y-values" (what Sarpras labelled "Amplitude" [falsely?]) and that the series have different trends (the upper wave seems to have a decreasing trend over time, the lower wave seems to have no trend). But even all this may not be relevant and instead the focus is on the correlation of the osscilations. Sapras really did not give any information and background, so it is rather moot to speculate here.
Hi,
As mentioned above, data should be completely described before applying any parametric or non parametric analysis.
In my opinion, if the data are time series, one may be interested in the joint behavior of the series, ie, whether the two series are stationary or not, or whether any of the difference of these series become stationary (co-integrating process).
Farah Yasmeen
You may test the hypothesis like this
Yupper(t) = a*Ylower(t) + f(t), where f(.) is a "nice" function of time, say a low order polynomial ("background"), and a is a real number. Hypothesis testing is a part of statistical analysis. Here, however, you have no well defined quantitative similarity measure: the sets are similar when the hypothesis is accepted and dissimilar otherwise.
Spatial Efficiency metric (SPAEF) is proven to be robust when comparing two raster maps. Python and Matlab codes are available at: http://space.geus.dk/tools_products/index.html
Two papers on the metric:
1) https://www.hydrol-earth-syst-sci.net/22/1299/2018/hess-22-1299-2018-assets.html
2) https://www.geosci-model-dev-discuss.net/gmd-2017-238/
- Badges
- Science topic