Dear colleagues,
I've been working with data from temporal series extracted from sensors.
I'm trying to identify anomalous values, i.e., values that distance much of the common values, and use ML to describe what has occurred.
Briefly, after setting a winding and make a clustering in each window, I set a reference window that serves as the basis for the sensors remained in the same clusters or moved between clusters.
However, I thought about generating a decision tree from each winding and comparing them to identify how different the trees are from others.
So, is there some idea to compare the sensors' values between the windowing and the movement of these sensors in the clusters?
Thanks for the contribution, and I hope this question can help many.
Greetings.
Dear Marcos W. Rodrigues ,
I would base my decisions on physical and logical reasoning.
I mean what are these sensors sense??
If they sense a random variable then one has to follow statistical methods to describe the sensor reading. One can calculate the most probable value and the standard deviation for example.
To detect the sensor movements across the clusters one can observe the motion
of the sensor across the boundary.
So i would propose that you introduce buffer regions interfacing the adjacent clusters and put counters in these interface of buffer regions.
This may be better that than one observe the cluster inside the cluster window itself.
What I brought here is jsut some thoughts and nor a rigorous solution.
Best wishes.
Dear Abdelhalim abdelnaby Zekry ,
Thanks for your contribution.
In fact, I found your reflection very important.
Yes, each sensor produces a random variable.
About Buffer Regions, can I understand as regions limited by the minimum and maximum values?
Would it be interesting to calculate each region's data gravity center to get the distance from the sensor's value to the center?
Thank you very much for your valuable contribution.
Greetings.
Dear Marcos,
If you are still working in this area, Dynamic time warping is used for analyzing patterns and comparing time series datasets,
In the following papers we invented a new algorithm based on DTW for comparing time series datasets in the same cluster to generate single time series as a cluster representative
https://www.researchgate.net/publication/346060936_Data_Reduction_Model_for_Balancing_Indexing_and_Securing_Resources_in_the_Internet_of_Things_Applications?_sg=nb5aWYtt_wlc9J0enMBfWst2DYMMIKIKG8MgeMLRjR7l4PE1Db6kZb2DL5ti93Yth4cqdnyfgOalnKldjMeot2aIs_3kO3JXjr5vXw3d.pFru8ip-hJNXFp9zVruddLWiOeJkEQNvp0bd-BzQGdIFhzq7QFX2kZKtD2OAPhfoIA0z-GkHGcRKxqmb6GO07w
- Badges
- Science topic