In order to test any event detection algorithm, it is common practice to compute a confusion matrix, in order to get performance parameters (e.g. sensitivity, and specificity).
It is well known that the confision matrix must be built (identifing and) counting the number of negatives/positives true/false detections, according to a pre-annotated dataset (Gold Standard).
Now, here is my question: how can i identify a true/false positive?
Let me show you a little example.
My algorith wants to detect QRS-Complex's peak, in order to get R-R intervals of a ECG recording. My pre-annotated dataset identify QRS complex not on QRS peak, but some samples before.
It is obvious that, testing my algorithm, i cannot identify a true positive detection "when my algorithm identify the same point as the gold standard", since there's always a time-shift between them. At the other side, all of my detections (because of the time-shift) would be identified as false positives (no time-coincidence with Gold Standard).
An easy solution would be to consider a "coincidence-window" around each point in the Gold Standard but, if i fix the problem with true/false positives, the problem with true/false negatives remains still unsolved.
Has anybody an idea? (Any reference would be great for me).
Thanks a lot.
Best regards,
Marco Bilucaglia
I recommend reading these surveys:
Chandola, V, Banerjee, A & Kumar, V 2009, ‘Anomaly detection: A survey’, ACM Comput. Surv., vol. 41, no. 3, pp. 1–58.
Chandola, V, Banerjee, A & Kumar, V 2012, ‘Anomaly Detection for Discrete Sequences: A Survey’, IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 5, pp. 823–839.
Varun Chandola, Deepthi Cheboli & Vipin Kumar 2009, Detecting Anomalies in a Time Series Database, Department of Computer Science and Engineering, University of Minnesota, 4-192 EECS Bulding, 200 Union Street SE, Minneapolis, MN 55455-0159 USA, Minneapolis.
If I remember correctly: If the process is stationary and the distribution of the random walk is parametric, then state-space methods for time series analysis can be appropriate to use.For example, the R package MARSS implements this. If the distribution of the random walk is non-parametric or the process is not stationary, then simulations need to be employed. Perhaps, particle filtering can be employed.
- Badges
- Science topic
- Similar topics
- Biological Science
- Physiology
- Biosignals
- Biomedical Signal Processing