When the time deviation between the annotation of the expert and what is labeled by a peak detection algorithm exceeds the tolerance limits (see the picture), should we consider it as a False Positive or as a False Negative?
Dear Erick,
In a matching windows T:
1. If a detection time B is signaled and an annotation time A (gold standard) is found within T, i.e. |A-B|≤T, then the detection must be counted as a true positive (TP).
2. If a detection B is signaled but there is no annotation A around within T, i.e. |A-B|>T, then the detection must be counted as a false positive (FP).
3. If an annotation A exists but there is no detection B around within T, then the detector has missed the annotation and thus a false negative (FN) must be counted.
In 2 and 3, |A-B|>T, but in 2 you ask about a detection that was signaled but there is no annotation close enough whereas in 3 you ask about an annotation that exists but there is no detection close enough. So, in your code, 1 and 2 are part of the same if-else statement that checks for detections signaled whereas as 3 is part of another if-else statement that checks for annotations that exist; however, if an annotation and a detection are encountered, and they were already considered in the if-else statement in parts 2-3, then a TP must not be counted since it was already counted.
Finally, to answer your question, on one hand, since the red dot (annotation) does not encounter any black cross (detection), the detector has missed the event and an FN occurs; on the other hand, since the black cross does not encounter any red dot, the detector has committed an FP. Therefore, you have one FN and one FP.
I consider that the book of Sörnmo and Laguna (page 508) explains very well this procedure, for the case of heartbeat detections.
Sörnmo, L., & Laguna, P. (2005). Bioelectrical signal processing in cardiac and neurological applications (Vol. 8). Academic Press.
Thank you Miguel,
In summary, and as shown in the Figure above, I may have either one FN or one FP, but not both, right?
You have both, one FN because there is no detection close enough around the annotation, and one FP because there is no annotation close enough around the detection.
- Badges
- Science topic
- Similar topics
- Biological Science
- Physiology
- Biosignals
- Biomedical Signal Processing