I am trying to implement anomaly detection using principal component classifier proposed in this paper (https://users.cs.fiu.edu/~chens/PDF/ICDM03_WS.pdf).It proposes that instead of using only the major principal components, it is better to use major as well as minor principal components. For example, the paper uses major principal components that explain 50% of the total variance and the minor components having eigenvalues less than 0.2.
What I am unclear about is the selection of hard cutoffs such as 50% and 0.2. Is there any science behind it? Can anyone please explain? Thanks in advance.
Hard cutoffs are often not the best way to choose which principal components are in the model space and which stay out. Cross validation by data splitting is the best way to select principal components. In cross validation, it is not unusual to find that some major components and some minor components are necessary to build a robust model for a given analyte, especially if that analyte is a minor constituent of a mixture. Sometimes you will find that components are skipped; for example, a model might require components one, two, and three, skip four and five, and then require six in order to create a robust calibration that cross validates.
For multivariate normal data you can look at model to observation Mahalanobis distances and use limits based on Hotelling's T-squared distribution.
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts