Working with health care application.
First applied PCA to reduce feature set then applied SVM (Non linear) to classify. got better results than using SVM alone.
But my doubt is. Initially I am reducing the feature size (PCA) then again I am mapping the feature to high dimensional space (Non-linear SVM) seems to be confusing. What is the role of PCA then?
Dear Gt,
The reduction dimension such as PCA have different goal. in the context of classification, the principal goal is to resume the information coming from high dimension data in a lower dimension representation with acceptable loss of information (you can estimate de % of inertia, information, from each new principal component). therefore with PCA you reduce the size of your data while retaining the essential information. Moreover in context of high dimension, you may suffer from the curse of dimensionality, also known as Hughes phenomenon. due to the concentration norm of data, the high dimensional data can provide a decrease in the performance of classifiers.
here the reference : Hughes, G.F. (January 1968). "On the mean accuracy of statistical pattern recognizers". IEEE Transactions on Information Theory 14 (1): 55–63. doi:10.1109/TIT.1968.1054102.
Best regards,
Ludovic
Instead of PCA you could also use autoencoders as done for example in deep neural networks
As Ludovic Journaux has said, those are two rather different types of space transformations. PCA is aimed at reducing the dimensionality of the problem while retaining as much information as possible (basically, from a variance point of view). The mapping that non-linear SVM does is aimed instead at producing linearly separable classes in the higher dimensional space, it does not deal explicitly (unless you use a problem-dependent kernel) with the information content of your data. Generally speaking, it is perfectly acceptable to use PCA and then non-linear SVM and, in my opinion, it is also worth the effort, as you firstly try to extract the most significant information from your data (note: other filtering and feature extraction techniques besides PCA may also be appropriate) and then perform classification on this new, hopefully more representative and simple-to-analyze, dataset.
For further information you may take a look at a paper that I and my colleagues recently published. I suggest it because it is focused on PCA and classification with SVM, ANN, and LDA of body-related data (in particular, muscle electrical activity), thus it may sound familiar to you as you are working in the field of health care.
Best regards,
FC
Article Optimization of EMG-based hand gesture recognition: Supervis...
I would like to comment on PCA a little bit (cf. Francesco's answer). In general, you can (and you should) apply feature extraction methods to create clusters with small intra-class distances of your objects and a large inter-class distance of multiple clusters. So far, so good. However, you must take care what method you will use. The classical PCA approach is often applied, but not very helpful. Why? PCA or the like does of course data reduction, but bey the maximisation of the variances! This is not what we want in classification, because we get (timewise) non-stable results! Be aware of this effect. You might use e.g. better approaches (Linear discriminance analysis based) like this: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6648126.
Use PCA for dimension reduction for your data set and use SVM for classification
- Badges
- Science topic
- Similar topics
- Geoscience
- Cartography
- Mapping