Hello,
For now, I am confused about the relationship between number of features that input into the classifier and the accuracy of this classifier. For example, I am using the random forest to do the prediction. I try to use the Haar features and histogram of the images and HOG features to train the random forest. However, I found that when I use only histogram features, the accuracy of this classifier is about 80%. I put the Haar features and the histogram features into the classifier, the accuracy of the classifier is improving to about 84%. When I trying to use these 3 features together, the accuracy of the classifier is decreased to about 82%. Could someone told me about how to solve this problem? I want to decrease the error rate, and don't know which feature is the best features for the classifier. Thank you very much
It sounds like you're overfitting to your training data.
How are you splitting your train/test data? And how are you choosing the parameters for the random forest? The answers to these will likely reveal where you are overfitting.
Before start combine two or more features make sure all features already normalized and give same weights for every feature or different weights for every feature (Different weighting will affect the different result).
Find the information by using keyword:
1. Normalization
2. Weighting features
Hopefully, this information will help you. Good luck.
Image recognition is not my speciality, but I can speak to general issues of data integration. Although it is commonly said that random forest does not suffer from issues of multicollinearity, this is not quite true. If there are enough variables that are highly correlated and this correlation structure is different for different variables, you are essentially weighting the trees to select more heavily based on those variables. If those variables happen to have less predictive power, this will decrease the accuracy of the random forest. This can be a particular problem when integrating datasets that contain some of the same information captured in different ways. You may be able to affect this by adjusting the number of variables selected per node in the random forest, or you may have to try removing redundant information between datasets.
You can try to use some feature selection method to determinate which features are really important. Irrelevant features sometimes bloat the forest, sometimes hit back the accuracy.
- Badges
- Science topic
- Similar topics
- Engineering
- Modeling