I have implemented Stacking classifier using Decision Tree, kNN and Naive bayes as base learner and Logistic Regression as metaclassifier (final predictor), stacking has increased the accuracy in comparison to individual classifiers. The problem is multiclass (6 classes) with categorical target (Target= Activity performed by user exa: walking, running, standing.... on UCI-HAR dataset). Now I am unable to understand:
1. How Logistic Regression is working on the output/prediction of base level classifier?
2. What will be the final output of Logistic Regression if: model 1 is giving class 1, model 2 is giving class 2 and model 3 is giving class 3 as their prediction. (exa: DT-Running; kNN-Walking; NB-Standing, then how logistic regression will decide the final output)? If possible kindly explain why and how.
Dear Sunita Chaurasia
These links seems to be useful, have a look:
1. https://www.mdpi.com/2076-3417/11/18/8489/pdf
2. https://stackoverflow.com/questions/69741791/stacking-classifier-with-dt-knn-and-nb-as-base-learner-and-logistic-regression
3. Article Early detection of coronary heart disease using ensemble techniques
4. Article Improved prediction of slope stability using a hybrid stacki...
5. https://link.springer.com/article/10.1007/s13369-021-06086-5
Kind Regards
Qamar Ul Islam
Hello, there are actually multiple ways to stack or ensemble machine learning classifiers. Could you please mention which method (e.g. Voting/Proabilisitic etc) you are using for considering the contributions of individual classifiers? Sunita Chaurasia
Muhammad Sakib Khan Inan what I know is when using voting classifier we can ensemble using Hard voting (maximum votes) and soft voting (using probabilities); but I have not used voting instead I have used stacking classifier i.e., three classifiers at level 0 (DT, kNN, NB) and on the output of these individual classifiers (predictions) I have used meta learner (logistic Regression) which is working as final classifier on this new dataset (prediction from individual classifier). (the meta-classifier is fitted based on the outputs -- meta-features -- of the individual classification models. I have taken prediction of individual classifiers to make new dataset). This stacking is increasing the accuracy also; but what I am not able to understand that what will happen if each of these three classifiers will predict different class. (how logistic regression will work); thank you for showing your concern.......
- Badges
- Science topic
- Similar topics
- Phytochemistry
- Extracts