I used Shannon entropy as Feature selection to reduce inputs of artificial neural networks, it's improving the accuracy of the model, and the number of neurons in the hidden layer is reduced.
Can I use this technique?
Best regards
You can use Shannon entropy/Information entropy as means of comprimation of the original system, which is often a complex one, into a restricted set of features. In other words, entropy serves as a probe of the full often complex system.
AN or ML techniques can be applied to such preselected features.
I found this approach very useful during the design of a predictive model that is capable to predict deadly heart arrhythmias up to one hour before their onset from ECG recordings processed by permutation entropy that is fed into Shannon formula.
Details can be found in the preprint of the paper on my RG profile "Application of Machine Learning and Complexity in Medicine: Prediction of Drug Induced Arrhythmias in Rabbit Model up to One Hour Before Their Onset ...". ML methods were evaluated by my coworker.
The paper itself is quite extensive. The full version should occur in the coming months.
***
Details can be found here in the case of interest
https://www.researchgate.net/publication/340460356_Application_of_Machine_Learning_and_Complexity_in_Medicine_Prediction_of_Drug_Induced_Arrhythmias_in_Rabbit_Model_up_to_One_Hour_Before_Their_Onset_early_version_requested_due_to_COVID-19_unfinished_m
Yes, but not in general. A good survey on feature selection is here https://upcommons.upc.edu/bitstream/handle/2117/97413/R02-62.pdf and Conference Paper A Survey Of Feature Selection And Feature Extraction Techniq...
shanon entropy is a good measure to analyze features randomness.
using only this parameter, you won't select best features.
Shanon entropy is a fundamental parameter in mutual information.
Instead of using shanon entropy, use shanon
-based mutual information for feature Selection.
Mutual information is a good method for feature Selection.
Maybe these researchs give insight into the subject.
https://pdfs.semanticscholar.org › ...PDF
Information theory methods for feature selection - Semantic Scholar
https://www.researchgate.net › 3388...
(PDF) Feature selection in machine learning: R\'enyi min-entropy vs Shannon entropy
Thanks @ Jiří Kroc @ Okuthe P Kogeda @ Muhammad Ali @ Mohammad Norizadeh Cherloo for this interesting answes, it's very helping for the research.
My best regards
It's possible. I did this, but used a different measure of information (the measure of A. Harkevich). Now something similar is called neural networks.
Thanks @Ujjwal, I will made a comparative study on SE vs MI and partial MI.
Best regards
Messaoud
- Badges
- Science method