I am trying to build a classification based machine learning model using efficient feature selection methods that predicts a dependent variable based on few independent variables.
The objective of feature selection algorithms is to generate discriminating features to accurately distinguish the type of independent variable. However, I am struggling to find efficient solution around this problem related to generating discriminating features (currently the features belonging to different labels highly overlap with each other) for each of the unique labels of the independent variable.
In addition, there is a variation observed among the independent variables across different datasets. Hence, there is recommendation required around the suitable supervised machine learning algorithms that perform learning and prediction based on overall generalizations of the data and not instance based.
Any thoughts or guidance towards the potential solutions around the highlighted problems would be greatly appreciated.
* The independent variables hold numerical floating point values
Dear Saurabh Malgaonkar
These articles might be helpful, have a look:
1. https://res.mdpi.com/d_attachment/genes/genes-11-00717/article_deploy/genes-11-00717-v2.pdf
2. https://towardsdatascience.com/understanding-dataset-shift-f2a5a262a766
Kind Regards
Qamar Ul Islam
Dear Saurabh Malgaonkar,
Related to your, statistical modelling based approaches are most effective and applicable to your project. One way is to use Bayesian parametric framework of Maximum Likelihood Estimation (MLE), however., working with Bayesian one of the noticeable hurdle is calculating the normalizing constant (otherwise the procedure is very straightforward). The second option is to consider non parametric clustering approach by considering von Misses Fisher distribution, Langevin distribution, Kent distribution or the Bingham distribution defined on the unit hypersphere. Also you treat these distributions by considering MCMC sampling techniques, or Gibs sampling techniques. See e.g., https://www.jmlr.org/papers/volume6/banerjee05a/banerjee05a.pdf etc....
One way is to avoid such noisy variables. Variable selection process needs to be revisited. After taking distinct variables the features can be assessed to get robust results.
Thanks Qamar, Muhammad and Anirban for providing key pointers towards the solution for the encountered research problem.
You may try Gradient Boosted Algorithms like XgBoost. XgBoost has a robust capability of handling missing values, variations and confounding factors. Also, you might experiment by incorporating undersampling methods for noise removal and ensuring well distributed discriminating factors in your dataset.
- Badges
- Science topic