A t the moment I'm developing ML based model for the prediction frauds. But, I have a small issue on feature selection for the my proposed model (Fraud Prediction).I have gone through Several articles and tools to get the best features that describes the model more accurately.
- Weka > Select attribute
- Scikitlearn > sklearn.feature_selection
- Scikitlearn > forest.feature_importances_
Unfortunately non of packages offers bench mark or rule of thumb to select best features that describe the model. I would be appreciate that if you could kindly let me know the best mechanise/method to identify best feature which I could use for the prediction model.
Hi,
The approach is to search/rank for the features, evaluate the selected feature subset, and pass the selected features to the classifier. This process has to be done under the evaluation of k-fold cross-validation, for instance. Bear in mind that this is an exploratory task, so you need to try many techniques before confirming the result.
HTH.
Dr. Samer Sarsam
https://datascience.stackexchange.com/questions/11390/any-rules-of-thumb-on-number-of-features-versus-number-of-instances-small-da
http://www.simafore.com/blog/3-key-considerations-for-machine-learning-on-big-data
https://www.quora.com/What-are-some-standard-approaches-rules-of-thumb-to-feature-engineering-in-competitive-machine-learning-or-otherwise
http://martin.zinkevich.org/rules_of_ml/rules_of_ml.pdf
https://machinelearningmastery.com/feature-selection-machine-learning-python/
Dear Hiran,
In principle, there is no rule of thumb or "magic" method to perform feature selection. There are filtering, wrapper and embedded methods that can be tested on a given dataset (have a look at the following link to get a better idea of different options available: https://towardsdatascience.com/why-how-and-when-to-apply-feature-selection-e9c69adfabf2).
It is very important to perform feature selection and model estimation on the train set and evaluate the performance of your final model on a separate test set that is not used neither to do feature selection nor model estimation (k-fold cross-validation, holdout, leave-one-out, bootstrapping or any other validation strategy). Otherwise, you may get good results which will not replicate in future samples.
For fraud detection, you will need more than maths to identify your best predictors (features, independent variables). You will also need to imbed yourself in the problem domain, and then use your head. Think carefully ... which of your statistically predictive features can't be easily changed by the perpetrator? I ask you this because, as soon as you put the resulting model into production, blocking the perpetrator's fraud attempts, the perp will get instant feedback from your intervention and try to modify his or her own behavior to avoid your trap. If your silver bullet variables represent behaviors that are easy for the perp to change, your model won't remain predictive and you will soon have to start all over again, cat and mouse. Try to choose a combination of predictors (non-linear combo style) that are too subtle to be easily modified by your adversary. The goal is to keep the perp engaged in the same stupid behavioral patterns as long as possible so as to make your fraud hunting more like fish farming than ocean searching.
- Badges
- Science method