Feature selection method is used to identify input columns that are not useful and do not contribute significantly to the performance of Neural Network System.
I want to know how the forward stepwise method works for input feature selection.
.
the "vanilla" approach is
- find the feature which performs best alone
- given an already selected subset of features, find the feature which gives the best performance improvement when added to the subset ; add it to the subset of selected features and iterate until no significant improvement is found
.
more sophisticated approaches would require your model to allow the estimation of the significance of a feature in the model
in that case, after adding a feature, you can also consider withdrawing some features from the selected subset
hence, you iterate add/drop operations between a selected subset and a rejected subset until no improvement is found, with some taboo-like heuristics (do not add a recently dropped feature, do not drop a recently added feature) to avoid getting trapped
.
In Forward selection procedure, one adds features to the model one at a time. At each step, each feature that is not already in the model is tested for inclusion in the model. The most significant of these feature is added to the model, so long as it's P-value is below some pre-set level i.e., 0.05.
Dear Deeman Yousif Mahmood ,
I recommend you read these works:
They proposed state-of-the-art feature selection techniques which compute the importance of all the features at once. In addition, their methods take into account all the possible subsets of features.
- Badges
- Science topic