Dear Arun Anoop Mandankandy

Generally, Feature selection is for filtering irrelevant or redundant features from your dataset. The key difference between feature selection and extraction is that feature selection keeps a subset of the original features while feature extraction creates brand new ones.

For more details explanation kindly look into these articles given below:

1. https://www.analyticsvidhya.com/blog/2020/04/feature-scaling-machine-learning-normalization-standardization/

2. https://towardsdatascience.com/all-about-feature-scaling-bcc0ad75cb35

Kind Regards

Qamar Ul Islam

Dear Arun Anoop Mandankandy ,

feature scaling/normalization is very useful in some cases, as each feature may have a different range, and this may impact the performance of some classification algorithms.

As for k-nearest neighbors (kNN), the processing includes distance computation, thus features with a greater range will have greater importance in predicting the final output.

As for artificial neural networks (ANN), they work better if the input data is normalized.

As for other algorithms (e.g. decision trees), input scaling does not affect the final performance.

Feature selection is a different preprocessing task, aiming to select the best feature subset (i.e. the one providing the best performance).

Some classification models have an intrinsic feature selection process, giving greater weight to the most important features (e.g. ANN). This is called embedded feature selection.

When this is not true, feature selection leads to better performance.

You can perform feature selection prior to classification (filter approach) or you can use a classification model to choose the best feature subset (e.g. using cross-validation), then use another classification model for classification.

Luigi Borzí Qamar Ul Islam

Thank you.

Best answer I received.

Yes in KNN, there is a distance method called 'Euclidian'.

During evaluation I used "Startified 10-fold cross validation" also. Based on my evaluation, " 'with standard scaling+classifier' & 'with cross validation+classifier' " result is almost same as "MinMax scaling+optimization algorithms".

I am agreeing with this "Feature selection is for filtering irrelevant or redundant features from your dataset." In order to that we get good classification accuracy. In these reason only I used optimization algorithm in my research work.

Without optimization, inside classifier algorithms itself, we can do 'tuning' process to get more accuracy. Optimization algorithm help us to get more accuracy at the time of considering huge dataset. Right??

My question is "Is it only the reason researchers going for optimization algorithms"?

Your most welcome Arun Anoop Mandankandy

Stay Home Stay Safe.

Hello,

Adding some practical tips over Luigi Borzí answer.

If you use Python for your investigations you have the excellent https://scikit-learn.org/ library that can solve feature scaling, selection and many much more data science algorithms, metrics...

For feature selection you can use some of the classes included in:

https://scikit-learn.org/stable/modules/classes.html#module-sklearn.feature_selection

And for feature scaling (translating the feature range to a known interval i.e. [0,1]) or standardizing (translating the feature range to mean 0 and standard deviation to 1) you can use the classes included in:

https://scikit-learn.org/stable/modules/classes.html#module-sklearn.preprocessing

In both cases you'll find many more functions to perform similar tasks, it is worth to invest some time to study the Scikit-learn API and its wealth of classes, functions and lot of practical examples and use cases.

I hope you find it useful.

Best regards

Jose Manuel Vericat yes sir thank you for your valuable comments. I used python scikit learn for scaling( standardization and Minmax) and classification processes.

With that alone is giving good accuracy, then what is the use of optimization algorithm?

I hope if our dataset is too large, may be better option is optimization algorithms.. am i right sir?

Hy Arun Anoop Mandankandy ,

When you have a dataset and you test several algorithms with default parameters, and one these algorithms performs reasonably well, optimization serves to tune the default parameters in order to improve the final performance of the selected algorithm.

I do not understand the relationship between the size of a dataset and optimization. Never a dataset is too large, data is your power. If a dataset is too large to fit in memory or other issues that can appear, you can randomily down sample it in order to get a representative set of observations and work with it.

Regards

Feature selection techniques are used for different reasons:

• facilitation of models to make them easier to interpret by researchers.
• shorter training times.
• avoid the curse of dimensionality,
• enhanced generalization by reducing overfitting

Feature scaling is a method used to normalize the range of independent variables (features) of data. it is also known as data normalization and is generally performed during the data preprocessing

Bahaa Saif great answer sir. Thank you for your valuable comments.

Not at all

feature selection is important in high diminsional data so that we can reduce the runing time of the training algorithm.feature scale are important in algorithms such as svm , logestic regresion,....

Fortunately, decion tree is not sensitive to the scale of the data

📷