Chapter Multi-objective Feature Selection: Hybrid of Salp Swarm and ...

The key solution is that you need to design the objective function well. For example, you can design the mean square deviation between the regression objective and the predicted value as the objective function, and calculate the objective function after dimensionality reduction by feature selection. Whichever objective function is small, you can choose which feature selection method.

Case（If you need anything, please contact me：[email protected]）：

import numpy as np

from sklearn.linear_model import LinearRegression

from sklearn.metrics import mean_squared_error

from sklearn.feature_selection import SelectKBest

from sklearn.feature_selection import f_regression

from sklearn.datasets import load_iris

# load data(150 samples, 4 features, 1 target)

X, Y = load_iris(return_X_y=True)

# regression before feature selection

model = LinearRegression()

model.fit(X, Y)

y_predict = model.predict(X)

mse = mean_squared_error(Y, y_predict)

# regression after feature selection

X_new = SelectKBest(f_regression, k=3).fit_transform(X, Y)

model.fit(X_new, Y)

y_predict_fs = model.predict(X_new)

mse_fs = mean_squared_error(Y, y_predict_fs)

print(mse, mse_fs)

Dear Joao Luiz Junho Pereira ,

Look the link, maybe useful.

Article A Meta-Heuristic Regression-Based Feature Selection for Pred...

Regards,

Shafagat

Dear Joao Luiz Junho Pereira ,

You may find these articles useful,

Article Hybridization of ring theory-based evolutionary algorithm an...

Article Feature selection based on meta-heuristics for biomedicine

Article A Meta-Heuristic Regression-Based Feature Selection for Pred...

https://staff.fmi.uvt.ro/~flavia.micota/AlgMeta/2017-2018/proiecte/biblio/aplicatii/feature_selection/TS%2BFS/Metaheuristics%2BfeatureSelection.pdf

Hope these articles will help you.

With Regards,

Sayan Surya Shaw.

One of the possibilities to apply is so-called drop-out loss. If there are 6 independent variables, it is to calculate output 6 times. In each attempt, just one independent variable is permuted (false data are assigned to the cases). The bigger MSE (or another kind of error) is achieved, the more important the variable is. The drop-out loss can be applied with any kind of model based on several independent variables.

As you mentioned, in relation to the feature selection of classification methods have been used approaches, for regression-based FS can be a similar structure. For instance, the combination of ANN with optimization methods while optimization goals are the amount of errors and the number of features or errors with R2. Depending on the number of objectives and the type of optimization, it is useful to use multi-objective (MOPSO, MOGWO, NSGAII) or single-objective optimization (GA, PSO) methods in multiple repetitions.

I recommend reading the following relevant paper which has useful information to answer your question:

Article Highly Accurate Prediction Model for Daily Runoff in Semi-Ar...

You can use the GPR instead of KNN in matlab for feature selection.