I want to classify my data with SVM with optimized features using PSO. can any one help me to define costfunction(fitness function or optimize function) for feature selection. Currently i have written following cost function.
1. i am confused what will be argument of cost function. My problem is to minimize number of features for svm classification .
def fitness_function(new_matrix):
svc_linear = svm.SVC(kernel='linear', C=1)
svc_linear.fit(new_matrix,y_train)
predicted= svc_linear.predict(X_test)
cnf_matrix = confusion_matrix(y_test, predicted)
FP=cnf_matrix.sum(axis=0)-np.diag(cnf_matrix)
FN=cnf_matrix.sum(axis=1)-np.diag(cnf_matrix)
TP=np.diag(cnf_matrix)
TN=cnf_matrix.sum()-(FP+FN+TP)
Error_rate=(FP+FN)/(FP+FN+TP+TN)
fitness=Error_rate
return(fitness)
Dear Suman,
It depends on the application you are doing. However, in general, the objective function would be the minimum number of attributes holding the maximum information or patterns which helps in the classification task.
Thanks,
Sobhan
Dear Suman,
It depends on the application you are doing. However, in general, the objective function would be the minimum number of attributes holding the maximum information or patterns which helps in the classification task.
Thanks,
Sobhan
@Sobham Sarkar : Thanks a lot, Then where i should specify(reduced) attribute(featues) for classification(this should selected by pso automatically i mean which features is more important)? My Training dataset contains 384 features with which SVM is working properly.
Hi Suman,
at fist, please be careful and do not use the testing data for feature selection cost function - performance measured on testing data should not be applied for any modification of the model...you should do all the modeling on training data only, otherwise the testing results make no sense. (I saw that in your script)
On the other hand...to maximize the performance while you want to minimize the number of features may be a task for multi-objective optimization. https://en.wikipedia.org/wiki/Multi-objective_optimization
or just for a 'prove of concept' and simplicity, you can try to reduce the dimension on found features into their intrinsic dimension (let's say by PCA for example) to suppress redundancy in the dataset. PCA may increase the performance sometimes, it's worthy to try.
God luck,
@Toamas Vantuch : Thank you so much , Your answers strikes me that i am not changing anything in my X_test(this one is original testing data set) and that i should not use.
Did you mean that i should divide my updated training dataset(new matrix here in my code ) into [training and testing sample again and then apply these lines?]
X_new_train, X_new_test, y_new_train, y_new_test = train_test_split(newmatrix, y, test_size=0.3, random_state=0)
predicted= svc_linear.predict(X_new_test)
cnf_matrix = confusion_matrix(y_new_test, predicted)
Sure , I am learing PCA and will check my accuracy . Thanks a lot again.
Yes, you should divide your entire dataset into training and testing parts...than forget about testing part and perform all modeling/feature extraction/training on the training dataset which can be additionally divided into training and validation subsets (just to validate the models during their creation)...once you will have your trained model and knowledge about all feature extraction/transformation procedures, you can apply them on testing part to know how applicable your model is...
however, this scenario is from "ideal world" and our world is not ideal at all :) most of the time we don't have enough data to cover this everything and someone can argue about the subject of testing - we are testing the best possible combination of features? the best possible classifier? the best adjustment for SVC? the best features for SVC? now it looks like you are looking for best features for SVC, but if I took different set of features and use XGB, maybe the performance may increase... In my opinion, the easiest way to do this is by "divide and conquer" :) - you better estimate the relevancy of the features without the application of a classifier - just statistically (check my last paper, you can see several ways how to play with data: Article An Examination of Thermal Features’ Relevance in the Task of...
)the easiest is to apply Mutual information estimation for a collection of selected features (with proper scaling and normalization and assuming the dataset contains lot of samples)... once, you will be sure about extracted features, you can apply more different models to see which perform the best...
I hope I helped :)
cheers,
Hello, it depends on type of your datasets. Rough set theory is very good for nominal datasets.
- Badges
- Science topic
- Similar topics
- Data Mining
- Classification