I'm trying to use SVM-RFE with libsvm library to run on the gene expression dataset. My algorithm is written in Matlab. The particular dataset able to produce 80++% of classification accuracy under the 5-fold CV without applied Feature selection. When I tried to apply svm-rfe on this dataset (same svm parameter setting and use 5-fold CV), the classification result become worse, can only achieve 60++% classification accuracy.
Here is my matlab coding, appreciate if anyone can shed some light on whats wrong with my codes. Thank you in advance.
[label, data] = libsvmread('libsvm_data.scale');
[N D] = size(data);
numfold=5;
indices = crossvalind ('Kfold',label, numfold);
cp = classperf(label);
for i= 1:numfold
disp(strcat('Fold-',int2str(i)));
testix = (indices == i); trainix = ~testix;
test_data = data(testix,:); test_label = label(testix);
train_data = data(trainix,:); train_label = label(trainix);
model = svmtrain(train_label, train_data, sprintf('-s 0 -t 0);
s = 1:D;
r = [];
iter = 1;
while ~isempty(s)
X = train_data(:,s);
fs_model = svmtrain(train_label, X, sprintf('-s 0 -t %f -c %f -g %f -b 1', kernel, cost, gamma));
w = fs_model.SVs' * fs_model.sv_coef;
c = w.^2;
[c_minvalue, f] = min(c);
r = [s(f),r];
ind = [1:f-1, f+1:length(s)];
s = s(ind);
iter = iter + 1;
end
predefined = 100;
important_feat = r(:,D-predefined+1:end);
for l=1:length(important_feat)
testdata(:,l) = test_data (:,important_feat(l));
end
[predict_label_itest, accuracy_itest, prob_values] = svmpredict(test_label, testdata, model,'-b 1');
acc_itest_fs (:,i) = accuracy_itest(1);
clear testdata;
end
Mean_itest_fs = mean((acc_itest_fs),2);
Mean_bac_fs = mean (bac_fs,2);
Each dataset is different so giving an answer could be risky without knowing the info contained on it.
What I would do is to apply SVM-RFE iteratively to reduce the number of features "slowly". That is from 2000 to 1750, then to 1500 and so on. Thus, you will be able to check whether the problem comes from your data or from the code that does not reduce the features properly.
Sorry for asking this, I am not really familiar with SVM-RFE method but it is strange to me that your code work. My understanding is that you generate an SVM model using svmtrain on the basis of original training data dimension, then you are using some sort of dimension reduction with iteratively changing the dimension and regenerating new SVM models (called fs_model), later you will have some sort of indexes representing important datapoints, finally using this index you update your testing set and call svmpredic with original model. So the question is how can a model that is trained on original data dimension be able to predict something with different dimension. shouldn't this throw an error for inconsistency on data dimension. Shouldn't you update your model by retraining it with the reduced training set first and then call the svmpredict with reduced testing set? As I mentioned I am not familiar with SVM-RFE and might be completely off track ;)
Dear JC
I appreciate that you are quit invested in SVM-RFE for your dimension reduction, however, you might also want to consider alternative methods. for example decomposition approaches such as CSP,SVD,PCA, and SFS,SBS or evolutionary wrapper methods such as GA,PSO,DE.
Good luck
Hello,
1) are you using a linear kernel for training svmrfe models ? Only linear kernel make sense, while it seems the kernel function is a parameter of your code.
2) Ideally the cost parameter c should be adapted at each iteration of svmrfe as the number of feature decrease. If not, the final model, trained with the reduced feature set with the same cost value, would result in a more "hard margin" model than the initially, which might not be expected. To solve this issue you should select the best C parameter at each iteration using inner cross validation. Alternatively, you may avoid this additional time consuming step by using nu-SVC models instead of C-SVC: nu parameter should be more consistent from one iteration to the next one, and it should not be necessary to update its value at each iteration of svmrfe.
I hope it will help,
Good luck
Julien
- Badges
- Science topic
- Similar topics
- Computer Science and Engineering
- Bioinformatics