The input data that I have is a matrix X (99*8) , where the rows of X correspond to observations and the 8 columns to correspond (predictors or variables). I need to apply the PCA on this matrix to choose a set of predictors (as a feature selection technique) .In Matlab, I know that I can use this function [coeff,score,latent]= pca(X) for applying the PCA on input matrix, but I don't know how to use the output of this function to create a new matrix that I need to use for training Support Vector Machine classifier. Please Help me!
Dear Huda,
Do you have to use Matlab? If you are open to other languages, I'll suggest you to use the scikit-learn lib in Python. It's very simple to make the transformation you want:
import numpy as np
from sklearn.decomposition import IncrementalPCA
X = # Load your data here (to build the model)
Y = # Load the new data here (to apply the PCA model)
pca = PCA(n_components=2) # Choose the number of components you want
pca.fit(X) # Build the model
pca.transform(Y) # Apply the model/components to the new data
You can read more about the function in http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html
Best Regards.
Dear Huda,
When you used the PCA, you will have principal components instead of features, namely feature extraction. First of all, you can use PCA like P1=x1c1+x2c2... With enough principal components (without loosing information), you will reduce number of features.
You can follow this information below...
https://onlinecourses.science.psu.edu/stat505/node/51
Thank you all for the help. I use Matlab but I'm new to it.
if I can use the SCOR output to train SVM directly, how to connect the information in the original matrix that was used as input in PCA and the Score output matrix. In other words, to train the SVM l should have two things features matrix and label ( two classes 1, 2 ). each row in feature matrix corresponding to 1 or 2.
when l apply the pca on feature matrix (116*8) (Rows of X correspond to observations and columns to variables or features) l got score matrix (5*1). Now, how l can know which rows in score corresponding to which rows in the original matrix ( feature matrix) so that l can re-create the label and then train the SVM by using Score and label. Please help me I'm really confusing!.
Hi Huda,
Let's say your data matrix is called X. For implementation of PCA, you can use eigen-decomposition of R = X*X' or singular value decomposition of X itself, and select several columns (features) that correspond to the largest eigenvalues (resp. singular values). I think this method has much better clarity and operability than using the "pca" function.
Hi Huda
I recommend you look at following links:
http://stackoverflow.com/questions/12067446/how-many-principal-components-to-take
http://stats.stackexchange.com/questions/192722/dimension-reduction-using-pca-in-matlab
I wrote a function for you to select those features which capture 95% of the total variance (please find the mfile attached ). You can change this value based on your dataset.
The "reduced_data" is the final data that you can use for SVM training. Please note that PCA doesn’t change the number of rows (which are samples). So your labels and reduced data have the same number of rows.
“Don’t forget to normalise your dataset before PCA’’
Hope my code helps you
You can use WEKA software both for PCA and SVM. Its free software and very easy to use it.
(1) If you need code, you can use mine. It's really simple:
%parameters:
%samples is the original sample;
%d is the number of dimensions you need.
%newsamples is the new samples generated by pca
function [EV,newsamples] = pca(samples, d)
means = mean(samples);
[r,c] = size(samples);
S = zeros(c,c);
for i=1:r
S = S + (samples(i,:)' - means')*(samples(i,:) - means);
end
[V,D] = eig(S);
n = size(D,1);
a = zeros(1,n);
for i=1:n
a(i) = D(i,i);
end
[Y,I] = sort(a);
EV = zeros(c,d);
for i=1:d
EV(:,i) = V(:,I(n-i-1));
end
newsamples = samples * EV;
(2) You have to pay attention to how many attributes you need for SVM. PCA cannot guarantee that you will always have a better result on SVM.
- Badges
- Science topic