I have a collection of 100 independent samples to assess after I get 12 responses each but some are more important than others. I want to know whether the analysis can eliminate some of these?
In general, the maximum number of variables supported by most software is 256.
There are two ways to eliminate unnecessary variables:
1- The variables, which are closer to the center of the correlation circles should be moved from the initial DB.
2- You can weighted variables according to their importance. Weight coefficient ranges between 0 to 1. Important variable has a weight coefficient near to 1, however non-important one has a weight coefficient near-zero. The sum these coefficients must be equal to 1.
with standard pca: number of obs must be greater than number of variables, so theoretically if you have e.g. 258 observation you can have up to 257 variables ;)
for p > n, other methods for high-dimensional data can be used.
If exist the right software for calculation, number of variables is not important. But the number of principal components obtained is very important. If exist a good linear relationship between the data, the 3 or 4 effective component is obtained.
The number of factors can be deduced by using the point of inflexion on the scree plot: variance=f(factors). In practice, only the three first PCAs are retained at max. This is because PCA is a linear dimensionality reduction technique. If the cumulative variance (%) explained by eigenvectors is too low in this case (less than 2/3 to 60% of the initial data of example), it should be very important to verify your input data (move some variables or observations, typographical error …). If not, the PCA is not the suitable technique for you study and, hence, you should try another method.
Yes, you can eliminate some variables. Usually it is done on criterion of small correlations with other variables (because we are looking for variables which form factors) or on criterion of extremly big correlations with other variables (because of the problem of multikolonearity which can cause computational problems).