Hi,
I try to analyze data set with numerous independent variables of which some are categorical (two or three levels each) and some are continuous. Unfortunately, several of the categorical (and continuous) variables are correlated among each other (e.g. level coded as 1 in one variable occurs together with level 1 in other variables). I try to find the optimal way to reduce the number of explanatory variables that can be later used in GLM analysis. Specifically, I wonder how to reduce the number of categorical variables in a way analogous to PCA to get uncorrelated variables. I have noticed that in SPSS there is a PCA for categorical variables. However I am curious if there are some other solutions or statistical methods, especially implemented in R. I would appreciate every suggestion how to cope with such kind of data.
Regards
Piotr
Hi Piotr,
Some remarks:
-Categorical variables can not have a correlation; the values are arbitrary. Perhaps you mean ordinal variables?
-If you are doing an explorative analysis you are on the right track, exploring the data to find a model. (and then, subsequently, replicate the research to test that model).
If you are doing a comfirmative analysis you should deduce the model from the theory and not from tha data. You use the data to test the model.
If it is a given, or existing, dataset, look at the constructs that are 'behind' the variables and base your model on thoughts about these constructs, and then test the model with the data.
-"PCA to get uncorrelated variables." ->uncorrelated factors? You could do that, but you should be able to interpret these new factors.
Greetings.
a couple of ideas:
* "level coded as 1 in one variable occurs together with level 1 in other variables" - you can try to deal with that unless you dont have confounded variables (e.g. day of week is confounded with part of the week, i.e. weekend vs non-weekend). In the latter you wont be able to get rid of the dependency. But I assume that you have just correleted, not confounded variables
* you can try to use Partial Least Squares instead of PCA - some substatial differences make the former much more useful in some cases and I saw categorical variables put into the PLS (e.g. here: https://www.r-bloggers.com/partial-least-squares-regression-in-r/)
* sometimes you can try to combine two variables into one (if the var are confounded) although interpretation gets complicated
* I think that you dont have to put all the variables into PCA - you can leave some uncorrelated vars in their original form and run PCA on the subset of badly correlated variables.
pozdro ;-)
Dear Piotr
What type of regression you need?, Linear, logistic, curve-linear?
In this context, you search code for calculate the Variance Inflation Factor?
into R, i recommend install the Rcmdr package, in there you can calculate the VIF for all types of regression analysis. Or use the vif() function into the car package.
if you search for the parsimony model, you use a ANOVA type II, and remove the variables not significant in each iteration. Backwards method.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS