I am exploring the performance of a measure that can be computed using different values of its parameters, so I have about 20 variables. I want to find out which of these variables are able to differentiate two groups. So, I want to know about the performance of the variables, rather than finding differences between the groups. Therefore, I don't really think that I am given myself multiple chances to find a difference between the two groups and so I am inflating the chances of getting significant differences by chance. Actually, I am considering from the begining that the groups are different. So, do I really need some correction if I compare the 20 variables between the groups?
The simple answer is yes. You either include them in a model type of statistical analysis or in old terms perform a multivariate analysis of variance (MANOVA).
Possibly SVD or PCA and Factor analysis of the two vectors might help , you can get the weightage factors for the variables from PCA.
Cheers
Thank you all!!!
But Jelinek,
If I perform a MANOVA, wouldn't I be considering the set of dependent variables as a whole? I need to analyze their performance independently. However, I have also performed a later discriminant analysis as part of the study.
But I insist, with multiple comparisons I cannot be increasing the chance of finding significant differences by chance, because I already know that the groups are different indeed. There would be no chance of getting a "false positive", because if it is positive it won't be false. Please, am I missing something?
My point was that there may be an association between the dependent variables and that s what the MANOVA corrects
I will try then what you suggest. Thank you very much Jelinek!
Best regards,
Claudia
Consider also using a stepwise logistic regression in which you use 0=group 1 and 1=group 2, with twenty independent variables. If there is multicollinearity among the independent variables, it may be better to remove some variables before running the logistic regression. The odds ratio will tell you which variables are most important.
I agree with Raid and you could do a correlation analysis to remove correlated variables.
Gerry, Nice summary but what is not clear to me now is really that Bonferoni or Scheffe for instance are used if I have multiple groups. A MANOVA or equivalent is used if I have multiple test results. These are not the same thing and I did not think that therefore Bonferoni etc are appropriate
Thank you very much Professor Amin!
I understand that. However, I still find a bit difficult and subjective to properly identify which is the "family of statistical test" for applying the correction, in order to minimize the Type II Error Rate.
- Badges
- Science topic
- Similar topics
- Correction