I'm wondering what test to use.
I have 9 IVs and 1 repeated DV (measured twice). The 9 IVs measure different aspects of identity, and include 2 composite scales and 7 subscales between them.
Initially, I had intended to median split each IV and conduct 2x2 mixed ANOVAs, i.e. High vs. Low IV on the DV. The issue is the inflation of Type 1 error (I hadn't factored that in when designing the experiment).
Is there another test that could include all IVs that wouldn't inflate the error rate?
Hello Ross,
You could try a regression model. That allows you to decide explicitly whether to include interaction terms (and, to what level; e.g., two-way only). As well, you won't lose potential information by dichotomizing the 9 measures into two levels.
Good luck with your work.
David's advice certainly makes sense, and I concur that you should consider the potential richness of interaction (equivalently, moderation) terms.
Maybe it has just been a long day, but I am a bit unsure what you mean by the 9 IVs include "2 composite scales and 7 subcales between them." In general subscales are composites themselves, so I am wondering what this means. This may not be a problem, but I am not sure in the way you word this, so I will just say that you need to make sure that none of the 9 scores are linearly dependent on any combination of the remaining 8 scores. Most methodologist's see our students get in to problems in using a total of scores that are also used as predictors in a model. This is not limited to summation; any such redundancy will cause the predictor variable matrix to be singular, thus no least-squares model can be rendered.
Beyond that, I would guess that some contemplation is needed in regard to how you conceptualize the repeated DV measures. Clearly, you could predict each separately, and then consider differences in models, but as well, you may have questions that link the two. Such method is heavily dependent on your theory, so I can't go very far with that without more substantive knowledge. You could for instance, predict the difference (but keep in mind the concerns about difference scores a' la Cronbach an Furby), predict the second DV including the first in the model, partial only the first from the second before use, etc.
As David Morse and John Morris have noted, some kind of regression model will give you complete control over what interactions are included in the model. You'll need to use a multilevel model or generalized estimating equations (GEE) to account for the repeated measures. What software are you using?
You mentioned median splits. That is usually a bad idea. Here are some good articles and notes on that topic. HTH.
- http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.462.859&rep=rep1&type=pdf
- https://www.stresshumain.ca/wp-content/uploads/2016/12/stats-critic-median-split.pdf
- http://www.psychology.sunysb.edu/attachment/measures/content/maccallum_on_dichotomizing.pdf
- http://core.ecu.edu/psyc/wuenschk/StatHelp/Dichot-Not.pdf
Bruce’s advice and excellent resources urging avoidance of dichotomization (and I would add any other “chotomization”) is something research methodologists face often. We strive for parsimony in science, but parsimony is not served by tossing out data.
At the moment anyway, I can only think of two situations that call for dichotomization (or in some cases more categories).
In some situations, it is necessary to think of a relatively continuous variable as a dichotomy for policy reasons. For instance, I had a student who was predicting “CLEP” scores from a variety of variables in a model. College credit was granted based on the CLEP score. The CLEP score ranged from 1 to 5, yet college credit was determined by a score of “3 or above.” Thus, from a bureaucratic perspective, it was important to consider whether the model predicted obtaining credit or not (policy), even though precision and power were lost. Of interest, this also changed the problem from modeling an ordinal variable to a classification problem. I believe I had the student do the analysis both ways however to satisfy power and policy.
In a more technical area, in moderation analyses with a relatively continuous moderator, if a moderation effect is identified, although the direction of the moderation is available from the sign of the coefficient of the product term, to really understand where the relationship between variables is “inflected” by the moderator, it is often necessary to consider the moderated relationship at different levels of the moderator, thus necessitating “chotomizing” it.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA