I have three IVs with two levels/groups. My primary hypothesis is that there will be group differences for all IVs in total score. However, total score is comprised of scores on 7 subscales and I want to do separate t-tests for each IV to check for subscale differences as well as total score. Total score difference was hypothesised a priori, while subscale differences testing was done out of curiosity. 2x2x2 anova is out of the question because some groups n
I did not understand the division of p. The p is the probability whether the mean will fall out side the distribution. So how does dividing p make senses?
Oniram,
Unless you can use step-wise testing procedure, it is impractical to control the Type 1 error rate, especially for overall alpha=0.01, which is unusually small. If you must, then division by 8 for each IV is reasonable. I would be pragmatic and state that no adjustment for multiple comparisons was used.
David
A few things:
David and Stephen, why are the corrections not necessary because the hypotheses are exploratory? Don't we inflate chances of accepting a false H1 without correction? Some references would be great!
P-values (and significance tests in general) are only meaningful for confirmatory analyses. For exploratory analyses, you can't make strong conclusions until you conduct a confirmatory replication anyway. See, e.g., http://www.sciencedirect.com/science/article/pii/S0022103115001237.
Oniram,
To the statistical purist, you are correct. Being more pragmatic, I differentiate between collecting data and making conclusions regarding the information from that data versus testing a hypothesis (with concern for Type 1 error). As Stephen noted, for a confirmatory analysis with a claim being made, interest lies beyond gathering information and in the former case, adjustment is important.
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA