Hello everyone, I have a 3 factorial experiment design. A few of my hypotheses are investigating effects of two IVs on several DVs and a few are investigating the effects of all three on same DVs. When I make use of a three-way ANCOVA, I find no statistical significance of my covariance and one IV. However, when I make use of a two-way ANCOVA, both are statistically significant. My advisor has been insisting that "the experiment design determines your statistical test and not your hypothesis". She is highly experienced with experimental studies. However, I seem to feebly remember that I have read articles where it's an n factorial study, yet authors have made use of (n-1)-way ANOVA (based on hypothesis development). A sincere request to all Experts in the domain to provide suggestions/recommended articles. It is very tempting to compare the effects of a few cells as I am finding statistically significant results. However, is that the right approach?
Ami Shah You advisor is correct. See: https://www.google.com/search?ei=UnZPXqTqM4_YsAWJ6JngCA&q=montgomery+D+C+Design+and+analysis+of+experiments&oq=montgomery+D+C+Design+and+analysis+of+experiments&gs_l=psy-ab.12...0.0..31860...0.0..0.0.0.......0......gws-wiz.m2nKP5NJPHI&ved=0ahUKEwik2-b5_-HnAhUPLKwKHQl0BowQ4dUDCAs
the chapters on factorial designs and interactions. Best, David Booth
I agree with David in the case of factorial ANOVA. However, it is possible that covariates have to be handled carefully in this design. In particular software can do strange things with covariates in repeated measures models.
So I would advise centering all covariates prior to including them in the model (i.e., substract the mean from all variables so that centered covariate has the same sd but a mean of zero).
This letter to the editor reports an example of the "strange things" Thom Baguley mentioned above, and like Thom recommends centering the covariates.
- https://www.cambridge.org/core/services/aop-cambridge-core/content/view/2BBC95DC4B17719654CEFD000F9477E0/S1355617707071147a.pdf/use_of_covariates_in_randomized_controlled_trials.pdf
An old but a good paper on covariates and strange things is by Meehl, here: http://meehl.umn.edu/sites/meehl.dl.umn.edu/files/084nuisancevariables.pdf
Are all three IVs based on random allocation? How many hypotheses are you evaluating? Do any overlap?
With all due respect to Thom Baguley , Daniel Wright , Bruce Weaver I believe that you are not discussing this from the point of view of an experimental study. The only things that I can think of to cause use of an ANCOVA with a factorial design is if a repeated measure is lurking somewhere or if a covariate such as rainfall is important to the outcome. Btw I believe Fisher's paper( https://digital.library.adelaide.edu.au/dspace/handle/2440/15184) was the first to do this. Ami Shah seems to be discussing hunting in the cells to find a p-value
I don't think there is enough information to work out what model is appropriate here. My point was specifically about repeated measures ANCOVA where AFAIK SPSS (and possibly other packages) still does strange things. Hunting in cells of ANOVA for significant effects isn't necessarily p-hacking (e.g., if appropriate error control is in place).
Collapsing n factorial to (n-1) ANOVA analysis can lead to inappropriate weighting and estimation, a contradiction similar to "Simpson's Paradox" I also encountered with collapsing 3D Contingency Table to 2Ds.
I would that you follow your supervisor as suggested by Prof. David Eugene Booth vid Eugene Booth
Good luck.
- Badges
- Science topic
- Similar topics
- Psychology
- Mental Processes
- Learning