In the manuscript of a recent study we describe some post hoc tests, looking at the estimated marginal means of our linear mixed model in SPSS, applying a bonferroni correction.
Now our reviewers are asking us to specify what "test" was used. Is it correct to simply call these comparisons t-tests?
I believe they are conditional F tests:
https://books.google.co.uk/books?id=y54QDUTmvDcC&pg=PA91&lpg=PA91&dq=Conditional+F-tests+are+preferred+for+LMMs&source=bl&ots=L_RVeumUQV&sig=WlHM0ZNHY_ahZYqPU3Y0PbFaUxs&hl=en&sa=X&ved=2ahUKEwiAyIuQkNHeAhVRfMAKHaL_BrgQ6AEwAnoECAcQAQ#v=onepage&q&f=false
However, I believe they use a potential very liberal heuristic for determining df. Because the LMM test statistic doesn't really have an F distribution and because the df are inaccurate these tests may not be very accurate for inference. The exception is a fully balanced ANOVA fitted withe REML when they should be equivalent to an ANOVA F test assuming sphericity.
Hi Rik
We can use post hoc tests to tell us which groups differ from the rest.
If ANOVA is found significant then one should proceed further for post hoc tests like LSD,Tukey'HSD test,Duncuns multiple test; Scheffe’s test etc.
best wishes
ajit
Thom S Baguley
Thank you for your answer. I believe the conditional F test refers to the test ran by the LMM directly. I am wondering exactly what test is ran by the post hoc pairwise comparisons. These results do not provide a test statistic.
I have seen people suggest simply computing the test statistic using the mean difference and standard error. The resulting values do seem to somewhat match the p values on a t distribution but somehow I am unable to find a conclusive answer regarding the actual test used.
Yes - I think they are likely to be Wald-style t tests. As you suggest these are formed by the estimate / SE. The same issue applies in relation to the error df calculation - the true df is somewhere between 1 and n - k (where k is number of parameters) but not straightforward to infer.
Do you mean the post hoc "test" or the entire journey from data to the adjusted p-values. If you mean the post hoc part only it j(Bonferroni's, you can devise post hoc tests where this is not true) just uses the p values and is agnostic to how they were created. So,, in R,
p.adjust(c(.01,.02,.03,.04,.05,.06,.07,.08,.09,.1),method="bonf")
produces .1,.2,.3,.4,.5,.6,.,7.,8.,.9, and 1.0. Just an add to these comments, it may be useful to justify to reviewers why you chose Bonferroni's correction as it seems out of favor among methods folks these days (e.g., Holm's is the default for the p.adjust function above).
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS