Dear all,
After having researched this question for days (ResearchGate, books, youtube, the general WWW....), I'm none the the wiser, even though it's probably an easy question. I'm new to linear mixed models, so my understanding is still very limited.
I'm running a 2-level linear mixed model in SPSS, where participants' search behavior in 2 different decision domains is nested within each individual (you could also imagine it as a repeated measure). I use a random intercept, and random effects for subjects. Other than that, I'm looking at age group (young, old), experimental condition (2 different types of information),health status (healthy, sick), and of course decision domain (health-related, not health-related) as "predictors" / factors. The DV is the amount of information participants have reviewed.
I have run my model but now I don't know how to interpret and report the results. It seems some sources I have consulted look at the Type III Test for Fixed Effects, which offers omnibus tests (F statistic) for main effects and interactions. I assume you can report these in writing, like I would with an ANOVA. However, some other people only report a table with Estimates for Fixed Effects, which offers t-tests. Some of the results in these tables "conflict" (maybe they don't, I just don't understand the difference yet), so I am unsure what to rely on and whether to report both.
To provide an example: My hypothesis was, among other things, that information seeking does not differ between age groups (i.e., no main effect). The F-test supports this, saying there is no overall effect of age group on information seeking. However, the estimates table lists age group as significant at p = .009, with information seeking as DV. The same is true for a couple more effects and interactions. Now I am very confused. There are not a lot of videos or texts covering both. One summary I found that addressed both types of tables did not provide insight into what to do if results differ. (Some people on ResearchGate and elsewhere also seem to refer to this t-test table as pairwise comparisons. If that is the case, does that mean that while age groups were significantly different in their search behavior when compared to each other, search behavior did not rely on age?)
I would be grateful for whatever insight you can share! Thank you.
If I understand correctly, you have an age variable which is categorical. Its main effect is found to be not significant, but in the estimates table you see a p value significant. If that is the case, that means your overall effect is not significant, but the comparison of that category of age to the reference category is significant. Your sentence between parantheses is correct. It could help if you share the estimates table.
Hi Mehmet, thank you for your answer! I am a bit confused by your answer - you say " If that is the case, that means your overall effect is significant, but the comparison of that category of age to the reference category is significant. " Do you mean "not significant" for the overall effect?
Someone pointed out to me that this threat covers the same question, which I didn't realize because tables have somewhat different names in SAS and I am not familiar with SAS at all: https://www.researchgate.net/post/Why_is_there_a_discrepancy_in_p-values_for_different_output_tables_in_a_mixed_model_when_an_interaction_term_is_included_SAS
Here, responses imply that F-tests are often interpreted the wrong way, and that one should rely on the pairwise comparisons/ t-tests instead.
It is a typo, I edited my reply.
T and F tests have different hypotheses and therefore they are not substitutes. Do remember that those t-tests are comparing a certain level to the reference level.
Hi Mehmet,
Thanks again for your clarification. I waited a little but as there are no additional responses, maybe you can answer an additional question?
I know that linear mixed models can be used to answer different types of questions. You can go for a slope-as-outcome approach and interpret results like a regression (here, I assume, relying on t-tests for parameter estimates makes more sense). But I know you can also consider mixed models as an alternative for repeated or nested ANOVA designs (likely using the F-tests I mentioned?), like I want to. Is this right?
We already established that F-tests and t-tests test different hypotheses. Is it best to report both, knowing that they differ, or should I just focus on F-tests given I want to go about my analysis like an ANOVA?
Thank you so much! I would still be stuck if it weren't for your response a few days ago.
Yes, mixed models are superior than those ANOVAs. For the main effects, you can report F results anyway. Those t-tests are comparisons to some reference level, therefore may not relate to your hypotheses.
@mehmet: I have a similar situation with a continuous IV. In that case, what is the reference category? In my case, the fixed effect for this continuous IV is significant, but the fixed estimate for the same variable is not.
Herve, you should be more explicit.
A continuous variable has no reference category. Could you share an output?
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS