I found that by using Proc Mixed in SAS to run a repeated measure ANOVA, the p-values from the table "Solution for Fixed Effects" are different from the table "Type 3 Tests of Fixed Effects" when interaction term is included.
E.g. for group, p-value in "Solution for Fixed Effects" is 0.4967. However, p-value is 0.9285 in " Type 3 Tests of Fixed Effects".
When there is no interaction term, the results from both tables will be the same. Should I only use the result from "Type 3 Tests of Fixed Effects"?
proc mixed data=result covtest noclprint; class gp sub time;
model Y = gp time gp*time / solution ;
lsmeans gp / pdiff cl;
repeated time / subject=sub type=un;
run;
Solution for Fixed Effects
Effect gp time Estimate Standard Error DF t Value (Pr > |t|)
Intercept 3.8000 1.1916 8 3.19 (0.0128)
gp 1 1.2000 1.6852 8 0.71 (0.4967)
gp 2 0 . . . .
time 1 1.2000 0.8888 8 1.35 (0.2139)
time 2 0.6000 0.5292 8 1.13 (0.2897)
time 3 0 . . . .
gp*time 1 1 -2.4000 1.2570 8 -1.91 (0.0926)
gp*time 1 2 -1.6000 0.7483 8 -2.14 (0.0650)
gp*time 1 3 0 . . . .
gp*time 2 1 0 . . . .
gp*time 2 2 0 . . . .
gp*time 2 3 0 . . . .
Type 3 Tests of Fixed Effects
Effect Num DF Den DF F Value Pr > F
gp 1 8 0.01 (0.9285)
time 2 8 3.29 (0.0906)
gp*time 2 8 2.67 (0.1296)
I may have found an answer to this question. See answer #3 to FAQ on the following website:
http://homes.ori.org/~keiths/Files/Methods/SAS-FAQ.html
Here is what it says:
"Why do the "Solution for Fixed Effects" and the "Tests of Fixed Effects" sometimes differ?
The F-tests are by default Type III tests, so that if there is an F-test for a main effect involved in an interaction, it is an approximate test for the main effect valid only if the interaction is zero (Murray, 1998, p. 293, note 23). If you look at the t-test in the solution table in the same analysis, the t-test for that same main effect is a test for the simple main effect of that variable at the zero level of the other term involved in the interaction. In other words, the t-tests in the solution table are always interpretable just as any test of a regression coefficient would be from a linear regression analysis, but the F-tests aren't.
Consider male (1=male, 0=female) and cond (condition; 1=intervention, 0=control). If we have an interaction model that includes male, cond and male × cond, the F-test for male is an approximate test of whether males are different from females in general, and is valid only if the interaction effect is zero. In the same analysis, the t-test for male in the solution table is a test of whether males are different from females when we restrict ourselves to students in the control condition. So the t-test and F-test are testing different null hypotheses in that circumstance and their p-values may be different, especially if the interaction effect is meaningful. In a purely main effects model, the t-test and F-test are testing the same null hypothesis and their p-values will be identical. (Answer by David Murray, 04-28-00 e-mail, edited.)
On Murray's (1998) note 23 on page 293, he comments that some controversy lies around Type III F-tests and that people have often misinterpreted them. In an analysis of condition and time, the F-test for condition tests the null hypothesis that condition does not vary at an average level of time. The Type III F-test of time tests for a time effect at the average level of condition. With an interaction, Type III F-tests reduce the denominator degrees of freedom and inflate the standard errors of tests of main effects for the included interaction.
For more information, please read an e-mail exchange on the s-news distribution list about Type III F-tests. "
Rafael, thank you very much for your comment.
What I don't understand is discrepancy in p-values in these two tables. Between these 2 tables, I know "Type 3 Tests of Fixed Effects" is the one I should first consider, but the discrepancy makes me feel that "Solution for Fixed Effects" is not reliable.
if I drop out the interaction term, p-value for gp will be the same (p=0.4084), which is what I expect. So is it true that when interaction term is included, we should not see the result from "Solution for Fixed Effects"? Thank you again
proc mixed data=result covtest noclprint; class gp sub time;
model Y = gp time / solution ;
lsmeans gp / pdiff cl;
repeated time / subject=sub type=un;
run;
Solution for Fixed Effects
Standard
Effect gp time Estimate Error DF t Value Pr > |t|
Intercept 3.8166 1.1329 8 3.37 (0.0098)
gp 0 1.1667 1.3373 8 0.87 (0.4084)
gp 1 0 . . . .
time 1 7E-16 0.7149 8 0.00 (1.0000)
time 2 -0.2000 0.4422 8 -0.45 (0.6631)
time 3 0 . . . .
Type 3 Tests of Fixed Effects
Num Den
Effect DF DF F Value Pr > F
gp 1 8 0.76 (0.4084)
time 2 8 3.23 (0.0935)
I may have found an answer to this question. See answer #3 to FAQ on the following website:
http://homes.ori.org/~keiths/Files/Methods/SAS-FAQ.html
Here is what it says:
"Why do the "Solution for Fixed Effects" and the "Tests of Fixed Effects" sometimes differ?
The F-tests are by default Type III tests, so that if there is an F-test for a main effect involved in an interaction, it is an approximate test for the main effect valid only if the interaction is zero (Murray, 1998, p. 293, note 23). If you look at the t-test in the solution table in the same analysis, the t-test for that same main effect is a test for the simple main effect of that variable at the zero level of the other term involved in the interaction. In other words, the t-tests in the solution table are always interpretable just as any test of a regression coefficient would be from a linear regression analysis, but the F-tests aren't.
Consider male (1=male, 0=female) and cond (condition; 1=intervention, 0=control). If we have an interaction model that includes male, cond and male × cond, the F-test for male is an approximate test of whether males are different from females in general, and is valid only if the interaction effect is zero. In the same analysis, the t-test for male in the solution table is a test of whether males are different from females when we restrict ourselves to students in the control condition. So the t-test and F-test are testing different null hypotheses in that circumstance and their p-values may be different, especially if the interaction effect is meaningful. In a purely main effects model, the t-test and F-test are testing the same null hypothesis and their p-values will be identical. (Answer by David Murray, 04-28-00 e-mail, edited.)
On Murray's (1998) note 23 on page 293, he comments that some controversy lies around Type III F-tests and that people have often misinterpreted them. In an analysis of condition and time, the F-test for condition tests the null hypothesis that condition does not vary at an average level of time. The Type III F-test of time tests for a time effect at the average level of condition. With an interaction, Type III F-tests reduce the denominator degrees of freedom and inflate the standard errors of tests of main effects for the included interaction.
For more information, please read an e-mail exchange on the s-news distribution list about Type III F-tests. "
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA