Dear Kerri,

if you met all assumptions to conduct a multiple regression, then you have nothing else to do, the results are simply as they are. I think your confusion comes from the different conclusions you have to draw from the ANOVA result and those of the single predictors. It is really very similar to a "standard" ANOVA and how to interpre this.

With the ANOVA/F-Test in the multiple regression you test the overall R for significance, i.e. if it (the model) is different from zero. In your stepwise regression, you should have 2 F-Tests, one for the first step only T1 and for the second stept T1+FFMQ. It does not matter how many predictors are included in each step (except for degrees of freedom), it is always the whole specified model that is tested. Similar to a standard ANOVA where you would test a factor regardless of the amount of levels. If this model is significant, then you can conclude that all predictors together in that step explain so much variance that it is significantly different from zero. In case of significance you can now look which predictors contribute significantly to this overall effect (again similar to ANOVA with post hoc tests, where not all pairwise comparisons have to be significant). In case of a non significant overall F-Test there is no need to have a closer look, since they do not explain more than zero (although power could be an issue in that case). So, the F-Test does not test the individual predictors (also not to be confused with the F-Test for the R-squared change, which is very small with 1.4% btw, which is identical to the t-test for that single predictor FFMQ. Test it, the F-value for R-squared chance should be identical to the squared t-value of FFMQ, p-values ARE identical as you reported)

Alright, half way through, you found a significant F-Test and inspected the two predictors. Your result shows that one predictor explains much variance (T1) and hence is significant, but the other (FFMQ) doesn't. It is simple as that. The maybe tricky thing that bothers you is that FFMQ had a quite impressive bivariate correlation of .51 with the criterion. But, as your stepwise model indicated, nearly all of the variance that can be explained by FFMQ is already explained by T1. FFMQ adds nothing over and above the information you already had with T1.

Multicollinearity can sometimes be an issue for the inferential statistics, but not in your case. Your tolerance corresponds to a VIF of roundabout 1.4 which is way below the recommended "critical" value of 10.

Best,

Rainer

If I follow you correctly, your data shows that the variable T1 (on the first level of your hierarchical linear regression) predicts your dependent variable very strongly (i.e., a correlation between .7 & .8).  But, while FFMQ might be correlated with your dependent variable, when you statistically control for T1, FFMQ does not add anything to your ability to predict your outcome.  Two plausible things could cause this.  First, FFMQ might simply have no predictive value for your dependent variable.  Second, FFMQ and T1 might be correlated with each other such that all the variance FFMQ could use to account for your dependent variable is already accounted for by T1.  To find out, I recommend you find the zero order correlations between the combinations of all three pairs of variables.  Then you might calculate the partial correlation between each pair of variables while controlling for the third. Hopefully that will make interpreting your results more intuitive.  Best wishes with your project, Kerri! ~ Kevin

Thank you for your quick response! So, I have checked the correlations between each pair of variables, and they are not strongly correlated. The variable T1 has an r of .730 with T2 (the dependent variable). FFMQ has .510.

I've also checked the collinearity statistics: Tolerance is .679 for FFMQ in the model. Could that possibly be the issue?

I'm just wondering: if the result simply means there is no significant predictive value then that's just what it is. But do the results indicate that I need to do some kind of different test, or bootstrapping? Not sure. I have several results that are similar to the one I've posted here. The indep. var. is not significant but the ANOVA model is significant. I don't understand how to interpret that. In other words, why would the ANOVA statistic (F) be significant but the F-Change value in the model summary NOT be significant.

I'm sorry to keep posting. Thanks for your help!

Dear Kerri,

if you met all assumptions to conduct a multiple regression, then you have nothing else to do, the results are simply as they are. I think your confusion comes from the different conclusions you have to draw from the ANOVA result and those of the single predictors. It is really very similar to a "standard" ANOVA and how to interpre this.

With the ANOVA/F-Test in the multiple regression you test the overall R for significance, i.e. if it (the model) is different from zero. In your stepwise regression, you should have 2 F-Tests, one for the first step only T1 and for the second stept T1+FFMQ. It does not matter how many predictors are included in each step (except for degrees of freedom), it is always the whole specified model that is tested. Similar to a standard ANOVA where you would test a factor regardless of the amount of levels. If this model is significant, then you can conclude that all predictors together in that step explain so much variance that it is significantly different from zero. In case of significance you can now look which predictors contribute significantly to this overall effect (again similar to ANOVA with post hoc tests, where not all pairwise comparisons have to be significant). In case of a non significant overall F-Test there is no need to have a closer look, since they do not explain more than zero (although power could be an issue in that case). So, the F-Test does not test the individual predictors (also not to be confused with the F-Test for the R-squared change, which is very small with 1.4% btw, which is identical to the t-test for that single predictor FFMQ. Test it, the F-value for R-squared chance should be identical to the squared t-value of FFMQ, p-values ARE identical as you reported)

Alright, half way through, you found a significant F-Test and inspected the two predictors. Your result shows that one predictor explains much variance (T1) and hence is significant, but the other (FFMQ) doesn't. It is simple as that. The maybe tricky thing that bothers you is that FFMQ had a quite impressive bivariate correlation of .51 with the criterion. But, as your stepwise model indicated, nearly all of the variance that can be explained by FFMQ is already explained by T1. FFMQ adds nothing over and above the information you already had with T1.

Multicollinearity can sometimes be an issue for the inferential statistics, but not in your case. Your tolerance corresponds to a VIF of roundabout 1.4 which is way below the recommended "critical" value of 10.

Best,

Rainer

Rainer, thank you tremendously for this answer. It was extremely helpful!

So I will, thus, conclude that, although the model is significant, FFMQ (which is a mindfulness measurement) does not add any significant predictive value to the outcome variable. I guess it is a simple as that!

Thank you.

All the best to you,

Kerri

You are welcome. The model is significant, because T1 explains so much variance.

Although that is great, unfortunately the T1 isn't the important variable... T1 is really the pre-treatment measurement (T2 the post) the FFMQ should have been the significant one. Oh well. But that's the way it is sometimes! Thanks again!

Thanks for the scholars who posted the question and those who posted the plausible answers. Reading Cummings' last comment raises several questions.1.according to your comment then why didn't you apply covariance analysis with T1as a covariate? 2. was your second model with T1 and FFMQ significant with T1 significant and FFMQ not significant? If this is so then Duesing Is  enough. 3.Can you (does your study allow) considering FFMQ as a mediator? If so you can analyze using SMART PLS3 and be confident of using a useful mehod.