Hi everyone,
My results show for some models that the model itself is not significant, but some independent variables within the model are significant (including the constant) in SPSS.
I was wondering how I should interpret these results? What can I say/conclude and what not?
Thank you in advance,
Minke
Hello Minke,
So, what can account for this apparantly "paradoxical" outcome (which, by the way, can only occur when you have more than one predictor in a model)?
The test for whether the overall model in MLR is statistically significant represents a different hypothesis (is R-squared = 0), with a different sampling distribution for determining significance, than the test for whether a single regression coefficient is equal to zero. As well, the more IVs you have in a model, the higher the number of significance tests you're running (for evaluating individual coefficients), and therefore, the higher the risk of a false positive result, unless you invoke some control by applying a more stringent criterion for significance.
In general, with large enough N, the two sets of results will be consonant (e.g., both overall test and at least one individual coefficient test will be significant, or all will be non-significant). But, since the sampling distributions are different, when N is modest, you can see these differences.
Good luck with your work.
@David Morse I was under the impression that the sampling distribution of R^2 was yet unknown.. Does SPSS then do a bootstrap approach to get a p-value for R^2 or?????. Thanks, David Booth.
Regarding the test for the constant, remember that the constant is the fitted value of Y when all explanatory variables are equal to 0 (or their reference categories in the case of categorical variables). So the value you see for the constant depends on whether any variables have been centered. And notice that centering explanatory variables affects the value of, and the test for the constant only. It does not affect the overall F-test for the model. Copy the following code into a syntax window and execute it. Then look at the following for both models:
- The model summary table
- The ANOVA summary table
- The table of coefficients--especially the constant.
* Change path on next line to folder where the sample datasets are stored on your hard disk.
GET FILE='C:\SPSSdata\survey_sample.sav'.
DESCRIPTIVES educ paeduc maeduc.
* Centre parental education variables on 12 years.
COMPUTE educ12 = educ-12.
COMPUTE paeduc12 = paeduc-12.
COMPUTE maeduc12 = maeduc-12.
REGRESSION
/STATISTICS COEFF OUTS CI(95) R ANOVA
/DEPENDENT age
/METHOD=ENTER educ paeduc maeduc.
REGRESSION
/STATISTICS COEFF OUTS CI(95) R ANOVA
/DEPENDENT age
/METHOD=ENTER educ12 paeduc12 maeduc12.
* --- end of syntax --- .
HTH.
@Minke de Joode, you say some IVs are significant, some are not. Pick on the variables that show a significant prediction potential on the outcome for the model. Exclude the non-significant ones and see what happens.
Joseph Kipkorir Cheruiyot, a manuscript checklist from the Biostats Dept at Vanderbilt University would suggest that removing non-significant variables is a bad practice. Here is the relevant excerpt:
Lack of insignificant variables in the final model
Unless the sample size is huge, this is usually the result of the authors using a stepwise variable selection or some other approach for filtering out "insignificant" variables. Hence the presence of a table of variables in which every variable is significant is usually the sign of a serious problem. [emphasis added]
Authors frequently use strategies involving removing insignificant terms from the model without making an attempt to derive valid confidence intervals or P-values that account for uncertainty in which terms were selected (using for example the bootstrap or penalized maximum likelihood estimation). J Clin Epi 2009-03-01, Volume 62, Issue 3, Pages 232-240 cited Ockham's razor as a principle to be followed when building a model, not realizing that parsimony resulting from utilizing of the data at hand to make modeling decisions only seems to result in parsimony. Removing insignificant terms causes bias, inaccurate (too narrow) confidence intervals, and failure to preserve type I error in the resulting model's P-values, which are calculated as though the model was completely pre-specified. [emphasis added again]
--- end of excerpt ---
Source: https://biostat.app.vumc.org/wiki/Main/ManuscriptChecklist
HTH.
- Badges
- Science topic
- Similar topics
- Mathematical Sciences
- Statistics
- Statistical Software
- SPSS