Hi all,
I completed stepwise regression analysis on 3 creativity tests as predictors with the dependent variable being wellbeing score.
The prediction model was reached after 2 steps, first step included one of the creativity tests (which was significant predictor and predicted nearly 9% of the variance in wellbeing scores), the second step included the same greativity test and also age. Age only accounted for very small part of the explained variance (2.4%). Two other tests and gender were excluded from the model.
Now, in scientific literature, one of the two creativity tests that was excluded from the model should actually predict it. Participants on this test scored really low (because of specific conditions in which this test was taken), so I thought to further check it by dividing the scores in it to three different categories and then running one way ANOVA with estimates of effect size. Indeed, this time the differences in wellbeing scores among groups were sizable (eta squared 5.4%). Doint the same thing with the second test that was excluded from the stepwise model reached similar results. Then, I did the same for the one test that was included in the stepwise model-again it was significant and with the largest eta squared value (equal to the adjusted R squared value it had in the second step).
So I wonder what I should do now? testing all these variables together in MANOVA? Looking for main effects for each and then to look for interactions?
And what should I do with the stepwise regression that I already used?
Thanks in advance...
So I wonder what I should do know
Gilad, IMO opinion you should abandon stepwise regression altogether. See the first link below for a list of well-known problems associated with it. See also Mike Babyak's nice article on over-fitting regression models (2nd link). Rather than letting an algorithm decide which variables are in the model (i.e., stepwise), I think you would be far better off making those decisions yourself, based on your knowledge of the literature. In the statistical dialect I speak, it is called hierarchical linear regression when you the researcher decides on the order in which variables are entered. You can find more info by Googling . HTH.
http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/
http://people.duke.edu/~mababyak/papers/babyakregression.pdf
You should make up your mind what question(s) you precisely like to answer with that data. If you make this very clear, I am sure that you will see the solution. I can't tell you the solution, because I don't know the precise questions you like to ask your data.
Gilad, IMO opinion you should abandon stepwise regression altogether. See the first link below for a list of well-known problems associated with it. See also Mike Babyak's nice article on over-fitting regression models (2nd link). Rather than letting an algorithm decide which variables are in the model (i.e., stepwise), I think you would be far better off making those decisions yourself, based on your knowledge of the literature. In the statistical dialect I speak, it is called hierarchical linear regression when you the researcher decides on the order in which variables are entered. You can find more info by Googling . HTH.
http://www.stata.com/support/faqs/statistics/stepwise-regression-problems/
http://people.duke.edu/~mababyak/papers/babyakregression.pdf
So I guess you have your predictors (creativity tests) and your outcome (wellbeing) and want to test as to whether this model (predictors predict outcome) is true in the specific condition you mention. The problem you have is, you don't get the results you desire.
When this happens, many people think - they did something wrong. After they reassure themselves that they made no mistake, the second thought comes to mind: "the data is unlucky". The last thought which often does not even come to mind is: "the model is bad".
You state that participants scored low on a creativity test because of some specific conditions. If you have that in data (not just in opinion) then you have your answer as to why you probably don't get higher results that you expect from literature.
If you test a model (test x predicts wellbeing) then be ready to scrap that model if the data tells you so. Else you do not test but just tell (theories).
An addition to your regression analysis:
Let me guess: after your second step you had no significant increase in R² when adding other creativity tests as predictors? I think this might be because of their strong corellation as they measure the same thing. Try to regress with the test from literature first, then add other tests in second step. you might get a similar image (other tests not helping much in clearing variance).
Thanks for your answers. One question that I assume sum it all. Is there a point to take a continuous variable that in the regression model was found to be a non significant predictor and to divide it to say three different groups, then using ANOVA to find if there are differences between the groups in the scores of the predicted variable, then using post hoc tests to see which group score higher than which?
Hello Gilad. Carving a continuous variable into categories prior to analysis is rarely a good idea. The Vanderbilt Biostats Dept maintains a list of problems with this practice--see the first link below. You can also find numerous articles that discourage this practice. I've given links to just a couple of them. HTH.
http://biostat.mc.vanderbilt.edu/wiki/Main/CatContinuous
https://ww1.cpa-apc.org/publications/archives/cjp/2002/april/streiner.PDF
http://www.ncbi.nlm.nih.gov/pubmed/16217841
Yeszz Bruce, I fully agree with you (on (more ingeneral): not letting a program decide upon matters (finding effects, cutting values) that the scientist should decide upon).
Gilad, read the links Bruce provides!
Cheers Pierer
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Modeling
- Analyses of Variance
- ANOVA