Of late, some journal editors are insistent on authors providing a justification for the ordering of the entering of the predictor variables in the hierarchical regression models.
a) Is there a particular way of ordering such variables in regression models?
b) Which are the statistical principles to guide the ordering of predictor variables in regression models?
c) Could someone suggest the literature or other bases for decisions regarding the ordering of predictor variables?
Hello Anthony Muriithi Ireri. If you are using the term hierarchical regression the same way I do, it means that you determine the order in which variables are entered (rather than letting some algorithm determine the order). If so, here are a couple of conventions.
The first approach, it seems to me, is quite common in psychology, so it may be the one you want, given that your area appears to be educational psychology. The second approach is more common in epidemiology and medical research.
I hope this helps.
Bruce Weaver , may I ask for some help? I don't understand why the order of variables in a regression model should matter. Has this to do with the way a software calculates p-values (because you write about F-tests)? Or is it on how (in what order) to report the model statistics in a table?
As I understand, the order of variables in a model is not relevant. The F-statistic is calculated from the comparison of nested models (this is not one model; these are two different models; just one being nested in the other). How the models are nested does not depend on the order of variables in these models. I think whatever software is used it should allow (or force) the user to clearly specify which models are to be compared, and that this should not depend on the order in which the variables were "entered" into any of these models.
I am also confused about the term "hierarchical". Is it used here to define a "hierarchy" of nesting rather than a hierarchy of modelling the variance? I know "hierarchical models" only as synonym for "multilevel models", in which some coefficients not fixed but itself are modeled as expected values of "higher-level" (linear) models, what makes these coefficients to "random terms" in the top-level model (so we are talking about random-effects or mixed-effects models here).
Thank you for helping me out here :)
Hello Jochen Wilhelm. This page describes the approach to hierarchical regression that is common in psychology:
- https://data.library.virginia.edu/hierarchical-linear-regression/
One point that was not emphasized is that the same estimation sample must be used for each block. SPSS takes care of that by allowing multiple METHOD=ENTER sub-commands to add variables as you move from one block/step to the next. You can see an example of this at the bottom of this UCLA page:
https://stats.oarc.ucla.edu/spss/seminars/introduction-to-regression-with-spss/introreg-lesson1/
The following command enters 3 explanatory variables (acs_k3, full, and meals) in block 1 and adds enroll in block 2.
REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA CHANGE /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT api00 /METHOD=ENTER acs_k3 full meals /METHOD=ENTER enroll.
To address one of your points, yes, I could enter all 4 explanatory variables in one block, and I would end up with exactly the same ANOVA table and coefficients that are shown for block 2.
REGRESSION /MISSING LISTWISE /STATISTICS COEFF OUTS R ANOVA CHANGE /CRITERIA=PIN(.05) POUT(.10) /NOORIGIN /DEPENDENT api00 /METHOD=ENTER acs_k3 full meals enroll.
I cannot immediately find a good example of the conventional approach in epidemiology, which is to estimate the crude effect of the focal variable in block 1, and then adjust for potential confounders in block 2. But here is an example I posted to RG some time ago:
* Estimate a model with X=male and Y=workft in Block 1,
* then add age and years of education to the model in Block 2.
LOGISTIC REGRESSION VARIABLES workft
/METHOD=ENTER male
/METHOD=ENTER age educ
/PRINT=CI(95)
/CRITERIA=PIN(0.05) POUT(0.10) ITERATE(20) CUT(0.5).
* The OR for male in Block 1 is the crude OR;
* the OR for male in Block 2 is adjusted for age and years of education.
Again, the final model here would be exactly the same as the model I would get if I entered all variables simultaneously. But the point (for epidemiologists) is to see both the crude and adjusted effects (odds ratios, in this case), and to get a test comparing the fits of the two models (a likelihood ratio test on the change in -2LL, in this case; an F-test on the change in R2 for the OLS model shown above).
To anticipate another possible question, I do not know when this hierarchical approach to model building first arose. Perhaps someone else does. HTH.
The order of variables in regression analysis does not matter for the summary of the linear model, which is based on t-tests that don’t change. However, when you do an ANOVA, the order of variables might affect the results depending on the type of sums used.
The order of variables in regression analysis does not matter for the summary of the linear model, which is based on t-tests that don’t change. However, when you do an ANOVA, the order of variables might affect the results depending on the type of sums used.
When you do an ANOVA, the order of variables might affect the results depending on the type of sums used.
Jochen Wilhelm some instances use the word "hierarchical" when not all variables are entered together, but separetely or in blocks of variables. For the former one automatic algorithms may be used (but not recommended, as you know) like stepwise regression, whereas for the latter one typically the researcher decides which (blocks of) variables are entered in which order (blockwise regression).
As Bruce Weaver already mentioned, this is done very frequently in psychology, although it may be questioned if this is necessary, yet useful, or just a habit because everyone does it. In most psychological papers in such cases they are only interested in the delta R^2, for example to show that the increase in explained variance of an interaction term is significant (but miss that this could also be done by the squared semipartial correlation and no blockwise regression is needed).
I remember a paper (I believe about job satifaction or something similar) where they entered sociodemographic variables in a first block, typical predictors in a second block and new predictor variables in a third block to show that the typical ones explain more than socidemographic variables alone. The third block to show that the new predictors explain variance over and above the former ones. (In light of causality, I would cast some doubt about the results....)
But you are right, within each block and for the full model, the order of the variables do not have any meaning.
Does it help?
Bruce Weaver thank you for your informative contribution and for the links. You are right, I used the term hierarchical to imply 'determining the order in which variables are entered' which is the popular meaning of the term in psychological research.
Thank you Jochen Wilhelm , Rainer Duesing , Chuck A Arize for the contributions. I am better informed out of this discussion.
- Badges
- Science topic
- Similar topics
- Psychology