This managed to throw me for a loop. When I'm assessing the presence of outliers before I conduct my moderation analysis I use SPSS to split the data file based upon the dichotomous moderator. I test my assumptions separately and perform a simple linear regression of the IV on the DV at each level of the moderator to generate studentized deleted residuals (SDR) and Cook's distances, so far so good!
The confusion I'm running into is that in Cohen et al. (2003) he recommends a different cutoff score for considering potential outliers: SDR +- 2 for small sample sizes and +- 3 for large sample sizes. This sort of information makes a lot of sense in the context of a standard multiple regression but when dealing with a dichotomous moderator in which assumptions are tested separately for each level I'm just wanting to do a bit of a sanity check.
I've assumed that if there is a small group size in one dichotomous level that I am applying these SDR cutoffs separately based upon size of the group and NOT overall sample size.
Similarly, I prefer using the 4/n cutoff of Cook's distance for considering potentially high leverage values rather than just looking for values greater than 1 (I could swear these are only present in artificial datasets).
All the articles I've come across so far always express n as sample size and so again I run into this slight confusion about if it's correct of me to have two different cut off scores of Cook's distance that is specific to the group size. I mean after all, the regression line, the residuals, and the produced Cook's distance are specific to that level of the moderator and therefore should have their own group cutoff.
Thanks in advance for taking the time to read and weight in!
----
Cohen, J., Cohen, P., West, S. G., & Aiken, L. S. (2003). Applied multiple regression/correlation analysis for the behavioral sciences (3rd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
Dear Robert,
can you provide a source that you should analyze for outliers/influencial cases for each level of the moderator separately? I have not heard of it, yet, and Cohen et al. dont say anything as far as I know.
Thx, Rainer
Hey Rainer, I examined this and got lost in the woods for a while. I don't believe I've found anything explicitly saying this, which is part of the problem and mainly why this issue was raised in the first place. I've had some more time to think about this.
Given that the definition of outliers in a regression context is values that are unusual relative to their predicted scores, and that this prediction is specific to the the regression line in question - outliers should be considered separately for each level/value of the moderator. The underlying assumption of moderation when dealing with a non-categorical moderator is that essentially a separate regression line is being drawn between X and Y for each discrete value of the moderator (W = 1, W = 2 etc).
A dichotomous moderator doesn't change this detail as mathematically when dealing with just two levels, it just assigns one category the value of W = 1 and the other being W = 2. Two regression equations are therefore still being calculated when we're doing this assumption testing.
Granted however that I don't believe this is explicitly stated anywhere, but I hope at least that this detailed explanation of my reasoning on the matter could spur someone to point out the error of my ways!
Dear Robert,
I would say everything you said is correct so far, but not your conclusion:"Two regression equations are therefore still being calculated when we're doing this assumption testing."
Why "two" regression equations? It is one equation, one model, in which you insert two different values and calculate the residuals. For example, take a simple independent t-test. Typically, the assumption of normality/no outliers is tested by testing the distribution for each group separately (in my opinion the reason why most of the peope believe that this assumption is about the normality of the data, since this is most often the first test to learn....) OR by calculation the residuals of the overall model. This is the same thing, as the model is only two scalars, the estimated mean values, which are subtracted from the original data within each group, to get the residuals. Therefore, the residuals are just shifted in their location.
Same thing applies to more complex models, in which it is not practical or possible to calculate the residuals for each level separately. A dichotomous moderator is just a special case of a metric one. Would you consider to calculate a regression line for each metric value of the moderator separately? I guess not. Hence, there is no need to calculate it separately, the residuals of the overall model ARE already calculated for each level of the moderator. The residual IS the difference between the datum and the predicted value, where the predicted value needs to account for the appropriate value of the moderator.
- Badges
- Science topic
- Similar topics
- Correction