I performed a logistic regression. I checked for interaction effects of gender with the IV. The interaction was not significant. However, when i stratify by gender, there is a significant influence of the IV on the DV for females, but not males. If that last statement is the case, why isn't there an interaction effect of gender?
Here are the optimal measures to take when considering stratified analysis in epidemiological analysis.
1. Preplan the stratified analysis [1] for factors for which there is a biological plausible explanation of the difference of effects of one factor on another depending of the exposure to a third factor [2].
2. Make sure the sample size is large enough to have sufficient power for each strata.
3. Report results for each strata and evaluate the presence of interaction by a test of heterogeneity or by the significance level of the interaction term if you are performing regression analysis.
4. If no interaction is present, look at the magnitude of the association within each strata and their CI95% to make sure you can exclude a clinical significant difference.
5. Never, never, go data dredging using stratified analysis -> you will always and easily find a subgroup in which an association becomes significant. This is however most likely due to chance and can lead to major misinterpretations [3].
The major error that is done when interpreting stratified analysis is to compare the significance level of the association within each strata. These depend of the sample size and cannot be compared. To test for modification effect, you need to rely on the test of heterogeneity or the log rank test of the interaction term. This will tell you wether there was a significant difference in effects between strata.
References
Confounding is different to interaction - could this be the case that this is the reason? not sure, just an idea.
When the interaction effect is not significant, it means, the effect of IV on DV for females is not significantly different with effect of IV on DV for males. Such cases occur usually when sample size is not sufficient or difference between effects in males and females is small. However, if it's important for you, you can stratified by gender and report two effects for males and females but say this difference is not statistically significant.
That really depends on your data, results and research interest. You may find similar results for men and women, but the coefficient for men only slightly smaller and because you have just half of the sample it won't be statistically significant. This is only a problem of statistical power. In this case it probably won't make sense to separate the sample. If, however, you find no effect at all for men, then you should analyse the two groups separately, especially if you could justify it theoretically.
If you consider stratifying by sex, depsite non-significant interaction, please consider also
- multiplicity correction issues (here, two instead of one, hence p< 0.025 instead of 0.05) ;
- application conditions of your logistic regression: ils (log-odd) linear relationship true in both male and females
- separation and quasi-separation issues
and of course power issues and all non-linear fitting issues that can affect logistic regression in general.
Just choosing to stratify because you see something in one group in this way and not with interactions seems a little bit « weak » to build solid conclusions; you should investigate why you obtain apparent discordant issues by the two methods, and only then you can select « confidently » one.
This would be true without the interaction, but the interaction term should take care of the differences between groups, and avoid the paradox, no?
I'm not so clear with forbidding stratification without interaction, because for instance in a linear model framework, using a whole model with interaction assumes equal variances in both males and females, and if this is clearly not the case, it may be simpler to treat separately males and females than develop a model with additional heteroskedasticity... But this is I guess matter of discussion, and not directly relevant to logistic regression.
You don't say that the results are different in men and women, only that one is significant and the other is not. Tests of signficance are not a useful metric as the are sample size dependent and small differences in the point estimate effect whether a result is significant or not. In general, one should stratify by sex as there are biological differences that often result in interactions. It might be the case that the sample size is insufficient for stratification, but this also will mean it is underpowered to detect interactions as well. In this case, it would be well to look at the models in men and women separately and see if there are any variables where the confidence intervals are different and where this difference is plausibly sex-dependent.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics