Suppose, we want to check association between A and Z using regression model. We add different covariates in model 1 let's say b, c, d, e. The regression shows significant association for A and Z after adjustment for these covariates. However, in model 2, we add another covariate let's say f, but when we run the model now, association between A and Z becomes non-significant? What caused this non-significance? What could be the reason for this?
One reason would be if the covariate f causes both A and Z. In that case, the relationship between A and Z is spurious, that is, it only exists because f is their common cause. Once f is in the model, the relationship between A and Z vanishes because it is entirely explained by f.
Another reason could be if you have many correlated predictor variables and/or covariates already in your model leading to overfitting and/or collinearity issues. In that situation, adding another predictor/covariate may increase the standard errors because of collinearity (which adds uncertainty to the model), limiting your power to properly detect individual relationships.
Christian Geiser So if f is the common cause that means that association between A and Z without F in the model can't be taken as true association? Means we can say that there is no association between A and Z? The relationship only exists because of F? What should be next step here for analysis? Should effect modification be checked to find out to how much extent Z modifies the association between A ad Z?
Collinearity has been checked and there isnt any with F in the model and even without F in the model
I think you need to recheck each of your covariates to make sure they are added correctly. As it stands, I think there are several scenarios (I'm assuming for now that the controls for b, c, d, and e are correct since I don't know the rest of the details): 1) you control for a confound that is new and causes a major effect, then at this point, as Christian Geiser says, you now have the correct result, the relationship between A and Z is spurious; 2) you control for a mediating variable on the causal path from A to Z. Then at this point, if there is no confounding between the mediating and outcome variables, you get the correct result as well, except that you now get a direct effect of A on Z; 3) you control for a colliding variable, thereby opening a new backdoor path, and the correlation on this path relation is in the opposite direction to the previous correlation, then your current result is wrong, which is the worst case scenario. So I think you need to check each of the covariates you use against your research question and research hypothesis to make sure they are correct and necessary.
Hello Hasan Bin Sohail. A classic resource, if you can get your hands on it, is Chapter 13 (Woes of Regression Coefficients) in this 1977 book by Mosteller & Tukey.
Mosteller, F., & Tukey, J. W. (1977). Data analysis and regression. A second course in statistics. Addison-Wesley series in behavioral science: quantitative methods.
If you cannot get the book, this blog post summarizes some of the key points:
- https://imstat.org/2012/07/18/terences-stuff-multiple-linear-regression-1/
HTH.
There are, I think, three possibilities here (not mutually exclusive).
1) Collinearity between A and f. You state "Collinearity has been checked and there isnt any with F in the model and even without F in the model". This is implausible except in designed experiments as it implies that there is exactly zero correlation with any of the covariates. Moore likely I think is that the collinearity is low. However, even a low correlation between A and f is reducing your effective sample size (relative to a model with zero collinearity). For example modest collinearity such as tolerance of 0.90 is a 10% reduction in effective sample size.
2) Each predictor uses up a degree of freedom in the model and reduces the error d.f. by 1. Even if there is zero collinearity this produces a modest decrease in statistical power because the estimate of the error variance (SS residual) is larger and the test statistics smaller. If overall n is low this effect can be very pronounced. If n is large then this effect is usually negligible.
3) There are missing data on f and model 2 has smaller n and may lose influential cases in the A-Z relationship.
I'll add my usual question: Why do you want to know? What really is your question about these data? Are you primarily concerned with the association between A and Z? Why? is that a scientific question and you plan to look for the mechanism by which they are associated? Or are you planning to change or control Z for some subjects and want to predict how it will affect their A? Statistical significance is rarely the answer to questions like these.
If "we want to check association between A and Z," then why are you adding covariates? As far as what method to use, it depends what type of association you are interested in. If linear, Pearson's is one approach, and then you could use bootstrapping for a significance test (if you think that is useful) and intervals. But this depends what you mean by association, and since you don't say that no one should be about to answer.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size