Why not try using a difference variable and get the p-value for that coefficient. Best. D. Booth

Dear Niklas Hansen , It depends on the distribution of your data.

You could use an independent samples t-test if you have only two groups (control/experiment) or ANOVA with posthoc tests if you have several groups (control/experiment 1/experiment 2, and so forth.). If your dependent variable is not normally distributed, several nonparametric tests are for two or more independent samples.

I don't know if this answered your question. Best of luck!

Regards

Thanks so far,

I should have been more precisely. Actually I just want to interpret a research paper, but I don`t have any acces to the data. So is there any rule of thumb or even a test, which one can use to evaluate whether the change of coefficients due to the inclusion of a control variable is significant and meaningful and therefore symptomatic for an omitted variable bias? Besides the coefficient of the independent variable of interest I just have the respective p value, so no standard error. And of course a summary statistics. I read that there is a t-test for testing the significance of the change in the independent variable of interest, but it requires the MSE of the estimator, which I don`t have.

best regards

Niklas

If I understand correctly, the p-value says the independent variable is significant, but you don’t know if it contributes to the model? Besides using the MSE, there are other ways to judge if the model fit has improved or not. You can do different tests based on the information criteria of the model (linear models). For this, however, you need to know the information criteria, with and without the variable(s) of interest, in addition to the degrees of freedom. You can also use the R2 if the model used gives you this value. Again, you would need to know the R2 prior and post the variable of interest. The problem with R2 is that it does not decrease when new regressors are added to the model. The type of information available depends on the test used. Confidence intervals can also be used. If the intervals are not overlapping or the confidence interval does not include the null hypothesis, they are statistically different. I don’t know if this was any help. If the paper provides limited information, the easiest way might be to email the author. Based on my personal experience, the authors are usually glad to help! All the best,

You can turn a p value into a standard error and confidence intervals

https://bookdown.org/MathiasHarrer/Doing_Meta_Analysis_in_R/k.html

Altman, Douglas G, and J Martin Bland. 2011. “How to Obtain the Confidence Interval from a P Value.” BMJ 343. British Medical Journal Publishing Group: d2090.

If you referer to the stability of the model (in case of parameters) you can test also CUSUM ,CUMUSQ tests, depending on the context