When you read an epidemiological research paper what are some of the red flags you encounter in phrasing, statistical tests used, and glossing over controlling for confounding? For example, when you evaluate the COVID reports or vaccine research what are key elements that if not present call into question the research or if included raise doubts?
Hey, so as i understand it and like to explain it, the specific phrases does not exist that you will find similar in all epidemiological studies. it depends on the paper and what the research was trying to do and find. For example, if they are doing a case control study and they have calculated relative risk for something then it is definitely wrong. For another example, see the result and then the sampling method and see if they both fit together or not as sometimes it becomes very convenient to have convenient sampling patterns in epidemiological studies and the sample not fully represents the population as there is some expectancy bias already present in the authors.
Dev Desai thank so much for your reply. I think when associations/correlations are used without any causal inference: propensity matching, difference-in-difference regression, DAG's, and/or COX Proportional Hazards with proper controlling for confounding we have serious limitations of yhe given research.
So basically you're discussing the use of observational data to answer causal questions (which are often best answered using randomized trials). In that case, the most important thing will be bias (selection bias, measurement error, and confounding bias). I will specifically look for the detail of study design and statistical analyses on this part, i.e., how the authors came up with measures to mitigate bias when they recruit and follow up participants, or if this is a retrospective study, how they can confirm the validity of the measurements.
Confounding is often seen as the most important problem in answering causal questions, so of course I'll look for the methods used to control for confounding. As you said in your comment, it may be multiple regression where the potential confounders are selected based on a DAG (or DAGs), some causal inference method such as propensity score or instrumental variable, or the confounders are controlled by design (restriction, matching, etc). However, I believe the use of one method is not sufficient; I'll read the description, too, to make sure that their actual implementation is valid. Unfortunately, you'll see that not all papers described their DAG in detail (e.g., how they came up with the variables and the relationships between variables); some didn't even publish their DAG. Or for propensity score methods, they didn't show the diagnostic results (SMDs, Love plot, histogram of weights).
It's hard to summarize everything I'll look for in a few keywords, but yeah, probably "did they handle bias properly?".
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis