For example: a trial that has measured anxiety, in the same population, three different ways. In anticipation of your asking "why would they do that?" - well lets say they wanted to use three different instruments, or self-report, parent-report and practitioner report. How do i get to one effect size for the study, before combining in my meta-analysis. I believe there is some way of generating a standardised mean difference from the three separate results ... please explain as if your are talking to a very young child!
Hey Tony
I don't know if it is appropriate to combine these different outcomes.
You could choose one of the outcomes, but you should argue for which one you pick, since it might introduce a reviewer selection bias (exaggerating the results). A great option would be to publish a selection hierarchy (e.g. 1. patient reported, 2. parent reported 3...) (in any case, you should provide a selection hierarchy a priori or post hoc for transparency reasons. An a priori hierarchy is not an option, you could choose e.g. the most valid one or the one that is most commonly used in the trials. You cold make sensitivity analysis to check if selecting one or another outcome for meta-analysis would change the conclusion.
See e.g.:
A Hierarchy of Patient-Reported Outcomes for Meta-Analysis of Knee Osteoarthritis Trials: Empirical Evidence from a Survey of High Impact Journals
https://www.hindawi.com/journals/arthritis/2012/136245/
Sincerely
Martin
Hi,
In general, self-report tools are known to be subjective and biased. I think artificially averaging or combining the three ratings would be erroneous. It would be better to analyse them separately. You could also see which is the predominant measure adopted by most studies reviewed and stick to that for consistency sake.
Alternatively, you can analyse each measure separately and see if this affects the conclusion of your meta-analysis - and discuss the possible reasons accounting for any observed differences.
Hope it helps!
Interesting question. My first question would be: why do you want to combine different data? Methodologically speaking, you need to be careful if you want to combine quantitative data that are separate measures of the same thing. Example: if a patient thinks they are anxious and their doctor thinks they are anxious, that does not mean the patient is twice as anxious. It can be tempting to do this to achieve more statistical power or better p-value & confidence intervals. However, it would be 'conflating' the data. Look up statistically methods to avoid conflating.
If you do combine measurement, these could only be comparative rather than absolute measures. As a general rule I would suggest only combining data if it makes logical/theoretical sense. For examples: to operationalize "health behaviours" you might combine: a) smoking, b) diet and c) physical activity.
Another point I would make is that before deciding to combine different measurements of anxiety. Check to see how different or similar they are. This may be a more interesting point to comment on.
Hope that makes and is helpful!
Paul
Thanks for those replies. I came across a youtube video by Michael Borenstein who describes how he believes it should be done - https://www.youtube.com/watch?v=fQ3t5RScLGw
- Badges
- Science method
- Similar topics
- Mathematics
- Statistics
- Statistical Techniques
- Meta-Analysis