Hi All,
I am currently undertaking a systematic review. Two of the studies, completed by nearly the same authors, seem to have used the same set of participants.
The differences mainly lies in the study design:
Study 1 included a double-blind, randomized, sham controlled trial (5 days/ week for 2 weeks) in twenty patients with 3:2 ratio. At baseline, each patient underwent a clinical evaluation. Assessments were then carried out immediately after either shaml or real stimulation (post-stimulation, T1), at one-month (T2) and at three-months follow-up (T3).
Study 2 included a randomized, double-blind, sham-controlled, crossover trial (5 d/wk for 2 weeks) in 20 patients with a 1:1 ratio. Each patient underwent a clinical evaluation before and after real or sham stimulation. A follow-up evaluation was performed at 1 and 3 months with a crossover washout period of 3 months. After the washout, After a washout period of 3 months after the last visit (i.e., T3), each patient received the opposite treatment (crossover phase)
and underwent the same standardized assessment as in the first
phase, at baseline, at 2 weeks, at 1 month, and at 3 months
Both studies made use of the same outcome measures and data analysis involves identical data analysis measures.
What is your view on the below:
1. Should both studies be included in the review?
Since the authors seem to have made use of the same participants (same research authors, same institution, identical sample size, same outcome measurements), could the inclusion of the two studies bias the findings of the review?
Thank you!
Pasquale
I would contact the authors and ask if those participants in two papers are same or not. If the participants are same in both papers, I would not use both of them. If the participants are different or the outcome differ (for example different time, different outcome ect) I would include both of papers. However, you must write the decision you have made, or otherwise readers considers with the same questions.
I would contact the authors and ask if those participants in two papers are same or not. If the participants are same in both papers, I would not use both of them. If the participants are different or the outcome differ (for example different time, different outcome ect) I would include both of papers. However, you must write the decision you have made, or otherwise readers considers with the same questions.
Very important question and good answer above, I think. There are two issues in this question. Handling of dependence from multiple outcomes in meta-analysis and research ethic related issue. The independent study is the primary unit of analysis in the meta-analysis. Therefore, for the studies that reported multiple outcomes, an approach of selecting a representative single outcome or making an average value, or shifting unit of analysis has been used for resolving dependence. If these two studies included same population, researchers who perform meta-analysis should seek for ways to solve the independence issue. If these two studies are considered as results from salami splicing of same population, researchers who perform meta-analysis should determine if this could be justified. The overall determination is dependent on the researchers doing meta-analysis who are the specialist in this topic. These processes and reasons for the decision should be reasonable and presented in the paper.
Good catch.
This highlights the issue of selecting studies (not reports) as the unit of interest in systematic reviews. As the Cochrane Handbook for SR of Interventions (Ver 5.1.0) highlights:
7.2.1 Studies (not reports) as the unit of interest
A Cochrane review is a review of studies that meet pre-specified criteria for inclusion in the review. Since each study may have been reported in several articles, abstracts or other reports, a comprehensive search for studies for the review may identify many reports from potentially relevant studies. Two distinct processes are therefore required to determine which studies can be included in the review. One is to link together multiple reports of the same study; and the other is to use the information available in the various reports to determine which studies are eligible for inclusion. Although sometimes there is a single report for each study, it should never be assumed that this is the case.
Paula Heikkilä's suggestion to confer with the study authors is the most efficient way to determine whether or not the two reports are actually referring to a single set of study participants or two separate sets of study participants. You can then make the decision as to whether or not to include data from both reports without risking "double counting" data from a single set of participants. IF the reports are from a single set of study participants, you may want to consider which of the 2 analyses best suit your SR objectives and your pre-specified study inclusion/exclusion criteria.
Hi Pasquale
I had come to the same opinion as our colleagues already answering before I scrolled down to see their answers.
My first priority would be, as Paula suggested, to contact the authors to see if the two cohorts are different and if not why they were recycled in a separate study with apparently similar methodlogy. Of course it depends upon the rarity of the condition being investigated whether enrolling new cohorts of patients can be achieved. Also it may be that the authors, having obtained one set of results with a direct comparison between two groups using active intervention versus sham, wished to see if the cross-over produced similar outcomes. So much depedns upon teh nature of the condition under investigation.
In our studies we often perform studies with apparently similar cohorts but the participants are not necessarily the same, although on occasions some individuals may appear in more thna one study. However, since we are looking at a clearly defined end point of an acute condition (eliminaton of head louse infestation), the person does not influence the outcome per se (it is the insects we are investigating) so we could even include a whole cohort a second time without there being any risk of "double counting".
Of course there may be clues in the reports as to whether the participants differ, for example if the authors report demographic characteristics you may be able to gain an idea, at least, as to whether the populations appear closely similar. so teh answer is dig a little and see what turns up.
Best wishes
Ian
Dear All,
Thank you for your responses. I was in contact with the author - practically some participants are included in both studies. The author can provide the exact number of participants that were included in both studies. Although there is some variation in collected outcomes, the study design is the main difference between the two.
Since there is extensive heterogeneity between included studies in terms of design and outcome measures , it is likely that it will not include a meta-analysis. It is likely that the review will be a narrative one.
As a result, I think that the fact that statistical analyses will not be included may warrant the possibility to include both studies in the review.
Any thoughts?
Thank you,
Pasquale
Hi Pasquale,
If the SR will be a narrative review and will not include a meta-analysis, then it would be appropriate to describe the two studies within the review.
For clarity and transparency, it would be good to mention that some participants in Study A were also included in Study B (It would be nice to have the specific number of participants in each arm, in each study that overlapped if possible; this would depend on the amount of detail the authors have shared with you) so that users of your research can make adjustments in their conclusions, as they deem necessary.
Hi Pasquale
Definitely agree with Elenore there. Of course, this is one of those occasions when open access data and data sharing would come into its own.
Regards
Ian
- Badges
- Science topic