I have to do a replication study on a model. How much different should it be from the original research? Obviously, it is going to have some aspects similar to the original research, but, in order to make the replication study meaningful, some new perspective is also expected. So my question has two aspects, regarding how much difference is expected so that the study is considered a replication and what novelty should be there.
First aspect: Which differences are acceptable? Like in respect to sample size, gender distribution of participants, age range, geographic distribution, cultural variations, study design etc.
Second aspect: What novelty can I add in the replication study? Like in respect to statistical analysis, study design etc.
The main characteristics for replication study are to follow the same design, methods of data collection, analyses. The main difference is different sample and time. The main purpose for replication to validate the findings in different settings.
I don’t think it is that much important to call it original vs. replication as long as it is significant and needed.
Replication study is done mostly for audit while original study focuses on empirical findings with new methodology that can be reproducible.
Dear Dr Tarun,
I hope this link will give you some general idea
https://explorable.com/replication-study
A replication study is possible and should be carried out, when
- The original research question is important and can contribute to the body of information supporting the discipline
- The existing literature and policies relating to the topic are supporting the topic for its relevance
- The replication study if carried out carries the potential to empirically support the results of the original study, either by clarifying issues raised by the original study or extending its generalizability.
- The team of researchers has all expertise in the subject area and also has the access to adequate information related to original study to be able to design and execute a replication.
- Any extension or modifications of the original study can be based on current knowledge in the same field.
- Lastly, the replication of the same rigor as was in original study is possible. Shared from the link https://explorable.com/replication-study
You want the sample size in the replication study to equal or exceed the original. If you are thinking that your results might contradict the findings in the original study, then having a much larger sample will help your cause.
The experimental design needs to be the same, or at least the core of the study needs to be the same. So the original study looked at fertilizer treatment on plant growth for 4 different weeds. You need the same fertilzer (maybe a different manufacturer, but same elemental composition), and same weeds. If you want to, there is no harm in adding another 3 weed species, or adding another fertilizer treatment. The original measured plant height, root biomass, and seed weight. You need to have those same measures, but you are free to include chlorophyll activity, and stem diameter. The problem may come in the analysis, because you now have replicated the original study (analysis 1) and have added more data (analysis 2). Somehow you will have to cram two experiments into the space reserved for one. It is hard to get a 60 page manuscript published. Depending on the original and your replication you might get two manuscripts out of one project.
First aspect: It depends on the research question. Does it make sense to conduct the same survey in Los Angeles and a replicate study in Lhasa 20 years later? It also depends on how the conclusions in original research was phrased and how it is percieved now. If the original methods selected only males, and drew conclusions about people in general, then your study should have a core section of males and additionally include females if that makes sense in the context of your goals or modern norms. The same goes for age, ethnicity, religion, and so forth.
Second aspect: You can add any novelty you like. Be careful that the added novelty is either a small addition (a couple of paragraphs) so that the overall manuscript length is not excessive, or that it is substantial and you are getting two manuscripts. The first manuscript is the retrospective, the second manuscript extends the results or restricts them. It is possible to do a confirmatory study where you take one conclusion from the original and try to replicate those results. The problem is that the outcome of any analysis may depend on the variables entered into the model. So if the original study modeled y=x1 + x2 + x3 + x4, and you only measure y=X2+x4, then you should expect your results may be different just because X1 and X3 are not in your model. Maybe in your population X1 does not exist. In that case, you need to be careful that the presentation of the original research was strong enough that you can correct (extract) the portion of results pertinent to the new population.
Can you get the original data from the original research, or are you stuck with trying to estimate LSmeans and standard errors from a bad xerox copy of the article?
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size