Social sciences have abandoned replication in favor of always "making it new," and that this undermines confidence in a number of findings. On the other hand, Kahneman recently calls for ‘daisy chain’ of psychology replications, mainly in social priming research?? http://www.nature.com/news/nobel-laureate-challenges-psychologists-to-clean-up-their-act-1.11535
Recently, "Perspectives on Psychogical Science", has published a free acces section about the need for replicability and the crisis of confidence that affects psychological research. Very useful for this debate: http://pps.sagepub.com/content/current
The Reproducibility Project (http://openscienceframework.org/project/EZcUj/wiki/home) appears to be a first step toward rectifying this problem.
Jaume: from the experience of my lab, we have had several failures in replicating some published research results even though we used the instructions, variables and parameter values as precisely as possible. This shows that replicability is an issue indeed. Thus, there should be some minimum number of experimental works showing the same or close results, given the same experimental conditions. On the other hand, there are some problems with replication research and its consequences. I would not expand my discussion here, but point out just two. First, the failure of replication need not necessarily mean that the original research was seriously flawed or fraud. There are so many potential variables that may cause inter-lab differences in results so that accusations on bad or unethical research would be possibly unfair. (Consider inter-regiobal individual differences in subjects, language nuances in interpreting instructions, subtle differences in equipment, the fact that with time passing subjects samples and cohorts inevitably have undergone different effects /eg, amount of having been influenced by flickering display monitors/, etc. Second, journals cannot be overloaded by replications. So there should be some agreed upon maximum of replication studies after which the new additional studies may have to find a special place, eg in an internet-accessible repository for replication studies files. Priority should be given to replication research where some unexpected twists in relation to well known, maybe already "calssical" paradigms have been noticed.
see Gary King's work on replication in political science is very valuable to read.
http://gking.harvard.edu/category/research-interests/applications/informatics-and-data-sharing
Thanks a lot Felicia! Data Sharing could be a valuable second step. Let's keep on walking...
Replication is critical in scientific research; without replication we do not have much of a science. Journals should be required to include replications especially the ones in which the original study was published even if it is 'overloaded by replications'.
Of course, this is a tricky issue. Ideally, we'd replicate everything several times. But if we actually did this we'd be too busy running replications in place of running new things to get published. We need publications to stay employed and journals only want what is new. This is flaw in the system. If everyone had to replicate in order to publish, then it would be in our personal interests to replicate. It should be like that, but sadly it isn't. It is hard for individual researchers to change without risking loosing our jobs. Journals, granting institutions, and hiring committees need to change. Hopefully this will happen in the near future (it has happened in other disciplines).
Yes, it is difficult to spend time replicating when it is not valued by the people in charge of your work life. Maybe it would be useful to have a replication index for each journal indicating the extent to which each article in a given journal has been replicated. Maybe this approach would place some pressure on the top journals to publish more replications.
- Badges
- Science topic
- Similar topics
- Statistics
- Time Series