Over the last decade, Wagenmakers has offered not only compelling critiques of standard approaches to data analysis in psychology, but potentially game-changing alternatives to analysis as well (e.g., Bayesian techniques). The days of the p value as an indicator of evidence seem numbered. But are they? How long will it take the field to embrace the limitations of null hypothesis significance testing, still so widely taught and practiced in the field?
Article An Agenda for Purely Confirmatory Research
I see the Bayesian framework as an extension of the frequentist framework where some of the assumptions are replaced by a probabilistic distribution, and their existence is thus rightfully questioned. Unfortunately, choosing an appropriate modelling of this uncertainty is far from trivial. Often, flat priors are used as reference which typically reduces the analysis to a frequentist approach. Furthermore, I think that the use of frequentist statistics facilitates scientific reasoning based on observations, as little knowledge is required to apply the methods. Unfortunately, this also implies that users do not need to comprehend what the calculated numbers actually mean (their literal interpretation is, in fact, very complicated. The actual meaning of a p-value is a very good example). In addition, by reducing a complex problem into a black number on a white piece of paper it is easy to make an appealing case for a large audience. This implies that there is a risk of oversimplification and publication bias. In short, I think the step from frequentist to Bayesian analyses is an important one but requires some changes:
- no requisite to report p-values in scientific journals (although reporting of parameter uncertainty remains important)
- increased collaboration between statisticians and clinicians.
- more emphasis on sensitivity analyses, particularly in frequentist settings
- encourage data sharing to allow replication of scientific findings
- Badges
- Science topic
- Similar topics
- Psychology