Statistics reform (or paradigm shift) seems to be in progress. The following excerpt from Preprint Statistics reform: practitioner's perspective
briefly describes its background.In recent years, the scientific community has become increasingly concerned about the replication crisis. It is generally accepted that one of main causes of the replication crisis is some of the most commonly used statistical methods. Specifically, the suite of null hypothesis significance testing (NHST) and its associated p-values, and claims of statistical significance, have come in most to blame (Nuzzo 2014). Siegfried (2010) wrote, “It’s science’s dirtiest secret: The ‘scientific method’ of testing hypotheses by statistical analysis stands on a flimsy foundation.” Siegfried (2014) stated, “statistical techniques for testing hypotheses …have more flaws than Facebook’s privacy policies.” Therefore, many authors suggested retiring or abandoning statistical significance and p-values (e.g. Amrhein et al. 2019, McShane et al. 2018, Halsey 2019, Wasserstein and Lazar 2016, Wasserstein et al. 2019). Basic and Applied Social Psychology has officially banned the NHST procedures since 2015 (Trafimow and Marks 2015). Furthermore, many scientists and statisticians call for statistics reform (e.g. Wagenmakers et al. 2011, Haig 2016, Colling and Szűcs 2021). Cumming (2014) proposed ‘New Statistics’ as a form of statistics reform. … Although some authors still defend NHST and p-values (e.g. Benjamini et al. 2021, Hand 2022, Lohse 2022), “A paradigm shift away from null hypothesis significance testing seems in progress (Berner and Amrhein 2022).”
As a practitioner in the fields of hydraulics and measurement science, I used statistical methods extensively in many environmental engineering and hydrological survey projects. I strongly concurs in the need for statistics reform. I join the statistics reform and the paradigm shift from NHST to the estimation paradigm. In my opinion, some statistical methods (tools) are good and should withstand statistics reform; some are bad and should be abandoned and removed from statistics textbooks and computer software packages. For my detailed perspectives on statistics reform, please see Preprint Statistics reform: practitioner's perspective
Would you (as a scientist, professor, researcher, statistician, or industrial expert) provide your perspective on statistics reform? Thanks.
Hi,
P-values can stay, but the focus should shift to broader analysis and responsible use.
Just my thoughts
Hi,
I agreed with some authors regarding the null hypothesis testing especially p-value should be avoided.
In practice, I always asked, if p-value less than golden number 5% or 1% (can this number generalized across industries/processes? ), so what?
Then, what does it mean for our operation & decision makers? Not very convincing.
Maybe works for some, but not for me. P-values is not robust for decision makers (at least my experience).
I do think, the p-value usage is merely like a placebo effect.
I also agreed with Hsin-Yuan Chen , focus should be broader analysis & responsible use. In my opinion, with big data in industries, the quality data plays a bigger role than p-values. In that sense, we need statistics reform.
Decisions cannot be based just on p-values. As indicated in the document, p-values can be ”engineered” with the sample size. Size effects are also important, and a small p-value may be meaningless if the size effect is negligible.
Uncertainties based on symmetric intervals constructed with the t-Student are fair under certain circumstances (small uncertainty, quasi-normally distributed variables), but, in general, profile likelihood confidence intervals are preferred.
I would not ban any tool when training people. It is better to explain the weaknesses and strengths of the many available tools, and then promote critical thinking, solid background, and common sense for applying them.
Article Why and how we should join the shift from significance testi...
- Badges
- Science topic
- Similar topics
- Geoscience
- Atmospheric Sciences
- Atmosphere