The Null Hypothesis "statistically" Significance Testing (NHST), in which the P-value serves as the index of “statistically significant,” is the most widely used [misinterpreted and abused] statistical method in psychology (Sterling et al., 1995; Cumming et al., 2007).
Rresearchers overstating conclusions beyond what the data would support and "careful fixing cheats if statistically significant", is quite common! However, a good reviewer (and experienced in same field/subject) would notice tht! Why? Increasing bait_like activities of predatory journals and a widespread lack of methodological sophistication, with researchers using poorly designed experiments with small sample sizes and inappropriate statistical models (Gelman and Carlin, 2014).
In the Neyman–Pearson framework optimally (which may not be possible in some instances) setting α and β assures long-term decision-making efficiency in light of our costs and benefits by committing Type I and Type II errors... this is frequestist approach, would bayesian approach make a difference where trial or studies repetition are limited/not possible? Perhabs, a prior calculations and report of the probability of replication to complement NHST! .
David Eugene Booth , What solution/approach is feasible, experimental design and that would ensure greater meaning of results and their validity? As Soderberg and Errington (2019) point out, there is a continuum of different types of replication, but a common distinction is between close or exact replications and more distant or conceptual replications. Reduce P-value threshold or add Probability of Replication to NHST(Significance tests without priors are the “flaw in our method.”) in reports .
Aim:...greater power for "the test" and what you would teach with respect to Statistical Inference procedures and how NHST and CIs (also their abuse) could influence the interpretation and validity of research results.
First, p-values have nothing to do with design of studies but lots to do with interpretation. There are as u point out already other possible ways to look at things many involving Bayesian ideas. Many in the social sciences. As a statistics professor I have not seen an increased demand for such studies this makes me think that the people abusing p-values would probably abuse Bayesian ideas too We must,IMO get away from the idea that there's some magic thing that makes a study publishable which is what drives p-hacking in my view. I suppose people like that are not interested in good science but rather in body count of papers. In any case education is part of the answer. BTW, greater power is achieved by finding better methods that would make power obsolete perhaps. In any case as you might've expected I will call for increased statistical research and education. I don't however expect the people that are abusing p-values to have much interest. Thus better education in both methods and ethics may be the best hope for future researchers. Best wishes, D. Booth
Thank you so much Prof David Eugene Booth We really need better education both in ethics, methods and core concepts be it that p-hackers are interested or not. "Facts are the air of scientists. Without them you can never fly." - Linus Pauling. Am really greatful for your response sir.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics
- Sampling Design
- Sample Size