I think the best way to address whether censoring authors in this way will have a good or a bad effect is to look what happened when it was done. See Article Assessing the Statistical Analyses Used in Basic and Applied...

NHST is logically and philosophically flawed mixture of two actually incompatible methods (see below). NHST, as proposed in text books, should be abandoned.

In research, tests aiming to reject H0 (Fisher's test of significance) is usually sufficient and should be used as a first line of defence against over-optimism about the data (not the final "approval" of something interesting!).

Neyman-Pearson acceptance tests (or A/B tests) are usually not applicable in a meaningful way in research since there are usually no two meaningful alternatives to test against each other.

I think NHST is a warrantable method per se, but that it is wrongly applied and/or misused by many people, either because they don't know better or they actively rig their data. The problem is not the method itself, but the people. As Gigerenzer pointed out, what we are doing/teaching are statistical rituals and not statistical thinking. Therefore, not the NHST should be eleminated, but the way statistics is taught, especially for the applied researchers, who do not have a formal mathematical background.

On the other hand, researchers need to accept statistics as a tool to get the most valuable infomation out of the data to answer the questions asked. But I have the impression many of them see statistics as a necessary evil and not a helpful tool. I myself did lots of statistics wrong in the past (and I am pretty sure still do), but I am curious to learn new methods and to revise old, wrong or not completely correct views, to improve my own analyses.

So in conclusion, there is work to do on several fronts.

Jochen Wilhelm I just read your comment. Would you say that NHST is only the Fisher/N&P hybrid model in a narrower sense, or would you say that both methods itself could be subsumed under this acronym?

And what do you mean with philosophically flawed? The method itself or the (wrong) interpretation that you get a probability for your model given the data?

I think the best way to address whether censoring authors in this way will have a good or a bad effect is to look what happened when it was done. See Article Assessing the Statistical Analyses Used in Basic and Applied...

Daniel Wright thank you for the article!

Rainer Duesing , NHST is a hybrid.* And I think that the logic is flawed. It's based on the modus tollens, but probabalistically - and then it all makes no sense anymore. The argument is as follows:

If the null hypothesis is correct, then this datum (D) can not occur. It has, however, occurred. Therefore the null hypothesis is false.

This is logically correct. But in NHST it is actually:

If the null hypothesis is correct, then these data are highly unlikely. These data have occurred. Therefore, the null hypothesis is highly unlikely.

This is flawed. Pollard & Richardson (1987) gave a simple demonstationwhy this argument is invalid:

If a person is an American, then he is probably not a member of Congress. This person is a member of Congress. Therefore, he is probably not an American.

The reason for this problem in NHST is that one hypothesis (H0) is contrasted to a pseudo-alternative (HA = "not H0"), and it is assumed that one of these two if true and the other is false (what already is a logical mistake).

---

*some authors use NHST for Fisher's tests of significance (calculate a p-value and then judge, informally, if it seems to be sufficiently small to continue with research). The hybrid uses fixed alpha, rejection regions and the term "alternative" from Neyman/Pearson test (but fails to account for the need of an utility function). The hybrid also sees the test result as a final decision, like the NP approach. Fisher's test of significance is not including any decision. It is the calculation of a p-value. This must then be interpreted in the context of the research and the experimental details, not to decide between H0 and the alternative but to see whether or not the data are sufficiently incompatible with H0.

NHST as acronym for the hybrid I agree with your statements and recommend the following article.

https://www.frontiersin.org/articles/10.3389/fnhum.2017.00390/full

Thank you Rainer Duesing for bringing up this very good paper!

I believe the construct of Pollard & Richardson (1987) for that hypothesis is flawed itself, it's not NHST problem but flawed logic position. The construct of "Ho" and "not Ho" is such that set of one is a complement of the set of the other(alt) in the Hypothesis Space. We are not seeking to accept the Ho posit (and hence logic) but to reject on "sufficient statistically significant evidenced" .

The problems with NHST were identified, and include sensitivity to sample size, exact (and contextual) repeatability, odds that the hypothesis is correct given absence of prior probability(significance without a prior, [and a good posterior for replicability] and also the probability/rate of error reports.

Steven D. LeMire (2010) arquements employing Toulmin (1958) model of argument (for/and using NHST) is plausible and really good.[Hope the journal is not predatory classified] http://jse.amstat.org/v18n2/lemire.pdf

The scientist that wrote the article, though extensive and appearing thourough wrote from position of applying Statistical tool, (medical trials with limited number of repeats, unlike industrial application), and really until recently replication studies have been very rare in many scientific fields; lack of replication efforts has been a particular problem in the psychological sciences [too](Makel et al., 2012). As in the article, "Crucially, the Neyman–Pearson approach is designed to work efficiently (Neyman and Pearson, 1933) in the context of long-run repeated testing (exact replication)".

Many of the reason he gave are contestable. P value is a random estimate, and is often mis-applied. Actually, and reasonably, a single significant result should not represent a “scientific fact” but should merely draw attention to a phenomenon [that is beyond just a chance] which seems worthy of further investigation including replication (Goodman, 2008). I don't want to "dance sentimentaly following" around limited prove of uselessness of p value or falsehood of NHST. Attention have been drawn, situation and alternatives should be proposed given prior distribution of data[Bayes' ], repeatedability of study, minimizing errors in inference.

Well Fisher and Neyman's work has lasted for a hundred + years so people have found some useful things about it. Statisticians are IMHO first a scientist in a subject area(s), 2nd an applied mathematician and third an applied logician. These skills should be kept in mind as we attempt to improve all of these things. An example of my view of how these things go together is my response to this question: https://www.researchgate.net/post/Significant_value_post_Bonferroni_correction

Best to all, David Booth PS I have in other posts given my view on why I think the combined view of the Fisher and Neyman- Pearson approaches is reasonable. My views are given by Erich Lehman et al here: Article The Fisher, Neyman-Pearson Theories of Testing Hypotheses: O...

Further I think that Daniel Wright raised some excellent points above.

I just want to make clear that I am not mistaken, when I said that NHST should be abandoned. This does not mean that testing, or a critical evaluation of the relevant information in the observed data, should be banned. This "first line of defence against over-interpretation" really is necessary, imo.

Instead of just saying "we observed a significant effect of.." (where "significant" actually means statistically significant), one should provide the smallest effect that is still statistically compatible with the data (this might be the limit of a 95% confidence interval that is closer to H0) and put this into its scientific context. If the confidence interval includes H0, any conclusive discussion is anyway impossible, as higher and lower effects than H0 are both statistically compatible with the data.

For instance, if I tested the effect of a drug on the blood pressure and observe a confidence interval from -30 to -1 mmHg (with a p-value for "zero effect" of

Related to Jochen Wilhelm 's answer, where is what I have in my teaching notes for testing band (non-nil) hypotheses. This is before I talk about Bayes Factors, where non-nil nulls seem more used (sorry fs don't copy from my pdf).

5.5.1 Testing Band Hypotheses rather than Point Hypotheses

The null hypothesis, as traditionally used, tests a single point hypothesis. In the context of this chapter it is whether the dierence between two population means is exactly zero. But researchers are not just interested in whether two means are exactly the same. Instead of testing a point null hypothesis, Serlin & Lapsley (1985) and Murphy & Myors (1999) discuss testing a null hypothesis

band. You would specify some region of values around the point that are, for your purposes, the same as saying that there is no eect. As these authors note, this addresses some diculties with NHST and better reects what scientists are doing when they test null hypothesis. It is surprising that this approach has not become more popular, but because of that it will not be used in this text.

One reason that it may not have become popular is that you need larger samples. Murphy et al. (2014) describe methods (power analysis, a concept discussed next chapter) and provide software to assist (see supplementary materials of www.routledge.com/9781848725881). The researcher has to dene this eectively zero band as well as the minimum eect to detect. Another reason

why the approach might not have caught on is that the p-value will always be higher than when testing the point hypothesis. Because reviewers expect to see the values from traditional NHST, it might be dicult to communicate that this higher p-value is more appropriate. If a discipline is thinking about tweaking some aspect of NHST, this choice should be considered as well as others, like changing the conventional threshold for calling an eect signicant from .05 to .005 as suggested in Benjamin et al. (2018). However, there are several ways to evaluate a hypothesis and dierent information, for and against the hypothesis, should be presented to the reader.

Since Bayes Factor has already been mentioned, I would like to hear your opinions about it. There are some obvious advantages, e.g. you can also get evidence for your null, increasing sample size does not only favour rejection of H0 and so on.

But somehow I have the impression that the whole analysis process may again be reduced to a single number, by citing Jeffreys, Wagenmakers or Raftery and using words like "anecdotal", "strong" or "decisive" which sounds to me like "significant" and "non-significant" in many flavours. In search for simple decision criteria, researchers may again forget to interpret the data, coefficients and results itself.

I personally find it more and more appealing to follow the approach by Kruschke (e.g. 2014) and focus more on the parameter estimation, for example by using regions of practical equivalence (ROPE) for each parameter of interest. Model comparisons like BF are optional and not in the focus of the analysis.

Please share your views. THX

Rainer Duesing , I agree. Some of the bad bits of p-values use, like applying verbal labels, gets repeated when listing those labels for BFs (I was teaching inter-rater reliability today and same issues with that). The arguments made by many modern Bayesians is that its notion of "probability" is simpler (or something) than the "probability" of classical statistics. I find both difficult; both require a lot of imagination. One plot I like that is in a few Efron's publications shows a triangle with Bayes, NP, and FIsher at the points, and how procedures fit within this (eg., FIg 8 of https://www.jstor.org/stable/2676745?seq=1#metadata_info_tab_contents, but I believe there is a more up-to-date version in his 2016 book with Hastie).

edit: It is also on p. 265 of https://web.stanford.edu/~hastie/CASI_files/PDF/casi.pdf, not behind a paywall. And in color! See p. 448 for a triangle with applications, mathematics, computing, and showing how with time the discipline is becoming less lenient on math and more on application and computation.