I was also in the middle of expanding on Jochen Wilhelm's reply when my reply vanished into the ether. I'll try once more.

When discussing hypothesis testing, people often use a legal analogy to address the question about how to interpret a non-significant result. In many legal systems, there is a presumption of innocence. For example, if I am charged with a crime under such a system, the default position is that I am innocent, and will only be found guilty if there is sufficient evidence. Importantly, if I am declared innocent at the end of the trial (due to lack of sufficient evidence to convict me), this is not at all the same thing as exonerating me (i.e., proving that I am innocent).

In hypothesis testing, the default position is that H0 is true, and H0 can only be rejected if there is sufficient evidence (i.e., if p alpha, you must stick with the default position--i.e., the presumption that H0 is true. But just as in the legal trial, sticking with the default does not prove that the default is true. All it means is that there is not sufficient evidence to reject it.

I'm sure you can find many other examples of this legal analogy online. Try searching on .

HTH.

That your data is inconclusive regarding the tested hypothesis.

Hello Adel,

To elaborate slightly upon Jochen's reply, the results suggest no dependable difference or distinction in outcome scores between the two treatment groups. Hence, you should behave (until evidence to the contrary appears) as if the treatments are comparable in their effect (at least, with respect to the outcome which you recorded). Is it possible that this conclusion is correct (for the population)? Yes.

Is it possible that this conclusion is wrong? Yes, and one might point to the small N as a possible reason for an incorrect conclusion in your case: That the statistical power of the comparison you ran was likely not very high (unless the magnitude of the effect was considerably larger in the population than what you observed in your sample). And for this reason, you can't assert that you have "proven" no difference between the two treatment conditions.

Good luck with your work.

I was also in the middle of expanding on Jochen Wilhelm's reply when my reply vanished into the ether. I'll try once more.

When discussing hypothesis testing, people often use a legal analogy to address the question about how to interpret a non-significant result. In many legal systems, there is a presumption of innocence. For example, if I am charged with a crime under such a system, the default position is that I am innocent, and will only be found guilty if there is sufficient evidence. Importantly, if I am declared innocent at the end of the trial (due to lack of sufficient evidence to convict me), this is not at all the same thing as exonerating me (i.e., proving that I am innocent).

In hypothesis testing, the default position is that H0 is true, and H0 can only be rejected if there is sufficient evidence (i.e., if p alpha, you must stick with the default position--i.e., the presumption that H0 is true. But just as in the legal trial, sticking with the default does not prove that the default is true. All it means is that there is not sufficient evidence to reject it.

I'm sure you can find many other examples of this legal analogy online. Try searching on .

HTH.

Hello,

Before discuss test results and for not reporte a conclusive decsison about your study, I think the sample size is very small (under the design you follow to run MW test);

and if you think the data are sufficient under such design , you simply accepte the H0 hypothesis according to it: "there is no difference between the two groups".

Best wishes

I'd like to respectfully "correct" David Morse :

"Hence, you should behave (until evidence to the contrary appears) as if the treatments are comparable in their effect (at least, with respect to the outcome which you recorded)"

I think this sentence is either wrong or easily mistaken. The simple and sole fact that p>0.05 is no indication at all that one should consider effects as being similar. This would imply to "accept H0", what definitively is a no-go.

If you want to claim that the effects may be similar, check if the confidence interval of the difference is completely inside the range considered "similar differences". Taken without any further information, p>0.05 only indicates that the data are inclonclusive regarding the tested hyothesis and nothing, really nothing more.

This is different in A/B or acceptance testing (Neyman/Pearson). Here, the experiment was designed to accept oneof two competing and substantially different hypotheses (A and B). So upon the result of the test (that can be decided without calculating a p-value!) one either behaves as if A was true or one behaves as if B was true. This imortant distinction between significance tests (using a p-value to estimate the surpringness of the data under H0) and A/B tests is blurred by the logically flawed NHST that is unfortunately thought in many books and classes.

Regarding the power: it is very obvious that the observed power was too low. Observed power and p-value are two sides of the very same coin. It's superfluous to talk about observed power at all. The concept of power is interesting onlyif it is used to design the experiment and needs an estimate of an effect size that can be reasonably expected. This can be expanded into the concept of A/B testing where a reasonable estimate of a loss-function is used to set reasonable confidence levels for the acceptance of either hypothesis.

Bruce Weaver The comparison of presumption-of-innocence to the null hypothesis is misleading and contributes to the misunderstanding of hypothesis testing and p-values.

Inferential statistics uses samples to infer the population. A p-value is one step in qualifying the sample for inference. The process of qualification is distorted by naming a part of sample qualification ‘statistical hypothesis testing.’

A scientific hypothesis evaluation requires asking questions. One question is similar to the statistical null hypothesis. Say, we wish to scientifically compare A to B. We first assume A = B. Now, the first question. What do we consider that makes A = B? We might decide the means and distributions are the same and only important measures. The second question. How must B differ from A to make a scientific difference? We might decide a quantitative difference in means, a difference in distributions, or both.

Statistical testing is similar. Assume A = B. Define a difference and a statistical test that establish a statistical difference. The statistical test and p-value define a minimum for observing a difference. This difference might not be quantitative nor equal to the scientific requirement. The sample for A might be different from the sample B, but the difference is smaller than statistically observable. There is no reason to accept A = B when the sample results are different. All that can be said is the samples did not show a statistical difference.

“We cannot make a determination based on the samples” is not equivalent to “we accept the assumption A = B.” While assumption and presumption appear similar, assumption is for mathematical convenience to establish the baseline and presumption is for until proven otherwise. We must accept the presumption. We have no obligation to the assumption.

Statistical language that states or implies acceptance or rejection of assumptions based on a statistical test is not appropriate. Worse, is language that equates assumption for testing purposes to hypothesis testing.

The first question is why you choose to calculate a p value (and that approach). Assuming you had a good reason for this, the next question is what minimum effect size to detect did you use in your power analysis. So it would be good to address these.

With these you can talk about what your results show and do not show. This is often easier to discuss when reporting the confidence interval. The final conclusion, as Jochen Wilhelm succinctly said: you are not confident even in what direction the true effect may be in.

Hello Joseph L Alvarez and thank you for your comments. I'll have to read them again and think about them some more before deciding if/how I need to change my practice of using the analogy to legal trials when teaching hypothesis testing. But for now, I'll just point to a couple of old newsgroup posts that have influenced my thinking about this issue. You can view them here. The first post post is particularly relevant.

https://sites.google.com/a/lakeheadu.ca/bweaver/Home/statistics/mcleanht

Thanks again.

Bruce

Hello all,

Mostly in response to Jochen Wilhelm 's post, I'd like to explain my statement in the first post above. Please note that I did not assert that the (no difference) conclusion was correct! My advice was, one should "behave as if treatments were comparable." That's not the same as if saying it is true, nor necessarily that one should (as a Bayesian might) believe it to be true. What was clear was the data provided no basis for saying it was false. As well, I did not say to "accept" the hypothesis (distinguishing here between failing to reject and acceptance).

In the strictest sense, every nil null hypothesis (e.g., differences = 0) is likely false. The questions of interest to researchers ought to be: (a) "by how much?"; and (b) "is that a noteworthy difference?" The first implies that interval estimates of parameter(s) of interest are far more likely to be informative (than reject/fail to reject hypothesis tests); the second implies careful thought about the variables, the population, and consequences. In other words, context matters. But, even interval estimates can be wrong (e.g., miss the true value completely).

Do we ever know for certain? Only when the entire population is accounted for in the data. However, whether with the humble hypothesis testing framework or parameter estimation, one has the liberty of setting risks of being wrong at virtually any desired level, and the resultant decisions will be right more often in the long run.

Dear David Morse, thank you for your clarifications. However, I still think that the choice of words is misleading. I don't see a difference in saying that "one should behave as if treatments were comparable" and "to accept the hypothesis". You saiy that "behave as if treatments were comparable" just means "failing to reject". In my ears this makes no sense. The wording "behave as if..." was just the formulation of Neyman and Pearson that meant to behaviorally accept one of the hypotheses (in this case here the hypothesis of "no effect" or "no difference"). I still think that it makes an important differencs if I behaved as if the treatments were comparable of if I behaved as if I don't have enough data to make a conclusion about which treatment I should believe has the stronger effect. I agree that there are practical situation where this is not relevant, but from a scientific point of view (what knowledge have we generated?) it is relevant, imho.

"The questions of interest to researchers ought to be: (a) "by how much?"; and (b) "is that a noteworthy difference?"" - Yes, this would be ideal. In life sciences (at least; what I can say from my experience), however, the models are mostly not quantitative. The researcher has usually no idea how much of a change should be expected and how much of an effect would be noteworthy or relevant (it's usually already unclear what relevance should refer to). All they can hope for if to get an idea if they can conclude that A>B or A

Dear Bruce Weaver, I just read the posts in the link you provided. To my understanding and opinion, the author sits on the mistake that "evidence of absence is evidence of absence" (what is wrong!). This is the characteristic sentence (but the entire post is going around this misconception):

"As I said earlier, a hypothesis test only helps the user to decide which of two models is probably better" (from Alan McLean, in his post "Hypothesis testing and magic - episode 2" at sci.stat.edu, April 13, 2000 7:56 AM)

Alan writes about significance tests, to there is one single hypothesis under scrunity that turns the larger model into the smaller, restricted model (he calles the "null model"). His message is that failing to reject the null model means to accept (even if just temporarily) the null model. And this is just not what significance tests are intended to do or to be used for!

It's simple to always use too few data to get any reasonably small p-value, so by using too little data we would always "accept" the null model (execpt in such extreme cases where the effects are so extraordiarily huge that the minimum technically required sample size will give a sufficiently small p-value). And oppositely, have huge amounts of data will almost always give a p-value that allowes us to reject the null model (and thus, so by the logic of Alan, "accept the alternative model"). At the end, it's just the amount of data that determines what model we are goind to accept, and not the correspondence of the two models with the state of nature!

To my opinion, Alan should have written these models slightly different:

i) height of a randomly chosen individual = overall mean + random fluctuation ii) height of a randomly chosen individual = mean for females * female indicator + mean for males * male indicator + random fluctuation

This way it is evident that "overall mean" from model i) is substituted by two separate means, one that applies to females and one that applies to males. This model may (and usually is) modified to include a coefficient that codes the mean difference between males and females:

iib) height of a randomly chosen individual = mean for females + mean difference between males and females * sex indicator + random fluctuation

The usual significance test is about "mean difference between males and females" in iib) and the maximal likelihood of the data under this model is compared to the maximal likelihood that can be obtained under the same model when that coefficient is restricted (e.g. to zero). The result is the relative surprisal of the data w.r.t that restriction of iib). Actually, model i) is not at all required, and it should not be interpreted as the alternative to model ii) or iib). There is no alternative model. There is only the model and a particular restriction of this model. And the test should show if the data provides sufficient information about that restriction, what is usually the question if the sample size is sufficient. If we cannot reject the hypothesis (H0) about the "mean difference" (D), if just means that the data are not-too-incompatible with this hypothesis, so that we are unable to decide if DH0 (no matter if the sample estimate of D (=d) is larger or smaller than H0). If we can reject H0, and if d>H0, than we can reject any D

Good morning Jochen Wilhelm. I am well aware of the expression absence of evidence is not evidence of absence, and I do stress that point when lecturing on hypothesis testing. I also assiduously avoid the expression accept H0, and say instead fail to reject H0.

I wonder if our disagreement is more apparent than real. I also wonder if the most efficient way to get to the heart of the matter is to put this question to you and to Joseph L Alvarez:

Q. Can an analogy to a legal trial be used to explain hypothesis testing to students? If so, please share your version of it. If not, what do you see as the fundamental problem(s) with the analogy?

If all models are wrong (as George Box noted), I expect that all (or at least most) analogies are wrong too. But I think that some of them can still be useful. Please bear this in mind while addressing the question.

Thanks for considering. Meanwhile, I'll try to follow this advice from Psychologist & Author Adam Grant:

“Argue like you’re right and listen like you’re wrong.”

Adam M. Grant, Originals: How Nonconformists Move the World

Cheers,

Bruce ;-)

Bruce Weaver , well, ok, since you asked...

Q. Can an analogy to a legal trial be used to explain hypothesis testing to students? If so, please share your version of it. If not, what do you see as the fundamental problem(s) with the analogy?

To me, the critical part with that analogy is the default dichotomy of "guilty" and "not guilty", both being perfect and accepted results in court. Actually, this is ok with the analogy, to take "not guilty" as default unless there is sufficient evidence against this hypothesis, in wich case it get rejected and the accused person is declared guilty. However - and this is my point -, this very easily leads to the wrong picture that the accused either is or is not guitly, and that the test should find out what is (or what we should believe is) true. This leads just to the flawed way Alan McLean proposed in his post of presenting two alternative models and the test is to decide which of these models is right (or better).

The case in significance testing is different. Here we have two people (A and B), and it is known that one of them is guilty, but we don't know which (they don't know each other, or they dont't like each other, so they are not both gulity, because in that case they must have worked together). To be able to commit the crime, the delinquent must have got special information, so the task of the curt is to find out how of them was able to get that information, or if one person could not have that information (e.g. the code of a bank save that was opened unauthorized by using the correct code). If there is sufficient evidence that A could not have that critical information, it is clear that B is guilty (as it was known already that either A or B must be guilty). If there is evidence that B could not have the information, A must be guilty. If it is not sufficiently clear that one of the two could not have that information, the curt can not make a decision. It is still known that one of them is guilty, but it remains unclear who it is.

I think ist would be a better analogy. Slightly more complicated, but this is required to get the important point clear. In a t-test, A and B would stand for the sign of the mean difference, which is either positive or negative. The possibility that neither A or B is the case (or that both, A and B are the case) is at no time considered. The evidence is collected to differentiate between A and B. There could be a lot of evidence suggesting that both could have the information, or that none of them could have it. In either case the evidence is simply not conclusive, and in no case this could be taken to suggest that both accused are guilty or both are innocent.

A would be happy to hear your critical feedback.

PS: Nice citation, so true :)

Thanks for your response, Jochen Wilhelm. I'll need to digest it a bit more before I offer critical feedback. But I will offer this. Suppose the context is a clinical trial comparing the standard treatment for some condition to a new treatment that is thought (hoped) to be better. Would you not agree that the usual practice in medicine is to stick with the standard treatment (i.e., the default) unless there is sufficient evidence to reject it in favour of the new treatment? (Note that I am purposely omitting discussion of equivalence & non-inferiority trials, which address different questions.)

Regarding that quote from Adam Grant, I found another variation by Karl Weick: https://www.goodreads.com/quotes/9821061-argue-as-if-you-are-right-and-listen-as-if

;-)

Dear Bruce, regarding your clinical trial example:

The test would be about the hypothesis H0 that the mean difference in the treatment effects (old treatment A, new treatment B) is zero. Therefore: A = the old treatment is better, B = the new treatment is better. Rejecting H0 tells us that we can decide between A or B, failing to reject means that we cannot make a decision.

You are correct that in this case we practically stay with the old treatment that was used, because there is much experience (compliance, side-effects, etc). But this is not a fall-back because the test said "if both treatments are similar, stay with A". The test is not about similarity. If we fail to reject H0 the data can be compatible with a condsiderable and very relevant superiority of B, but also with a superiority of A (always up to a given level of confidence; there is never certainty - for sure:) ).

Do you agree that this is not a counter-example but actually also a good example for my analogy?

Bruce Weaver An important difference is presumption vs assumption in the legal trial. No one presumes the defendant is innocent nor assumes innocence. We all know (assume) the defendant is guilty. Presumption is a problem for the prosecution and the jury. The prosecution must negate the presumption to the satisfaction of the jury. The defendant is acquitted if the jury is not convinced. The defendant is released with a determination of not guilty of the charge. Not guilty of the charge does not equal innocent. Here you can draw the similarity to 'not significant' to aquital. Also, the similarity to prosecution and investigator.

The similarities do not justify the analogy. Jochen Wilhelm attempted to correct the analogy by describing two defendants. Good point, Jochen.

Constrain the problem to a t-test of means between populations. We do not have a single defendant. We have two samples, one from population A and one from population B. We are not comparing populations, unless the samples are of the entire population. The traditional approach is to assume the null, A = B, and then show otherwise. We also presume this is a hypothesis. It is equally possible to say, "Use A as the baseline and compare B to A." Would you call A the hypothesis?

You must first establish the properties of A that will be the basis for comparison, regardless of A = B or A as the baseline. (Note: A is the baseline in A =B.)

Compare the properties of A to B using the t-test and p=0.05. The result is statistically significant or not statistically significant.

If not satistically significant you must report the investigation as inconclusive. You should critically evaluate your assumptions, sampling procedure, and sample size. You should perform this evaluation based on prior observations and your scientific question asked at the beginning of the investigation.

If statistically significant the samples and test indicate a possible difference and a possible sufficiency of sample size. You can report possible sample result in a possible direction. The same critical evaluation as in the previous paragraph is necessary before proceeding with additional sampling and testing.

A statistical test is not similar to a criminal trial. There is not a pass/fail outcome. The statistical test suggests what you should do next. Those who teach that statistical tests are pass/fail are the guilty.

Thank you for your comments, Joseph L Alvarez. I am happy to use the independent samples t-test as the context for the discussion. But even in that context, I think that you and Jochen Wilhelm are unnecessarily confusing things by talking about 2 defendants. Let me try to explain why I say that. I teach my students that all t-tests have a common format as follows:

t = (statistic - parameter | H0) / SEstatistic

In the case of the independent samples t-test:

• The statistic = Xbar1 - Xbar2
• The parameter = mu1 - mu2 (according to H0)
• SEstatistic = SE of the difference between two independent means

The fact that the statistic is the difference between two sample means, and the parameter the hypothesized difference between two population means does not change anything important about how the test statistic is computed and interpreted, IMO.

Maybe I'm just not understanding the distinctions that you and Jochen are making, but I feel as if words are being put into my mouth (and maybe David Morse's and Alan McLean's mouths too) that are just not there. I certainly agree that failure to reject H0 does not in any way prove that H0 is true. I also wholeheartedly agree that if p > alpha (for an unpaired t-test), the sign of the difference between population means cannot be determined. Or as Jochen put it in his first post above, the data is "inconclusive regarding the tested hypothesis".

It seems to me that the real point of (apparent) contention must be my description of H0 as the "default" position, and Alan McLean's description of the simpler model (of two nested models) as the one that is (usually) "privileged". I take that suggestion from McLean to be in line with the principle of parsimony, which is why I am struggling to understand the objection to it.

Thanks for the thought-provoking discussion. ;-)

Bruce

I also thank all of you for the critical discussion!

Bruce, my problem with talking about a "default" or "privileged" model or position is that this is not what the test is about. The test is about one model only, this model includes a coefficient for the mean difference. There is no second model, there is only the hypothesis or restriction of this mean difference having a fixed, given value. Notably, this restriction only leads to a simpler model when µ1=µ2, so it can not be a (general) argument that the default model is default because its simpler. It's also not more parsimonious because it makes an additional assumption about the mean difference.

Jochen, I think I am now beginning to get a clearer understanding of your objection to "default" or "privileged" model in the context of "testing". The reason I am not particularly troubled by that language is that the pooled variance version of the unpaired t-test is equivalent to the test comparing two nested OLS linear regression models, one being the intercept-only model and the other including both the intercept and an indicator variable for Group. In that context, I think you would agree that two models are being compared, right? ;-)

Cheers,

Bruce

EDIT: The original post said "not beginning" in the first sentence. I meant now beginning! :-)

Bruce Weaver Are you including the moving target model? Pooled variance, nested OLS, ... You were happy with a simple t-test context.

Two samples, two defendants. The criminal trial argument was yours. Jochen attempted to stay within the analogy. The analogy is not appropriate because, even when you include acquittal with guilt and innocence, it is a pass/fail statistical argument. It is not a scientific argument that uses statistical tests to evaluate data for continued investigation.

Hello Joseph L Alvarez. From my pragmatic point of view, a pooled variance unpaired t-test and the F-test comparing two nested OLS models (intercept-only model vs a model including the intercept and group indicator variable) are equivalent tests (t2 = F). But you and Jochen appear to be arguing that there is some difference between the two that affects how the "results" are interpreted. If that is what you're saying, I'm afraid the difference is too subtle for me.

Cheers,

Bruce

I agree with Bruce Weaver here: there is no difference between a t-test (with u d.f.) and an F-test (with 1 and u d.f.). The F-test ismore general as it allows to extend to 2+ and u d.f., that don't have an equivalent as t-test. But for the two-sample case, they are just the same.

The "nested OLS models" are "nested" just because these are two versions of one model. In one version of this model, one or more coefficients are restricted.

Very important: the restriction must not neccesarily mean βi=0. It could mean βi=2.81 or anything else! When βi= is restricted to some value other than 0 (βi≠0), the "nested model" is not simpler. Example t-test:

The model is Y ~ N(µ = β0 + β1X, σ²) where X is an indicator variable specifying the group. The intercept (β0) is the expected response for the "reference group" that is coded with X=0, and β1 ist the expected difference between the groups (X=1 vs. X=0). We are interested β1.

Focussing only on the expectation (i.e. on the linear predictor) we have

(i) µ = β0 + β1X

Within this model, we restrict β1 to a particular value, say β1=h, so the restricted model is

(ii) µ = β0 + hX

Now, (ii) is a nested version of (i), because β1 is "free" and might be choosen (according to the observed data) to be just h, in which case (i) and (ii) would be identical. The version (ii) doesnothave thisflexibility; the value of h is fixed and cannot be choosen freely, so (ii) cannot be tuned to match (i). This is why (ii) is the restricted version. This is all very obvious, you surely know all that. But I want to make it as clear as possible.

Note that the term ith X can be omitted only in the case when h=0. But generally, it cannot be omitted. Therefore, I think it's wrong to say that (ii) is "simpler" or "smaller" than (i). It's just a restricted version of (i).

The test is about comparing the maximum likelihoods that can be obtained for the observed data under (i) and under (ii). The question is if the restriction leads to a "statistically significant" reduction of the maximum likelihood. Since Y~N, maximizing the likelihood is the same like minimizing the sum of squared residuals, and it is possible to get exact p-values from t- or F-distributions.

A low p-value then says that the reduction in the maximumlikelihood of the observed data is surprisingly large under the restricted model. The conclusion is that the data contains recognizable information about that restriction.

Only for the (mathematicallybut not practically!) special case that h=0, (ii) can be simplified to

(iii) µ = β0

Also, (i) can be reformulated as

(iv) µ = βAA + βBB

where A and B are individual indicator variables for each of the two groups.

and then β0 in (iii) might be interpreted as the expected response of both groups (not distinuishing the group membership), and βA and βB are the expected responses in the two groups. It's hard to see that these to models are nested. It'salso not really obvious that (iii) is a "simpler version" of (iv), as there are no coefficients in these two models that have the same meaning.

I think the simplfication in language and communication about these models introduces a loss of these (relevant) details and leads to an over-simplified picture and eventually to all these logical misconceptions related to testing.

Joseph L Alvarez , "even when you include acquittal with guilt and innocence, it is a pass/fail statistical argument" -- Yes, that's the point. You write one sentence and express something for what I need 10 paragraphs... :)

Bruce Weaver , I looked for examples for your legal analogy and found one in the Prism Guide (https://www.graphpad.com/guides/prism/8/statistics/stat_a_legal_perspective_guilty_or_.htm). They write:

"If the P value is higher than your threshold (usually 0.05), you don't reject the null hypothesis. This doesn't mean the evidence convinced you that the treatment had no effect, only that the evidence was not persuasive enough to convince you that there is an effect."

It's clearer here that "fail to reject H0" does not mean "believe the defendant is innocent". But still, the action taken is to treat the defendant as if he/she was innocent.

This is the problem Neyman critisized on Fishers significance tests. "Fail to reject" does not imply to take any action (no default or other action whatsoever). The only action that might be considered is to go for a better experiment or more data. Therefor Neyman put down the acceptance test where two alternative hypotheses are contrasted and the experiment is designed to finally allow the accpetance of one of the two alternatives with a pre-specified confidence.

Hello Jochen Wilhelm. I think I made the same point as the Prism Guide in my original post in this thread when I said this:

"But just as in the legal trial, sticking with the default does not prove that the default is true. All it means is that there is not sufficient evidence to reject it."

;-)

"Not to prove the default" is not to indicate "akt as if the default was true" or "belief or suggest that the default is true". That a statistic never provides any proof is trivial. The relevant point is what we do, given we fail to reject H0. It is an important difference is we "akt as if H0 was true" (i.e. "going with the default") or if we "akt as if we have not enough information to decide" (end the process without conclusion or start over and collect better or more evidence).

I did not mean to say that the paragraph in Prism is more corrrect.* The legal analogy still has that issue, imho, and this problem is just clearer to me after reading the Prism version (actually, although the guide says that "fail to reject" does not imply "accept", Prism does just this when interpreting the test of assumptions like normal distributionand variancehomogeneity! - this demonstrated that this analogy seems to facilitate a wrong understanding and finally leads to flawed procedures).

I still think that the clearest point I can make about this is that H0 does not mean to be a "no effect" or "zero difference". H0 is arbitrary (it's our choice).** So the "default" in the analogy is arbitrary, too. It's not reasonable to "stick with an arbitrary default".

---

* The Prism Guide has several wrong passages. I do generally not consider this a good source of knowledge regarding statistical theory.

** Yes, it often is practical and useful that H0 is about "no effect" or a "zero difference", but this is not the basis of the test, it is not its logical justification, and it is not required. Confidence intervals are constructed by inverting a test over all (usually infinitively many like the set of the real numbers) possible hypotheses.

Jochen Wilhelm You said, "Yes, it often is practical and useful that H0 is about "no effect" or a "zero difference", but this is not the basis of the test, it is not its logical justification, and it is not required."

Excellent summation of the interpretation of H0.

Failure to understand H0 is reason to object to calling the assumption of "no effect" or "zero difference" a null hypothesis. It is not a hypothesis. It is an assumption for purposes of defining minimal conditions for differentiation.

The next confusion is introduced by calling the attainment of minimal conditions "significant."

That is enough to boggle the minds of non-statisticians. Yet, there is the further introduction of HA. Now, sampling for inference about populations becomes decision making. Thunderous applause, this is like a jury trial.

Joseph L Alvarez, thank you for the flattering feedback :) and also for this sentence: "It [H0] is not a hypothesis. It is an assumption for purposes of defining minimal conditions for differentiation.". I'd like to use this sentence to make this clear to students. I would just omit "minimal". The restiriction [H0] just is a condition for differentiation, imo. It's a borderline we set at some (arbitrary) point. There is no "minimum", "maximum", "more" or "less" of a condition. Would you agree?

Jochen Wilhelm My reason for including 'minimal' is for sample size determination. The minimum sample size, n, to observe an 'effect' (difference), d, and reach a given p-value. I agree, there is no qualifying condition. H0 is arbitrary for setting a baseline. A minimum distance from baseline is equivalent to your often stated 'condition' for observing the sign of the difference.

Language is very important to the weight we assign to statistical processes. Processes and practices (what we do with the processes) become significant when we declare 'significance.'

Please use and edit my sentence to convey meaning and understanding.

Joseph L Alvarez , I see. Thank you!