The meaning of H0 is not specified and can be selected by the researcher. This is the hypothesis the researcher aims to reject. to show that the observed data is not compatible with such a hypothesis. Imprtantly, the logic of such (significance) tests does not provide any idea about the compatibility of the data with other hypotheses. And this is the point that is hitting the mark of your question:

By specifying "a non-zero H0" (or, in an ANOVA setup, a null hypothesis that tests an F-value different from 1) will give a significance for the data under this particular hypothesis, and if this significance is large enough the researcher or reader may be inclined to reject this particular hypothesis. However, it does not provide any information about another hypothesis, in your case about the hypothesis of a zero-difference or an F-value of 1. Independent of the fact whether or not the tested hypothesis has been rejected, under this hypothesis the data may be more or less significant. One can not use the rejection of a "non-zero H0" to infere the confidence in a "zero H0" (or in any other hypothesis). This is the key point here. The argumentation you describe is kind of a "flawed logic".

Significance tests are unable to test identity. This is a simple truth.

Identity a mathematical, logical construct that is not part of the empirical world. To adress this construct with probability theory and real data one has to give a practical definition of identity, what is often called equivalence. Equivalence means, that the differences are smaller than some practically relevant limit. Such questions can be adressed stochatically using hypothesis tests, where this practically relevant difference is expressed in form of two substantially different hypotheses. Another more intuitive approach is to define a region (interval) of the difference being considered as irrelevant and check if the (1-alpha)*100%-confidence interval is fully included in this interval. If this is the case, there is at least (1-alpha)*100%-confidence in the decision that the difference is irrelevant. I am not sure how this is done in an Anova setting (e.g., how to decide on an interval for F, etc.).

A null hypothesis can be any value. Think of a one-sample t-test for example. The formula is generally

Mean / SE

which amounts to testing whether the mean for this group is significantly different than zero (technically, the numerator is "Mean - 0"). However, you can do a different t-test comparing the mean against some other number (some other null hypothesis), e.g. 5:

(Mean - 5) / SE

Such a t-value would be testing the null hypothesis that the group mean is 5 (where the alternative hypothesis is that the mean is not 5, in the case of a two-tailed test). All that is to say, the null hypothesis is usually zero, but it doesn't have to be. A nonzero null hypothesis is often used in, for example, equivalence hypothesis testing, where someone wants to test whether the effect is smaller than some previously-decided number.

The meaning of H0 is not specified and can be selected by the researcher. This is the hypothesis the researcher aims to reject. to show that the observed data is not compatible with such a hypothesis. Imprtantly, the logic of such (significance) tests does not provide any idea about the compatibility of the data with other hypotheses. And this is the point that is hitting the mark of your question:

By specifying "a non-zero H0" (or, in an ANOVA setup, a null hypothesis that tests an F-value different from 1) will give a significance for the data under this particular hypothesis, and if this significance is large enough the researcher or reader may be inclined to reject this particular hypothesis. However, it does not provide any information about another hypothesis, in your case about the hypothesis of a zero-difference or an F-value of 1. Independent of the fact whether or not the tested hypothesis has been rejected, under this hypothesis the data may be more or less significant. One can not use the rejection of a "non-zero H0" to infere the confidence in a "zero H0" (or in any other hypothesis). This is the key point here. The argumentation you describe is kind of a "flawed logic".

Significance tests are unable to test identity. This is a simple truth.

Identity a mathematical, logical construct that is not part of the empirical world. To adress this construct with probability theory and real data one has to give a practical definition of identity, what is often called equivalence. Equivalence means, that the differences are smaller than some practically relevant limit. Such questions can be adressed stochatically using hypothesis tests, where this practically relevant difference is expressed in form of two substantially different hypotheses. Another more intuitive approach is to define a region (interval) of the difference being considered as irrelevant and check if the (1-alpha)*100%-confidence interval is fully included in this interval. If this is the case, there is at least (1-alpha)*100%-confidence in the decision that the difference is irrelevant. I am not sure how this is done in an Anova setting (e.g., how to decide on an interval for F, etc.).

As said by jochen Wilhelm and Stephen , a null hypothesis is something that a researcher aims tomprove wrong or reject. My understanding is whatever the value of test statistics is, it proves the actually nature of given data, so that we can infer from the numerals of any statistical test like ANOVA, chisaquare,  test etc

I'm 99% sure that the answer is, No, the student doesn't understand what he is doing. I could be wrong, but you are right to question this.  The whole of the research could be invalid. I would recommend someone review the approach with the student before he is up for defense or whatever.

Null hypothesis testing - where we either reject the null or fail to reject the null - is inspired by a classic philosophy of science.  Karl Popper noted that we can never really prove anything (i.e., we don’t prove the research hypothesis), but we can prove things false (i.e., the null hypothesis).  So, if you’re being philosophical, no you can not hypothesize the null hypothesis because a null hypothesis is what you set out to falsify.  As Stephen notes, you could do a one-sample t-test comparing a mean to any fixed value.  But in this case, your null hypothesis is that the measured quantity does not differ from that fixed value.  Null hypotheses do not need to be about 0, just about 0 difference.  Similarly, when we do a correlation, the null hypothesis is normally (implicitly) r=0, but you could make your null that a correlation does not differ from r=.5 or the correlations in two samples do not differ.  In these cases you might use a Fisher r-to-z transformation to test is you can reject the null.  Once again, though, you can not hypothesize the null, you simply reject or fail to reject the null (i.e., of 2 correlations being the same).

Beyond philosophy, and being practical, we do very often wish to support the null.  Usually it’s in the context of hypothesizing a difference on one dependent measure X and then wanting to say something like, “the difference in X is not because of Y, because the groups do not differ on Y.”  In practice, what most people do is a Power Analysis, where they say that given my sample size and assuming an effect size of at least Z, there would be a ## percent chance of finding a significant difference.  But since I didn’t find a significant difference, I suggest there really is no difference.

What I’m saying is similar to what Jochen wrote.  I’m just writing it another way to unpack some of the ideas a bit.  Hope this helps Kevin.

Kevin

Thanks for all your thoughts everyone!  I appreciate the insight and discussion on this issue.

Kevin, Please consult standard published textbooks that discuss the null hypothesis (that Fisher defined typically hoping to reject at a specified significance level to imply a inequality, and thus an experimental effect (which is always to be expected, however minute) and the alternate hypothesis (one or two tailed). Better yet, do not use ANOVA, but instead construct a linear regression model, and use MultR for significance testing, because MultR does not require equal cell samples, and is a more robust analytic tool.

Jack, I have two remarks:

i) in the logic of Fisherian significance testing only the significance of the data under the specified null-hypothesis is considered. There is nothing line a one- or two-sided alternative.

ii) ANOVA is nothing but a multiple regression model using 0/1 coding for the factor levels, the maths "under the hood" are exactly the same, only that regression focusses more on the estimated coefficient values whereas ANOVA focusses more on the amount of reduction of the "residual sum-of-squares" that can be attributed to the factors in the model. Hence, one can not be more robust that the other. These are just two sides of the same thing: a (general) linear model.

Dear Kevin,

indeed the original formula of null hypothesis testing, originally developed by Sir Ronald Fisher, does not address the alternative hypothesis, but rather focuses on the null hypothesis. Thus, despite the fact that it is widely used in scientific studies, null hypothesis testing does not have potential to test alternative hypotheses reliably. Instead, as cumulating evidence suggests, new approaches such as Bayes Factor have prpvide more rlieble tools to test alternative hypotheses. Morevoer, Bayes Factor may also enable one to compare the results of, for example, an experimental group and a control group on the basis of the probability to obtain their results in a given sample. Additionally, when compared to null hypothesis testing, Bayes Factor does not require to stick with a narrow confidence interval (e.g., 95%) which is highly sensitive to sample size. Please see recent studies on Bayes Factor to have more information (e.g., Mulder & Wagenmakers, 2016; Nathoo & Masson, 2015).

The nulle hypothesis impies 'no difference'. It is usually not stated or specified. It is rather assumed by the researcher in any kind of experiment. There is no need for anticipation since this would imply subjective bias. In formative research, the null hypothesis can be stated for the sake of practice and training but no more than that. In proper scientific research only the inferential statistics applied to the experiment will tell the difference (for or against treatment) or confirm the null hypothesis.

My five cents

Jorge

Null=No (difference/relation)

Hypo=Assumption (awaiting verification with data)

A 'difference' therefore belongs to the alternative hypothesis.

No, Soh,

Null = Nil = hypothesis to be nilified by the data (read Fisher!). It can be any amount of difference or relation. It is only that mostly the zero difference is of particular interest for the researcher to nilify.

In significance testing (where a null-hypothesis is tested [=tried to nilify]) there is no alternative. The test is only and exclusively about the tested null. Don't confuse this with a Neymanian hypothesis test where two alternative hypotheses are tested against each other. These hypotheses are substantially different from each other, like "the mean difference is x" vs. "the mean difference is y" (put any numbers for x and y, as long as x and y are not identical).

Thanks for the clarification. Let me pose a side issue:

Significance tests assume random sampling, inter alia. But very often this is not done in educational research for practical reasons; more often than not, the group of students is a "convenience sample" (which is not really a "sample" of a well-defined population). In fact, there is no population in the real sense. Then, should we then be concerned with significance (inferential stats) rather than magnitude (effect size; (descriptive stats)?

We should be concerned with maginitude always and in the first place. Significance is far less interesting, and a p-value can be interpreted only in the context of many other things, among which is the effect size (and the sample size, and the model, and the experiemtal details, and .....).

I do not agree that looking at a magnitude is "descriptive" whereas looking at significance is "inferential". Inference means to formulate an idea or expectation about something (usually a coefficient in some model; what you call "magnitide" is the numeric value of such a coefficient [e.g. a mean difference, or an odds ratio, etc]). An estimate is something that is inferred, and it is (neccessarily) based on some prior ideas about the model and also on the available data. Model and experimental conditions (e.g. sampling protocol) further imply the "domain" ("region of applicability") of the estimate.

The concept of "population" and "sample" is not required for statistical inferences. We may see an estimate as a guess on a value of a "population parameter" based on a sample statistic. But this is a very narrow view. More generally, we want to find a useful model that helps us to understand relations between observations. In a mathematical sense, such a model has coefficients, and the values of these coefficients are tried to be estimated based on some observed data. It is convenient to talk about "samples" and "sample size" for the observed data and the amount of observed data, even if this should not imply that the inference refers to some "population" (the inference referes to a particular model and the coefficients therein).

Actually, the "estimate" is expressed as an entire probability distribution, what is usually hard to communicate. To simplify, estimates are usually given as a point estimate together with some kind of interval specifying the "precision" of the estimate. Again, the point estimate should actually be a "maximum posterior density (or probability)" (MPD) estimate, and the interval should be an interval off some given amount of credibility. But these are Bayesian measures and as such not always obtainable in a simple way and they are disliked by people who think that probability statements must reflect metaphysical long-run frequency properties. As a kind of a compromise, only the likelihood of the data is considered, and the point estimate is usually taken to be the value at which the likelihood is maximal (-> maximum likelihood estimate, MLE*). The precision can be given as a likelihood interval (the interval where the likelihood is higher than some specified proportion of the maximum), but more conveniently often as the square root of the inverse negative observed Fisher information (a.k.a "standard error") is given.**

Inference from small samples are generally problematic, because the data just does not provide much information about the coefficients. Here, the choice of the model and all the experimental details (can) have a large effect on what the data can and should tell us. Relying only on MLEs and standard errors (or confidence intervals) can be suboptimal, and a full Bayesian analysis can be better.

---

*The least-squares estimate LSE is identical to the MLE when the conditional distribution of the data is normal. In this case, the log likelihood is a parabola (2nd order polynomial). As the sample size (i.e.: the amount of observed data) increases, the shape of the likelihood approximates the parabolic shape independent of the conditional distribution of the data (-> central limit theorem), and therefore the LSE is always a "large sample approximation" for the MLE.

** For the large-sample approximation the standard error is the standard deviation of the normal posterior distribution, and the estimate is the mean. Therefore, in the large-sample case the estimate and its standard error fully specify the posterior distribution, MLE and MPD as well as confidence interval and credibitity interval coincide.

Achieving representative, unbiased samples, by random sampling is a major problem for educational research. In general, if the purpose of a study or experiment is to generalize beyond the sample available for research, and random sampling from the target population cannot be achieved, then generalization cannot be satisfied whatever statistical analytic results are generated.

W Edwards Deming made a valuable distinction between two possible purposes for action oriented research (chapter in the Handbook of Evaluation Research, edited by Struening and Guttentag) :  

Enumerative:  action on the frame, and Analytic: Action keyed to a generalizable result. In other words, research may be focused on a specific group, including a sample of a group (e.g., a lunch or breakfast program may need changing), or on generalization about cause-effect , such as the level of formal instruction appropriate as a function of student age.

A null Hypothesis is always a fixed number. Whether it comes from a hypothesis, a difference, a correlation... does not matter. In other terms, H0 does not always equal zero.

H(0)=There is no relationship between the independent variables and the dependent variable. (You pick the effect size and parameters and select your population size to ensure to  meet the statistical significance level.)

This is a good question and it has got me thinking. My own take is as follows. The null hypothesis owes (some of) its lineage to Popper´s belief, amply supported, that hypothesis testing should be based on an honest scientific effort to disprove one´s initial hypothesis (A) rather than to confirm it. Ergo in order to to demonstrate A, you find out if it is Not A. In this setting, A stands for the research hypothesis and Not A is your null hypothesis. I think it is wise to note that not being able to disprove Not A is not the same as accepting A.

Paul

The problem with Popperian logic here is that it is even not possible to disprove "Not A". All you can say is that the observation might be quite unexpected under "Not A", what is not a disprove of A. Eventually, interpreting "Not A" as "dispoved" just because observed data is unexpected in a model based on the "Not A"-assumption is at least as "weak" as identifying the (range of the) most likely or credible value(s) for the hypothesis (parameter) based on the observed data. The latter is much more direct and avoids statements about "proof" or "disproofs".

Well stated Jochen. Thank you. Paul

Interesting discussion. Jochen you have just challenged my longstanding view that null hypotheses are for people who get into research to disprove two things are not different...

David and Kevin G, It is well to keep in mind that any discussion of statistical hypothesis testing, such as using a null and alternate hypothesis (Neyman Pearson model) or null alone (Fisher's preference, so the researcher need not commit to the direction of deviation from the assumed statistic, such as a population mean to be compared against a sample mean) is not about mathematical or logical proof (or disproof). Instead, it is about rejecting a null hypothesis at some level of statistical significance (subject to Type I error), or failing to reject the null hypothesis-- which is not in the least logically equivalent to accepting the null hypothesis to be true, much less proving the null hypothesis to be true. Karl Popper simply emphasized that if no test could be constructed capable of falsifying a theory (e.g., rejecting a null hypothesis), then the construction of the theory was not of scientific merit (such theorizing might be merely tautological, and not empirical).

Kevin K, A statistical inference test framed by applying the null hypothesis tradition of Neyman-Pearson and Fisher relates an assumed population statistic, such as a mean, u, to a sampled value, such as x bar. The alternative may be nondirectional (Fisher's preference) or directional (Neyman-Pearson preference). If the student is being asked to adhere to traditional testing terminology, than his terminology is incorrectly stated. You may find this wiki article helpful (  https://en.wikipedia.org/wiki/Statistical hypothesis_testing ).

It is also helpful to realize that not all empirical research has available to it enough data to declare with any certainty the precise values for population parameters that are empirically determined. But there is one powerfully important exception, and that is provided by the Central Limit Theorem which offers the mathematical certainty that sigma  converges to one(1), and the mean to zero (0) in a normal, Gaussian, distribution  see  https://en.wikipedia.org/wiki/Central_limit_theorem ) for very large individual samples.

Dear Jack, only one comment on your last statement: the central limit theorem only says that the distribution of a sum (or average) of a large number of independent random variables approximate the normal distribution.The law of large numbers is used to show that the average will approximate the parameter µ of the normal distribution that can be thought of as generating the average (what is a kind of self-referential - I was unable to resolve this self-reference yet, so I question the value of this statement) in probability. The limiting value of the mean (the "exected value") can be any value and does not need to be zero. Under a "true" null hypothesis, zero is the expected value, for sure. But still the variance is arbitrary, and only after suitable standardization fo the values to be averaged, the variance will also approximate a specific value (in probability, again), what usually is also zero (if you had an inifinitifely large sample, you would know the expected value precisely, without any uncertainty, so the sigma²=0; the normal distribution "collapses" to a Dirac delta function at x=µ).

Jochen, thank you for your valid comment-- but notice too that I included the term "converges."  My phrasing was intended to provide a helpful simplification for folks who obviously have not "mastered" statistical theory and are asking for help. :)

Kevin, it is okay as long as the null hypothesis is framed as a negative statement. It can, in other words, predict differences in groups but in a negative way. 

Kevin,

If you are doing a directional test, the null hypothesis can be a difference.\

Norm