Before I get a bunch of well deserved negative comments: The tests are univariate and a quantile plot of residuals would suggest that the model fits.

Well, we typically design statistical tests so that our preferred hypothesis isn't the null hypothesis. And we never actually accept the null hypothesis, but rather say that we can't refute it. Hence, I'm against doing alpha>0.95 tests. Rather, we should test the preferred hypothesis with multiple data sets, to see if statistical significance is robust.

 -Bob Vadas, Jr.

The experiment was to test if an insect was sensitive to having an electric current passed through its body. An expert picked a ridiculously high voltage (600 mV) with the idea that this would surely elicit a response. The typical voltage is 20 to 75 mV, but we often use 150 mV. Thus the treatments were 20 mV, 150 mV, and 600 mV.

The experiment has 85 variables. Only two of them showed significant treatment effects. However, there were six with alpha>0.95.

We then tried a multivariate meta analysis using controls from past experiments. All of the voltage treatments formed a tight cluster with small and non-significant Mahalanobis distances. So there was less difference between the voltage treatments than there was between a set of 20 controls run 4 months apart all using insects from the same colony.

So the experimental design was to show voltage differences. Normally, a failure to do so would end the project. However, in this case it would be useful if we could convince readers that the statistical outcome was strongly in favor of a true null-hypothesis (useful both from a scientific stand point, but also because it resurrects what might otherwise be a dead manuscript).

That said, I agree with Robert that the standard option is to either refute the null hypothesis, or end up in no man's land of "we don't know." I am also not quite sure if I would be brave enough to design an experiment where I started off suspecting that I would fail to reject the null-hypothesis. I would be concerned that I hadn't thought the problem through well enough and the outcome was more indicative of a bad experimental design. However, I have now strayed from the original question.

Scientists often worry about negative results, as such tend not to be published, even though they can still provide useful information in many cases. That's why multivariable analyses make great sense, to figure out what really is biologically significant.

Rejecting a null hypothesis is a subjective decision that should be based on the context of the experiment. The very same argumentation can be used to defend the acceptance of a null hypothesis.

From my point of view, both strategies are scientifically flawed. The rejection is notning that could (or should!) be used to assertain some knowledge. It is only a method to rate if it is considered worth to continue research on that topic. Observing a rather small p-value for a treatment is not helpful in judging whether or not the treatment was effective. This is not the purpose of this test nor can such a question be answered by such a test. The test only indicates that -given all the experimental conditions and all the assumptions and models- the data is sufficient to make the effect statistically discenible from the expected noise, what may be a good reason to continue research (because one is likely able to measure interesting things). Hence a large p-value only indicates that the data is not sufficient - what is not the same as to say that there is no (absolutely! exactly zero!) effect.

Neyman and Pearson proposed a strategy to decide between two alternative actions, based on a utility function over some given ("minimaly relevant"), non-zero(!) effect size. One may implement an intermediate procedure by testing a non-zero null hypothesis (or two of them with opposite signs). Rejecting this hypothesis / these hypotheses in a appropriate one-sided tests may then be used as an indication to act as if the effect was smaller than the tested effect size.

I think the question of the existence of a (non-zero) effect is simply a question outside the realm of statistics, even outside the realm of empirical science. The scientifically relevant question is: Given the model (and any other assumptions), and given the data: what is a credible value or range of the effect? That is an estimation problem and not a testing problem. And a decision on how to proceed should be linked to the size and the precision of the estimate (and considering the context of the experiment) rather than on the probability with which we would expect more extreme test statistics under some point-null hypothesis.

Respected Timothy A Ebert while looking to your publication list i am happy to see one of the paper on tuckrellidae mite, i am also working on one of the species Tuckrella kumeonsis which is a pest of sapots in our area 

so nice ti see this paper as till now i have found very limited research papers on this 

The question of power comes into this. Power is defined by a particular reasonable minimum difference given a particular variance and sample size. It varies with the sample size. Assuming there was such a meaningful difference you had only the percent power dictated by the sample size and variance of what's being measured or counted. We are at a crisis point in statistics because the researchers are tending to be unaware of this or to estimate the sample size based on "small, medium, or large" which are discredited and would have to be calibrated for each experiment and lab anyway. If the sample size is insufficient then the researcher cannot detect any but huge differences. 95% is not automatically your power because you chose 5% criterion.

At the least you should use some of your data to estimate variances for future experiments. Also, even if you had an 80% power (probability of detecting what you saw as meaningful) you would still have have a 20% chance of not detecting it. Bad stuff happens even at my alma mater department. 

One could rephrase the problem: Say that I have two populations, and that they have a mean difference of size g. As g becomes smaller and smaller, it takes an ever increasing sample size to detect the difference (as g gets closer to zero, the required sample size goes to infinity). Is there a point at which one can say that for all practical purposes g=0?

You can also work the problem starting with an observation that I can only take 200 samples (or some other fixed value of your choice). Given that fixed n, there will be a difference g such that the probability of withdrawing a sample of 200 will have a less than (some value) p chance of observing a significant difference. If you have chosen a sample size that is considerably larger than previous research, then it is unlikely that anyone will be able to arrive at a different conclusion.

One obvious problem in all of this is that it is easy to fake the result in support of the null-hypothesis. I simply select a model that is inappropriate for the data, and my p-value gets close to 1. Given the level of abuse of statistics I can see a cause for concern.

Your null hypothesis is that there is no difference. But your alternative hypothesis is, in effect that there is a difference no smaller then some distance set by your sample size, etc. For applied statistics, rather then some mathematical ideal, that can never approach anything near zero. Even if you then can do a very good job of determining a sample size for some minimum reasonable difference (say the minimum your technique can detect), you are still going to be stuck with alpha % chance of false significance given H0 were actually zero (the calibration point upon which your estimate of t is based. So, a wonderful design with an enormous sample size can still be messed up by chance. In the opposite direction you can have a power (a sensitivity) to detect your reasonable minimum difference (whether you consciously chose one or not) and  that power is also a probability. So, there is a probability that chance will ruin all your work whether you do a great job or not. This is not some crime of Neyman and Pearson, This is the sometimes cruel world in action. No number of experts can change the reality that chance is often working against researchers. 

You seem to have initially been referring to a real experiment, and that worries me. No number of experts can change the reality that chance is often working against researchers. Whatever, if anything, that went wrong with that experiment, a new approach to statistics is not going to change how it turned out. 

Yes this topic is driven by a real experiment. So I have an insect, and I apply an electrical current through the body of the insect. When the insect take a bite of it food it gets a shock because it is completing the circuit. Sort of like touching your tongue to a 9V battery. If the current is sufficiently uncomfortably the insect would rather starve that try to eat again. If I decrease the voltage I might see less dramatic effects. Further decreases in voltage result in fewer and fewer effects, until the insect is no longer able to detect (or can detect but doesn't respond to) the voltage being applied.

So there is considerable debate as to what voltages are sufficient to cause a behavioral effect (Yes, I know that it isn't voltage, but power that is making the difference, but our equipment lets us directly change voltage). So there is one group that assumes that the voltages we use are so small that there is no problem based on theoretical argument. Another group has a different piece of equipment, and different insects, and has shown that there is a behavioral response to the voltages that we are using. I come into the middle of this, and with the blessing of the second group try voltages of 20, 150, and 600 mV with a general agreement that 600 mV will give a nice response.                  Well, it didn't. Not only that, but the differences in the voltage treatments were smaller than if I rerun the 150 mV treatment a month later. I have 85 variables that I collect from the system. Two variables showed a significant difference. This is fewer that we would expect from chance alone. Six variables were in support of the null hypothesis.

     The result is useful because not all researchers use the same voltage. So if I use 150 mV and you use 20 mV can we compare results?

Lets cut to the chase. As another commenter implies, you seem to be asking whether you could test

H0: Food avoidance is learned from electrical shock training in species Foo,

vs

HA: The training had no effect.

That would be unconventional, and many people would criticize it. Maybe some Bayesian statistician might know a way to do it. But, if you were to use your already examined preexisting data it would be a post hoc hypothesis pair, and that is a definite no no. You cannot observe something new while analyzing the data for something else and then test the same data to 'prove' it. Subsequent new data is required.

If you actually find a way to design a somehow valid followup experiment using, say, your highest voltage vs a control at the 1% level or even smaller (5% is pushing it in such an unconventional situation), you should also consider simply describing, or using descriptive statistics to describe, the data you already had before this definitive experiment. That data might be presented as the basis and justification for your unconventional hypothesis. Depending on the details of the collection of the data you already have, you might be able to provide a lively introduction to a publication. Maybe the Bayesian approach could use your old data to hypothesize some sort of prior value.

You say, “The experiment has 85 variables.” OK, I recall a dissertation at the University of Nevada Reno in the very early part of this decade with a title like “How Often do You Have To Observe Something Before You Conclude It Is True?”. Unfortunately, I doubt that is the exact title, but it was by a very talented student whose first name was Janine, in either the Psychology Dept. or the Political Science Dept. If those UNR departments can help you find it, it might help justify an unconventional challenge of what you do not believe rather than conventionally challenging what you do believe. (If anyone knows the author's last name I would appreciate if you would post it.)

Finally I must warn everyone reading this that if your experiment for a single test is  too complicated, or too expensive, chance will probably bite you on the bottom.

Ah, the first mistake is that we are not talking about a learned behavior. Electricity is used for monitoring (like EEG or EKG), not for training. There is no standardization, so I might use 150 mV and someone else might use 10 mV or 500 mV. Some researchers do not report the voltage that they used (because they assume that the applied voltage is undetectable by the insect). Are these results comparable between researchers? Now you have to do an experiment. Some people will say this is a stupid activity because they expect that there is no effect, and others will disagree because they have seen that it can have an effect on insect behavior. However, neither side has published evidence in support of their side, and neither side is using the same insect (either with each other or with what I am using).

A new version of "Shock the Monkey"! Anyway, there was a series of 1980s articles in 'American Naturalist', in which (mostly) CA ichthyologists argued about whether lotic warmwater-fish assemblages were predictable (‘deterministic’) or stochastic in interannual composition. Some authors rightly criticized acceptance of the null hypothesis to conclude that stochasticity prevailed. One could just as easily argue that there'd be more predictability if limiting environmental factors were taken into account. So we need to be careful about what we mean by "random" and "stochastic". Coldwater (salmonid-dominated) fish assemblages were assumed to be more stable in this past debate, but I’ve recently documented otherwise because hydrologic variation affects those assemblages, too:

Vadas, R.L. Jr., H.A. Beecher, S.N. Boessow, and J.H. Kohr. 2016. Coastal Cutthroat Trout redd counts impacted by natural water supply variations. North American Journal of Fisheries Management 36: 900-912 (cf. http://www.tandfonline.com/doi/full/10.1080/02755947.2016.1173138).

As the saying goes, “Absence of evidence is not evidence of absence” (see http://c2.com/cgi/wiki?AbsenceOfEvidenceIsNotEvidenceOfAbsence).

-Bob V.

Absence of evidence: Yes, sort of.

So in the first example where they flip a coin ... If I flip the coin twice and get heads both times I could conclude that the coin heads on both sides. This would be stupid because the probability of getting two heads in two flips is very good. I could get "lucky" and flip the coin 500,000,000,000 and get heads every time. I could postulate a coin where heads always appears on the face up side. I could have a mental block and only see heads regardless of actual outcome. The coin could be weighted in some way to only come up heads. However at some point my reality is that the coin only has heads.

However, this seems to be a problem in finding what we look for. I think there is a treatment effect so I will design an experiment to show a treatment effect and disbelieve any result that does not show a treatment effect. To make it valid I will formulate a null-hypothesis that I know beforehand is false just to make it all convincing and logical.

As I have said before, one must be careful because there are other ways that one can get a negative result other than having the negative result being true. Bad laboratory practices, forgery, sabotage, insufficient sample size, and so forth all can give an erroneous result (positive or negative).

Of course, the easiest way to determine whether a coin has 2 heads is to examine both sides, instead of flipping it. Although very obvious, my bigger point is that we should strive to design simple, easy-to-interpret experiments to test hypotheses, when possible. A good example of confirmatory experiments meant to support the status quo, rather than to actually test hypotheses, is the honey bee dance-language controversy:

Vadas, R.L. Jr. 1994. The anatomy of an ecological controversy: honey bee searching

behavior. Oikos 69: 158-166 (http://beesource.com/point-of-view/adrian-wenner/ the-anatomy-of-an-ecological-controversy-honey-bee-searching-behavior).

Bee cool,  Bob Vadas, Jr.

Oops, correct link: http://beesource.com/point-of-view/adrian-wenner/the-anatomy-of-an-ecological-controversy-honey-bee-searching-behavior