“The problem is not that people use P-values poorly, it is that the vast majority of data analysis is not performed by people properly trained to perform data analysis” (Leek, 2014).

I think that everything is there...

As Jochen says, "people" are mainly looking for stars whatever the tool, the data or the initial idea...

Finally, I didn't see ASA banned p-values, they revealed a misunderstanding or misuse... Only one journal banned him.

What I understood is that p-value can still be used for H0 rejection but NOT to determine if H0 is true or whether to evaluate how results are important.

"Does it necessitate a whole new way of reporting statistical results excluding p values entirely or does it indicate a more careful reporting of p values in future studies?"

It certainly does necessitate a new way of reporting p values and careful consideration of them and giving extra information like confidence intervals, effect sizes and may be sample size and power clarifications.

This topic is discussed in previous threads like the following

https://www.researchgate.net/post/Is_rejecting_the_null_hypothesis_because_of_p005_valid_statistically_or_clinically

https://www.researchgate.net/post/What_are_the_alternatives_for_p-values

https://www.researchgate.net/post/When_the_significant_differences_P_value_less_005_should_be_consider_as_clinical_important_seriously

The problem is not in the p-values. The problem is our own behaviour, the bad scientific practices upon the report and interpretation of p-values is based. Some of my issues are discussed in the linked article.

To my opinion it indicates that we must improve (i.e. pretty much change) the way how statistics is tought to practioners / empirical scientists, and that one must stipulate a better "statistical thinking" of researchers. The current educational system and the current curriculum does not do a good job.

I think is further indicates that we must change many common (actually bad) practices in research and build a framework which ensures that research is done according to "good scientific practice". We must build a system where all attempts of finding somthing are reported, not only those that gave a "publishable result". We must enforce that the "research story" is not told after the data. We must take care that alternative models and explanations are examined with the same care as the research hypothesis (also see the linked article).

And finally I think that we must avoid, where ever possible, to distinguish "significant" from "non-significant", i.e. we should avoid simple binary decisions about statistical hypotheses. Instead, we must focus much more on estimation. Whatever we aim to estimate should have some resonable interpretation, and estimates should be given with some appropriate measure of uncertainty. I wouldn't say that confidence intervals are a good choice here. We should better use capture intervals; also, likelihood intervals would be a possibility. The best would be to give a full Bayesian posterior density for the relevant estimate.

my 2 pence

http://www.nature.com/news/how-scientists-fool-themselves-and-how-they-can-stop-1.18517

As Jochen has rightly put it ... "The problem is not in the p-values. The problem is our own behaviour, the bad scientific practices upon which the report and interpretation of p-values is based."

A nice reflection on the ASA statement is discussed in the attached link in case someone is interested. 

http://www.r-bloggers.com/its-not-the-p-values-fault-reflections-on-the-recent-asa-statement-relevant-r-resources/

“The problem is not that people use P-values poorly, it is that the vast majority of data analysis is not performed by people properly trained to perform data analysis” (Leek, 2014).

I think that everything is there...

As Jochen says, "people" are mainly looking for stars whatever the tool, the data or the initial idea...

Finally, I didn't see ASA banned p-values, they revealed a misunderstanding or misuse... Only one journal banned him.

What I understood is that p-value can still be used for H0 rejection but NOT to determine if H0 is true or whether to evaluate how results are important.

I basically agree with Jochen Wilhelm.

Personally, I never use the p-value in my publications, and I always tried to make it clear to my interlocutors, especially in the biomedical field, that there are more accurate and reliable statistics.

For years I tests, especially among occupational physicians and epidemiology, a strong resistance in accepting different statistics from the p-value.

In fact, in the biomedical field the p-value is widely used, often in rather inappropriate way.

Unfortunately, the work of statisticians is often used incorrectly by the decision-makers, especially in the biomedical field.

The statistician, and statistics, must be included in the decision-making process of a search or a scientific work.

The greatest mistake lies in the treatment of servitude that is often reserved for the statistician.

We have to work on this point, in my humble opinion.

p-value is ASSOCIATED with the null and alternatives hypotheses. Calculations of p-values are done based on the specified numbers in the hypotheses. In published articles we see PARTIAL information i.e., only the p-value is printed and the reader has to arrive at the “correct” null and alternative hypothesis.  The easy way-out is to “blame” p-value.

For example:  p-value = 0.02 is published as “significant” result in a clinical trial. How to interpret it?

1.       Treatment A equal to Treatment B (Null hypothesis) and significant p-value will indicate that A & B are different because Null hypothesis is assumed as A = B vs A ≠B.

2.       Are the age groups are the same in the treatment arms (Null hypothesis)? If p-value is 0.02 then indication that the study is poorly designed i.e., the age groups are not balanced. Only when p-value > 0.05 is a “significant” result.

Once has to base conclusions on [p-value + hypotheses + assumptions] and not only on p-value.  

In research p-values are highly misused; however there are no other methods available to arrive at a ‘definite’ conclusion or to generate hypothesis.

A p value may be interpreted as the "strength of evidence" for the null hypothesis. Having said that, any conclusion based on a p value alone should not be taken as conclusive. A p value is only one piece of information to help arrive at a conclusion - there will always be other factors that will need to be appropriately accounted for.

For simplicity, assume H0: x=0, and H1: x>0. For each hypothesis, there is an associated  distribution of values.. If we calculate a test statistic, that value lies somewhere within the distribution for H0. The p-value is the area under that distribution from the point of the test statistic to the asymptotic tail in the direction of the distribution for H1. Since prior to our study, we have established how often we would be willing to commit a type I error, we may reject H0 if the p-value is less than that level. Otherwise, we fail to reject H0. The case for H1: x!=0 is similar, except there is a distribution for H1 on both sides of the distribution for H0.

Dear John,

your H1 does not have a defined distribution, because your H1 is not a defined hypothesis. There is further no finite type-I error defined for this case (also, the notion of a type-II error makes no sense here). Althogh this is described in many stats books, it makes no sense to define H1 as x>0 or x!=0. Either you define ONLY H0 and you infomally check the significance of your data under this H0, or you define two substantial hypothesis A and B, where may be given as B = A + "effect", but this effect has to be a given, particular, known, finite, non-zero value. Only then you can talk about type-I and type-II errors.

The distribution for H1 is substantially the same as the distribution for H0, and in the case x>0 differs in that its central tendency is to the right of that for H0. This is what gives the notion of direction and hence the ability to interpret a p-value vs. a  type-I error. The type-I error is certainly finite, because it is a subset of the distribution of H0 which is finite. Type-II errors did not enter into my discussion, because this was about p-values, and not effect size or power.

If H1 is just "not exactly H0", then H1 is logically always wrong in practice, as one can proof by increasing the sample size. We know a priory that H0 is false, so we know that the rate of "type-I errors" we commit is 0%. It makes no sense to argue with such error rates when H0 is not contrasted against some specific, substantial alternative. Significance tests are no tools to control error rates. This is achieved by hypothesis tests, and those require a substantial alternative. The combination of these procedures in "null-hypothesis-significance-tests" is an illogical zombie.