After reading the editorial published in Nepal Journal of Epidemiology by Dr. Shrikant I Bangdiwala, Professor, Department of Biostatistics, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, NC, USA. I had a mind to think more about significance, p value and Confidence Interval.
Interested scientists are requested to write a letter to the editor for Dr. Shrikant's Editorial which can come up with a new change in the research.
On 7 March 2016, the American Statistical Association (ASA) released a statement to improve the interpretation of statistical significance and p-values and their role in scientific research. In their concluding remarks, they state “Good statistical practice, as an essential component of good scientific practice, emphasizes principles of good study design and conduct, a variety of numerical and graphical summaries of data, understanding of the phenomenon under study, interpretation of results in context, complete reporting and proper logical and quantitative understanding of what data summaries mean. No single index should substitute for scientific reasoning”. The ASA statement also states: “Informally, a p-value is the probability under a specified statistical model that a statistical summary of the data (for example, the sample mean difference between two compared groups) would be equal to or more extreme than its observed value” . The p-value is thus a single index, so where does it and ‘statistical significance’ stand within ‘good statistical practice’?
http://www.nepjol.info/index.php/NJE/article/view/14732
I think the statement on p-values issued by the ASA could be fairly summarised by
Use with caution.
There are many well-documented problems with the p-value, and many acceptable uses as long as one keeps in mind the limits of the experiment. Below is a post that explains this well--- and notes that it is simplistic and probably wrong to believe that Fisher would necessarily today be "pro p-value".
As with so many questions, the answer is neither yes nor no, but somewhere in the grey. Note also that replacing p-values with confidence intervals may be better, but may not be sufficient. It depends on your question.
http://blog.minitab.com/blog/adventures-in-statistics/the-american-statistical-associations-statement-on-the-use-of-p-values
I think the statement on p-values issued by the ASA could be fairly summarised by
Use with caution.
There are many well-documented problems with the p-value, and many acceptable uses as long as one keeps in mind the limits of the experiment. Below is a post that explains this well--- and notes that it is simplistic and probably wrong to believe that Fisher would necessarily today be "pro p-value".
As with so many questions, the answer is neither yes nor no, but somewhere in the grey. Note also that replacing p-values with confidence intervals may be better, but may not be sufficient. It depends on your question.
http://blog.minitab.com/blog/adventures-in-statistics/the-american-statistical-associations-statement-on-the-use-of-p-values
Brijesh -
You generally don't want just a p-value. "Significance" is a misnomer.
A type II error analysis could help, but hypothesis tests can be hard to interpret clearly. If you can substitute a confidence interval, it will be more practically interpretable.
https://www.researchgate.net/publication/262971440_Practical_Interpretation_of_Hypothesis_Tests_-_letter_to_the_editor_-_TAS
Cheers - Jim
Article Practical Interpretation of Hypothesis Tests - letter to the...
I have noted earlier that ASA statement does not change anything. It didn't tell us anything we didn't know before. All it said, is that one should not rely on only p-value. Anyone , who is well-trained, knows that rejecting null does not make sense, if various assumptions on which the estimator are based are violated. Assuming that all the assumptions are met, predictive failure and forecastability metrics need to be examined before interpreting the economic or practical significance of the estimates. Statistics theory remains the same, and one can never prove the null hypothesis; although, one can present evidence that may cast doubt about the null. Simply rejecting the null and concluding the alternative has never been enough. We knew all these before ASA report. I am with James that "you generally don't want just p-value" and Adam is correct with his advice to "use with caution"
You can find a lot of good discussion on this on Cross Validated. I linked a few good discussions. Hope this helps.
http://stats.stackexchange.com/questions/139290/a-psychology-journal-banned-p-values-and-confidence-intervals-is-it-indeed-wise
http://stats.stackexchange.com/questions/200500/asa-discusses-limitations-of-p-values-what-are-the-alternatives?lq=1
http://stats.stackexchange.com/questions/200745/how-much-do-we-know-about-p-hacking-in-the-wild?lq=1
http://stats.stackexchange.com/questions/201146/what-is-a-good-convincing-example-in-which-p-values-are-useful?lq=1
Chuck - I wholeheartedly agree, but I think the importance of the ASA statement is that now we statisticians have a professional society statement to reference as we try to explain things like this to journal editors and reviewers. I work in clinical outcomes research, and MANY times I get papers returned that include some comments from a reviewer pushing one direction or the other about the "significance" of our findings based almost entirely on the p-values reported. I'm grateful to have a professional statement from the ASA that urges more of a nuanced approach, because I'll be able to reference that in my responses to reviewer comments.
Andrew, you have an excellent point. Honestly, that did not cross my mind. Yes, it gives us some edge because some editors do not understand statistics but with ASA statement attached in response to the reviewer's rejoinder , the probability of a yes island enhanced.
Why do we read many articles with bad statistics? : what does the new American Statistical Association's statement on p-values mean?
Sangseok Lee
http://synapse.koreamed.org/search.php?where=aview&id=10.4097/kjae.2016.69.2.109&code=0011KJAE&vmode=FULL
Thank you so much for all the valuable answers. Please share further material related to this topic, because it will improve the statistical decision making of researchers.
I do not understand what you mean by bad statistics? Bad because it is not what you expected or what?
Dear Sir,
May be methodological error.
It was an editorial of Sangseok Lee. Please read full editorial and email the author.
Dear Sir,
Please see the attachment as pdf on Sangseok Lee editorial.
Thank you very much for taking the time to help me read Lee's write-up. I agree wit much of his comments. I do not agree with some minor statements such as "express ticket to publication (line 9 from top and the first line of his last paragraph.
However, as. Mr. Andrew D Althouse stated, it is nice to have a stated position of ASA for use when responding to research output. I have stated earlier, that it is the responsibility of the teacher to choose alpha , alternative hypothesis, methodologies and other metrics I mentioned earlier to exhaustively show that all the assumptions underlying the model are satisfied. Rejection of the null using probability value (p-value) is useless once the assumptions are severely violated or most importantly non-spherical error are allowed. While confidence interval and effect size are good, they cannot be acceptable without extensive diagnostic analyses of the assumptions of the estimator.
- Badges
- Science topic
- Similar topics
- Mathematics
- Statistics