There was a comment today on twitter highlighting that statistical significance and clinical significance are two different things, clearly. However, It is not uncommon for authors of reviews to extrapolate a minimal positive or negative statistical effect from a research piece and use it to support whatever point that they are trying to endorse. Researchers may tell us that a treatment is or is not statistically valid, but how will this translate into the clinical environment?
This makes it difficult for the clinicians who want to try to be lead by research in their practice, but are perhaps less adept or experienced in sifting through research and determine quality, and thus end up accepting or misreading low statistical effect for benefit it the clinical setting. How can we ensure that dissemination of information is correctly interpreted for better or for worse in relation to patient care, and that clinical significance is differentiated correctly from statistical significance?
I believe the major problem here is the low statistical knowledge fo the the researchers, which is even worse in the practitioners... And this is not a privilege from clinicians!
In general, the disciplines of statistics offered in undergraduate and graduate are superficial and small workloads. Students do not like and do the minimum required to pass.
If, at least, clinicians could understand that statistical significance has no information about clinical significance, but only about probability, and that clinical significance can only be really significant if has statistical significance, because without that the results are not related to the target treatment, it would be easier to obtain information from scientific research.
My experience as a clinician, a manager of rehabilitation staff and as a Senior leader for practice is that the clinician reads one article, often not a review, that shows promising results from some intervention(S) and want to move forward with that intervention(s) or practice(s). The understanding of the results from a statistical perspective is one area that is perhaps lacking, the other is determining if there was a research question and identifying whether the approach/methodology chosen is/was appropriate to answer that question. Then the issue of understanding the data, the process used to analyze the data and whether that is/was the appropriate chose. And last, the significance of that analize.
I have also found, in my experience, that often time clinicians do not read the entire publication - they skip the data collection process, analyze and move to the discussion and conclusion, because that is easier to understand. Whether the results have clinical significance may not be connected to the statistical significance at all.
I think this is where the argument for clinical academics can come in. You need appropriate individuals within the clinical context to help review and lead the implementation of new ideas. It's not always best to await a definitive trial as this can be years off.
Also there are other methodologies that might be appropriate to ascertain how an intervention can transfer into clinical practice- service evaluations?
Evaluation is also valuable within the clinical setting and involving clinicians in the evaluation process, from beginning to end, helps create greater understanding of the intervention. I also believe creating partnerships between the researchers and the clinicians in developing the research, participating in the research supports further development of the understanding of statistical results, utilization of evidence appropriately, critically appraising and thinking about the applicability of any research.
Boutron and colleagues (2010 in JAMA) did a very interesting study looking at "spin" in research that had non-significant results. By their definition, around 60% of these articles contained some form of spin. If expert researchers can't interpret their own work correctly, how can clinicians do it.
I think the key is to simplify and demystify statistical analysis as much as possible and reduce the over emphasis on the pvalue as the most important statistic. Clinicians should be taught to use the pvalue as one decision tool as they work their way through a research article
Researchers should have to include the descriptive statistics for their results along with exact p-values - or better, 95% confidence intervals. Effect sizes or MCID should be provided to help show clinical relevance. Articles without this information should not be published. This would allow clinicians to be able to interpret results in a clinically useful
I agree entirely nancy. For me as a novice researcher but experienced clinician, interpretation of papers is always a challenging aspect of incorporating research into practice. As you say, p values give limited info yet its often the aspect that is used to justify results and interpret data. Often publications will be found in journals that support was is perceived as positive result eg acupuncture in acupuncture journals (no offence intended, its just an example). This element of bias by reviewers usually interested in finding positive results means that interpretation of limited data always sees the light of day, where perhaps more rigorous reviwieng would see through the spin.
I have also read that there isn't enough published by researchers when the results aren't what they hoped for, therefore adding some bias to what clinicians can find when researching or looking for best evidence on a topic. When there is scarce research related to a practice, no matter the quality or lack of understanding,we clinicians read with hope.
Absolutely. It is much harder to get research with a "non significant" results published. Personally, I want to know if something doesn't work as well as if it does
I agree with Seth that individuals within the clinical context are needed to help review and lead the implementation of new ideas. Somebody with patient to read and critical evaluate the articles/studies. Clinical research are very important.
- Badges
- Science topic
- Similar topics
- Correction