Very interesting question. I too look forward to the answers.

Dear Heba,

your question affects the kernel of research, in my opinion. The task of research can be defined as investigation of reality. If this demand is taken seriously by researchers, then it's a matter of fidelity or honourableness to act in this way, respectively, to act honestly. Furthermore, to act honestly is important in order to be considered reliable / credible. And so you can ask legitimately: "What could be more important to be considered reliable in order to have an excellent reputation?" Finally, you can ask: "Who should trust in researcher's work, if it's not done honestly?"

Perhaps, you can feel ethical level of my statement, because there are values like seriousness, fidelity, honourableness or honesty. That's the reason, why I think, your question affects the kernel of research, because your question asks for the ethical preconditions / presuppositions of research. In my opinion, to act in order to fulfill these values, should be an ethical self-imposed commitment for researchers. - And, in order to be honest - my statement represents a normative statement. Perhaps, there are other researchers, who don't share this ethical position.

So, how to deal with this problem? Starting from my explicated ethical position, a researcher should renounce to publish. But you can't force no one to act in this way. In my opinion, there's only another option / opportunity: Why not founding professional associations for researchers, developing ethical guidelines for research projects and obligating everybody, who wants to become a member of this professional association, to act in accordance with the developed ethical guidelines. (Background: There are yet various professional groups, which have got ethical guidelines!)

But there's also a "pragmatic" solution, in my opinion. If a researcher has conducted a study with "statistically insignificant results", he also could analyse the data, but he has to describe the problems or causes, that lead to "statistically insignificant results", very carefully. Perhaps the published problems of conducting a study could help other researchers, who work in the same field.

Perhaps, the "statistically insignifant results" can be analysed in a qualitative way and so be published. (Background: "statistically signifant results" are the highest standard, that can be achieved! Often / regularly there has to be done much research in advance and this kind of research can perhaps be published in form of a case report or case series)

Finally, a study with "statistically insignificant results" can contribute to a better study design, in order to get the desired results. For this case, a researcher can also describe in his publication the problems of the failed study attempt. This can be very important in order to emphasise the great difficulty of a research topic.

Good luck with your research projects! Kind regards, Detlef

Detlef! I highly appreciate your effort and input. My question is there is a huge pressure on researchers in tenure track to publish for the sake of promotions - whatever the result is significant or not statistically. At the same time, researchers have to conduct ethical research. I just wonder if it is even a logic from editors to call for statistically significant papers? It is not only causes publication bias, but also force some researchers to do unethical research.

Heba -

So it seems that there are two parts to your question. I will briefly note the first issue, and concentrate on an aspect of the second issue.

First, I think that ethical behavior is something we should all strive to follow, though difficult. When it is difficult, it may be all the more important.

Second, I agree completely with your following statement: "I just wonder if it is even a logic from editors to call for statistically significant papers?"

Under this second issue, I see two sub-issues: (1) As related to something that Detlef noted, and I think ResearchGate stated somewhere a related comment as policy: Both experiments that "fail" and those that "succeed" may have scientific value. That is how progress is made. But (2), more fundamentally to the issue of "significance," the term "statistical significance" as interpreted by an "insignificant p-value," has done a great disservice to science in general. Many editors may ask for p-values, not realizing that without a power analysis or other sensitivity analysis, a single p-value is generally not very meaningful. Many want to always set a level of 0.05 for "significance," but as a p-value is a function of sample size, if one just uses that level, then a null hypothesis which is close enough to true for all practical purposes can generally be 'rejected' with a large enough sample size, but with a small enough sample size, the null hypothesis could be far from true, and yet be 'accepted.' Even done with the correct analyses, hypothesis tests are often very impractical to interpret. With continuous data, you are often much better served by looking at confidence intervals. (Note the letter to the editor of The American Statistician, which is attached, and the notes added under the ResearchGate 'abstract' for that letter.)

Best wishes - Jim 

Article Practical Interpretation of Hypothesis Tests - letter to the...

Thank you a lot, James!