Many experimental results never see the light of publication day. For a large number of these, it comes down to the data being “negative”, i.e. the expected and/or wanted effect was not observed. Despite their potential, negative results are repeatedly relegated to the lab books, the drawers and the trash bins. Prof. Dr. Anthony Cerami once said "Many of the biggest discoveries of my career were the results of failure of another research project. ... Failure strikes a negative tone, but it appeared in my personal history that it was an essential experience on the path to important discoveries.".Should researchers publish their negative results?
Some new journals like "Journal of Negative Results in Biomedicine" or "New Negatives in Plant Science" encouraging researchers to publish their negative results. But as you know, there is a negativity towards negative results. Why negative results not published by journals as freely as positive results? Is it time to publish research “failures”, too?
http://www.jnrbm.com/about
http://www.journals.elsevier.com/new-negatives-in-plant-science
My dear @Mahmoud, let me share the related questions to this one. Many good responses may be found. Beside, I find the publishing of "negative" results very useful!
https://www.researchgate.net/post/Would_you_publish_your_negative_results_If_no_why
https://www.researchgate.net/post/What_about_Journal_for_Negative_Results
https://www.researchgate.net/post/Shouldnt_there_be_a_platform_to_share_negative_results
https://www.researchgate.net/post/What_is_the_level_of_acceptability_of_negative_results_in_research
https://www.researchgate.net/post/Should_negative_results_be_treated_with_the_same_rigor_as_positive_results15
One story of such failures was given by Goodchild van Hilten [1] recently: "After hours of testing, retesting, checking and confirming, Dr. Jones closes her lab book; her experiments have not disproved the null hypothesis and they don’t support her new theory. She leaves her negative results behind, moving on to work on something new in the hope that she’ll have a positive outcome next time.
Meanwhile, what happened in her lab will stay in her lab. No point in publishing negative results, after all."
[1] Lucy Goodchild van Hilten (2015). Why it's time to publish research “failures” Publishing bias favors positive results; now there's a movement to change that.
https://www.elsevier.com/connect/scientists-we-want-your-negative-results-too
If there will be publish negative results, it would be contrary to the traditional approaches.
The question arises:
Academic science is ready for change
academic science can break the tradition.
As I mentioned in the body of my question there are some new journals started publishing negative results, too. For instance, "Journal of Negative Results in Biomedicine" or "New Negatives in Plant Science" are indeed encouraging researchers to publish their negative results with them. The questions are: Is this a new attitude/changes in publishing everything? How scientists are happy with it? Is it a good practice? Is it OK by you?
If you don't publish 'negative' results, people will continue to work on the topic because the topic was perceived as interesting before the 'negative' results were obtained?
Dear Mahmoud Omid: Thanks for sharing this intriguing question with me. Sincerely I never heard before about publishing negative results. If the results are negative is probably because the assumptions are wrong and should be previously corrected. According to your words, it is perhaps possible publish negative results as negative, however I am sure that it is much better to correct all the errors before.
You can publish 'negative' results when the experimental design that caused the results is scientifically acceptable?
My dear @Mahmoud, let me share the related questions to this one. Many good responses may be found. Beside, I find the publishing of "negative" results very useful!
https://www.researchgate.net/post/Would_you_publish_your_negative_results_If_no_why
https://www.researchgate.net/post/What_about_Journal_for_Negative_Results
https://www.researchgate.net/post/Shouldnt_there_be_a_platform_to_share_negative_results
https://www.researchgate.net/post/What_is_the_level_of_acceptability_of_negative_results_in_research
https://www.researchgate.net/post/Should_negative_results_be_treated_with_the_same_rigor_as_positive_results15
The topic of negative results is not only flourishing - it has its own journal!:
http://www.jir.com
Dear Mahmoud,
Negative results are negative only against a hypothesis. But they are positive because they reduce the odds, discarded some ideas, or clarify new directions for research.
If negative results are not published..., What is the reason for doing experiments?
Sir, With new OA publishing models research papers with negative results are getting published easily. The time of just the big names, only positive results has surely changed.
www.dmrjournal.org
Dear All,
To summarize the comments so far, I may conclude that negative results provide very good (or sometimes more) feeds to scientific progress as positive results would offer. So, why researchers are reluctant to publish their negative results?
I think it is easier to get through the review process to publish results that were negative against a hypothesis (like, "No, x is not related to y, or, no, there is no difference between A and B') than it would be to publish results that were negative outcomes of some engineering objective (like, "No, we could not get this to work.").
Things that work make an easy story to tell. Methods, Analysis, Results, Conclusions: the storyline is easy to figure out. For negative results it's more like, this method, that method, this other method, with this and that tweak, short results (nothing worked). I think it is sometimes very hard to write up projects like that in a way that makes a concise and well-organized paper that will fair well in the review process (and not sound like you just didn't work hard enough to figure it out).
The above is my impression of one reason authors may self-censor reporting of negative results, but I completely agree that it's important to the community to do so anyway.
Assessment of positive and negative results is very subjective and depends on many factors. Ingenious inventions of Leonardo da Vinci lay more than 400 years like "section - negative". And then these inventions began to rediscover our contemporaries. Jules Verne invented a way of publishing devices that were considered a fantasy and fiction, that is negatively perceived by the community. Any true result is important for the development of science and society.
I thank Dr. Kaleita and Dr Kulchitsky for their contributions in this topic. There were good observations and points I was learned from them. Beside their good points, maybe there are other reasons why researchers don't try hard enough to publish their negative results, such as publishing negative results might harm their reputations. A good example is misconceptions about fuzzy logic when it was presented Professor Zadeh in 1965. ‘‘Fuzzy theory is wrong, wrong, and pernicious.” said Professor William Kahan. He continues to say "What Zadeh is saying is the same sort of things ‘Technology got us into this mess and now it can’t get us out.” Similarly, Professor Rudolf Kalman in 1972 said "I would like to comment briefly on Professor Zadeh’s presentation. His proposals could be severely, ferociously, even brutally criticized from a technical point of view. This would be out of place here. But a thinking? No doubt Professor Zadeh’s enthusiasm for fuzziness has been reinforced by the prevailing climate in the US – one of unprecedented permissiveness. ‘‘Fuzzification” is a kind of scientific permissiveness; it tends to result in socially appealing slogans unaccompanied by the discipline of hard scientific work and patient observation. blunt question remains: Is Professor Zadeh presenting important ideas or is he indulging in wishful." However, Professor Lotfi A. Zadeh stood by his theory and nowadays you would appreciate the importance of his discovery. As Zadeh [1] stated "progression from bivalent logic to fuzzy logic is a significant positive step in the evolution of science."
[1]Lotfi A. Zadeh, (2008). Is there a need for fuzzy logic? Information Sciences, 178 2751–2779.
No doubt that publishing "negative results" is very good thing for research community.
Why science needs to publish negative results?
"...Scientists should move away from positive bias to ensure all research results are shared through peer review..."
@Luis Fernando is right, why experiments are conducted if results were not published.
https://www.elsevier.com/authors-update/story/innovation-in-publishing/why-science-needs-to-publish-negative-results
As everything in life, whether positive or with success, or negative or with temporary disappointment, the most important is the character of the research, and the veracity or authenticity.
Dear Prof. Omid,
of course, YES! I suppose when we come to a positive conclusion, it means that other researchers do not have to continue or redo the same research while each negative result is a first step to following that study up. Perhaps in this case the number of reports for failures would increase to infinity.
Dear Artur: Is not a question of reputation, but of character. The conclusion you cite is abusive. Each researcher has its own individuality, and speaks for himself. I cannot accept that the individuality or freedom of choice of any researcher can be questioned. My position is clearly explained in my first answer. I could never publish negative results, because I never had them in the final of any research work. However my duty is to respect any other option.
Fighting publication bias: introducing the Negative Results section!
Nature call: "We invite authors to submit data that did not substantiate their alternative hypotheses (i.e., a difference between experimental groups) and/or did not reproduce published findings. A common criticism of the publication of negative results is that the experimentation involved may not have been as extensive as in research with positive results, which are often further complemented by additional, mechanistic experiments. A survey of the existing literature exposes this as wishful thinking, as most experimental studies are grossly underpowered. Importantly, the quality of the data submitted to our Negative Results section must meet the same rigorous standards that our journal applies to all other submissions. In fact, it may be said that the standards must even exceed those applied currently, as type II error (false negatives) considerations need to be included."
http://www.nature.com/jcbfm/journal/v30/n7/full/jcbfm201051a.html
The purpose of a research is either to extend a known result or establish new results from existing ones or in rare cases to disqualify a result to solve problems or augment knowledge. In this process extend-ability is one of the attracting parameters of a scientific result.
Negative results, although by their very nature of being results, tell which direction of inquiry is wrong, but do not belong to the general category of research to solve problems and they are not extendable to any thing, which makes them less interesting to follow. They are like dead end roads or streets in which only the impossible sign to continue is posted.
If journals do not publish a negative results is weakness in the CCAMLR Liberation and its ability to disseminate science. I do not think any regulations published in the journal demanding not to publish negative results, and not to publish negative results is a bug in the scientific integrity of the researcher and publisher
This is fine reosurce for this thread! Negativity towards negative results: a discussion of the disconnect between scientific worth and scientific culture!
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3917235/
Negative results and unanswered questions as to why something doesn’t work may lay the foundations for future research and advancement if science.
At the present, publication bias affects the body of scientific knowledge in different ways, including skewing it towards statistically significant or “positive” results. This means that for example, the results of thousands of experiments that fail to confirm the efficacy of a treatment or vaccine( including the outcomes of clinical trials) fail to be published.
Why is the publication of negative results so important?
In general, the researcher is following a hypothesis which he or she suspects has a probability of occurring. Based on my own experience, in some cases this is not necessarily what will happen at the end of the study, and this is more or less an unexpected surprise for the researcher themselves as well as for the whole research group. A certain proportion of these unexpected results are negative results, which means a scientist is not able to show the phenomenon that they would like to demonstrate in respect to a positive effect of the experiment. Also, the interpretation of these negative data might be difficult. Doubts about the setting, the methods, and the expected outcome arise. In my opinion, if negative results are really confirmed, it is more important to present these data to the scientific community so that we can learn for future research in medical treatment.
How does the publication of negative results benefit the scientific community?
As an example, let’s talk about orphan diseases: These are very rare, which means that clinically randomized controlled studies will not be realistic to perform. If we only consider positive results, it will result in a biased impression of the effectiveness of a specific treatment. Therefore because these negative data haven’t been published, nobody knows that this specific treatment failed in many more cases. So the patient will be given the wrong treatment, which may not help or even cause serious problems. This is just one example from a therapeutic point of view, but of course this is true for the whole of biology and medicine.
http://blogs.biomedcentral.com/on-medicine/2012/08/28/why-publish-your-negative-results-2/
http://exchanges.wiley.com/blog/2015/03/31/ask-about-what-doesnt-work-a-guide-for-peer-reviewers/
http://mobile.the-scientist.com/article/31356/a-negative-results-search-tool
Dear Manuel: Falsity is bad. Let me remember your recent answer about falsity that may explain something:
“Diplomacy is a system or structure that is built on values of falsity and continues not only to exist but to flourish, very useful as alternative to war. Don't you agree?”
https://www.researchgate.net/post/What_is_falsity_the_size_of_domain_of_existence_and_its_temporal_span/1
Dear Manuel: There is no possible excuse for "values of falsity" in any field. The falsity is in the origin of the wars. There are people that flourish in falsity. A Professor cannot support any sort of falsity. Is a shame for all?
Dear All,
Thank you for well focused answers on the topic. Based on most comments till now we may say that researchers should move away from positive bias to ensure all research results (positive or negative) are considered through peer review process. These are just a few positive reactions around the launch of New Negatives in Plant Science by Elsevier (A newly launched peer-review open access journal) :
- Be bold, and simply let the world know what you ‘negatively‘ know. Jickerson P. Lado
- It will bring openness to the scientific community and stimulate innovation. Leonard Rusinamhodzi
- I would prefer to read negative as well as positive results in a very well-balanced way so that I can receive as much information as possible … Saudan Singh
http://www.journals.elsevier.com/new-negatives-in-plant-science
Dear Mahmoud. Results must be always trustable, without any error or bias. It may depend of the area of knowledge, but if the models or assumptions are wrong (and that can be easily evaluated given that it is supposed that the researchers have scientific capacity) the details of the resulting data may be absolutely irrelevant.
It is probably due to the dominance of 'positive thinking' in the american way of living, the main reason. Other reasons:
- we are too egotists and don't accept that we could plan a search for nothing (if we don't reproduce what we expect, then we silently drop all the project away)
- nobody accepts the failure as a requirement for success, although all claim that you learn from your failures (They -silently- say to us: "That holds for you, me I am an excellent, glory me !")
How do we judge a publishable negative result? A so called positive result can easily be validated at least by some theoretical understanding but a negative result can not enjoy that feature. An experimental negative result, for example, may be obtained from a defective experimental set up! Therefore the paper related to that may mislead the researchers.
Dear Demetris: Fundamental or theoretical research as experimental research or applied research requires an iterative process that reduces systematically failures and errors until a criterion of convergence is reached (with a given level of accuracy, or with a minimum error). Is necessary a lot of humility in all process, because we need to discover and recognize these errors and correct, as rapidly as possible to continue the research. Frequently, the biggest difficulty is to find what is wrong. We can never impose our logic to the logic of the nature or to the logic of the problem. We must try to understand slowly and with hard work and patience the logic of the nature or of the problem. If at the end of a research we can reach any level of Excellency is not because we are better than the others, is just because we have a lot of humility when looking to the knowledge and we worked much. Francis Bacon, one of the precursors of the scientific method, said that if we began with many sureness we will end in doubts, but if we began with many doubts we will end in sureness.
@
A 'positive' result might also be caused because of different unidentified underlying mechanisms, so highlighting only one of the potential mechanisms causing the 'positive' result might be misleading too?
Does a negative result exists? Suppose a hypothesis is conjectured. Depending on the context result of an experiment or a theoretical example refutes it. Can we call this experimental result or the theoretical example as a negative result?
Thus, a negative result might simultaneously refute an hypothesis A, but then again support an alternative hypothesis?
The issue is still unresolved – what is a negative result?
Dear Anup,
According to Wikipedia, a negative result may refer to:
Proof of impossibility, a proof that a particular problem cannot be solved.
Null result, a result which shows no evidence of the intended effect.
https://en.wikipedia.org/wiki/Negative_result
A null result is not a negative result, is it? E. g. a negative result rejects an hypothesis, which is to my opinion not listed above?
Dear Marcel,
The following article may resolve the confusion:
Jonathan Knight, Negative results: Null and void, NATURE 422(6932):554-5 · MAY 2003.
Link available in RG.
Article Negative results: Null and void
In history, some research of scientists met with not to publish in the belief that it negative, but after their death proved true what science has reached. I think that this issue put forward is very important. It is the truth concerning scientific integrity. The researcher must write that the findings of the outcome, whatever the negative or positive after confirmation of the validity of the tests.
Journals publish properly framed negative results. A proper frame requires a well-constructed hypothesis. A well-constructed hypothesis is one that is strictly falsifiable.
Most hypotheses in publications are introduced by discussion and do not state the requirements for acceptability. Indeed, vagueness seems a requirement for funding (how can you fail if you do not commit to anything) so the lack of commitment in funding proposals carries through to publishing. Nevertheless, something positive must be found for research to be successful. A positive result is ensured if there are no requirements for acceptability.
A falsifiable hypothesis is one that is stated following the rules of logic to infer a conclusion from a set of premises. The premises justify or support the conclusion. The premises for a scientific argument must include the conditions limiting the argument. The limiting conditions define what is necessary to justify or support the conclusion. The conclusion is falsified if the limiting conditions are not met. (Note: Some medical journals require publishing the experimental plan before actual experimentation begins.)
Some journals do avoid negative results, specially those that fail to replicate another study. This could be justified if a logical, falsifiable hypothesis was not presented. Nevertheless, the only criterion for a positive result appears to be P < 0.05. P < 0.05 hardly constitutes a reason for publication if a logical, falsifiable hypothesis was not presented.
Mahmoud Omid stated in the question, "Despite their potential, negative results are repeatedly relegated to the lab books, the drawers and the trash bins." Here is where the power of a logical, falsifiable hypothesis can make a negative result publishable.
Negative and positive results like waves in the sea. Negative stimulate new solutions. Positive results are a cause for celebrations that are often protracted.
Dear Mahmoud Omid,
You mentioned the following two types are the negative results.
(i) Proof of impossibility, a proof that a particular problem cannot be solved.
(ii) Null result, a result which shows no evidence of the intended effect.
In theoretical research the first type viz., the impossibility proof is considered to be a very important contribution and people do publish them in regular journals. There exists no negativity for this type.
Best example for the second type is Michelson–Morley experiment. They also did not have any hesitation to publish their result.
What is important is, I quote from Fikrat Hassan,s answer “It is the truth concerning scientific integrity. The researcher must write that the findings of the outcome, whatever the negative or positive after confirmation of the validity of the tests”.
Dear Anup,
My previous answer to your question about "what is negative result?" was based on the definition in Wiki and I gave reference for it already.
Anyway, I did not say or do not think negative results or non-confirmatory are worthless. The truth is the Michelson–Morley experiment (you mentioned) is a very rare example. Albert Einstein’s theory of relativity spawned from the negative results of Michelson and Morley. But, many such experiments are never known to the scientific community. Often, only the very eye-opening negative results such as Michelson–Morley experiment are published, while the rest are filtered out. This trend can be seen across all disciplines. Why? For a journal's editor it is very difficult to conclude/decide that whether the negative findings attained in a new manuscript are in fact because of a flawed hypothesis, or rather a simple experimental flaw, etc.. So peer-review journals may prefer to reject such papers.
Dear Mahmoud,
Dear all:
If is possible that, in most situations, the researcher wants to find evidence that an effect occurs --not that the effect does not occur. This may introduce a bias in the evaluation that researchers make of negative results. However, some negative results may be very significant (or, as some of you said in the thread, involve "positive" consequences) .
I can offer my own modest example. A number of researchers supports the hypothesis that online newspapers, because of its interactive nature, have a special ability to promote interest in politics and political discussion. By using data from a series of surveys that I conducted , I built regression models to test the hypothesis that people who read newspapers online were more interested in politics than those who read printed newspapers. The results of the tests in the four surveys were negative. There was no difference between the two types of readers. I found it interesting to publish the results, since my surveys were based on probabilistic area sampling and personal interviews in the households --they were not interviews with students or online surveys.
Dear Professor Omid,
From my humble experience, negative results are usually thought of as resulting from questionable research design. As much as I agree with that popular assumption, I personally believe that negative results may also point to the flaws in the existing theories, or that the theories may also be viewed in a new light or extended. I think academia seems to be comfortable with status quo; therefore, it won't surprise me even one bit if my experiment yields a negative result, and it is rejected by the mainstream journals.
Best regards,
Cameen
Dear @Mahmoud, 555 publications available here at Reseachgate on negative results publishing. Link follows. I have pinpointed some articles.
https://www.researchgate.net/publicliterature.PublicLiterature.search.html?type=keyword&search-keyword=publishing+negative+results
Article A call for greater power in an era of publishing negative results
Conference Paper The benefits of publishing negative results
Article Publishing Negative Results: The Problem of Publication Bias
Article The value of publishing negative results from a randomized c...
Dear @Ljubomir, Thank you for the statistics. I have some comments on your search results and in particular about the 555 publications you found here at RG on negative results publishing.
Currently, there are more than 19 millions fulltexts and 80 millions publications (see link) available in RG! So 555 means, less than 0.003% (in terms of total fulltext) and less than 0.0007% (in terms of total publications) are of negative results types of publishing. In your field of study, as you may confirm, the 555 becomes much less. So it means virtually zero percent if you take into account various disciplines separately.
https://www.researchgate.net/press
In Engineering, things must always work. But in more advanced areas of science, human knowledge looks still very limited. So, if the research works are well structured and the research is well constructed, has consistence and is trustable, the results being positive or negative may not be the most important factor for the publishing decision. There are many works published as positive and with very low level of reliability. If they could be reformulated and be published as negative, perhaps could be more valuable and be a little more reliable.
Also, in some areas, the difference between positive and negative results may be not easily determined. For example, the results presented in The Prince of Machiavelli and the Machiavellian practices may be considered positive or negative findings? If they were considered negative, they would never be published?
https://en.wikiquote.org/wiki/Niccol%C3%B2_Machiavelli
Dear António says " I agree with our RG friend Mahmoud that there is much rubbish published as positive and with very low level of reliability." But, I don't. I never said positive results/publications are rubbish! All I wanted to say was the tendency of majority of researchers is toward positive results. You may think of this attitude as similar to seeing the glass half full (instead of seeing it as half empty). If we think this way then not only makes us happier, it makes us healthier and wealthier, too.
Okay, Dear Mahmoud: Thanks for your speedy correction. I removed your name and the word “rubbish”.
Hello Everyone,
I think that positive thinking has actually nothing to do with publishing (or rather hiding) the negative results. In my opinion it goes about courage. If a scientist states the hypothesis that prove to be negatively tested, it can imply that he/she was simply wrong in his logical reasoning at the stage of constructing/ creating them.
Well, I am strongly in favour of publishing all interesting results, both negative and positive. When we show negative results, then someone else can see eg. the flaws in our resoning, but that' s a good thing though, that's what improves science actually.
We've got right to look for the truth and make mistakes, but it's a shame to defend them, once they prove to be ones.
The interest of a research is for something which improves the world, the science or the knowledge for a better life and society. This is called usually excellence. But when there is not any improvement for anyone, dedicate resources and time for worse life and society does not seems intelligent or reasonable.
This is also the reason for do not publish false or ilogic contents, if you like improve your life or the society.
Negative results are not because of inferior research methods always. One cannot ignore these, does not matter how infrequently these are observed.
Are more experienced people less afraid to publish so-called 'negative' results? If so, why? Perhaps Young people are more hypothesis-focused or hypothesis-attached?
Generally scientists are proud with their results when they have some logical statement; objectivity, and positive. When the results are negative scientists are not enthusiast to show them, as it could sound as source of less reasoning, failure in experiments, methods or process.
i am at a loss to understand whet is positive or negative result. a result is a result that is all-- accept or not the other man's worry - i disagreed with many yet i am surviving - do not bother- do your work and publish
Dear Nimata. Thank you for your comments. But I think you should have emphasized that publishing of negative results should be encouraged "if experiment/study was designed and executed properly" to exclude the possibility of flaw in the experiment. Thank you.
References can never be forgotten. I remember some recent sentences that I wrote in two RG discussions:
“Someone that guides always his /her behavior on solid principles and universal values and not on his/her interests of the moment or cheap smartness or any other “weak parameter” as I heard from the first time from “my sister” Fairouz is surely trustworthy. “
“It looks than the domain of falsity is much larger and much easy to reach. Is the path to correctness much harder and painful?”
https://www.researchgate.net/post/What_is_trust_what_is_the_definition_of_trust_how_do_you_know_you_trust_someone_Are_you_trust_worthy/7
https://www.researchgate.net/post/What_is_falsity_the_size_of_domain_of_existence_and_its_temporal_span/1
Based on many good comments we had so far we can say publishing should be made more easier and peer-review more rigorous. However, publication decisions in scientific journals are still largely depend on the study outcomes.
Disseminating negative results might show what did not work; augment our knowledge of what is not there; save other researchers from wasting time trying operations that have already failed; and demonstrate the limits and upper bounds of previously reported results. Most important, it might help to increase the speed and transparency with which new ideas are introduced, tested and filtered in social science research (Lehrer et al., 2007). They believe in order to be acceptable to one's scientific peers, research findings must be original, replicable, significant and relevant to the existing body of theory. Little room is left for the result of a sound research process that does not fulfil all of the above criteria.
David Lehrera, Janine Leschkeb, Stefan Lhachimic, Ana Vasiliud and Brigitte Weiffene (2007). Negative results in social science, European Political Science (2007) 6, 51–68.
http://www.palgrave-journals.com/eps/journal/v6/n1/full/2210114a.html
http://ejop.psychopen.eu/article/view/590/html
Dear friends. Publish. No one has a final say in science
Feel free
The first and most obvious benefits of publishing negative results are a reduction in the duplication of effort between researchers, leading to the acceleration of scientific progress, and greater transparency and openness.
Furthermore, the publication of well-documented failures allows for negative results to be discussed, confirmed, or refuted by others, and in some cases might also reveal fundamental flaws in commonly used methods, drugs, or reagents.
More broadly, publication of negative data might also contribute to a more realistic appreciation of the “messy” nature of science. Scientific endeavors rarely result in perfect discoveries of elements of “truth” about the world. This is largely because they are frequently based on methods with real limitations, imperfect experimental models, and hypotheses based on uncertain premise.
http://www.the-scientist.com/?articles.view/articleNo/33968/title/Opinion--Publish-Negative-Results/
Let's face it, we all live in a "publish-or-perish" culture, for better or for worse. Statistics show the total population of researchers and their publications continues to rise every year, and they become increasingly subject to the principles of "publish-or-perish". Further more, research is meant to be a noble and rewarding process. Under such circumstances, peer review is the primary tool we currently have for assessing papers (negative or positive results dont matter) prior to publication. So we should be more concerned of research integrity and standards.
Very well written question and answer, Mahmoud. In my industry it is not publish-or-parish, it is produce-or-perish, that is, complete the project on budget and on time or you're out. We always sought negative results so that we could change direction quickly to get the job done, and I'm talking about getting new products into production. Therefore we read every negative publication we could find about a material or technology we planned to employ. They were, as you suggest, sometimes difficult to find.
dear John
in tech development we share information like this- like pitting in plating
Prof. Muralidharan: You are right. And pitting during plating is a very difficult/interesting problem, usually explained to me by a plater during a product development program. These days I am seek criticism of my work in studying how the slow brain can do what it does. John
good John
dear all
you have done the work- it is your results- on this earth you have every right to understand your work as you think
repeat the work- if you are sure do not bother about others- he is not paying you
in technology we share failure results - we have a branch known as failure analysis
bye
I do like very much term brought by @John, produce or perish.
"To produce research results or perish is today a continual threat to the research worker who in the boom days of the past was only sometimes subjected to the threat of publish or perish. Research management must become more effective to ensure higher productivity and continued funding. We live in an age of change, and successful technological innovation is essential for survival. The selection, motivation, and continuing education of people can contribute much to successful R & D. The systems approach and the use of industrial economics and engineering concepts can contribute to optimum utilization of research workers and facilities..."
http://wfs.swst.org/index.php/wfs/article/view/657
From Cambridge university:
There is a “publish or perish” culture in science that is biased towards selectively reporting positive results and undervalues the importance of replications and negative results.
Because of these publication biases, many findings go unreported and data gets stored away and ignored in a hypothetical file-drawer.
Not only does this put unhelpful pressure on scientists to get positive results, it also undoubtedly causes wasted time and resources when others unknowingly repeat unsuccessful experiments that have not been published.
Recommendation to bring positive changes in the current scientific system
- Establish a pre-publication archive database for all research: positive results, negative results and replications. Fields such as mathematics, physics and astronomy use these types of archives which some researchers respect almost as much as traditional peer-reviewed journals. They allow all research to be freely available and provide a more transparent and complete picture of data in its entirety because all data is welcome regardless of the results.
- All students and scientists should be required to take research ethics and research methodology seminars. At the end of the day, the responsibility lies with the individual researcher to make unbiased and ethical decisions. Open discussion of these issues can only help the scientific community.
https://www.gatescambridge.org/multimedia/blog/students-stand-science
From www.anu.edu.au: Science is a human endeavour. And like humans everywhere, those who work in it will do what they are rewarded for. So we need to make sure those reward structures are encouraging good quality research, not the opposite.
http://www.anu.edu.au/news/all-news/publish-or-perish-culture-encourages-scientists-to-cut-corners
Negative results may mean a new discovery
"There are two possible outcomes: If the result confirms the hypothesis, then you've made a measurement. If the result is contrary to the hypothesis, then you've made a discovery."
-Enrico Fermi
Negative result is also a result. It must be taken into account.
If we want to avoid repeating old mistakes,then all results – both positive and negative, have to be published. In recent years, the issues surrounding the negative finding culture are certainly gaining more traction (by people as well as new journals), but publication bias is still an issue. Therefore a shift in the scientific culture is required. One solution to this issue would be to reverse the anti-negative-finding culture. This lies partly in educating young scientists about the importance of disseminating all results.
Negative results are helpful and a most important source of knowledge. One of my favorite social science books is Charles L. Briggs' "Learning how to ask" where he mainly analyzes interview questions which did not lead to the expected results.
Dear Mahmoud et all,
And imagine a result about weather, where temperatures could be lower than 0º degrees. The negative results are "normal" in some countries and during some seasons of the year. Then , as in life, if we make associations between lots of different things, we would find always situations where negative is good and others where negative results are not good at all.
What this mean? Like in statistics, when we want to make comparisons we usually apply relative measures instead of absolute ones.
But there are situations where it is possible to use absolute measures (eg abosluto zero kelvin), which means that using logic we can always defend as useful or useless negative responses.
In science if we knew everything then it would not be worth further investigation, which meant we had reached all knowledge. Many discoveries have been born of successive errors (negative). If the negative results have discouraged the reseachers and stop them doing investigation, then many discoveries would still not be made.
Have a nice time
Helena
Publishing negative results is important to expand the research towards finding new approaches to exceed the probable negative findings again.
Dr. Berg, Dr. Helena, Dr. Charles, Dr. Watheq, Dr. Omid,
Yes, all of you are telling it correctly that null results are also results. In this context, I cannot stop myself from mentioning one aspect: We need to think laterally. Previous findings might have got a straight line relation between two parameters. But that does not mean that they will always behave in a linear manner. Unfortunately, scientists, students, teachers, researchers- all try to fit to a linear relation only even if they find some other relation. Instead, we need to find out the reason of getting a non-linear relation, if any. This may lead to a new discovery.
In research serendipity is an important concept. You find what you are not looking for. Negative results are essential for serendipity.
At the end the following anecdote aptly illustrates the problem that plagues scientific literature—publication bias or reporting bias:
“… there's this desert prison, with an old prisoner, resigned to his life, and a young one just arrived. The young one talks constantly of escape, and, after a few months, he makes a break. He's gone a week, and then he's brought back by the guards. He's half dead, crazy with hunger and thirst. He describes how awful it was to the old prisoner. The endless stretches of sand, no oasis, no signs of life anywhere. The old prisoner listens for a while, then says, ‘Yep. I know. I tried to escape myself, twenty years ago.’ The young prisoner says, ‘You did? Why didn't you tell me, all these months I was planning my escape? Why didn't you let me know it was impossible?’ And the old prisoner shrugs, and says, ‘So who publishes negative results?’” (Hudson, 1968, p. 168)
G. William Walster & T. Anne Cleary (1970). A Proposal for a New Editorial Policy in the Social Sciences, The American Statistician, Vol 24(2):16-19.
http://www.tandfonline.com/doi/abs/10.1080/00031305.1970.10478884
A new feature has been added to ResearchGate recently: Thanks for your support on this question, now beside regular peer-reviewed papers you can upload many formats to your profile including Negative Results, Experimental Findings, Patent, Poster, Raw Data, Research Proposal. Technical Reports, Working Paper, ... too.
Mark Twain once said: “What gets us into trouble is not what we don’t know, it’s what we know for sure that just ain’t so.”
New study (please see the link) shows the direction of scientific research should not be determined by the pressure to win the ‘significance lottery’, but rather systematic, hypothesis-driven attempts to fill holes in our knowledge. At the core, it is our duty as scientists to both: (1) publish all data, no matter what the outcome, because a negative finding is still an important finding; and (2) have a hypothesis to explain the finding.
Happy New Year! and Best wishes,
Mahmoud
http://dmm.biologists.org/content/7/2/171
- Badges
- Science topic
- Similar topics
- Academic Journals