It ought not be the norm, but there is much nonsense published in Nature, Science, other fancy journals, and if we can see who approves these publications, we may learn many interesting things...

Wonderful idea but I think everything depends on the referees and editors. For example in my journals there is no possibility to recognize the idendity of the referee at all, because we have additional inner mechanisms to hide the idendity. In my opinion you ask about one of the most important problems of the contemporary science- the right and fair criteria of judgement of an article and scientific work at all. I think that the science should be open as much as possible. As the editor of the journals I very rarely  judge the quality of the text but I have right reviewers, who are specialists. Sometimes I get the article with unfair opinion or not well reviewed. In this case I change the reviewer or I don't work with such kind of 'specialist' in the future. In my opinion only  publishing of the reviews is a cure against unfair and slipshod review.

Blind review process, all said and done, has its merits and really helps the editors a lot. There could be some weakness/ exceptions or cases of bias. But, in case of true (worthy) contribution/ feedback by a reviewer/ referee, the authors tries to improve it and also acknowledges it in the revised paper. In some cases critical comments are cited/ published (in a select few journals).

I remember my own case, my first ever research article sent for publication (some 16 years back) to an International Journal. The article was hardly of 7/8 pages. Reviewer comments were 5 page long. Though it was a shock for me then; later I had to appreciate the effort given by that 'honourable' reviewer/ referee. 

I of course appreciate your feelings and suggestion.

Regards

The idea of blind review can contribute to the impartial assessment and publication of articles that really deserve. But, maybe inclusion of reviewers' names and the name of corresponding editor in an accepted manuscript can positively contribute to impartial assessment based on a real merit. In that case, anybody can see who justified  publishing of a manuscript.

For professional disciplinary journals the present system of anonymous peer review is the best, even if it can never be perfect. The problem today is rather to maintain the quality of peer reviewing as the number of manuscripts, the burden of editors and the competition between journals and their financial problems keep growing.

We do not want editors to create favours to "their friends" or to themselves – and this must be counteracted by a good, reasonable and transparent organization of the flow of manuscripts as well as containing appropriate check points along the way and above all the dedicated work of a devoted, careful, knowledgeable Editor-in-Chief, EIC, that is highly recognized by the discipline.

In special cases the reviewer might be allowed contact the author(s), when particular problems develop. Rather than having the reviews published after each paper a joint publication, a letter to the editor etc. can be the natural solution to a "difficult case", of course in full agreement between the parties the EIC and the Board.

Why not a double blind process? Authors' names are hidden and referees' comments published...

Double blind peer reviewing is always an alternative, however I have two minor comments against it as given from the EIC point of view:

1. The double blind review process is more time consuming than the ordinary one. This may be a problem as it becomes more difficult to recruit reviewers to accept nonpaid evaluation work. To get serious referee reports on every ms is becoming increasingly problematic for editors.

2. It is easier to recruit a referee if he knows who is writing the ms and also knows the author(s) from previous encounters and writings.

Point 1 is a more practical issue, but looking around with the dramatic increase in publishing one can see, based on the growing number of bad, I mean really bad, papers being published, that the problem of peer reviewing is so big that any refereeing should be better than the present ignorance of screening.

Point 2 has the danger of opening up the arena for personal controversies and "club thinking", nevertheless the ethics of a good EIC and Editorial Board should  easily counteract and prohibit such activities. It could also lead to creative discussions and letters to the editor, notes added in proof, etc.

Nevertheless I think that double blind reviewing might perhaps work better in cases like Nature and Science, while smaller disciplinary journals should evolve healthily with the single blind.

Of course, “…for an expert in the field it is easy to recognize the identity of the (main) author”. From the other side, as authors are also experts in the field, they also may recognize the identity of the expert :) Therefore hiding names of experts sometimes has no sense, but (as a general system) it prevents authors from contacting experts directly. So editors don’t miss details of author-expert conversation and can judge more objectively regarding quality of the article.

Not experts but editors are the judges. Someone may be expert for one article and author for another and vice versa. When expert express his/her opinion, this is merely his/her opinion and nothing more. Author may disagree with expert and should have the possibility to defend own opinion. After all, expert is a live human and could fall into mistakes. So the editor (editor board) should make final decision in every case. And it is very important to make authors possible to defend their opinions, but of course all discussions must be correct. If editors are able to allow and judge such correct discussions, then this is the optimum case. At that, hiding authors name would not improve the process very much.

Publishing comments of each referee would be good, but this would increase published volumes twice or even more. Although in this case referees indeed would be more careful with the comments, especially if their names are published.

Hi Piero

A big problem can be reviewers who express a negative opinion because they feel it competes with their work.  If a paper is rejected as a result, which happens, the reviews will not appear as you propose, so authors have no chance to defend their opinions. Or should the reviews be published anyway if requested to allow a public rebuttal?

Dear Piero,

I appreciate your idea. If your idea is followed by the publishers then it will be very useful for the young researchers those who are going to cite the particular paper. Nice idea.

In the recent years, new specialised filed of specialisation has come in practice. It is appropriate to connect the known people. The filed expertise can be shared and synergy in research can be obtained. It is almost equal in effect on both ways like the blind review and open review. 

Hi Philip.

You are certainly right that the referees' comments may not appear if the manuscript is eventually rejected. However, the revewer who writes the comment cannot know that in advance (even if sending a negative report) and should be careful in writing something objective, because if the paper is accepted the comment will be eventually published. Anyway, papers that can be deposited in archives (like research gate), could be anyway joined by the reviewers' comments at all stages of the evaluation, including the rebuttal.

Hi Piero,

Concerning the availability of referees' comments ...

Some journals have introduced "open peer review" system (eg., PeerJ) where authors can opt to make the entire review process open so that the reviewers' comments and the authors' responses are available to everyone. Likewise, the reviewers have an option to make their names known to the authors (and to everyone else because everything appears online). 

This format might be feasible only for mega / online journals where there is no worry about space. However, this idea -- in a modified form perhaps -- could be adopted by other journals. This would address yours (and many authors') concern about ensuring fairness of the review process.    

Perhaps "anything goes" (Feyerabend) fits well here as in science. However, I'm a bit skeptical about a publishing system (the traditional) where authors are known (to the referees) and referees are anonymous (to the authors). As the authors take responsability in what they write with their names (challenging or not), I believe that the referees should do the same, at least in the face of the authors (not necessarily to the entire world!). I think that a fair review should come more likely from either an double open (both authors and referees known) or a double blind (all anonymous, somehow putting aside credibility and more about searching for objectivity in the review process; but what about the Editors ... should we go for a triple blind?).

A combination, i.e. that the authors are requested to name two or three prospective referees that are knowledgeable in the field and familiar with the topic under question, leaving the editors the choice to select one from their own list and one from the authors, seems to be common practise today, partcularly with journals with impact factors about 1-3.

The problem today appears to be the multitude of new open "journals" that must accept almost anything in order to get going. In these cases our discussion is unfortunately not relevant.

Blind reviews where the reviewers are not given the name of the author are nothing new, and some journals use them routinely. Blind reviews where the author is not given the name of the reviewers (unless the latter agree to this) are very common. Double-blind reviews are just a combination of the two, and also not a new idea.

It has been repeatedly discussed among scientists that, in a narrow field with relatively few specialists, often one can make a very good guess about the identity of an author or a reviewer, based on what they write. The same applies, even in fields with thousands of specialists, to authors or reviewers who subscribe to extreme views or non-mainstream ideas. In these cases, there can be no reasonable expectation of anonymity.

The same problem applies to the idea of publishing reviewers' comments. At least some readers will be able to identify the reviewers from their comments, and anonymity of the reviewers in these conditions disappears.

I think a much bigger problem with the idea of publishing the reviewers' comments is that these comments apply to the MS in its original form, not to the published paper. The author may have taken care of the comments by making the appropriate changes in a revised version of the MS, so making the reviewers' comments public, at this point, would be pointless. In the case of comments that the author decided not to follow, the author may have explained convincingly to the editor, in private communications, that the comments are not relevant or are based on a misunderstanding of the MS by the reviewer(s). There is a lot more information being exchanged in a review process than just the reviewer's comments, and publishing only the latter gives an incomplete view of the process. The author himself may not want the reviewer's comments to be published, because they may publicly expose embarrassing errors or omissions in the author's original submission, that have been corrected by following the reviewers' comments.

A further factor to consider is that, in the present review system, review comments (like many other forms of personal communication) are supposed to remain private in order to give to the parts involved the possibility of freely speaking up their minds. The same persons may be reticent (for the right or the wrong reasons) to make the same comments in public, even in a supposedly anonymous way.

Some journals even make it possible for reviewers to submit private comments to the editor only, which are not forwarded to the author. Also this type of communication is private in nature (even more so than comments to the author), and has a potential for both good use and misuse.

In a community that presupposes the integrity of its members, like the scientific community (which is the case so far, in spite of alarming exceptions where this assumption was not warranted), the potential for misuse of private communication should not, alone, be a reason for giving up its obvious advantages.

Certaines revues fonctionnent - après accord de tous - avec des expertises nominatives. Les auteurs et les experts peuvent même échanger.

J'ai eu ce genre de travail à faire avec la revue "éducation et didactique".

Cordialement

Bernard Calmettes

Suppose, there is a theory that has been accepted decades ago. Suppose, you have observed that there is a flaw in the theory. Now, you may write an article stating that flaw and suggesting the correction to be incorporated.

Try getting it published, and see how the referees react! 

In fact, one becomes a referee only after one has published a good number of articles in a field. In your case, the referee would be someone who has been following the theory which you are trying to refute! So accepting your findings would mean that the referee has to reject his own works first! Why would he possibly do that?

If Galileo would have written a paper stating that the Universe is not geocentric as  believed, but heliocentric instead, do you think any referee of those times would have accepted the article for publication? 

Bonsoir.

Votre proposition est intéressante. Il existe déjà, pour certaines revues, des dispositifs d'expertises qui lèvent au moins pour partie l'anonymat. On peut procéder par exemple, notamment pour les articles jugés "publiables avec peu de modifications" à des mises en relation directe entre les auteurs et les experts. Il ne s'agit pas, bien sûr, pour les experts d'écrire à la place de l'auteur. Les experts se placent dans une posture "compréhensive". Il s'agit alors de procéder, à partir d'échanges argumentés et précis, à l'amélioration de l'écriture de l'article, de sa forme plus que du contenu.

Piyabute. Yes the purpose of publishing the reviewers comments is to make the reviewers more responsible, although the reviewer comments should remain anonymous. This is in fact the idea: if the reviewers know that their comments are going to be revealed, they would avoid the typical tricks to make them "recognizable" (if their comment on the paper is positive) and will probably refuse to referee a paper if their competence is not adequate for that subject (which is much better than having meaningless comments from people who know nothing). Having less referees (but of better quality) may imply that less papers will be published, which is indeed better: in the jungle of articles and journals we have now, there is high chance to miss important contributions.

I agree with M. Calmette that the suggestion of Mr. Macchi is a very interesting one indeed. But I think that this should refer to all published papers and not be restricted to papers that are judged to be acceptable but require minor modifications "publiables avec peu de modifications". However, I do not understand why the identity of the reviewers should be kept anonymous. Surely, "the referees will probably be … more careful with the comments (if their identities are revealed) because everybody may read what they write and the referees will probably refuse to referee a paper if their competence is not adequate for that subject (which is much better than having meaningless comments from people who know nothing)". Furthermore, as Mr. Macchi points out, the editors' decisions will have to be more balanced, because everyone may read the comments received from the referees. However, this leaves out the papers that are rejected on the basis of reviewers' comments.  

Yes I agree that papers rejected will remain unknown. However, if full track of a manuscript is kept (meaning comments from all referees of all journals to which the manuscript has been submitted), there are very high chances that eventually the reasons for rejection on a given journal will be known.

I do like the suggestion by Piero. I do not need to read the author's name to guess it from the contents. I would think that it is more important to make reviewers responsible. I find it equally important to be able to see the comments published if a paper has been rejected. I do not know how to achieve this in practice. But the level of comments in "fashionable journals" is often really shocking, and if people could see on which basis a good paper has been rejected by biased or ignorant decision-makers, the prestige of "the high-impact factor journals" would - duely - go down, and the professional journals with high standards of reviewing get the respect, readership and authorship which they deserve.

Having read and carefully considered the various comments and opinions expressed regarding the open peer review and the double blind processes and the merits or otherwise of both, I came to think that a better process would be something like this:

1. If the comments of the reviewers are positive and the paper is accepted for publication without revision, then the names and comments of the reviewers need not be mentioned or published.

2. If the comments of the reviewers are negative and the paper is rejected, then naturally the names and comments of the reviewers will not be published.

3. However, the paper submitted may be judged by the reviewers to be acceptable but after major or minor revision. Three scenarios may then emerge:

3.1 The demanded revision may be found unnecessary after an exchange of ideas between the reviewers and the author or “if the reviewers’ comments are found to be irrelevant or based on a misunderstanding of the MS by the reviewers”. If the paper is eventually published without revision, the names and the original comments of the reviewers need not be published in this case, as the comments of the reviewers have no effect on the published paper.

3.2 The demanded revision is found acceptable and reasonable by the author of the paper and duly undertaken by him and the published paper is improved as a result. The reviewers’ comments need not be published in this case as they would “apply to the MS in its original form, not to the published paper” and their publication would be pointless. However, due acknowledgement of the reviewers part in improving the paper is appropriate and fitting in this case, as it gives the reviewers their due and be some recompense for the thankless task of reviewing.

3.3 Finally there may arise a difference of opinion between the author of the paper and the reviewers regarding the demanded revision, and if the editor decides eventually to publish the paper disregarding the reviewers’ comments and recommendations, then the reviewers’ names and comments along with the author’s response may be published In this manner, the rights and dues of both authors and reviewers are preserved, and the matter will be left to the judgement of the readership of the journal.            

"If the comments of the reviewers are positive and the paper is accepted for publication without revision, then the names and comments of the reviewers need not be mentioned or published." Why not Hassan? This is exactly what happens! Papers that contain poor science (if not errors) are often passed by referees who do their job in a very nonscientific (if not corrupted) way. In case a paper is passed without remarks, then the referees share with the authors the same responsibility on the correctness of the scientific work (of course they cannot be blamed for data falsification or fabrication, which would remain the sole responsibility of the authors).

I agree completely with this!

I also think that you have a point there, but only if the paper published is below par, which ought not be the norm in reputable journals.

It ought not be the norm, but there is much nonsense published in Nature, Science, other fancy journals, and if we can see who approves these publications, we may learn many interesting things...

I fully agree with Elena. Nature and Science have the highest retracting rate....

If the whole editorial process was accessible (not the name of the referees but their statements), we may know how accurate was the refereeing (and even if a referee pointed out a good remark, perhaps not taken by the authors and ignored by the editor). This may force editors to take fairer decisions (because everybody could read the documents in the hands of the editor) and referees to avoid (or at least reduce) some dirty tricks (like making themselves recognizable if behaving nicely with the authors, or hyper-negative towards "enemies", etc).

Yes, this is also true. The authors are sometimes helpless in the hands of dishonest reviewers and editors, who delay or reject a paper, to give way to a competitor publish his own staff. I wonder how the cases of rejected papers with all excellent reviews could be made public, To compare with another paper appearing soon afterwards under another name...

It is a common complaint of many authors that the (name and) country of the authors' institution play a major role in the thoroughness of the review and the decision of the editors. Editors and reviewers probably think twice before recommending rejection of a manuscript from an ivy league institution, but may have no such qualms for a comparable manuscript from a "third world" country. I don't have anything more than anecdotal data to back this up, but even if blind refereeing can minimize just the perception of reviewer bias, it is well worth it. Although it may sometimes (or even often) be easy to guess the identity of the author(s), I see no real benefit in explicitly reporting the authors' names and institutions to the referees.

Very interesting discussion. I do like, too, the suggestion of Piero. I believe, however, that its practical implications are in contrast with what is happening now in the scientific journal arena. In the struggle for funding, higher and higher bibliometric records are required, meaning that malpractices of over-publishing and cheating (unjustified self-citactions, cross-citations to "friend"'s works, unfair reviews - as Elena said - and so on) are somewhat encouraged. In turn, this pushes the system toward the uncontrollable blossoming of new journals, often predatory, which foster dubious pay-per-pubish practices. The idea of publishing referees' comments together with an accepted paper is good, but I am afraid that it will have poor chances of being implemented in practice. Actually, it would obviously imply that less papers, though of higher average quality, will be available. Perhaps, we should also start rethinking the criteria on which the funding allocation procedures rely. There are neither simple nor obvious recipes for that, but I believe that most problems you cited could be solved if we ourselves - and funding agencies as well - will eventually give less importance to the publish-or-perish mantra.

I'm not sure what this anonymous appendix is intended to accomplish. Shame the editors into rejecting papers with bad reviews? I doubt it would be effective towards that end! Editors (and authors) of predatory journals are impervious to shaming. And as Elena asked, what about the rejected papers with all excellent reviews? How many authors read more than the abstract and conclusions of the vast majority of papers, or even of a paper they cite in a review? How many read or download the supplementary information? So then how many readers will have time to read the anonymous appendices / reviews of any but the most exciting or controversial papers? And for those most interesting papers, this is already happening when the editors of journals like Nature and Science highlight these with special perspectives.

The interesting point is that in my university rules (and I guess in those of many others), unethical behaviors are listed and of course falsification, fabrication, plagiarism are the most blamed, but also false refereeing is mentioned and sanctioned. Problem is that in practice only plagiarism (easy to detect with modern software) and (more rarely) data fabrication or falsification are really spotted and punished. The unethical behavior of a referee is almost always unpunished and even not detected, unless the referee plagiarizes ideas read in the manuscripts or projects submitted. Though, this is once again plagiarism.

Suppose you saw your unpublished ideas are technically theft by your colleague, or collaborator, and published it (including the image of your chemical system) without your name on the paper, your concern, and your knowledge, no matter, whether it is published in high impact journal or even in a journal with impact factor < 0.5!! 

Would it be a scientific misconduct? Would it be plagiarism? How to define it - "theft of intellectual property"? 

It is called plagiarism of ideas. Difficult to prove, though, if ideas were not published or at least submitted to a journal, an agency or so.

Interesting question.  I think the major value of knowing the authors is self-selection for biases: it's never a good idea to peer review for one's friends, or one's un-friends, for a variety of reasons.  Of secondary importance is in analyzing study novelty: there's a big difference between author B trying to publish a study similar to what A did (ie: reproducing the data and maybe expanding a bit), and author A trying to publish a study similar to what author A did (ie: trying to get two publications out of the same results).  How valuable either factor is will vary by reviewer.

The benefits of not revealing the authors, as a few have mentioned, are fairly obvious: it's harder to have a bias against the authors if you don't know who they are.  Of course, one could still kill a paper for personal reasons - which is a terrible thing to do, but certainly seems to happen - but it would require a lot more guesswork.  As for being able to easily identify the author based on specialty, though...  editors don't seem to pick specialists carefully enough for that to matter in all but a handful of the narrowest fields.  

Overall, I don't know that blind reviewing will be of much benefit overall (especially given that the authors have to be careful not to reference their old studies as 'our' or 'we did this', and will probably screw up at least once in that respect), but I doubt it will be of much harm, either.

I like the idea of public rebuttal with reviewers comments used for rejection published.

Dear Jean-Pierre Djukic, 

thank you very much for your kind opinion. In fact, I have witnessed cases like this recently. What if the academic institution does not admit to accept this kind of unethical practices by its scientific staff ? Such a thing is not surprising since the institution will always attempt to save its international reputation.