Hi Kaisu,

You make a good point, but I still think there's a fundamental difference between the role of the author and the reviewer, which may explain the discrepancy you note.

As an active researcher publishing papers, I get paid for writing my papers though my salary - it's a part of my job. I'm expected to devote some of my time to writing and it's also my moral obligation as a publicly funded scientist to disseminate my data.

At the same time, I'm regularly being asked to review research outputs. This includes various granting agencies asking me to evaluate research applications, universities asking me to evaluate thesis work or applications for academic promotions, and, of course, various journals asking me to review manuscripts.

What strikes me is that in most cases, work on evaluating applications, theses and promotions, is being paid for, while reviewing manuscripts for journals is not paid for.

The academic system in Sweden is perhaps weird in this aspect, but I actually have to cover a significant part of my own salary with external funding, if I want to focus on research instead of teaching. Therefore, the time I have for research is very valuable to me and I really have to prioritize how I use it. Therefore, it irritates me at times when journals who are e.g. part of the Elsevier and Springer money-making machines (Elsevier had e.g. 780 million £ in operating profit 2012), are asking me to do refereeing work for free.

Not at all.This gives seriousness to research. There are filters everywhere and and we should all learn to deal with them. We are not babies that should get what we want but we should be published when our work deserves publishing. If our paper gets rejected then we must choose another journal.

Dear Henk and Jose, When I read your responses, I understand that I've formulated my question pretty badly, and I apologize for this. I am not suggesting that manuscripts should be published without a critical, professional review, quite the contrary. Thus I've added to the main question what Ithought would be noted Iin my comments.

The question is similar as in professional sports. If you're competing at a professional level, you want the referees to be paid professionals too.

If the journals are making 20-30% profit on every paper they publish, why shouldn't they be able to afford to pay a panel of professional reviewers to do a really good job of critically reviewing submittied manuscripts?

Yikes Björn. This is much trickier topic than I thought. I see your point, but I am not interested in the pecuniary side of this business. I do believe though that reviewers should be given something but not money. Let's say free access to the journal in return? This only an example.

Dear Henk and Jose, I really only brought money into the equation in order to clarify that a professional, paid refereeing system is an option, which it wouldn't be if the journals were poor, struggling operations.

Basically, I simply wanted to start a discussion on the pros and cons of the current peer review system, and what alternatives are there?

It is OK with me. No rephrasing is necessary. Dialogue is in many a way part of the equation.

Its fine, if the journal is making good money, which most of them does. Reviewing articles for a particular journal for a very long time and getting nothing in return, say in terms of Free publication or free access or reputation or money or anything else and still very much satisfied by serving the scientific community....... You must be one with very big heart or very little work at hand.

Hi Kaisu,

You make a good point, but I still think there's a fundamental difference between the role of the author and the reviewer, which may explain the discrepancy you note.

As an active researcher publishing papers, I get paid for writing my papers though my salary - it's a part of my job. I'm expected to devote some of my time to writing and it's also my moral obligation as a publicly funded scientist to disseminate my data.

At the same time, I'm regularly being asked to review research outputs. This includes various granting agencies asking me to evaluate research applications, universities asking me to evaluate thesis work or applications for academic promotions, and, of course, various journals asking me to review manuscripts.

What strikes me is that in most cases, work on evaluating applications, theses and promotions, is being paid for, while reviewing manuscripts for journals is not paid for.

The academic system in Sweden is perhaps weird in this aspect, but I actually have to cover a significant part of my own salary with external funding, if I want to focus on research instead of teaching. Therefore, the time I have for research is very valuable to me and I really have to prioritize how I use it. Therefore, it irritates me at times when journals who are e.g. part of the Elsevier and Springer money-making machines (Elsevier had e.g. 780 million £ in operating profit 2012), are asking me to do refereeing work for free.

@Björn Thrandur Björnsson , Yes, it is a good thought. Responsibility also comes when you are getting paid for what you are doing (in this case, peer reviewing). Peer review quality may also increase.

You might also be intrested in another Peer-review discussion; https://www.researchgate.net/post/Which_one_is_prefered_Blinded_or_Open_Peer-review?ev=home_person_like_comment_question_target

It would be a good idea. But it costs more and still there is no guarantee that their comments are sensible. As I said before the best way is to publish reviewers comments--before or after the publication of a paper-- publicly and preferably non-anonymous. This way they have to be sensible for the sake of their own reputation.

On many occasions, the researcher is only told about the peer-review, the review itself is not shared in its original form, but a summary is transmitted by the editor. I have been a reviewer myself, and I feel it is necessary that the complete review is shared verbatim with the researcher. I thought publication houses would have streamlined the process of review, but unfortunately that did not happen for the better. Peer-review needs to be standardized first and since it involves a significant amount of time, it is time the reviewer is acknowledged in a monetary manner for the collegial responsibility that he/she performs. the benefit that could possibly come out is that the reviewer can be asked to make a reasoned and explanatory comment and not just a general template-based comment given by the publication house

It is a good idea if the publishing houses will agree to that. However on the flip side of the coin, we also have to look at the fact that paying professional reviewers may lead to the shrinking in the number of reviewers that a publishing company would handle. Still on that line, research seems to be progressing and we can't overlook the fact that currently some peer reviewers are not tolerant to new ideas. I have had instances where I had to overhaul a research to meet a reviewers demands.

So, I agree to the fact that we could have reviewers that are paid. But I am also skeptical if the publishing companies will adopt this idea and if they do, what effect would it have on the progress of scientific research.

Peer reviewing, and in particular making careful, mentoring reviews, is the way we all have to help the scientific community (to which we belong) to grow and improve. Paying for reviews would in my opinion have two consequences: 1) creating "professional reviewers". Would these really make more careful and useful reviews than those done by scientists engaged in research? 2) Lead publishers to increase the cost of scientific publications, which is already excessively high for many budgets. Both of them would likely have a negative, rather than positive, effect on the good functioning and development of research.

The point is, in my opinion, a different one: the publication of scientific literature should not be delegated to enterprises which try to make the most possible money out of it. Of course there are costs entailed by publishing and disseminating, but the prices currently charged for scientific literature are mostly higher than one could reasonably expect .

Dear Giuliana,

The first part of your answer is basically only a rephrasing of my original question, but on the second point, I can certainly agree with you that the risk is that they might pass those costs on to us rather than taking them out of the profit margin.

Paid referees could still remain anonymous.

@Björn: excellent question. I do agree that COMPETENT reviewers should be paid for their work - which is a huge work when done properly (= verify the references!) ..and since SCIENCE is about "transparency" and any kind of declaration of " conflict of interest" I also think that there is NO REASON to hide reviewers' name.

Hiding the reviewers name is not at all important but well hiding the authors name is necessary to reduce pressure from reviewers which they will feel on seeing big names of the trade.

Dear Ian and Alessandro, Both of you point to an aspect of my question which is the anonymity (or not) of the referees, whether they are "free" colleagues or "paid" professionals.

I can certainly sympathize with Alessando's view that's "there is no reason" why referees should be anonymous, that would be my choice too if we lived in a perfect world. However, we're "only human" and human nature is pretty flawed. So in this imperfect world, I do think there are valid arguments for anonymity of referees. Some referees lack empathy and therefore have difficult time formulation their comments in balanced manner. We've all received referee comments which we feel are uncalled for or even downright nasty. On the other hand, there are authors who cannot criticism, even if it's balanced and fair. In these instances, lifting anonymity can easily open a "collegial can of worms" of escalating arguments and retaliations, e.g. next time the roles are reversed and the author becomes the reviewer.

I remember when I was publishing my first papers with my young colleagues, how much time we spent speculating on who the referees of our papers were, searching for clues in the comments and reading between the lines. I think it was just because it was very exciting to be publishing and getting peer-reviewed. Today I can ask myself the question - would it ever have made a tiny bit of difference if I'd know the identities of my reviewers? In scientific terms, I really don't think so, and in human terms, I think I'm just better off NOT knowing who wrote that occasionally nasty, over-critical comment about one of my manuscripts.

Now that I am an established scientist in my field, probably having coauthored papers with 200-300 colleagues from all over the world, most of my papers are probably being refereed by colleagues who I know fairly well, some who have been my collaborators and coauthors. I think it's just a part of human nature that for these colleagues, it would be more difficult to be highly critical, e.g. recommending rejection of my manuscripts, if they could not be anonymous. Because I really do want thorough, constructive and tough criticism of my papers, rather than being treated with silk gloves, I think that I'm actually better off with a system of anonymous referees. However, getting back to my main question - I don't think it's unfair that they'd get paid for their efforts.

Perhaps a useful review (one that an editor actually can use to decide the fate of a manuscript) could be rewarded with a gift voucher from the journal/publisher (for future publishing costs, open access, colour figures, access to papers, subscriptions etc...)? It could easily be implemented as a code you receive after submitting your review, which in turn can be entered when ordering a service. Code is valid one time only. This system is used in many online-shops and seems to work fine...

I found this piece in The Scientist http://www.the-scientist.com/?articles.view/articleNo/32378/title/Whither-Science-Publishing-/ and I think it gives food for thought on several issues. QUESTION 2: "Are there problems with the existing peer-review system?" and the answers given, is especially relevant to this discussion thread.

One aspect raised is the excessive time manuscripts are sitting on the desks of the referees who have accepted the task, but really don't have the time to carry out the reviews in a timely manner. The comment by Michael Eisen (UC Berkeley) starts: "Peer review as practiced in 21st-century biomedical research poisons science. It is conservative, cumbersome, capricious, and intrusive. It slows down the communication of new ideas...", read on by following the link.

Wow! I just reviewed a manuscript this week submitted to a well- respected professional journal. The paper was submitted by a team of researchers from Beijing, China and was a lengthy retrospective analysis covering 20 years of data. When I review papers I frequently get caught up editing the submission to assure that spelling, grammar and sentence structure are as flawless as possible before tackling the actual "review". I have been told to leave this task to the editors, but in cases like this recent submission, the paper was so fraught with simple grammatical errors and misspellings that I could make little sense of the data. In the final analysis, this was a very good paper, deserving of publication. I agree with those who feel this can be quite time-consuming and tedious.

As a clinical researcher it gives me a great deal of personal satisfaction to know that I have been entrusted to review the work of my peers. When my own work is reviewed I am exceeding pleased to read "my" reviewers comments and suggestions. More often than not, their suggestions for revision strengthen the manuscript. Peer review is an excellent service that we, as researchers, provide free of cost to one another. In medicine and in research, dissemination of good information is the goal of publication. I am in agreement that many reviewers accept the task of reviewing papers knowing that will be tossed aside until a free slot of time opens up. The review becomes very low on the priority list and deadlines come and go with regularity. While payment may incentivize reviewers and promote rapid turn-around times, I can see where the more respected and well-published researchers would demand premium fees for this service. Publications may then be forced to select panels of "expert reviewers" based upon cost and efficiency. The genuine "Rolex" reviewer will be replaced by a cheap imitation, and the quality of published manuscripts will suffer.

I think peer reviewers comments should be published. The courageous peer-reviewers should dare not be anonymous. It is not much different when one writes a critical book review!

Yes I agree with you that peer reviewing should be professional and that reviewers should be rewarded: And in all the cases review history must be also made freely available

This is quite an old discussion which circulates quite often. I wont cover all of the points here but add something new to those that exist:

You have to think of economics. If you are paying referees and you have a rejection rate above 50% you are paying for more articles than you publish. If I went to the bank manager with that business model he would likely laugh me out of the door.

@Paul: It would be up to the Journal to pay the reviewers...not the authors.

At least this would "free" from any big "conflicts"

I certainly feel strongly about two points

1.the reviewer and the author must not know each other's names. Obviously, if i get an article by my senior, or a highly-published and respected doctor, i would not be able to give an honest review.i would be conscious of his/her seniority, status a s a researcher etc.\

if an author knew who was going to review his/her article, there might be an attempt to influence or maybe put pressure...

2. Reviewers should work for free. Absolutely.

Paid reviewers may sound good, but the value of reviewing may be lost.

Of course, one must try to find reviewers who do the job honestly, dispassionately, fairly and quickly.

Reviewers should not be paid. Receiving payment would definitely make research more money oriented than it is now.

@Alessandro - that was my point, the journal would pay the referees. if the journal has a rejection rate of over 50% then it will be paying more for articles it doesn't publish than those it does. As a journal business model it does not work.

If not reward reviewers should be benifited with many other things including support to attend conferences etc.,

Hi Paul, I'm not sure I understand your "argument" that paying referees does not work as a business model because of a certain rejection rate. Any publishing house publishing books has a number of PAID staff editors who's main occupation is to read book manuscripts submitted to the company, most of which will subsequently be rejected.

Why wouldn't a publishing house which has a 20-50% profit margin publishing scientific journals not be able to use the same business model, and have a paid staff of reviewers?

Some journals get more than a thousand of submissions a year and only accept 5% or less of them... paying to reviewers could be quite expensive, and result in increasing costs for libraries that pay for such journals. Also, if the journal would hire e.g. 10 full-time reviewers to read all these 1000 papers then are you sure that they would be competent in reviewing all these? Some journals have so-called area editors (e.g. reading all submissions from Asia or Africa) who pre-select papers they think are fit for reviewing (and desk-reject all others), and they might not be totally competent in deciding about every article if it is e.g. original enough. Such professional reviewers may make a similar mistake as it is not possible to know absolutely everything about a certain field (especially if it is not very narrow): what is new, who are the main authors, which are the main articles every author should read, which methods have been successfully applied by whom etc etc etc. If the editor is competent and can select perfect reviewers for a specific article (who know exactly this topic/narrow field, method etc) then the authors may benefit more. In addition, as reviewing takes time, reviewers can also refuse to review if they feel that they cannot do it because of time constraints or because they do not know this topic/methd well enough, but if you would offer payment for every review, then some might agree as they need money.

Publishing reviewer comments as supplementary material and assigning a dOI or publishing the names of reviewers and editors in the paper may be looked in to in my opinion

Due to the wide variation in the quality of reviews, I think we could benefit from some form of review rating eg by the submitter. This would need to be adjusted according to whether the MS was rejected, recommended for minor or major rewriting, but should include tardiness too. This might increase the overall standard of reviews - be they from high IF journals, or low, open or traditional. And I think reviewers should get some sort of recompense eg free access to the publishers articles for say 6 months, or conference support, etc if not to be paid directly.

I would recommend that the professional, paid reviewers must also be anonymous. Again the manuscripts must be reviewed by at least 3 reviewers who are geographically isolated as much as possible.

Hello,

As an update to the conversation, you might be interested in today's editorial by Freddie deBoer, Can Double-Blind Peer Review Be Reformed?

Best wishes,

Barb

http://dish.andrewsullivan.com/2014/08/18/can-double-blind-peer-review-be-reformed/

Award of Reward for better reviewing will surely improve the quality of reviewing, quality of paper and over all standard of journal will surely improve.

In my opinion a well reviewed manuscript will surely add to the journals reputation. 

Now a days with increasing number of journals i noticed many mistakes in journal publications which can be avoided with improved standard of reviewing.

Open access, increase in journals, pay for publication all have

been instrumental in today's degraded journal publication standards.

@Alessandro Diana - in the model I was referring to, it was the journal that would pay the referees. Still, the model does not stand up.

In the era of open acess publication where authors have to pay for publication, publishers earn money and the refrees who actually do all the hard work to improve te manuscript (In good/etical journals) are doing teir job for free.

How can we justify this---I am for monetary reward for refrees---who spend their valuable time

.