To paraphrase Churchill (on democracy): 'Peer review is the worst form of scientific quality control, except for all other forms that have been proposed from time to time'

Peer review is spam-filter in science.

This is save time and money, because some experts do it for thousands of scientists.

To solve the dilemma "VOX POPULI vs PEER REVIEW", we need to understand where is the place of science in a range artscience technology. So we can forgive to artist some uncertainty in painting, but we hope that drug formula, building or jet plane drawing was developed by professionals from certified units of scientific knowledge.

P.S. http://www.theguardian.com/higher-education-network/blog/2013/oct/23/peer-review-system-science-research

I definitely agree with everyone that blind peer review is definitely the only available way to ensure the quality of research. This is some sort of checkpoint for the finished product before it becomes available to the scientific community. Without peer review it would be really difficult to rely on the published data which most of the time works as a foundation for future advancement.

To paraphrase Churchill (on democracy): 'Peer review is the worst form of scientific quality control, except for all other forms that have been proposed from time to time'

We need a better criteria to perform peer-review process. It would be very useful to analyze the content of the reviews of different reviewers around the word. This kind of "content analysis" would be very useful to understand the latent variables that explain how reviewers do their job

Peer review is important for publishing quality research. during peer review process the reviewer estimate whether that research are applicable and original.

It depends on the system that should substitute peer-review. Reviewed by all, texts as permantently changing, rewritten projects, is an interesting thought, but I don't think that this will work for all kind of texts. But I would prefer a detailed editorial editing. And the blind reviewing is just a charade, at least in the humanties, we know each other, it's no problem to find out who has written the article. Finally I don't think that publicaton systems works for different disciplines in the same way.

We need systematic research on peer-review pocess.

Yes indeed, we need data about the consquences of the peer review system and of alternatives. To my knowledge this is still an open field.

Peer review is important for publishing quality paper but always remember on time. Time for peer reviewing process must remain within 1 month. During peer review process the reviewer estimate whether that research manuscript are suitable for publication or not considering on many aspects.

Hi,

Peer review is the best available to ensure high quality reliable research

When I seek out referees I first look outside my editorial board. I use keywords from the title of the manuscript that has been submitted. And I find scientists with related work from all over the world. I solicit usually a dozen of these scientists to referee the paper. And I will usually get two or three affirmative responses. It is possible that some referees are biased, but when you have more referees looking at a paper I think the majority will be fair. And for those who are biased, it is often evident that they are grandstanding and not very knowledgeable. So when I make my decision about a manuscript I take into account all of these issues, and I believe the authors get some helpful comments overall. I am glad even to have five or six referees for a submission. Quite a few authors get back to us and thank for the comments. So I think peer review is very useful, the editor just needs to be diligent and watchful for bias.

Peer review is not just about copy editing corrections only, but it primarily brings only the articles with quality science in it. So its more of a check point before anything and everything gets published in the name of research. From my own experiences, peer review helps in revising the draft to bring it to a better shape. However, some of the draw backs of peer review is, there is always a chance that our research article might get delayed deliberately if it is reviewed by any competitor in the field. Sometimes, the journals send it to scientists who take unusually longer time in getting back and these cause delays. Regardless, peer review is still essential for quality control of articles getting published.

I think that some peer review is mafia in science. If they are not very keen enough then all of us will go to hell. Please eliminate the relationship between the author and his teacher. They select their old advisor or friend to read his article in every journal that he submitted. This is not correct for all of us.

I agree with the comment of Napa Siwarunson. The editor should not send paper to the author's selected reviewer. However, Reviewer must be neutral during review of manuscript.

Napa Siwarungson point is very important. In my view, doing a "blind review" will be the answer. In this case reviewers do not know whose work they are reviewing to eliminate bias, that is if peer review is necessary at all.

I consider that the procedure should be reviewed not eliminated, since sometimes indeed are other interests in the (un)acceptance of your particular contribution. In this way the revision of your work between related pairs is a through step to enhance the quality of your contribution. The main issue herein is not dependent on money or savings. Best regards.

For half a century, I have found it very helpful to have others look at what ever I've written. But I don't like the idea of anonymous reviews. Knowing the reviewer, I can often better evaluate the review, and it is possible to contact them directly for further disucssion. In my textbook writing, we started with anonymous reviewers, but we found out that there was really no need to be anonymous except just to be nasty. So our publisher stopped anonymous reviewing, and things went much better!

Peer review is a very good method of quality control in science when done honestly and correctly and a vehicle for scientific corruption when not. I have had the somewhat dubious honour of finding clearly fraudulent research in plenty of peer review journals, especially where the article is closely associated with product sales in pharmaceuticals and med devices. I would have to agree with Jack Pronk in paraphrasing Churchill. Peer review will continue to be as necessary as democracy just as long as, also like democracy we don't suffer any illusions about its dark side.

Peer review is necessary. I agree with other contributors on the positive aspects; removing spam, improving the quality of the final paper, and filtering out unimportant contributions. This latter aspect is very important in view of the fact that there are now more scientists publishing today than have ever before existed during the whole course of science over thousands of years and we do not have the time to read even all the most important data, even in one's own narrow field. Secondly we were asked about our own experience: I was recently surprised to hear that in physics and mathematics reviewers are avoided who are known to be critical of the (revolutionary?) proposal. I have not noticed this fortunately in molecular biology. One anecdote as a word of caution. In very competitive fields it is a great temptation for some reviewers, with large labs, under pressure to be at the forefront in their own work, to delay manuscripts of revolutionary developments until they get up to speed in their own labs using the confidential data. So far I have experienced this several times from american reviewers. I find such behaviour despicable. I hope that it is not an experience of too many of you who sent your manuscripts to well-known experts in your own field in the hope of recognition.

This is the best form of ensuring quality.

Hi John: You have given the answer to your own question. I too agree that Peer Review is a healthy practice. However, One thing needs to be mentioned that the reviewer must adhere to the time given by the editor. In case it is not possible then they should return the manuscript to the editor who then can get it reviewed by some other reviewer. Just holding on to the manuscript,for a long time (with some excuse or the other) might put the authors in very difficult situations (for eg. if it is a part of Ph.D. thesis). The peer reviewer must realize their responsibility that they undertake.

As previously said, peer review is the best way we currently account for scientific publishing. It is true that sometimes (very few) the reviewer comments appear nonsense or bad intentioned, but the majority of reviewers contribute a lot for improving our manuscripts. If someone takes time for detailed comments is because the content is interesting for him, and it should be positive for authors.

Peer review definitely saves time and valuable money.

Like John Collins I have experienced reviewers who take advantage of their position, and this has cost me at times. However this is rare. Furthermore I have also been impressed and surprised when competitors fight to save a grant proposal as I've seen done on Study Sections. From unknown reviewers of my manuscripts I've gotten ideas and improvements and also forced to do better controls and make the work stronger than before. So scientists are people who can be amazingly generous or less often too selfish like any other group of people. By and large I think we win more than we lose from peer review - the contributions outweigh the cleaver manipulations ten to one in my experience.

I agree with the comments of James Taylor. The referees must accept to review the manuscript in their own research field and should be given accurate and unambiguous comments on the manuscript.

I agree with you to eliminate peer review.

Peer review needs strengthening not weakening. The literature is flooded with publications that were allegedly peer review that sometimes do not pass the laugh test. The quality pf peer reviews seems to have waned either because editors are not rigorous enough in their selections of reviewers or in their expectations of the reviews, or in their demands on the quality of the publications. In just this last year I have written 2 critiques of papers that were clearly wrong in their methodologies and conclusions. Both were from allegedly peer reviewed journals. I recall another epidemiology paper whose methodology was obviously lacking and whose findings conclusions were totally unsupported and bore no resemblance to the substance of the paper. I am a technical editor in one journal, and barely half of the submissions are accepted, and many of the accepted publications require significant reworking before being well described and supportable.

Peer review isn't perfect but the alternative is proliferation of even more trash. One option would be if journals really solicited post publication critiques and published them extensively in an open forum , with rebuttals from the original authors. Publications that do not survive should be visibly retracted by the journals if they wish to maintain credibility in the scientific community.

So, the bottom line is we need more experienced and rigorous editors with high standards, and greater interactive public activity in the scientific community to police itself.

I think Jack's paraphrase of Churchill sums up the frustrating nature of peer review - - the best of all imperfect choices. The critical role of the editor in the review process has not been highlighted so far, in this thread. A major problem today is that the editors of some of the top journals are no longer active scientists (and in many cases were only postdocs when recruited to become editors). They cannot tell when the proverbial wool is pulled over their eyes by a dishonest or ill - intentioned reviewer. They also may be reluctant to discount the review of a famous scientist, who, unfortunately, may be highly biased. One solution would be to put the name of the editor along with any paper published, as is the practice at PNAS. One hopes that would ensure that the editor will be more careful of not blindly accepting reviews. However, while a "real" scientist may be embarrassed by being outed for doing a poor job as gate keeper, I'm not so sure about the junior people hired by major journals. Another idea would be to reveal the reviewers' names, after a defined wait period. This would put more pressure on reviewers to be fair and would discourage outright dishonesty. Another possibility would be that we share our publications online, and let a select panel of reviews comment. Oh, time to stop procrastinating and get back to writing a paper ... that will be subjected to the peer review system...

I agree with Joseph Cotruvo except by the open postpublication critiques instead peer review. We need to take more care for reviewing.

I also think peer review is necessary. I also think that something would need to change to improve the process. Some of them have been already said in the previous posts. As the names of the referees are unknown, it would be fair and right to keep the names of the authors unknown to the reviewers. This way the decision would be less influenced by relationships between scienists and would be more based on the real quality of the work. Also, I think that the pratice of not giving any compensation to the reviewers for that that is a real job, that costs a lot of time and effort when done well, takes the responsability out of them. If they were given some compensation for their job, I think the quality of the review would increase as one would always consider it a real job that needs to be done well, as in fact is.

Although imperfect peer review is essential. I have reviewed many papers that are either poorly written or scientifically weak and in some cases, both! Once a paper is published iit should still be critically examined and commented on, most editors will welcome comments papers that correct errors.

I agree that peer review is necessary but it should not be treated as doctrine. There is a good deal of material that gets through the 'spam filter' described by Andrei. I have seen published papers comparing drugs that were farcical from hypothesis to conclusion and some where the methodology was clearly unethical. Peer review as a filter is necessary but like other kinds of filter you need to knock the dust out of them occasionally.

Harnad, S. (1998/2000/2004) The invisible hand of peer review. Nature [online] (5 Nov. 1998), Exploit Interactive 5 (2000): and in Shatz, B. (2004) (ed.) Peer Review: A Critical Inquiry. Rowland & Littlefield. Pp. 235-242. http://cogprints.org/1646/

http://www.nature.com/nature/webmatters/invisible/invisible.html

I agree that peer-review is a generally good thing but has many flaws. From my own personal experience, I have learned a lot about analyzing my own data from the comments offered by the reviewers. (Sorry to John Milledge if he read some of my poorly written scientific papers =). However, I would have to gently disagree with Barbara Zambelli about compensation. Although it would definitely be nice to get a little reimbursement I frown upon it since it potentially encourages people to do the reviews for their own benefit rather than because they are the best suited to evaluate the manuscript. I currently receive around two or three manuscript requests a month. At first I accepted almost everything. Then I got smart and prudent and started rejecting requests that truly did not fit my area of expertise. Lately, the requests from the same journals have been much more suited to my topics, showing that you can teach an old dog (the editorial staff) new tricks (to request me to review only appropriate manuscripts).

Peer review needs to be reinforced. A thorough, objective and scientific peer assessment should be mandatory for all papers published in scientific journals. However, there are reviewers that instead of reviewing the paper as it is written want to impose content on the author (example: the paper should cover this specific topic-sometimes completely unrelated). This is the reason why more than one reviewer is needed, Three is an ideal number because even numbers of reviews would be inappropiate. It would be also ideal that reviewers are from differnt institutions and different countries.

In short, I do not think it is a good idea to eliminate peer review.

Peer review is necessary if its been reviewed by the subject experts, they can suggest to enhance the quality of the manuscript.

I am a strong supporter of peer review. It insures quality of papers and enhances ranking of the journal. However, I have a couple of problems with it. At present, the reviewers are not compensated for the time they spend on such a stressful work. Those reviewers who are active in research themselves, find it very hard to take time out from their own projects. On the other hand, publishers make a good amount of money from the subscriptions to their journals, Therefore, it is unfair not to compensate the reviewers in some way.. Once the reviewers are compensated, they may be asked to follow some guidelines, including fixing the time limit of one month for reviewing a manuscript . This will help editors to reduce the wait period for authors in getting decisions on their papers.

Despite keeping the author's names secret, sometimes the topic or keywords used in the title, such as geographic names and places, or the institution names reveal the identity of authors. In such cases, the editors need to make sure that the the assigned reviewer is neutral

Each part of a paper should be reviewed by an active scientist with a solid knowledge of the topic. But at the end of the day only the authors have an in depth acquaintance of the subject NOT the reviewers, therefore a positive communication is absolutely necessary prior to any final decision!

The system of peer review of research papers prior to their publication was introduced with the intension not only to sieve out substandard papers but also to improve the quality of manuscript and maintain the standard of publication. However, this system is not being followed by a number of journals in a true spirit. Selection of Reviewer for a particular paper is very important as only a subject-specific expert can understand the work on respective aspect, correctly review the paper and suggest pertinent correction/modification if any. Many times, it is seen that a very good paper couldn't find place in the high impact journal that it deserves due to inappropriate review. Mushroomy appearance of publishing houses across the world, many of them asking for publication charges from the authors/contributors, are not doing justice to the system of peer view by sending the papers to unconcerned reviewers. Hence, the peer review system in practice now-a-days appears to be faulty that need improvement. Editorial offices should have a subject and topic specific panel of reviewers prepared strictly based on research and publication profile of the experts.

Though there is certain loopholes in peer review, but the benefit of peer review stands clear as it not only improves the quality of article by polishing the article and removing ambiguity from the article. It also improves in language, presentation and style for better understanding. Good quality reviewer and editor not only improve the standard of the article, but they also help improve the flow, discussions portions thereby the whole article, itself. Though, nonstandard reviewers and conflict of interest affect some good articles or some good but unwanted/ real results being published, still in present context, it is must.

This is such an interesting topic and I've enjoyed reading the contributions. One problem with peer review is that when used as the basis for grants and publications it tends to act as a conservative force in research and can serve to narrow the scope of . Aled Edwards from the Structural Genomics Consortium (an outfit that produces excellent science without a peer review mechanism feeding into funding decisions) has written about this and blogged about it on the SGC website. RAND Europe has done some work and published on alternatives to peer review in funding decisions and whilst few are talking about abandoning peer review, research funders and supporters do seem to be increasingly interested in exploring options. In publication, we should also remember that peer review is not a guarantee of accuracy of course as Begley and Ellis point out in their Nature article of 2012.

There is always a difference between theory and practice. In theory all supporters of peer reviews have the argument. However, in practice the system is currently in shambles. The reason is that there is a lack of quality control over the peer review system. This lack of control has led to publication of rubbish papers and to rejections of highly innovative content (but inconvenient to the peers). There is also lack of progress. Time/technology has moved on, and certain rules and principles urgently need revisited.

As being a peer-reviwer by myself, as well as my articles being peer-reviewed, I think that this is a VERY difficult question. I think that both Peer Reviewed journals and Open Acess publications may and should exist and co-exist, and REFERENCED as well. Right now, we have two extremes when one side accsues the other in non-professionalism, whilethe other one blames its oponents in "close-knit" elitism", which sometime is fashionably called "high impact journal mafia" (= "shelter" in the Sicilian dialect - Sicult, no pun intended) of "wise-men", who review AND promote mostly each other and its own "circle of followers". Throw a stone to the the authors of these words one who's never seen or heard a situation when one "pack" publishes its papers (and reviwes the manuscripts!) in one journal while its rivalry group does the same in another journal.

At the same time, if we don't have professional filters and openess of non-compensating peer reviews, we will continue to see those low quality mushrooming Open Acess journals, whichwe are witnession nowadays.

Thus, the answer, as it is often happens, is in the middle, and it lies in the selction of the reviewers and transparency of their reviews. I think PLoS One is one of the attempts to combine the best of tne two worlds, though I can imagine how many objections THAT notion can receive from both sides

I don't think that reveiwing should be compensated, otherwise, some of us will become "professinal judges. I strongly believe that it must remain a scientific priviledge as recognition of one's neutralitiy, integrity, and professional expertize (NOTE the order of the merits!). Neither it should be ANONOMOUS, and both must go TOGETHER! We have enough with all those anonomous grant reveiwing cases when reviwers write total nonsense and have no responsibility to the cientific community for that bias and often sheer incompetence. I think, we, as the community, are the best judge, both to the manuscripts, and to the reviwers as well. In any sport, court, or other comeptitions, we KNOW the names of our refreeres, why then for God sake we have no rights to know them in peer reviewing of grants and papers? If a referee wiites a total garbage that hurts one's chances and often a whole career, let's make that judge be responsible in front of all us. We are not stupid, we wi understand when a person got retaliation for that, that must be equally intolerable! So, let's make the reviewing process much more transparent, and start with the papers first.

Igor,

I agree with your comments but I can not understand why you opponed peer reviewed journals to open acess publications. Nowadays the principal scientific journal have the possibility to publish open access y you as author pay for.

Hi Ana,

Pls, don't get me wrong, I pesonally do NOT oppon them, I say they BOTH have rights to exist.

The Open Acess has several meanings, and you are right, you can pay and publish in a high IF journal (but not all of them!) w/o a "standard" reviewing process so the community can judge the paper's merit in the futre. In this case, it must go as an "advertise" strictly speaking.

HOWEVER, the principal scientific journals still have the same group of people, who will basically judge whether a particular manuscript should or should not be published even though the authors are willing to pay for it. And it is a right approach. But, WHO are those judges? We've gotta know who make that decision! And if one had missed a future Nobel Prize publication just because he/she personally thought it was not worthwhile, we gotta know the name, whether it were a peer refrerees, or journal managers

That is my basic point..

Igor, I think any paper can be published earlier or later, is a question of sending once and again after fail. It is true, it takes time, but no any Nobel Prize will be lose because bad editors or reviewers. We need to have confidence.

Transparency is key for the peer-review system to survive. It this remains obscure I fear the end of the peer-review is nigh. It is one of the aspects that needs urgent adjustment. We also need to address the competence of the journal's editors. Many data are not accompanied with proper details for the readers to be able to reproduce the results. Better guidelines from the editors to the authors to what details to include is also a must. Peer-reviewers massively overlook this major flaw.

Please also have a look and your input at http://tinyurl.com/pan8yzv

If the argument to justify the elimination of peer-review is economical, then the proposal should be properly supported with figures. Here are mine:

---The cost of running the editorial system and handling submissions should be around $800 to $2000 US dollars per paper, as judging from the cost of publishing in open-access recognized journals. Peer-reviewing does not make it more expensive, because: (i) only "accepted" papers are channeled to the editorial system, i.e., the less the articles accepted, the lower the production costs (of course there is an equilibrium point here - to little papers accepted make cost per paper impractical due to shared fixed-costs). (ii) Reviewers do not change for the work. (iii) A smaller number of "good quality" papers make scientific knowledge databases (SKD) more manageable, and helps scientist to follow better the new developments in their field, which in turn contributes to the advancement of science.

---The current cost of handling a functional SKD should be arround $2000 US dollars per journal per year (on average), as judging from the cost of having a journal indexed in a recognized DB. This system is necessary to every scientist, to scan the large number of Journals and papers currently being published every day in search for new developments.

------So, what would be the effect of retiring the Peer-review system?

---On average only 1 out of 3 submissions is accepted for publication (this may vary a lot with the journal, but many papers rejected by one journal are resubmitted and get finally accepted in other journal, we all know that, and it is not necessary bad).

---The production cost of the scientific editorial system should increase by a factor of 3 (perhaps 2.5 after correction for the drop in the ratio in fixed-costs/Nº of papers). ---The cost of SKD indexation would increase by a factor higher than 3, because the computational cost of searching and maintaining larger relational databases grows nearly exponentially with the size of the database, which is not fully compensated by the reduction in the cost of hardware and software.

------Conclusion

---Even when my estimates are rather imprecise, the considerations do not support the initial statement "Eliminating peer review may save time and money", but the opposite "Elimination of peer-review would lead to a more costly editorial system and would make SKD harder to handle, and eventually unmanageable"

------Colophon

---In a perfect world, Scientists would not force to publish unfinished or poorly supported science. They would not feel bad if the someone else publishes first. One scientist will not abuse of privileged knowledge of a colleague's unpublished data to be the first to publish, and we all collaborate with each other to make science better and more useful to all mankind without the pursuit of profit. Peer-review would be irrelevant and editorials would have been superseded by an authoritative almost flawless Wikipedia.

---But the human world if far from perfect, and it does not seem to be moving in that direction either.

---So Peer-review is unlikely to disappear, at most, it may evolve to gain fairness.

Original Q was only half of the story: can you imagine if everybody, who has money or needs, will be published anywhere they want, including unprepared papers unfinished work? Money aside, it'd be a scientific disaster! Let's eliminate the driving license Depts then, it'll save a lot of money too? So, money is not the most important issue here but the quality of the papers is. Besidfe, science per se is not a profitable enterprize but GOOd scinece can bring huge advantages in the long run. So, people must have an ability to filter graphomaniac garbadge and white noise; but, the other extreme could be that the good scinse is entitled (between the lines) only to fo amous scientists or prestige universities and it is not always a case it all.

I propose to intruduce a "symetrical approach ":

either BOTH the authors and the reviewers are anonomous

or BOTH are not (and this is preferable). We know the judges, we know the names of officers, whio stopped uf for speeding, we know the name of critics who gave bad reviews on a particular movie, etc.

Why for God sake, scientific reviwers are free to writy any nonsense hiding behind the wall of anonimty?? The situtaion when the authors are known to the reviewers but not vise versa is a direct way to abuse!

Yes, the reviewers can easily find out who the author is, If they do not like him or her, they make sure that the manuscript is not accepted. On the other hand, the author does not know who the reviewers are. If he or she tries to caution the editors about this possibility, they do not like it.. .

Actually, in my field, they journal reviwers KNOW exactly who the authors are! Not vise versa though.

In the grant review, you would have at least the list of the Reviewing cmtee (SRG in NIH) in a month ahead of the SRG meeting (it's called "ROSTER"). If you see a clear conflict, a person, who was criticized by you, etc, 'doesn't like you" (but you have to prove it!), etc, you can send that notion to the SRA (head iof SRG). It won't necessary mean that he/she would not assign that particular reviewer for your grant, but if your cause is well justified, it could be considered. In some European grant Agencies, you would give a list of 3 persons as NO for the reviwers, those pepole can't be assigned as your reviwers at all, no questions asked. In both cases, we can let our voices heard. In the journal review, it is completely in dark. It shouldn't be! And the reviewers' comments should be acessible to the public so a third party can judge whether the paper was not good or the reviewer(s) was(were) biased.

That combination of openness and high quality of the reviwers' reports would help eliminate clear biases I think.

QUALITY is indeed the issue and nothing else. Please see the discussion on LinkedIn http://tinyurl.com/pan8yzv and see that the current system fails to provide the required QUALITY. Hence, drastic revision is needed if the peer review system is to survive.

Peer review is a format of scientific communication to ensure the quality of manuscripts and science. The problem with current scientific publishing is that it only work more for profits and their own convenience than for science. As the scientific discipline get more specialised, more publication decisions should reasonably be given to immediately relevant peers but not the editors.

Reform of the current system to a more academicaly autonomous and integrated one also help solve the failure of peer review that a flawed manuscript get published in a lower impact factor journals.

It is true that the peer review process eliminates a lot of substandard and poor manuscrips (not always, though), but biased referees also eliminate or delay good papers, which in my opinion is just as bad if not worse. As pointed out, the reviewers can figure out who the authors are, but even if they couldn't they still can give an unfavourable review because f.ex. the manuscript does not support their own work, or the reviewer is in the process of publishing similar findings and wants to be the first to get the credit.

In my view, the best solution would be that the referees are non-anonymous, which would better ensure an objective review. It is likely that it could be more difficult to find reviewers, but perhaps some way of giving the reviewers credit in the publication would encourage them into the process. After all, a good reviewer can make the manuscript substantially better for which he/she should get public credit.

Peter makes some god points. However like the standard QWERTY keyboard layout, changing the peer review in place that does providing useful improvements in publication will be disruptive at a minimum. Moreover I wonder if improvements will come at the cost of new problems that may prove more difficult to deal with, such as making it even harder to find unbiased and expert reviewers and having to take revisers who will to get credit or create favor with the authors.

Hey John, can you see who was put your message "high" and "down"? Would you want to? How would you feel about the one if you knew who did it?

So, It was me, your neighbour across the street. You know why I did it? Because the essence of the message was obvious but the wording was very confusing.

Feel better now?I How will you treat me when my paper comes to you as an anonimous reveiwer? Or, I will aplly a tenure track to the Scripps In-t? Answer to yourswelf in frotn of the mirror at your bathroom, not here.

I believe, I answer the question. Nothing personal, just a test ;-)))

Igor just demonstrated why peer review does not work anymore: Rejection will be met with a grudge rather than with scientific integrity. Although I would support transparency and frown upon anonymity, both options fail when a submission is rejected based on personal vendettas. This is why I would rather see scientific publications reviewed online by everyone and its contents judged by pure scientific arguments out in the open. Any false argument will then be publicly dismissed and any solid argument will find public support.

That's exactly what I wanted to show, thank you Jan!

But in case of "total anonomous online revieweving", there is a danger that it can slip to "ochlocracy, where first, the votes can be easily manipualted.Secondly, one vote for a knowlegeaible in the field person = one vote for a "rookie"? That's not good in science,as it it is not good in any professionalInstitution. So,, I see no choice but still keep it as a matter of a team of respected meritocratic referees set by a panel (not soleyly by Editor) but it MUST be transparent in both ways. it should be also a system of scores for the reviwers,(given by the peer), which should be count in their performance, and public should know the numbers. . Putting in one's CVthat "Reviewer score say, 89 of 100, must play a role in one's career as people usually do it free of money. But, a good reviewer shouldn't mean a "kin/spinelessd: reviewer. My reputation is probaly "tough but fair and constructive in his critiques", or I maybe dreaming, just "mean and brutal" ;-) Correct, John? ;-)))

An online reviewing system could be viable when set up properly and when sufficiently refined. Obviously we cannot balance the approval of a carpenter equally against the dismissal by a well established expert in the relevant scientific subject. This is exactly why each referee needs to be identified. In the LinkedIn discussion groups this is the case with the likes and in other social media likes or thumbs up cannot be submitted without identity either. So this would be a good starting point. Then we need a weighting formula based on the level of expertise of each referee. The advantage of such online scrutiny is that the weighed judgments will result in a new form of impact factor independent from any impact factor of the publisher. This way the contents of the paper is judged solely on its scientific merit and no longer on the publisher's reputation, or on the (artificial) reputation of the authors.

May be this is a good idea in theory, but having a flood of the "carpenter's reviews" would be difficult to stop. And that weight system per se is also a big problem, e.g., who will decide that weight "impact factor" for the reviewers, the carpenters? Catch 20/20. But the idea per se is interesting and worth to develop further..

I agree, this is something that needs sorting out. The weight system needs to be robust for it to work properly. There will be other hiccups no doubt (integrity of the host, rivalry among the traditional publishers, etc), but the current situation does not work (think of the cold fusion report and the claim of memory in water as illustrations of peer reviewed rubbish in high impact journals). For the new system we need all publishers in consensus. Besides, the weight system will initially be based on the current impact factors as that is all we have. But over time, both reviewers and authors will gain reputation by the support of their peers online rather than by an anonymous elite linked to a particular publisher.

I am inclined to think that peer review and peer reviewed journal are two seperate things.

Peer review is the evaluation of a performance and should be seen as the basis of everything, and not just science.

Peer reviewed journal is a commercial service offered to the science community. It adds commercial value, quality control and credibility to the research outcome.

We currently have a two two layer quality control. The Journal chooses the reviewers, and the reviewers assert the quality of the scientific paper.

The reader in turn trusts the quality of judgment of the journal an/or of the reviewer.

We can very well seperate the two elements if we can make the reader trust the reviewer only, as it should be the case. After that, it is all about marketing the paper, which is the real work of a journal (edditing, formatting, publishing, distributing, obtaining data about readership, citation etc...).

Peer review can be (and should be) organized at reserach hubs. Universities, research institutes etc... and all publication (online should suffice) should be accompanied by the underlying data. As an additional distribution service, printed journal can be used, with or without a second peer review.

I am for impartial peer review - without peer review literature will get polluted - good and useful peer review will help authors - you can learn new things -

As a new entrant in research I am so very disappointed by all these comments. I have two publications in barely known journals and a major ISI listed journal turned me down after three rectifications. However, I was not disappointed, I realized that I probably did not meet their standards but I was hopping that one of these days my work will get me there. Further, I was very impressed by the comments of the reviewers and I realized that they did tell me things and pointed out flaws that I was not aware of earlier. However, from the comments mentioned above it looks like there is nepotism and favoritism even in these prominent journals. But my vote is still for peer review.

P.S: There aren't many mass communication researchers in Pakistan and we honestly don't have access to varied information. Peer review is a way for us to learn new things.

dear Qurratulann Malik · , i am playing this came from 1968 - do not get upset if your paper is rejected or donot feel happy if it is accepted - peer review helps you

dear Qurratulann Malik ·,when is was young i felt like you - now have open mind - if your paper is rejected do not open the paper for a month - leisurely open when you are free you will agree with reviewer

I totally agree, Professor sahib. Peer review is extremely important especially for new researchers.

Dear Qurratulann Malik, the point of discussion is not about rejecting peer review but to improve it. I totally agree that reviewers help making the manuscript better, and in your case the reviewers seem to have done what they are supposed to. Which they certainly do in most cases. However, the point is that there are occasions when reviewers aim at rejecting or postponing the process for the wrong reasons. Probably the main reason is that they consider the authors competitors in the same scientific niche and rejecting or delaying the manuscript would benefit them. For instance, the reviewer may have similar novel findings about to be submitted and wants to be first to get the scientific credit and perhaps giving him an competitive edge in competing for grants. So it can be a dirty game. The reviewers non-anonymity and publicly disclosing comments and criticism would force the reviewer to do his job more sincerely and make this game more fair.

Peter, I could not have said it better! One way to improve the peer review system is proposed by me above. If there are other bright ideas to improve the current flawed system, here is a great opportunity to share them with us. Contributions to this discussion on http://tinyurl.com/ntd7l2d is also warmly welcomed..

To answer this question one need only look at the flood of conflicting articles on any number of subjects in the non-peer reviewed literature. There are valid criticisms of peer review (cronyism). However, despite a few well publicised outright ridiculous papers that made it into high level journals, peer review works most of the time. As a sometime reviewer, I hope my critiques of papers have led to better science. Without these critiques a lot more rubbish would be published. Jack Pronk's Churchill paraphrase is right on the money.

I notice how many people confirm the importance of peer review without even attempting to propose something to improve it. We all agree that a form of quality control is required, but many of us also acknowledge that the current system does no longer meet today's standard. We have highlighted the flaws and I proposed something to improve. Let's not go around in circles, but come up with more bright ideas how to improve the system in order to get rid of the current flaws. Anonymity will have to go and objections will have to be solidly argued in a timely manner. We have to address conflicts of interest with the reviewer in order to avoid jeopardy of the submitted work. And what about the reviewer who cannot process the deliberately complex statistical gibberish that has to justify the conclusions and claims in the paper, but the reviewer is too embarrassed to admit lack of understanding such statistics (because beyond anyone's expertise) and approves the paper?

I would prefer that anonymity be eliminated, but then people are not saints, neither the reviewers nor writers. I reviewed the paper that led to the original discussion. Yes, there are lots of horror stories (cronyism, slow reviews, arbitrary rejections). I have some of my own, but I know precious few people who can't get published in some peer reviewed journal, maybe just not their first choice. The paper makes an interesting point that "There is arguably danger in bad medical papers." I don't think this can be understated. I rely on my papers being vetted to some degree so I have already separated some of the wheat from the chaff so to speak. Peer review does this. Other methods may do it also, but I haven't seen any suggestions that don't carry their own baggage. Who has the time to read every paper, or all the comments about papers. I have no problem with people disseminating their work via non-reviewed journals or web-sites, just make sure it is clear this is the case. Peer review was not created to disseminate knowledge, but to disseminate truth. I want my biology, i.e. medicine, to be true.

The question aked is for saving money and time. Surely, eliminating peer review will save a lot of money and Time...absolutely! Nevertheless, peer review is not established for those reasons, but to assure that most papers published report realiable information and save us from rubish. Bref, Saving money and time will also save us from knowledge (if is not truth, is not knowledge)

One thing that is being added now by some journals is the possibly to comment on published papers. I am not sure if this helps or hurts but it does provide another means to communicate issues with published work besides writing a new paper. 

Once the peer review system becomes transparent, its flaws will cease to exist and all its benefits as argued above will be met. I know of one publisher who goes all the way in this respect, and it deserves to be followed up by all other publishers: F1000Research.com. (I have no financial or personal interest in this publisher, but as a scientist I strongly believe that total transparency, as this publisher shows, is the only way forward).

Quality peer review is essential to assure dissemination of valid scientific information. Otherwise we see proliferation of useless and misleading information that is costly and wasteful, and scientific rigor disappears.  We have all seen papers that were published with obvious weaknesses and then wondered how they cleared any peer review. Peer reviewers who are qualified and conscientious are essential. Editors should eliminate from their lists those reviewers who do not provide good service to the scientific community.

Peer review is not simply synonymous with quality. Many landmark scientific papers were never subjected to peer review; on the other hand many heavily cited papers, including some describing work which won a Nobel Prize, were originally rejected by peer review. Reviewers are often not really conversant with the published literature; they are biased toward papers or the innovation that affirm their prior convictions. Reviewers also seem biased in favour of authors from prestigious institutions. A study in this context describes that papers that had been published in journals by authors from prestigious institutions were retyped and resubmitted with a non-prestigious affiliation indicated for the author. Not only did referees mostly fail to recognize these previously published papers in their field, they recommended rejection. Perhaps the most powerful criticism of peer review is that it fails to achieve its core objective, i.e. quality control. However, till the other effective means are in place to maintain quality of research and publication, it would be too immature to dismiss the significant achievements of the current system, even as we acknowledge its many shortcomings and prepare to take full advantage of the new technologies of publishing.

I keep hearing arguments against abolishing the peer review system, while the proper action is to IMPROVE it. All arguments to keep it despite its flaws have been said many times but more and more of the same are being submitted.  If scientists are interested to get really engaged in this discussion then please read what others have submitted so to prevent going around in circles.

Good points Yogesh. I have noticed that when there are paradoxes resulting from seemingly conflicting results there may be errors that are accepted as correct more often when they  come from top groups.  In human factors this means that those from less established groups and places are held to a higher standard so perhaps in the end their papers will be better when finally published and this will help them and their institution over time. There was a study that showed  that papers that had been rejected and sent to another journal in the same field were more cited than those that were not rejected, but I have forgotten its source. Perhaps someone recalls this work which seemed solid to me. 

Save money? This kind of activity never is paid. By the way, I would like to hear another kind of means to evaluate research and organize publsihing. Do we have any better process for planning publishing and research evaluation?

Saving time and money could impact the quality of information, and even open the door to pseudo-scientific  or main superstition as real evidence. Even with sctrict reviewers around there are hoax-articles; would you imagine unrestricted publishing?

regards,

We need more formation to perform the peer-review process. The method is valid bur the problem maybe that some reviewers do not know how to evaluate a manuscript or a research project. In my views, this is the main problem. We also need more papers about the peer-revier process. See, for example, "The Seven Deadly Sins of Communication Research" published in the Journal of Communication (Neuman et al., 2008; http://onlinelibrary.wiley.com/doi/10.1111/j.1460-2466.2008.00382.x/abstract)