The peer review system is often unfair and biased. Very often, when I am asked to review a paper, I personally know the authors. Shall I abuse this to "get even"? Shall I be more strict because I actually value the author as a scientist, as a person, as a colleague? Remaining impartial is close to impossible. Also, since the author does not know who I am, I cannot have an open discussion, honestly pointing out my view of things, or how I understand the theory.
Sometimes, to circumvent these problems, I bluntly state my name in the review. A move which is usually suppressed by editors-in-chief. More often, however, I feel I have to turn down the review.
Conversely, one often finds reviewers doing a really sloppy job: they cannot be criticised, so they review a paper in (what appears to be) a few minutes, criticise parts of the paper unjustly (due to missing parts of the argument laid out in the paper), or are not familiar enough with the field to criticise justly. This is what one almost invariably observes in robotics conference reviews in, e.g., ICRA, IROS. This is the major reason why I do not like to publish there and why such papers are often not (and indeed should not be) acknowledged as publications on par with journal papers.
The often-mentioned double-closed approach seemingly solves one of the two issues---but only seemingly, since even there I can often, in about 50% of the cases, guess who the author is.
A way out would be to use a double-open approach, in which reviewers are acknowledged and, better still, reviews or commentaries are published. The new, open-access journal series Frontiersin uses the former approach, and it appears to works well.
An improvement on this approach would be to use a grading system, in which everyone---with names being named, of course---can comment on or grade papers online. This will not only lead to majority voting, but also to accreditation of accreditors. Of course, blind votes must be prevented.
Wikipedia lists such ventures under http://en.wikipedia.org/wiki/Open_Peer_Commentary, of which Behavioral and Brain Sciences, a journal initiated by (the legendary open-access promoter) Stevan Harnad, was a brilliant example (unfortunately, Cambridge University Press seem to have decided to close its traditional open access, and now papers cost a daring $45). The journal "Papers in physics" uses a comparative approach, I think, but I don't know that journal (nor the field) so I cannot comment on it.
The suggestion would therefore be, that a "journal" would consist of an open access web platform where logins are given to accredited users (that step requires some thinking). Each user can post an article or a commentary to an article, and a rating system along the lines of RG's silly score (silly, since at RG it is heavily based on discussion ratings, and only very lightly on publication and impact factor (caveat!)) will lead to something like an impact factor for the paper. Can a person build a reputation on reviewing only? I think not; the effort in writing a scientific article is much higher, while its impact (=influence on others) is much more profound, and should be honoured accordingly. Yet, only publishing without engaging in discussions is not right, too.
Advantages: fair reviews; but also a better understanding of the papers, since commentaries of peers are included. And, much more impact of papers since those papers which are controversial or important will have high grades. High grades are better earned in such a way than by being referred to from papers in obscure conferences!
It is easy to be critical when your name is not revealed, no matter how unjust it is. It is easy not to accept a paper if one does not take the time to read it. But when I review a paper with repairable flaws and do not point them out, my review is worthless. That does not help me, too. In effect, careless reviewing is punished (rather, not honoured).
What kind of criticism do you want to out which you dare not share openly? Refer to an own paper. Why not, it would add to putting the submission in perspective. islike the paper? That is great, as long as you can find the right arguments. You are right in saying that one would think twice before outing a negative review, but also before outing a positive one! And exactly that is the goal.
Patrick, my humble opinion is that the reviewer should be blind to the author(s) identity. granted in some specialty, specialty areas, the author may be correctly guessed, but it is now a global research community, and one does not always know who the people are that are publishibg, even in your topics of expertise. As someone who submits manuscripts in good faith, believing that my reviewers will provide a sound evalauation of the science, methodology and results of my work, I trust that, with blind review, my reviewers are similarly opinioned.
granted I am a dinosaur, so perhaps the way of the future is open access to all, though why was it we moved from references to referees, who could be asked in confidence?
From what I read you seem to more or less agree with my plea, putting forth somewhat different, contemplative arguments with the same final goal. What's more, indeed the current review and publishing process, with high-impact journals usually being closed (in my fields of research), is based on past models, and we can easily solve the issues at hand. But for that a common acknowledgement is required that the existing review and publication approaches are oudated and harmful.
Also, most people reacting in this thread seem to be associated with research institutes whcih can afford the too expensive journal fees, and we all have access, albeit sometimes somewhat cumbersome, to most journals.
But what about rising countries or less affluent universities? They have no chance to access publications by, e.g., elsevier, and are left out of a key part of research. This clearly leads to an increased schism between research at richer and poorer universities, *for the wrong reasons*.
I think it is our duty to put PDFs of accepted papers on our own websites, and to my best of knowledge I do so myself and convince my lab personnel---I also applaud RGs effort in that direction (but their impact will be, I think, low).
By combining open access with open review, we may actually move towards a scheme where it is easier to assess the quality if publications---of course, there are too many papers on any odd topic to know them all---and we could, in such a scheme, do our best to draw attention to our own paper and so get better votes. The post-publication review system, as Lucas rightly calls it, will be a way ahead.
Your arguments are strong and convincing---against the author-pays principle, not for the reader-pays principle, since your method (which everyone uses) breaks the official rules.
But it only brings me back to the original idea. Getting rid of peer review publishing through arxiv-like platforms allows us to have open access for all, and a review process which gives more than a few peers a chance.
I have been and am editor for various journals and conferences. In this process, finding a good reviewer is always a daunting job, esp. if the article is not precisely in one's area of expertise. It takes many days, and often the first choice reviewers reject to review.
However, as editor one usually wants to get the best reviewer, meaning choosing the more senior peers. That often works for journals, but not for conferences (which is one reason why conferences should never count as ISI publications).
By selecting three (some editors only take two!) reviewers, one leaves out on many others, who possibly are more knowledgeable in the area or for the specific paper. My role as an editor is paramount, but my knowledge is, of course, limited. If I pick a wrong reviewer...
Whats the use of peer review, If opinion of reviewers is not respected? Sometimes bias is too apparent.
I submitted a paper in a reputed journal (not open access) , in about 5 days it was sent for peer review. After about a month editor conveys that the paper is to be published after revision. Reviewer's comments were also attached. Reviewer 1 : minor changes in formatting and references , Reviewer 2 : suggested 4 corrections and improvements. I submit revised manuscript. After about 3 days I get mail that the revised manuscript has been sent to reviewers again. 2 months gone by, one fine evening I get this mail from the editor "Though the reviewers comments are positive about your paper we will not be able to publish........................................". It was not very pleasing to see that the paper went to reviewers twice, their recommendations were positive and but still editor rejected it on flimsy grounds.
- Badges
- Science topic
- Similar topics
- Composition Studies
- Articles