Shows the manipulative nature of peer review very well. Reviewers often have concealed conflicts of interest and where they do they need calling out.

Hiding behind confidentiality in these circumstances is the mask of the scoundrel.

Why not withdraw the manuscript, and submit it elsewhere - hopefully with a better outcome? And hopefully not with the same evaluators ... :-)

I cannot withdraw already rejected paper:) And sure one can always run away from problems instead of facing them.

Anyway, that was not the point. The point is whether a non-researcher and a member of an advocacy group, with only one "publication" we critically mention (!) in our paper can serve as a reviewer of our paper and submit a peer-review without any constructive suggestion about how to improve paper and explicitly declare that our paper is unimportant?

Would you, for example, invite a non-researcher and a member or pro-gun organization to review a paper that provides clear arguments in favor of gun control and which is written by 4 experts from 3 different continents?

Should we just accept this and submit our paper somewhere else or whether this constitutes a serious issue and requires a formal response?

A member of an advocacy group cannot be said to be a neutral observer or unbiased referee.

Would environmentalists like having a member of the oil lobby deliberating on their paper on climate change?

If a journal invites biased reviewers onto the panel there is little point in submitting to it. If advocacy groups control peer review then it is not peer review but dogma and ideology.

Sadly that has been in the peer review system for a very long time now. Perhaps this journal should be referred to the extending list of journals that cannot be trusted.

https://www.theguardian.com/higher-education-network/2017/jun/06/why-we-cant-trust-academic-journals-to-tell-the-scientific-truth

Indeed. Unfortunately our case happened in one of the most prestigious journals! Impact factor (if that means something) is above 20!

In psychiatry this is actually quite normal, with reviewers paid by pharmaceutical companies passing both experiments as proven and papers confirming the nature of the proof.

Double-blind peer review system is useless if editors are negligent or biased when selecting reviewers.

One editor once wrote me that in situation when his journal has a 5% acceptance rate, it is very unlikely that a paper without two highly supportive reviews will be accepted.

What was the chance of getting a supportive review from a non-researcher, an advocacy group member, whose the only "publication" we criticize in our paper?

My remark about double-blind peer review system was written in response to the previous comment.

Our case happened in a journal that uses fully open reviews with an important restriction for rejected papers: you cannot reveal reviewers' identities and their review. And there is my dilemma: how to transparently and publicly discuss about this issue if I cannot provide all details about the case?

In my opinion, the current system of reviewing scientific articles with two or three reviewers has long compromised itself with its idiocy in assessing the scientific merits of articles. For example, the famous mathematician Grigory Perelman published his three famous articles not in a journal, but on an electronic network without any review, but this did not prevent him from becoming famous all over the world. After publishing his articles on the net, two groups of mathematicians in the United States were created, who meticulously tried to find any mistake in Perelman's proof and as a result came to the conclusion that the proof does not contain an error. However, Perelman still refused to accept $ 1 million for reasons of principle, since he was very offended by the mathematical community. That is, in other words, one probably need to follow the example of Perelman and do not give a damn about review journals.

I take this opportunity to express my attitude to the title of PhD, for which the resrachers are trying to publish articles. I just duplicate my answer from https://www.researchgate.net/post/Is_a_PhD_degree_painful_to_a_student_and_his_mentor_Why_or_Why_not

"In my opinion, the PhD title should be abolished altogether, because today this title has ceased to be respected and revered in society. Yes, there were times when this title sounded proudly as a degree of recognition of scientific merits, but now this title sounds more and more as a barrier to overcoming any obstacles to enter science through the back door. We pay too much attention to the title of PhD, however, this title has lost its relevance today, because so many fake scientists are hiding behind this title, who acquired this title in various dishonest ways such as blat, corruption, blackmail, flattery, and other dirty options. I often ask: Why is it that the goods that no one needs are never sold to the market and this is perceived normally in society, and in science, fake scientists are always successful despite their zero scientific contribution? Therefore, I propose to assign this title as an honorary degree already on the basis of the scientific contribution recognized in society. For example, the great composers Grieg and Tchaikovsky were honorary doctors of music. Grigory Perelman, who showed the whole world an example of honesty and decency in science, is a very good example. He demonstrated that rare case when the essence of scientific work far exceeds its visibility. However, in the vast majority of cases we see that visibility of scientific work far exceeds its essence. So, one need to review today's PhD value scale and propose a different value scale to be more fair and respected in society."

Thanks Gennady for your general view on the issue of peer-review and PhDs; however, I am still seeking for advice regarding my cases.

If I write a blog and identify the journal, the reviewer and their review, will I be facing legal or some other consequences? I don't want to do anything unethical myself, but, in my view, this is so serious issue and deserves a full and transparent discussion.

Would we want court of justice with anonymous judges with masks hiding ther faces. Judge by being publicly known are thus responsible for their actions. If they would act irresponsibly, they would be sanctioned or even demoted. Anonymity should never be allowed in scientific reviewing. Too much can be done behind closed doors. When our action are public then we are under scrutiny, if we say something then we can be asked to justify further certain point. Actually, the author of a paper should approve the choice of the reviewers. The journal should send them a first revewer list and they could check if they consider these as competent in the field cover by the paper. If judged as not so by the authors, then some changes should be made. Of course authors should not be able to choose their bodies for avoiding inbreeding.

Indeed. I think the review process has been seriously compromised and that the editors have not adequately responded to our appeal. What is the problem if I publicly and transparently with arguments engage in discussion with the journal and the reviewers? Is there a problem?

I submitted two papers to two different journals. The peer I encountered, had similar comments twice for both papers. It seems that he was master in those skills, which he asked me to do. I wish blind review should be avoided now.

But what if you get an identical review in two journals? That also happened to us:)

The second journal uses “double-blind” reviews while the first one uses “single blind”, so our identities were known to this reviewer before he/she submitted their review. And this reviewer served as an additional, third, reviewer.

When we brought these actions to the attention of the EiC, she was unwilling to take any corrective action. She asserted that the third reviewer “made absolutely no difference in practice as I was about to reject the manuscript anyway on the basis of the first two reviews.”

So why do you invite the third, usually a decisive, review if you have already made a decision?

Why do you waste the reviewer's time (even though he/she also copy/paste an old review) if you have already made a decision?

Why do you keep our paper in your journal for another month if you have already made a decision?

To make things worst, we revised our paper between two submissions so the third review was relevant for an earlier draft not for the submitted (and revised) version of the manuscript.

What do you say about this case?

Saad Javed ! No. One reviewer reviewed two different manuscripts, submitted in two different journals. However, he raised similar concerns for both manuscripts.

Hi Igor, just because someone has done something wrong, you don't have to do it as well.

First of all, reviewing is voluntary work, no one gets paid for it. He/she spends time to read a few times, take notes and provide a critical review. When he/she submits a paper, others might reciprocate their action.

Secondly, if a journal fails to invite a qualified expert in a particular domain of research, it is wrongdoing from the journal's point of view. That said, standard research journals wouldn't resort to this sort of review. A mostly cheap open-access publisher who charges fees and gets the reviews done quickly might resort to this type of review.

Thirdly, an ethically and morally motivated person will disclose the publisher that he/she cannot review a particular reason as they have a conflict of interest. It is a self-imposed action.

Fourth, it is up to the reviewer to share how much opinion that he/she wants. Some provide very helpful and constructive comments (e.g. major issues / minor issues) that they see in a paper. They have the right to do so! (Refer point #1)

Fifth, a journal can reject a paper without even sending out for review at the editorial level. It depends on the impact factor and the rejection rate. An author may think that their paper might be appropriate for the journal they submitted, the editor may decide otherwise. An author cannot question the journal at this stage. They just have to look options elsewhere.

Sixth, for a reviewing expert, the reviewer doesn't have to accept all the writing/interpretations of the authors - hypothesis/theory/opinion/findings. No two reviewers might have to agree to each other. The reviewer has to be convinced in the way in which the paper is written. The paper has to address the positive and negative aspects of their paper (e.g. limitations of the study). The authors have to find a sweet spot and make compromising decisions.

Seventh, the study authors can supply the names of opposing reviewers (e.g. the names of people who they don't want the paper to be reviewed by). Usually, they adhere to this request. However, the may or may not adhere to all the potential reviewers. This is because the names could be biased.

Eighth, the forum hears on your appeal with regards to the reviewer's conflict of interest. However, we are hearing one side of the story. We might not make an informed decision about who is right.

Ninth, anyone can make a mistake - the authors, reviewers, or editors. There might be an inherent politics built around a journal. It needs to be either explained or negotiated. If the escalation doesn't work, just move on with another journal. Manuscript rejection is part and parcel of academic life. Every researcher knows about it. It is not a big deal at all. You don't have to spend sleepless nights over an issue that doesn't even deserve it.

My advice would be - JUST MOVE ON!!!

Thanks Seithikurippu R. Pandi-Perumal

You wrote many things. Here is a "short" reply.

Yes, I am trying to avoid doing something wrong. That's why I asked the research community for advice. I am willing to share the whole aspect of the story, but I am reluctant to do so before I am sure I can actually do that. Until that you can hear only (small parts) my side of the story. Sorry.

The journal's associate editor wrote me that I have no right to reveal the name of the reviewer and their review. Regardless of who is wrong or right, I asked for advice here whether I can reveal my story in full or whether I can only continue whining here with the risk of reading "we hear only your side of the story." I want to make everything transparent. Then you can make your own verdict:)

Doing something on a voluntary basis is not an excuse for doing it wrong.

Reviewers can write in their peer-review whatever they want, what their ethics, education, upbringing, and manners dictate; however, many journals have clear guidelines about what constitutes a good review. COPE also gives an example (which I cited) about what constitutes a conflict of interest. Many journals commit to COPE guidelines on good publication practice.

Further question that arose in my case is whether a non researcher and a member of an advocacy group is qualified enough to review a scientific paper.

Imagine this hypothetical situation: you submit a paper to Sleep Medicine, the EiC invites a non-researcher and a member of an advocacy group that promotes "good sleep" to review your paper in which you criticize their misinformed preaching... and then this reviewer declares you paper as unimportant without writing a single positive word about it. Would you just laugh and submit your paper to Journal of Sleep Research?

I hear all the time: find another journal, move on etc. Science is not a game.

Hi Igor,

Let me enumerate my answers based on the sequence that I read from your message.

You should not 'reveal' anything in this public forum. I don't have any second opinion.

You should not reveal the name or the actual review. I don't have a second opinion either. There are several publications follow such policies. You could opt for such journals. You submitted your paper based on the understanding of the journal policy.

'Transparency' should be between you and the publisher, not the forum members.

Doing it wrong - it is coming from you. The forum members will not know the other side of the story. We cannot make any judgment based on listening to one side of the issue. Very poor review only shows the quality of the publisher, editor, and reviewers. As I mentioned, most likely, I would assume that it is not a good journal to start with.

No one says that the reviewers can, 'write anything they want'. A sensible reviewer will address generally and specifically. You talked about COPE policies in many journals. You didn't specifically mention that if COPE policies are followed in the journal in question.

With regards to your question, "whether a non-researcher and a member of an advocacy group are qualified enough to review a scientific paper." It is a bit vague statement. My answer would be, 'that depends'. Someone who is qualified enough (e.g. a Ph.D.) may not be doing any research now. Does it prevent him/her to review your paper? Some subject matter experts (SMEs) are statisticians. They might not be an expert on any specific domain. If a statistician sees a poor design, wrong use of statistics, non-significant findings; they could request for rejection.

With regards to the 'part of an advocacy group' let me give an example. Let's assume that I work as an advisory group member of a cigarette company and get an honorarium from them. You have submitted a paper stating that in your prospective study, cigarette smoking caused cancer on your study participants. So, what's wrong here? Being an advisory group member, do you think, I would reject the paper because I am getting paid by a cigarette company? When I receive a request to review your paper, I will read a paper, then simply refuse to review and decline the request. This is because I have a 'conflict of interest' here! So, in my opinion, anyone in my position would do a similar action.

On what basis you assume that the reviewer was not qualified? Is that based on his education? I have a non-academic affiliation. So what? Can't I be an ethically and morally-driven person?

With regards to your example, what I might do? Normally, the peer reviews are done by two reviewers. I will evaluate to see if both reviewers trashed my paper or only one person trashed it for a particular reason. If it is a blinded review I might not know who that might be. However, if I see the name in the reviewers' panel I can easily connect the dots. Let's say, I used a particular instrument. I found that it was not reliable and the paper addresses it. But the reviewer owns the company or works for it. You can request for a third opinion. Journal editor will adhere to you if you're not satisfied and especially if you feel that you were victimized by a reviewer. If he/she makes bad or unjust remarks, of course, you can make a point by point refutal. If not you cannot do anything.

If the editor is genuine, you can make a formal complaint, along with a copy to the publisher. These things are viewed seriously. Because, the general rule of thumb is, one unsatisfied customer will tell 10 people. They don't want a bad reputation for their journal or the editor himself.

Finally, science, in fact, is a game. It has its own politics -, especially how successful you're or how successful you become. You just need to know how to swim with or against the tide.

Pandi

http://pandi-perumal.blogspot.com

Thanks Seithikurippu R. Pandi-Perumal for elaborating further.

I hope you don't think I did something wrong so far. Nobody can recognize the reviewer and the journal.

As I previously wrote , "Unfortunately our case happened in one of the most prestigious journals! Impact factor (if that means something) is above 20!" so your remark "As I mentioned, most likely, I would assume that it is not a good journal to start with" does not apply in this case.

I made a formal complaint and the EiC has not replied or acknowledged receiving it. So far.

You write "On what basis you assume that the reviewer was not qualified?" I hope I expressed my concerns as a question rather than explicitly stating that the reviewer had no qualifications. Actually, I have serious doubts about their qualifications, but obviously I cannot write more details here. That makes it difficult.

As I previously wrote, I didn't ask anyone here to judge who is right and who is wrong. I was simply asking whether I can do something about it, if I feel we received an unfair treatment arising from a serious conflict of interest. I tried to describe my dilemma as best as I could without revealing any confidential information.

I asked " We strongly disagree. What should I do now? Can I do anything about it? I want to make this public but this means I must reveal the name of the reviewer. The journal explicitly forbids that. Does anyone have any advice? "

If I understand you correctly, you say that in no circumstances am I allowed to reveal the name or the actual review. Is that correct?

I can reformulate my question for you: in what circumstances revealing the name and the actual review would be justified in your view? Absolute no or it depends?

Another interesting case.

http://www.wame.org/reviewer-conflict-of-interest

"The role of open (signed) review — the point was made by an editor whose journal uses open (unmasked) review that the only reason this conflict came to light was that the name of the reviewer was publicly revealed, thus allowing the conflict to be recognized. In the more traditional blinded review, the author would never have known of this conflict and the paper might well have been rejected for (undetected) reasons of bias. "

My experience is that reviews, even from the same journal, are not particularly similar. My paper in Automata 2008 was reviewed by no less than three researchers of the field of cellular automata and self-replicating machines, and no two of those three was in any way mutually equivalent. Indeed, for some questions, the reviews were diametrically opposed to one another.