Most researchers have made individual experiences with publishing in peer-reviewed journals. What is your opinion about the peer-reviewing process in general? In your opinion, is it "censorship", a necessary evil or does it help improving the quality of your articles? What are your individual experiences?
The closest I ever can to "censorship" was when a reviewer asked to counter with an editorial if my paper was published "as is." Certainly, censorship is far from the principal goal of peer-review. Nor is peer-review a just a "necessary evil", although it can be painful. It is a process designed to improve the quality of publications for authors, journals, readers, and science in general.
مرحبا سؤال جميل واستفزازي في ان واحد والامر يعتمد على نوع العمل واهميته وعلى اخلاق الاقران وعلميتهم
I do not exclude it can sometimes be 'censorship' , for instance when manuscript contents are provocative?
Dear Frank T. Edelmann ,
The peer-review process is a far from perfect system however saying it is a system by default would be too strong.
Of course, we all experienced peer reviewers with bad comments (simply wrong or clearly not an expert in your own particular field etc.).
However at least one or two peer reviewers give good feedback that ultimately improved the paper. Though initiatives of post review are tried it is for now the best and most solid system to ensure scientific rigour and fair judgements about your submitted manuscript.
Best regards.
Peer review gives a seal of approval, stamp of authority, and acceptance.
A related paper for this interesting thread is available at :
Article The Future of Academic Journals in a COVID-19 World
https://www.mdpi.com/2413-4155/2/4/76
Sci ; 2(4):76, 2020.
Article | WHO COVID | ID: covidwho-864836
The closest I ever can to "censorship" was when a reviewer asked to counter with an editorial if my paper was published "as is." Certainly, censorship is far from the principal goal of peer-review. Nor is peer-review a just a "necessary evil", although it can be painful. It is a process designed to improve the quality of publications for authors, journals, readers, and science in general.
Dear Prof. Frank T. Edelmann, first I start by translating Dr. Amani Harith three answer, and after that I add my point of view.
1st- welcome, nice question and provocating at the same time, it dépends on the type and importance of the work and the ethics of collaborators and their scientific background.
2nd- I agree with Marcel M. Lambrechts, Regards
3rd- exactly, I agree perfectly with Rob Keller
My contribution:
I don't have such experience, but according to m'y colleagues there is usually a kind of segregation with respect to the origins of the authors, mainly when they are from developping countries. To avoid this, most Researchers perform a short period training at a well recognized lab and workgroups and associate their names even if they don't have any participation. This way they are sure that the publication will have a great chance to be accepted. Kindest Regards Dear Professor for allowing me to take part of this thread.
اتفق معك في جزء من الكلام ستيفان سي مان واعترض على كلمة مصممة هذا التصميم ان لم يكن عادلا فهو شر ضروري وقبله على مضض
Dear Frank T. Edelmann I don't consider the peer review system either "censorship" or a "necessary evil". The concept of developing a perfect review system is mirage, a never attainable goal. It's my personal experience that reviewers' comments mostly improved the quality of my papers. In a few cases, it appeared that reviewers' comments were not appropriate; I put up my point, and it was accepted by the editor. Overall, I do hold good opinion about the review system presently followed by reputed international journals. Personally, I favor an open review system.
Regards!
Dear Dr. Frank T. Edelmann,
this is a very interesting and debatable question. Personally I don't think the peer review process is a kind of "censorship" but, if conducted in a critical, and also precise, intelligent and constructive way, I think it is a way to improve the level of a publication. However, the peer review process does not always happen in this way; sometimes the revision proposals leave us very perplexed as they are sterile and do not give indications for improving the paper or provide too synthetic suggestions, with few suggestions, sometimes impossible to carry out. When I take on the responsibility of revising a paper - and it has happened to me quite often - it is, in my opinion, my job to evaluate it but also to try to make the authors make it even better, if possible. Without a peer review process, a paper would be self-referential and, except in exceptional cases, it might not show all the potential it contains.
My best regards, Pierluigi Traverso.
Dear Frank T. Edelmann; Dear All
Peer review is an absolute necessity and an obligation to validate an article; a doctoral thesis; or any promotion file. This review is usually done by 2 or 3 peers for the articles; and goes up to 6 or 7 peers for doctorate. for promotions there are sometimes commissions of 10 people. In summary ; you have to trust your peers.
Best regards
Emeritus Professor Ahmed KETTAB
Peer review helps to ensure that the manuscripts are sound and scientifically correct prior to publication.
Overall, peer review helps ensure that manuscripts to be published are of acceptable quality
Peer review has become the foundation of the scholarly publication system because it effectively subjects an author's work to the scrutiny of other experts in the field. Thus, it encourages authors to strive to produce high quality research that will advance the field. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4975196/
Rules are made for betterment but always rules are manipulated by strong persons for their betterment this is still almost evil of universe
Peer review is necessary to ensure the quality of published research. Most of the recommendations and suggestions help to amend the reviewed research and motivate researchers to achieve higher standards.
During the last couple of years, the number of initials of my name determine where I will publish. As M. Lambrechts it is an English-oriented journal (e.g. J. Anim. Ecol.), and as M. M. Lambrechts it is a German-oriented journal (Springer)...
Is this a concrete example of censorship?
By the way, following claim disappeared from my previous RG track record
I do not exclude it can sometimes be 'censorship' , for instance when manuscript contents are provocative?
I thought that scientists were fighting for freedom of expression, but perhaps only in certain conditions?
Academic papers that are published in non-peer-reviewed journals are not taken seriously.
Frank: Thank you very much for the invitation to comment, but I expressed my opinion already many times, usually without other disputants even trying to address my arguments! So, only in short: obligatory and binding "peer"-review is evidently, by definition, a censosrship ["censorship" = the situation when somebody, taking advantage of her/his position, can do and does prevent somebody else from presenting her/his opinions in her/his preferred way" - do you know any other reasonable definition of censorship?]. As censorship is evil, "peer"-reviewing as applied now is evil. And this evil is neither "necessary" nor advantageous to science - but I will not repeat here the arguments for this statement: I did this already many times; nobody seemed to be interested, but if now somebody wishes to know them, they are summarized in the paper downloadable from RG and, for the convention of those eventually interested, attached here.
I believe that in most cases peer-review is necessary. There will obviously be exceptions - but theses are likely the stories of 'disgruntled' posts on RG. It is particularly neccessary where peer-review is a valid and valued process - rather than a token service (especially with some open access journals focused more on money generation than quality) of 'just getting things through'. Multiple peer-reviewers i.e. 3 per manuscript is useful for countering censorship. If one peer-reviewer looks to censure - the other two will generally rule that out. It also depends what you mean by 'censorship'. For instance, I recently reviewed that a manuscript was 'unsafe' for professional and public consumption - related to clinical healthcare provision. The authors 'protested' (on the basis they had published on the subject previously) - so it was sent for further 'moderating' review. The final verdict was unanimous rejection. If you ask me - that is 'good censorship' for the good and safety of the public at large.
Hi;
I find that peer review increases the quality of scientific research, despite its shortcomings. Authors should always consider, including in the case of rejection for publication of their article, that the evaluator's work aims at improving the quality of publications through objective and constructive remarks.
With my best regards
Peer review has become necessary of the scholarly publication system to ensure the quality of published research.
For more details we have this paper:
Peer Review in Scientific Publications: Benefits, Critiques, & A Survival Guide
Jacalyn Kelly, Tara Sadeghieh, and Khosrow Adeli
Best regards
As I, based on previous experience, supposed nobody is interested in my arguments, other disputants prefer to just repeat slogans of "ensuring the quality" &c., so thank you again, Frank, for the invitation but I do not see any sense in my further participation and I withdraw from the discussion (at least until I see any of my concrete argument addressed with concrete counter-argument!
When I peer review, I often give advice about the structure of the paper and the clarity of the presentation. This is a category of review that is completely separate from the feedback about the research design, analysis, and conclusions.
I like to think that the papers are much easier to read and understand after the authors have complied with my advice. So I do not think this is censorship. It is making the paper stronger.
The quality of peer-review is declining. More and more I see recent articles in my field with utterly wrong science. The same is true for the quality of the final proof-reading. There was a time I could trust a paper. But now I find so many inexcusable mistakes or flat-out misinformation that I spend more time reading other papers just to determine if the one I wanted to read is credible.
This will only get worse as it becomes easier for people to pay to get their low quality "research" published in vanity open source rags so that they can simply boast they published something, anything. Modern science has become irrevocably tainted.
Peer-review as necessary, althought we need to rethink about it. Sometimes I see people looking for reviewers that are known as "friendly". Sometimes I see reviewers that you pick thinking they know about your theme, as they already published things similar to your article, but they don't discuss your article with you. Sometimes I see reviewers being mean, just to be mean with considerations like: this article shouldn't be accepted because it's not a good work (as a scientist I've learned that good or bad is a perspective, a fact would be something like: the authors didn't detail their experimental session, making it impossible to be reproductible).
There's an excelent article called "Dear Reviewer 2: Go F' Yourself", and I recommend this one to every scientist I know!
Dear Frank T. Edelmann
It depends. But, I think most engineering journals are a necessary evil. This is for the benefit of readers.
Peer review involves subjecting the author's scholarly work and research to the scrutiny of other experts in the same field to check its validity and evaluate its suitability for publication. ... A peer review helps the publisher decide whether a work should be accepted. https://www.publisso.de/en/advice/publishing-advice-faqs/peer-review/
An essential aspect of the scientific process in the life sciences is the thorough examination of manuscripts by other scientists. They read the article critically and then either suggest that it is accepted, rejected, or—most frequently—revised and improved before it is published. In fact, most scientists will not consider a scientific pronouncement as valid unless it has been approved by this anonymous process, known as peer review. Without such an external seal of approval, they would consider any results presented as preliminary, potentially flawed and generally of the same self-serving status as a press release. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1084042/
"Censorship is the suppression of speech, public communication, or other information, on the basis that such material is considered objectionable, harmful, sensitive, or "inconvenient." Censorship can be conducted by governments, private institutions, and other controlling bodies."
https://en.wikipedia.org/wiki/Censorship
Many journals engage in censorship.
If you are not satisfied with your treatment at a journal, find another outlet.
According to my opinion peer review is a very important process for maintaining the integrity of science by filtering out invalid or poor quality articles. Yes, usually it helps to improve the quality of articles, though there are some exceptions.
Let's go back to the remark about 'engineering'. Peer review combined with censorship is essential when human lives are at stake? This can be in the fields of engineering, medicine, etc...?
Example:
When a publication is proposing a new method to build a bridge, the pressure on editors and reviewers should be very high, right?
Every scientist feels a proprietary affection for his or her ideas and findings. For an academic article to be acceptable , it needs to be closely assessed and criticized by the individuals who are are very knowledgeable about the target field. A scholar cannot do anything that is not checked and rechecked by scientists of this network before it is approved.Therefore, peer review is not considered a scencorship ; rather, it is a platform for enhancing the quality of research.
What an interesting question. In my experience, I have never felt 'censored', that's for sure. Generally, any comments I have received back from peer-review have been helpful and ultimately, added to the final draft / paper.
That said, the length of time the peer review process can take can be a little frustrating - especially if there is an element of contemporary relevance or timeliness about your paper.
Overall, I remain generally supportive of the peer-review process.
It is a norm nowadays, but was not in human publication history. The Royal Society of Edinburgh introduced peer review in 1731. Yet, many scientific journals practice peer review only after World War II:
https://blogs.scientificamerican.com/information-culture/the-birth-of-modern-peer-review/
Although peer-review is a "golden standard" as stated by the most of the above comments, there are still exceptions. I would say that Luigi Cornaro was lucky that peer review is introduced much latter after him, otherwise his book would never be published, as no one else can reproduce his life-long experiment:
http://hdl.handle.net/2027/nnc2.ark:/13960/t1sf3j34q
And here is another question: Does the replication crisis really come from the authors?
https://en.wikipedia.org/wiki/Replication_crisis
From my experience, peer review enhances the quality of the manuscript. The issue that you may face in peer review is that if one of the reviewers doesn't like your manuscript for some reason that you could not understand.
It's my personal experience that peer review system is sign of help in quality of articles. I have learned a lot from Comments or recommendations by reviewers. I recommend proper review system.
Marcel: "If you are not satisfied with your treatment at a journal, find another outlet" - and then you can find another which do the same, so you find yet another, and so da capo al fine... - nice game, some young children would perhaps enjoy... But the job of a scientists is (or at least should be) not to play blindman's buff: having finished the work on a publication, she/he should start to work on her/his next project, not wasting time and energy on searching for a publisher who will find the profit expectable from publishing the paper sufficiently high to magnanimously condescend...
"Peer review combined with censorship is essential when human lives are at stake? This can be in the fields of engineering, medicine, etc...?" - I, for one, would not be happy to be cured by a doctor basing the applied curation on single ("peer"-reviewed or not) paper!
Dear Roman Bohdan Hołyński "I withdraw from the discussion (at least until I see any of my concrete argument addressed with concrete counter-argument!" I'm really glad to see that did not withdraw from this discussion. I (almost always) highly value your thoughtful comments which are often contrary to the "Zeitgeist". So please continue participating in this thread! I think we should for example examine in more detail how much it depends on the various disciplines if peer-review is more or less problematic. I will try to come up soon with some arguments for our discipline (chemistry).
Frank: Thank you very much for your kind words! In fact, I am an "addict" disputant and it is always very difficult for me to withdraw. But if I quote the definition of censorship, ask what in it does not, in anybody's opinion, exactly fit the obligate and binding "peer"-reviewing system, and instead of answer I find repetition of the "assurance" that ""peer"-reviewing is not censorship"; if I write detailed argumentation showing that "peer"-reviewing is harmful for science, and nobody tries even marginally address any of my arguments, repeating only like a damaged record "necessary to assure the quality", &c., then what is the sense of my "participation"?
Now Marcel made relatively new points, so I tried to react,, but am already tired in repeating again and again the same arguments without any reaction: play ping-pong against a wall may be enjoyable only for short time... I do not know whether this disinclination for a serious discussion is because my opponents have no arguments, know well that I am right but do not wish to admit it, or there is any other explanation, but anyway the result is not what I would call a discussion...
Dear Frank T. Edelmann , thanks for kind invitation to your thread. Sorry, I was out of RG for 2 days, but it is never late to share the opinion about peer-review.
I consider peer-review as very necessary step in publication process which help authors to improve their text. No evil at all.
For those who do not want their article to be reviewed, there are many ways to publish, including pre-publications, different portals, institution portals etc.
Amani Harith has posted 4 answers already that I can not understand dear Frank T. Edelmann . Do You? Is it a new spammer?
https://www.researchgate.net/profile/Amani_Harith/answers
Dear Ljubomir Jacić
it's nice to see you in this question asked by our colleague Dr. Frank and see your comment. peer reviews are a necessity and provide a plus. In my institution and I think in many institutions: only articles validated by peers are admissible for promotions.
Best regards
To Ljubomir Jacić :
I agree that peer review is essential to establish a minimal threshold for any academic research paper.
Any academic who chooses not to have their article peer reviewed will not receive a minimal seal of approval, whether they want it or not.
Peer review is crucial. It helps preserve research integrity, authenticity, and validity.
Dear Frank T. Edelmann,
The vast majority of my published papers have been reviewed with minor corrections. On the other hand, some of my papers were rejected for publication with a short review, which mainly contained explanations such as: "The results of the paper are new and their proofs are correct, but may not be of interest to most readers of our journal…"
What about competition between research teams, and potential consequences for censorship?
Can ideas obtained by reviewers during the reviewing process, but censored by the reviewers, be exploited by reviewers to adjust their research activities?
Dear Marcel M. Lambrechts "Can ideas obtained by reviewers during the reviewing process, but censored by the reviewers, be exploited by reviewers to adjust their research activities?" Thank you for mentioning this interesting aspect. This is of course not only a question of censorship but a case of scientific fraud. In fact, it is probably one of the worst nightmares of many researchers: A reviewer reads a manuscript reporting an exciting finding, writes an unfair review suggesting rejection, and then runs into the lab quickly repeating the work. I have no idea if and how often this happened in the past.
Frank: This might be a problem when referees remain unidentified, and there are no open-access traces of referee reports or nobody takes the time to look in detail at these potential problems....
Dear Frank T. Edelmann - thank you for your question. In my field of consciousness research, I would welcome the review of peers. However, censorship takes place at the editorial level. The topic of censorship mentioned by Roman Bohdan Hołyński does take place, I know this from my own articles. I have at last been able to polish my latest contributions to such an extent that they cannot be thrown out for technical reasons. Now I get different answers like our readers will not be interested, or interesting novel position but you have not mentioned enough other studies which support your point of view. If one is presenting a point of view that is novel, there are no or very few other authors who support the view presented. A real catch 22 position!!
But I "plod" on, eventually publishing in journals which are not considered the best and I am now seriously thinking of writing a book.
Warm regards Tina
Peer review helps maintain a certain level of quality and minimum standards. Since the reviews are from humans one may expect a certain level of personal bias depending on the journal and reviewer. However if the work and results are well presented and beyond very little criticism the author has little to worry about the fairness. If an article is rejected for some reason the author has the opportunity to revise it, address concerns and resubmit to a different journal. I have done this myself and there is no shame about it although there is no comparison to the joy experienced if the journal
editor accepts the paper as is with the first attempt.
In the past 50 to 70 years, peer review was introduced not only for ensuring the quality of the papers, but also because of the enormous increase of number of specialized articles, yet limited by the journal space for publication. Because of this, a lot of valuable articles are rejected by peer review.
Now, the publication of most of the papers is electronical, and there is no limitation in journal space at all. So why cannot we still keep the peer review as a quality control method to improve the quality of the paper, but publish all papers after peer-review without rejection? In this way, we can avoid the cases that a paper being rejected by the reviewer, simply because the reviewer doesn't have the pertinent knowledge and experience to recognise the real value of the paper.
Dear Prof A. Kettab , I would say it is very hard to ascertain that peer-reviewed papers have "quality" when "most of the published research findings are false":
Article Why Most Published Research Findings Are False
If we set a quality scale from 0 to 100, all the papers are in the range between 0 and 100, but none of them is 0 or 100, as what you have just said before: " no document is perfect".
As for most people in the science business, after 50+ published paper my personal experience is a rather mixed bag. In general, in my research area the reviewers' comments have been helpful and well meant. On the other hand, there have been cases of total misunderstanding or even clear will to kill a paper that was somehow developing a research line conflicting with that of the reviewers'. In my opinion, these attempts are very short sighted nowadays, when there are so many ways to make one's viewpoint known. It is hard to talk about censorship when you can upload your (possibly completely wrong) paper on arXiv and RG and let everybody see it (and judge it). In some areas, like particle physics, people have claimed with some right that peer review is dead, since high level scientific debates have taken place on the internet within a few weeks or months, which is the usual time span of a review. Even though there is some truth in this, I believe that peer review should still be maintained as a standard, but made more transparent, for example by having open discussions on the papers, to which the reviewers also openly participate revealing their identity and potential conflicts of interest (avoiding them is impossible, one can only hope to identify them clearly). Some communities and journals, see for example the journal Geophysical Model Development, are already doing this in a way that seems appropriate to me.
The questionable statement :
"most of the published research findings are false"
is misleading, at best.
The cited paper from 2005 concentrated on research in the medical sciences.
i keep bad memories, shall not submit to peers again - nor review myself
I think peer review process is good for improving manuscript quality but must follow the rules and regulations before reviewing any articles.
The provocative title “most of the published research findings are false” refers indeed to an interesting paper. However as said by Michael John McAleer it deals with a particular subset of studies in the field of medicine.
The paper never implied a ‘failure’ of the peer review system but way more points out the limitations of the set-up of certain studies and particular the way statistics are used and interpreted.
Good point that the term "false" in the paper "why most published research findings are false?" should be interpreted as "limitations". And I think this is the initial intension of the author: "Science has limitations and needs perpetual improvements." And I think the same applies to peer review. It's a "golden standard", yet it is not perfect.
According to my opinion, peer-review increases the quality and visibility of the manuscript.
According to my opinion, peer-review is a necessity and a benchmark for quality research papers.
Dear Frank T. Edelmann , maybe someone would be interested to attend this workshop.
Peer review online workshop , Thursday 19 November 2020
The workshop will work through real cases raised by COPE Members to identify issues in peer review and how to handle them...
The cases for discussion will be based on these real cases:
Case 1 Editor and reviewers requiring authors to cite their own work
Case 2 Compromised peer review system in published papers
Case 3 Author requests for certain experts not to be included in the editorial process
Case 4 Author of rejected paper publicly names and criticises peer reviewer
https://mailchi.mp/publicationethics/peer-review-workshop-nov2020?e=19bdda085b
It is a matter of gatekeeping, with respect to the cohesion of scientific discourse communities, driven by efficient and effective ignorance.
https://www.college.columbia.edu/cct/archive/winter12/columbia_forum
It is neither an evil nor censorship. If we apply T.S.Kuhn, normal or official science employs this tool to keep the paradigm clean or to avoid cognitive dissonance a la Festinger.
I see no evil and hear no evil where there is none. I fail to see the evil behind peer-review. It is what it is: a review by peers.
I think all open access journals will become peer-review soon due to availability for citation and review.
Unbiased peer review is always helpful to revise the manuscript pointwise . But there are certain peer reviews which are biased and unmeaningful . The comments like ' unscholarly ' , ' out-of scope ' all these are fake terms . These terms needs further explanation for rejection of manuscript to the point .
Dear Ljubomir Jacić many thanks for sharing this information about the online workshop on peer-reviewing. I'm sure it will be very interesting and useful. For me, I can say that I have fulfilled my debt. For more than 30 years I regularly reviewed research papers for ca. 40+ chemistry journals. Now, after my official retirement I more often decline review request that accept. Occasionally, however, I still write reviews of manuscripts when I find the topic very interesting. Moreover, I'm on the editorial board of the traditional chemical journal "Zeitschrift für Anorganische und Allgemeine Chemie" (ZAAC) which is published since 1892. I never decline review requests from this journal and regard it as my duty to do peer-reviewing for them.
Dear Jaydip Datta "The comments like 'unscholarly' , 'out-of scope' all these are fake terms. These terms needs further explanation for rejection of manuscript to the point." Yes, with this you raised an important point. Personally, I would not call statements like "out of scope" fake terms, but you are absolutely right in that reviewers who use such terms must explain their verdict in detail. The same is true for the popular term "lack of novelty". A reviewer who writes this must also give a plausible explanation. The only time when I successfully argued with an editor about a rejected manuscript was when a reviewer had mentioned "lack of novelty" without any further details.
Frank T. Edelmann
Dear Professor,
Thank you for the beautiful question. I want to answer this question from the point of view of developing countries.
Most of the time, a reviewer either does not know anything about the researcher/ author or knows very little. It is very difficult to review anything if the reviewer does not get enough scope to know about the researcher/ author, to know about her/ his psychology and tastes, preferences etc. (this is particularly applicable in case of social sciences).
There was a great film director who told us that all art is autobiographical, a pearl is an oyster's autobiography. If we are to believe that this is true, then most of the time, reviewers actually are not in a state to review what they are supposed to review. So review does not become a censorship, neither is it a necessary evil because most of the time it does not happen in reality.
I sincerely felt from the bottom of my heart that peer review is a business concept. These are some compulsory hurdles created to influence the perceptions about the quality of the product (journal) and the barriers one have to cross to get a research material published there. I think it remained a marketing gimmick in many cases. It is an open secret. At least in many developing countries this is the reason that quality of research does not go higher and that is why those countries remain developing only.
In this thread, a lot of great scientists and researchers have shared their opinion. If my opinion appears childish, then please neglect it and forgive me.
Thank you.
Best regards,
Anamitra.
A. Kettab
Dear Professor,
I once pursued a management program in a B-School. When I wanted to publish a paper on the basis of a research pursued there, I was told the same by the reviewer. He gave me some of his own articles so that I could cite from there. I told the reviewer that by doing this he may increase the number of his citations but he was working against the progress of the world.
Most of the time, we select reviewers on the basis of their fame and glory as a researcher. We forget that reviewing is a professional's job. A reviewer may not be a legendary professor or teacher or researcher. She/ he needs to be a good reviewer only.
Till the time, we do not give respect to the job of reviewing, we will get people as reviewers who would prefer to do this in their leisure. And, in that case, yes, peer review will remain censorship and a necessary evil.
Thank you.
Best regards,
Anamitra.
Dear Dr. Frank T. Edelmann ,
Actually whatever I felt that whenever I have chased them into logic there is another term from there end " not suitable " , " deemed for rejection " -all these are
non-specialised ,or most of these journals are using such fashionable words but
never appeared in superspecialised ,logic oriented research platform . A good author will never tolerate one way verdict .
Dear Anamitra Roy , I think Prof A. Kettab has a good point that if the reviewer offer us relevant articles of his own to improve our manuscript, then this is a good thing. I will be very happy to cite these articles in the revised manuscript. Otherwise, if the reviewer wants us to cite his irrelevant articles, we can just reject such suggestions of the reviewer.