Indeed a very interesting and actual topic Ilkka Korpela !

In my oppinion all your points are directing into one direction: Citations are not a really objective and useful measure to quantify the scientific value of a person´s work. At present many people believe or feel that this is the case, so they try to find many ways to boost their citations through the points you mentioned (self citations, low quality but high frequent publications, questionable co-authorships, etc.).

This can indeed disillusion ones view on today´s scientific community and thus many people (see previous comments) try to find a solution how to fix this. I guess the best solution accepting right from the start that citations are not a useful measure to rate a person. In your and previous comments it was mentioned that more citations lead to more merit and more money. To my experience this is not true. At least most "decision makers" I came across are interested in the scientific field, the scientifc quality and impact as well as the personality of a person.

So instead of complaning about the actions some people take to boost their citations, one shouldn´t care too much about citations in the first place. I am sure that on the long run good scientifc practice is more sustainable for ones career (be it in form of money or any other reward).

I pretty much agree with your concerns. Science is increasingly made to get publications, not to create new knowledge. What I do I do not submit papers and review for publishers that I regard as questionable OA publishers, even though they are not mentioned on Beall's list.

Thank you Ilkka for sharing your thoughts. IMO it is all about how academia works nowadays - more publications = more merit, more money. Peer review work is a nuisance that restricts everyone's access to these resources, so why should anyone do it rigorously? Maybe research merit could be measured better by the number of citations that published papers received - provided that self-citations are excluded.

Hi Lauris

I talked about the general topic with forest ecologists at work yesterday - and the choice of discussion topic wasn't inspired by me. It seems that this is not an issue within certain disciplines only so 'we are not alone'.

One of their corollary thoughts made me fell of my chair - current pressure (hype) with 'promoting OA' implicitly suggests that 'science, as we know it, hasn't been open in the last 200 years'.

Like You Lauri (M), I do not review for some publishers (whose journals seem to be very popular nowadays), and let them know about the reasons (them having been on Beall's list, previously, before 'someone' threatened to file a lawsuit against the guy), and ask that they'd stop sending me requests. No impact - I keep getting them.

Hinted by discussions with old-timers make me realize that in all likelihood ethical issues around publishing aren't a new thing. But at least nowadays we have better means of discussing them. And what is alarming is that the discussion is so lame and very few seem to be concerned, but silently accept the evolution.

I hope that I'm wrong.

Cheers, ilkka

Thank you Ilkka Korpela for opening this discussion. I very much agree with your points above. I personally haven't been involved in research that long, but am glad that I managed to start at the time when review statements were exclusively of the 'old-school type': thorough, thoughtful, constructively critical reports that really opened eyes for points that needed more clarity; or caveats that needed additional experiments. It's impossible to describe how much it has improved (and still improves) my research + I have tried to give something similar back to others (in so far nearly 100 reviews during ~10 years)

But sadly, I also have experience on (OA!) journals that largely neglect reviewer feedback when deciding on published papers. It's really frustrating for those who write detailed peer feedback. It's a reasonable future forecast that most competent reviewers decline to review for journals with such practices -> the manuscripts do not receive best possible feedback and peer reviewer scrutiny -> the scientific quality of the papers will increasingly reduce. There are indications that this is happening already, like you say.

You ask how to get rid of undesired review requests. My recipe was simple, although unintended; (1) Write a comment paper - basically an extended review on one such neglected case that (i.a.) asked the Editorial Board's views on the negligence, and because of demonstrated importance of the comments, instructions on which aspects to focus on when reviewing (i.e. whether it should be language-oriented or also on more fundamental aspects). (2) When no visible responses from the EB, write a follow-up e-mail asking if this criticism could indeed be addressed by the journal administration to show that they take this issue seriously. Since then, no reviewer requests - unfortunately, no visible reactions to the concerns either, which in turn is concerning. If interested in what this is all about, the paper with a title 'Comment ...' can be found somewhere from my RG profile. I am quite tired with this topic and cannot but to agree with you that the discussion has been lame.

You ask what qualifies for authorship. Journals actually instruct very well (but interpretations of the instructions seem to vary). It's a long list of qualifications; e.g. quote from instructions for authors of one OA journal (source https://www.mdpi.com/journal/remotesensing/instructions#authorship): "Each author is expected to have made substantial contributions to the conception or design of the work; acquisition, analysis, or interpretation of data; the creation of new software used in the work; and/or writing or substantively revising the manuscript. In addition, all authors must have approved the submitted version (and any substantially modified version that involves the author’s contribution to the study); AND agrees to be personally accountable for the author’s own contributions and for ensuring that questions related to the accuracy or integrity of any part of the work, even those in which the author was not personally involved, are appropriately investigated, resolved, and documented in the literature. Note that acquisition of funding, collection of data, or general supervision of the research group do not, by themselves, justify authorship.Those who contributed to the work but do not qualify for authorship should be listed in the acknowledgements."

Indeed a very interesting and actual topic Ilkka Korpela !

In my oppinion all your points are directing into one direction: Citations are not a really objective and useful measure to quantify the scientific value of a person´s work. At present many people believe or feel that this is the case, so they try to find many ways to boost their citations through the points you mentioned (self citations, low quality but high frequent publications, questionable co-authorships, etc.).

This can indeed disillusion ones view on today´s scientific community and thus many people (see previous comments) try to find a solution how to fix this. I guess the best solution accepting right from the start that citations are not a useful measure to rate a person. In your and previous comments it was mentioned that more citations lead to more merit and more money. To my experience this is not true. At least most "decision makers" I came across are interested in the scientific field, the scientifc quality and impact as well as the personality of a person.

So instead of complaning about the actions some people take to boost their citations, one shouldn´t care too much about citations in the first place. I am sure that on the long run good scientifc practice is more sustainable for ones career (be it in form of money or any other reward).

Dear Ilkka,

thanks for opening this discussion. I think already many important points have been mentioned and I just would like to add a few small points.

1. Concerning who should be author of a paper, my colleague Somidh Saha just recently pointed me to this comment that was published a couple of years ago:

https://www.nature.com/articles/352187a0

It describes a numerical key to assess whether a collaborator should be part of a manuscript or not - I found it quite convincing.

2. One very important point that cannot been overstated is that a lower quality of peer-review is not only lowering the quality of published papers but also a missed opportunity for authors to learn something (I very much agree with Jari on this - my feeling is that particularly in the first years of scientific work, researchers can dramatically benefit from reviewer's comments, particularly also in the times of super-busy supervisors...).

At a recent conference we discussed that due to the popularity of some OA journals with very mixed qualities in the published papers but a high impact factor (yes - why not mention MDPI) there is a certain risk that a sort of "parallel world" is evolving. Young researchers submit their work to these highly-ranked journals, experience the review-process, are successful with it and think this is a standard to follow also for their own reviews. This is particularly problematic as more and more papers are being submitted, older scientist have no time to provide too many reviews and hence the younger people are more involved in reviewing but might lack good examples about what a good review is.

I personally am still very grateful to a few anonymous reviewers who sent me several pages of comments for my first 2-3 papers I submitted to RSE. They helped so much to improve my work and set the standard for my own reviewing activities.

3. From my experience as Editor, I would say that in at least 50% of the manuscripts I handle, the reviews are not of sufficient quality to make a good decision and particularly also help the authors to improve their work. While the "quality control" part works in many cases, it is rare that reviewers actually try to give reommendations to improve upon a presented approach.

At the moment I don't think there are real solutions to this problem, except for discussing the topic. Particularly also with students and young PhD students. So that maybe, the next time they receive an "Ilkka-style 20 pages of comments review" they are actually happy to be challenged and for getting the opportunity to learn something instead of being disappointed to not have received minor revisions in the first round.

One more thing: Regardless of change in publication models, I don't think that the scientific method itself has changed. It still involves developing and testing hypotheses/research questions, and documenting the analyses objectively and in such a detail that the analyses can be reproduced based on that description.

If a paper comes with a stamp 'peer-reviewed', a reader should be able to assume that all these steps have been subject to peer review. From another angle: a researcher who wishes to add one item with a stamp ‘peer-reviewed’ in her/his list of publications, should not forget that it calls for editor(s) + (usually) 2-3 peer reviewers, in addition to one's own work. I guess this is where Ilkka's 1:4 ratio comes from.

Just increasing open access of peer-reviewed research becomes totally meaningless, if the word ‘peer-reviewed’ is inflated, i.e. if peer reviews are no longer carried out with high standards. A “parallel world”, where a reader would need to judge whether a publication has been properly peer-reviewed, does not sound attractive either.

In addition to focusing on number of one's own publications (and citations) – why not include the number (and citations) from publications one has peer-reviewed and edited! Or, like done already for a while by a service called Peerage of Science, derive quality indices for both manuscripts and peer-reviewers from "peer reviews [that] are themselves peer reviewed"; https://www.peerageofscience.org/how-it-works/quality-indices/ Yet, as recognized above, this is unlikely a matter of missing indices, but how to convert those to (financial) incentives ensuring that high-quality peer-reviewing would not rely only on those conscientious.

Hi Fabian & Jari

Thanks Fabian for sharing your experience as an editor (50%, the type of feedback in the reviews) and for the link to the authorship scoring system example.

All we can do is to discuss and bring up a difficult topic (in Finnish we say "lift the cat to the table" :)) and influence things that way. I still feel that one of the things that we should really promote are OA journals by scientific organizations.

Jari, haa! I have recently put my name in the reviews and quit giving the wanted recommendation, but written that is the job of the editor based on the comments (praise or criticism) given. I know that some of my reviews suck, yet I have felt relieved doing the review with my name on it. That way I don't have to worry if I'm on this planet or in the parallel world mentioned by Fabian :)

Regarding citations, yep, I also do 'feel good' when my work is cited - .hmmm - in the right context. I don't know if I should cry or laugh when I check citations to some of the papers that I've been involved with. But, I can also do better here on my part.

Let's keep them hypotheses tested and let's not be afraid to leave out the sentence "The results were promising and warrant future work" if the actually correct statement should read "In regards to the deficits in our experimental design, the hypotheses deserves another well-thought try" :)

BR ilkka

Agreed with members

I think it is going to fail soon. In my opimion the peer-review ISI-WoS system is crushing