(2)

Another – maybe even worse – consequence: political decision makers overtook this stupid variable (as (1) they had been advised by profit-orientated publishers and groups and (2) as terms such as benchmarking – another quantity variable was overtaken in health care and science) in regard to grant application and money for science and research. Actually I have another example right now on my table: an absolutely high sophisticated scientific project, but not supported by the government in another country due to the fact, that the group did not publish in a JIF high-ranked journal. This is not just a tragedy, this is absolutely nonsense. I of course told my distinguished colleague, that I will support them and started already to work on that as well.

(3)

Last weeks I judged two grant applications for major grant societies: did I scrutinize the scientist by JIF? No – no way. I did what I always do: at first I reviewed the content of the key words to get to know what this is about, and then downloaded prior papers in this regard published by the distinguished authors and reviewed these papers and afterwards their application followed by searching for additional potential aspects and re-judged both. At the end I asked myself: is something missing in my thinking / understanding and / or by the application. For this, someone has to read the papers and the content in detail – not some metrics. For this someone need to think and to reflect.

Now, what the EU did:

they together with Research Grant institutions demanded, that we should get away from it, and by the minor examples above. One major basic for this decision was the missing reproducibility of some 80% in science (!).

This finds my support.

@ Björn L.D.M. Brücher Thank you for your invite. In my opinion, there could be a confusion here in terms of the enclosed statement. Authors are evaluated based on their h-index and i10 index. Journal Impact factor is something different. If a researcher publish in a high impact journal, by default, many assume that the citations will be high. Although it is true to some extend, it need not be so. My highest cited paper is not the highest impact factor journal. It can be seen from my own publications: https://scholar.google.co.in/citations?user=MO90ZXkAAAAJ&hl=en

First, impact factor (I mean hear author impact = h-index), might not be best to evaluate all the time, however, this is the best we have today. Of course, there are variations of such measures as to how one can calculate the academic impact.

EUA and Science Europe made some bold statements as to how the dialogue between their members might lead, the type of support necessary changes for a better balance between qualitative and quantitative research assessment approaches, and aiming at evaluating the merits of scholarly research. It is easy to say, but difficult derive to a consensus. Surely, there is no one-size-fit-all!

Finally, coming up with a research assessment practices that offers rewards and incentives structures of research funders and research performing organisations, as well as universities is not that easy. Certainly divergent and conflicting views, dichotomy in terms of expectations and outcomes, implementation and monitoring issues, etc. is going to make things further more challenging.

What if someone leaves EU and takes up an employment in other countries. How would they get evaluated?

Right now, the EUA and Science Europe don't have any better alternatives for now, and nothing on their table yet! They just have aspirational standards, which they have to develop down the line. This may eventually turn into mandatory standards.

What we don't may not be the best, but still works! They still offer some of the current expectations. Whatever transpires in the future, still subject to critiques and comments. It may not be a fool-proof alternative after all!

http://pandi-perumal.blogspot.com

Thank you Seithikurippu - your answer and comment is really appreciated.

That is why i posted it in such a provocation way.

As you correctely point out: it is a difference if the h-index or the original impact factor is used for a discussion. 

I have as well the same experience as you mentioned at my own Google scholar analysis.

But:

In the statement the original IF is ment, not the h-index etc.

One disappointing point for so many postdocs and scientists is that too many faculties overtook a quantitative variable to (ab)use it judging their scientific quality.

Getting it worse, even some faculties demand from scientists a defined value of IF before being allowed to submit and / or defend their PhD (some official / some inofficially).

Here despite the inequality of various disciplines - imagine you would demand an IF value of 20 in molecular biology, how should this be achieved by a scientist in linguistics or Marine biology?

Therefore the mass of scientists reject the IF - which is here the topic:

The Clarivate Analytics Impact Factor (former ISI Impact Factor, laterThomson Reuters Impact Factor).

I hope this answer clarifies, that here the IF is the topic, not the later elaborated various variables, such as the h-index etc

Therefore the European statements are in terms of the IF and is this from my point of view correct and should find support.

Just because not another solution is provided is not synonym (at least) to me, we should reject such a statement

Correct me if i am wrong please, but by now we do not have any reliable quality measurements and maybe we never will have.

Some may argue, we should use the following variables you mentioned, such as h-index or i10-index.

Furthermore i am not sure if such really will work appropriately, especially as the reflection of various disciplines provide not a correct picture here as well.

And if one scientific group is large, it is known that reciting literature from the own group is biased as well.

This, as well as many more arguments resulted into the San Francisco declaration by some 13,000 scientists and organization's.

Some good points raised by Dr. Pandi-Perumal. The metrics widely used may be flawed and even recognized as such but how, in the absence of any metrics, can a portfolio of research be evaluated? I seem some analogy to the current "p-value" debate. Perhaps, as an interim measure until a consensus can be reached across the scientific community, we can agree to put forward different suggestions and, over time, see "what sticks"?

It is an index, useful for bureaucrats, if not to scientists. They rule the nation and money.

@ Björn,

When analyzing scientific caliber, it is not possible to cross-compare scientists working in different field. For example, a palentologist or an archeologist or an exobiologist or an sociobiologist cannot be compared to a biotechnologist or a nanotechnologist, because the citations may vary. Inherently, some fields will have less scientists or less publications so that it will receive less citations. More people will be working on 'hot fields' than a 'less glamorous' fields. So, Comparisons within the same field only be entertained. Making such comparison is totally wrong. Over the years, many fields might have established certain standards. Such standards can be taken into account.

Now, let's talk about individuals. In many universities across the world, the faculties work in four different ways. One is focus on research, and they have dedicated percentage of research time. The second, focuses on teaching and education. The third type has more of an admin responsibilities. The fourth and final type has the combination of any of these. If someone carefully have to evaluate such CVs, one could characterize the type of responsibility that the particular scientist has. It is not a problem. So, within a field, let's assume that I have a h-index of 40 and you have a h-index of 25; when someone evaluate our CVs, they would easily identify that you had more admin and teaching responsibilities than mine. That could have caused the difference in terms of our impact factor. So, when we compete for a same position, they would take into consideration all of them. For a position that requires more research-based, I might be given priority; while, if they were looking for someone with both admin and teaching, you might be given a priority. So, in my opinion, despite the h-index, the opportunity doesn't change if the people equally talented.

Despite these 4 of my hypothetical classification, there lies an elusive 5th one. These are the procrastinators. The pretend they are doing the research, admin, teaching, etc. and show themselves super busy. That aid, 'The proof of the pudding is in the eating.' One can easily spot such individuals. So, the impact factor cannot taken into account in isolation, but all other related criteria as I have outlined above.

From my opinion, the impact factor of the journal is immaterial, but the impact of the researcher and his research work. I could bolster myself that I have published in a high impact journal such as NEJM, JAMA, Science, Nature, etc. etc. What if the citation count is low? Despite high journal impact, your article metric might throw a different picture altogether! Isn't it? For an individual who hasn't published in high impact journals, their impact make no sense. It is only how they fared well with their counterparts.

The purpose of all these metrics is to provide a crude and quick way to measure someone. As I said, this is not an absolute fool-proof method. Let me give an example here. I want to buy a kilo of apple. I can take 4-5 apples to the counter and ask them to weigh them or just take 4 apples to the counter and ask them charge for the apples. Certainly there might be a variation depends on the size of the apples. However, intuitively we all know that a kilo of apple would be equivalent to 4 apples in a decent size. That's the metrics that we are talking about! Additionally, it works! Additionally, it has survived these years!

On a final note, one could chose to publish in any journal (indexed, non-indexed, high impact, low impact, or no pact or open access).

People will not usually reject any alternate ideas. However, it has to stand the test of time. What if no consensus was reached with the emerging idea? What will happen then?

Though welcome move, too little and too late.

Will burying JIF take away the subconsiuos association of Quality with C,N,S journals? Esp. in hiring committee decisions? I am still skeptical and not too optimistic. Though, I like the idea/ statement.

Nice discussion and various impacts – I love that.

We may point out again, that the EU statement is about the Journal Impact Factor.

Attached some thoughts:

(1)

The use of metrics alone is not a science-value based and / or neutral judgment – such are quantity rather than quality variables plus they are not predictive at all. Otherwise any judgement may also never be neutral and / or non-biased. That may be one major obstacle in science. Should we even transform to so-called Science influencers, in which clicking the like or recommendation button is the major goal?

I understand that scientists who see their value in science by JIF and especially who may compare themselves by having a high JIF score with other colleagues will not have an interest that this way of judging is dismissed – logical – that is psychology; and of course everyone is living in his bubble and network. However, I by myself have a high JIF score but I have no interest in it. Honestly Scientists are much more than variables.

If you have a directed communication and exchange (which is triggered by JIF), someone may be aware, that one key of wishes but also of motivation is identification. However for this we already learned from business organizations (such as publishers as well as companies such as google etc.): filter bubbles are necessary, but on the other hand democracy needs permeability of these filters bubbles. Otherwise arguments of truth no longer penetrate here and on a long term eve threaten democracies. This also reveals that controlled communication can split societies. But here comes an interesting psychological behavior: everyone prefer quality and not quantity variables. But, in case this has an impact of their own (pseudo) success (using such variables), the way of pluralism is not wanted anymore. We undermine the possibility of accountability if we focus too much on comparison of quantitative variables as the highest good here - ethically this is to be rejected.

Is there a strong need to use this variable to compare? Most scientists do not see, that terms such as ‘benchmarking’ are increasingly abused – as in health care, and especially by those who do not perform the clinical and / or education and / or scientific job. Someone may read the Editorial and may find content to think / re-think.

https://www.4open-sciences.org/articles/fopen/full_html/2018/01/fopen170002s/fopen170002s.html

However, as a young scientist we had to count the sum of our JIFs, as this was one (too often “the”) door to be allowed to defend our PhD. So I came out of the lab at the Helmholtz one Saturday night and asked myself: “why are we judged by those variables?” Later the week I went to a faculty member and asked again -the answer was: “we have no other way of using metrics”. I asked “as scientists – does it have to be our goal to use metrics to value science?” And the answer was “maybe someday someone will create such variables of higher value – until we use it, point”. So I (as everyone else) counted my IFs. Now I counted it again, I have a high score, but honestly: if someone consistently produces scientific output and content than you see it alone by the number of publications and especially (more important) by the content, but for that someone has to work hard: someone has to read the content. Of course we live in a time, in which (too often) abstract are read only – or should we say: surface reading started getting the standard?

Honestly, I do not want our young promising scientist to go through this kind of stupidity and I support them as they correctly demand that they are judged by the value and content of their work – not by a non-dimensional metric variable. We could even spin this further – what especially those scientist using JIF to value themselves do not want to hear - as this development (JIF) resulted not just into an inflation of the JIF itself, it also resulted into an inflation of judging science, labs, groups and postdocs and this even resulted (unfortunately) into the fact, that too many postdocs had been abused as the doormat of scientific groups and faculties.

(2)

Another – maybe even worse – consequence: political decision makers overtook this stupid variable (as (1) they had been advised by profit-orientated publishers and groups and (2) as terms such as benchmarking – another quantity variable was overtaken in health care and science) in regard to grant application and money for science and research. Actually I have another example right now on my table: an absolutely high sophisticated scientific project, but not supported by the government in another country due to the fact, that the group did not publish in a JIF high-ranked journal. This is not just a tragedy, this is absolutely nonsense. I of course told my distinguished colleague, that I will support them and started already to work on that as well.

(3)

Last weeks I judged two grant applications for major grant societies: did I scrutinize the scientist by JIF? No – no way. I did what I always do: at first I reviewed the content of the key words to get to know what this is about, and then downloaded prior papers in this regard published by the distinguished authors and reviewed these papers and afterwards their application followed by searching for additional potential aspects and re-judged both. At the end I asked myself: is something missing in my thinking / understanding and / or by the application. For this, someone has to read the papers and the content in detail – not some metrics. For this someone need to think and to reflect.

Now, what the EU did:

they together with Research Grant institutions demanded, that we should get away from it, and by the minor examples above. One major basic for this decision was the missing reproducibility of some 80% in science (!).

This finds my support.

The influence of biochemical factor 'AMC' on another factor 'DPR' in the conditions of the deficit of factor 'RNE'. That is the example of a microscopic problem. The percentage of non-reproducibility will grow in such microfields. Deep specialization, when specialists in hands appear who can no longer heal our feet, is an objective cause of the appearance of impact factor and other factors of this kind. The Brownian movement of scientific microproblems makes it more and more difficult to objectively evaluate the work of a scientist. If it is impossible to eliminate narrow specialization, then it is impossible to weaken the influence of scientometric indicators.

1. It fails the technical screening.

Before they even go to the editor-in-chief, articles are checked for technical elements. The main reasons they are rejected are:

The article contains elements that are suspected to be plagiarized, or it is currently under review at another journal. (Republishing articles or parts of articles, submitting to one or more journals at the same time or using text or images without permission is not allowed. See our ethical guidelines.)

The manuscript is not complete; it may be lacking key elements such as the title, authors, affiliations, keywords, main text, references and all tables and figures).

The English is not sufficient for the peer review process,

The figures are not complete or are not clear enough to read.

The article does not conform to the Guide for Authors for the journal it is submitted to.

References are incomplete or very old.

2. It does not fall within the Aims and Scope.

For the journal Carbon, the material studied may contain carbon, but is not carbon.

The study uses a carbon material but the focus is on something different.

There is no new carbon science.

3. It's incomplete.

The article contains observations but is not a full study.

It discusses findings in relation to some of the work in the field but ignores other important work.

4. The procedures and/or analysis of the data is seen to be defective.

The study lacked clear control groups or other comparison metrics.

The study did not conform to recognized procedures or methodology that can be repeated.

The analysis is not statistically valid or does not follow the norms of the field.

5. The conclusions cannot be justified on the basis of the rest of the paper.

The arguments are illogical, unstructured or invalid.

The data does not support the conclusions.

The conclusions ignore large portions of the literature.

6. It's simply a small extension of a different paper, often from the same authors.

Findings are incremental and do not advance the field.

The work is clearly part of a larger study, chopped up to make as many articles as possible.

7. It's incomprehensible.

The language, structure, or figures are so poor that the merit can't be assessed. Have a native English speaker read the paper. Even if you ARE a native English speaker. Need help? We offer language services.

8. It's boring.

It is not archival, is incremental or of marginal interest to the field (see point 6).

The question behind the work is not of interest in the field.

The work is not of interest to the readers of the specific journals.

How to publish an article in a journal with a high IF? Instructions for authors: 1). Strictly adhere to generally accepted ideas. 2). Include so many figures and tables in the manuscript that the reviewer will be terrified. 3). The discussion should be poorly structured. The reviewer's thoughts are easy to guess: I understood little, but the article is in line with generally accepted ideas and contains a huge amount of data; most likely, the publication of this work will not bring harm. This is how garbage is born in the literature.

IF has not been buried yet because there is no other good (clearly better) index to measure our own contribution to science. Probably the best index might be into account several variables: 1. Not only the number of citations but the impact of the journal which cite your article. 2. Your position in the author list, 3. A correction factor depending on the number of authors, 4. The IF during the last 5 years.

It is very difficult to find the perfect index.

This is the same because the arterial blood gases analysis still exists.....because unfortunately there is nothing clearly better.

All Professor always consider high impact factor journal.......

https://www.researchgate.net/job/933255_Postdoctoral_Researcher_in_Organic_Chemistry

True - not buried until now - therefore the provocative question as explained earlier.

Many still look at metrics (quantity) - not on quality.

My best paper has been published in low impact factor journal.

Article Stable and reproducible nano-electrospray ionization of aque...

My bad paper has been published in high impact factor journal

Article A simple desorption atmospheric pressure chemical ionization...

First, we need a substitute factor to rank and evaluate contributions. If so, IF will be buried soon.