Does this type of analysis have beneficial results which can help to advance scientific research? Are there too many other factors not taken into consideration in making such a calculation?
http://the-scientist.com/2012/09/12/predicting-publishing-futures/
I mostly agree with Joseph Coco’s answer and may I can add a few points.
Some years ago (June 2008), the “Joint IMU/ICIAM/IMS-Committee on Quantitative Assessment of Research” has issued a very interesting report on the quantitative assessment of research (http://www.iciam.org/QAR/CitationStatistics-FINAL.PDF). Although the report was focused at showing the many possible of misuses of citation data to assess quality of research, the text was very sound and well balanced. I learned quite a bit reading through it.
It doesn’t matter whatever indexes we use, they cannot be used alone to judge quality of research or of researchers. They can be a complementary means to be used with due care. Suppose that you want to select the most notorious papers of a specific field to read, then from a thousand of papers you could use the h index as a screening process. But even though we would have to make some renormalization to account for the fact of: (a) different publishing dates (may be by segmenting in bins) and also (b) the variation of the average number of references per article that varies a lot from journal to journal.
On the other hand if you are pursuing a career were a given index is like business card or it is ingrained in peer’s culture, then you should monitor your performance in this index and consider how to play the game without compromise your research ethics.
As I understand it, H-indices are intended to boil down the pool of scientists to brightest minds. With the concern about the growing ratio of Ph.D. applicants to associate professorships I think such measurements have a place. I dissuade the direct use of such indices as they are likely not accurate enough to judge actual scientific contribution, nor to judge future contribution well. But I think it is a good tool to narrow a pool of applicants down to a size which can manually be reviewed to determine the best qualified candidate.
A concern of mine with research institutions directly using this score is their giving scientists something to directly optimize. Scientists are of a similar breed as engineers. So if you give them an optimization problem, they are going to game you. A good such example is my resume. I spoke with some professional recruiters and determined I would be seen as a much more desirable candidate in the software development industry if I littered my resume with unseen metadata including skills and leadership keywords (even if I didn't have such skills). I likely appear as a top candidate for many job posting resume searches even if I am not qualified, but I also appear at the top if I am qualified.
As long as these scores exist there will be some people who win a position over others simply because their H-index is higher, but as long as such behavior is not the norm I believe H-indices can be beneficial.
Dear Lucas, Thanks for your answer. I agree with your conclusion about how this quantification could (and will) become political. In my opinion, this is something which should be avoided or worked against. Your questions are very much appreciated and have stimulated more thinking on this topic. Cheers, Jack.
I mostly agree with Joseph Coco’s answer and may I can add a few points.
Some years ago (June 2008), the “Joint IMU/ICIAM/IMS-Committee on Quantitative Assessment of Research” has issued a very interesting report on the quantitative assessment of research (http://www.iciam.org/QAR/CitationStatistics-FINAL.PDF). Although the report was focused at showing the many possible of misuses of citation data to assess quality of research, the text was very sound and well balanced. I learned quite a bit reading through it.
It doesn’t matter whatever indexes we use, they cannot be used alone to judge quality of research or of researchers. They can be a complementary means to be used with due care. Suppose that you want to select the most notorious papers of a specific field to read, then from a thousand of papers you could use the h index as a screening process. But even though we would have to make some renormalization to account for the fact of: (a) different publishing dates (may be by segmenting in bins) and also (b) the variation of the average number of references per article that varies a lot from journal to journal.
On the other hand if you are pursuing a career were a given index is like business card or it is ingrained in peer’s culture, then you should monitor your performance in this index and consider how to play the game without compromise your research ethics.
As i do not have access to the original article from Nature this is an answer especially directed to the H-index and to the way Web of Science, Scopus etc. calculate scores.
In short, all the major indexes use a formula that looks like citations/nr. of papers authored with the only differences being the time over which this is calculated and what database is used (Web of Science, Scopus, Google Scholar etc.).
While, so far, Impact Factors and Relative Impact Factors have worked so far for journals, especially seeing that is the smallest evils, not the best situation I think IFs and IRs are acceptable.
However most people agree that the H-index is only useful for established researchers (over 40 years, a H-index of over 7-8 generally).
I will not re-express the opinions of my colleagues, with who i agree, but i think that, at least in Europe, governments have been trying permanently to get to a better way of calculating the "score" of a researcher for receiving university positions and state funds for research.
- Badges
- Science topic
- Similar topics
- Education
- Higher Education
- Academic Writing