Hi Dear Michael

I think the impact factor is very important to other index

Dear Michael,

This is a hard and often discussed question which is apt to divide people because of their different interests. Basically, the contribution should get the most impact which may have the most important “positive” influence on human welfare or which have used most frequently by scientists for new and constructive ideas or discoveries. However, scientific journals, their editors as well as reviewers may have their own interests and these do not agree always with the interests of society. Certainly, journals are evaluated but this evaluation can be only approximate and are influenced by many and various interests. In addition, unfortunately there is cheating in many phases of the publication process look at https://www.researchgate.net/post/Who_should_be_among_co-authors_of_the_scientific_paper for details. One thing is sure future conditions, scientists and a well trained, deep thinking public may have, will have a thorough and just evaluation on the true value of a scientific contribution. Another approach: it would be useful a continuous change of editors and reviewers and the diversity of these people would provide a more just evaluation in the long term. But all of them need time and hard work and may offend or damage the interests of influential people. Thus, this is a trap situation.

In general, the best way to measure the impact of a scholar is to ASK HIS/HER PEERS within the same field. This is why, for example, when a scholar is being evaluated, the LAST PLACE to look for is Google Scholar. The first please is the LETTERS from other established scholars ...

Also, one more thing : A scholar can only be accurately evaluated by somebody within the same (or similar) fields ... There could even be a major difference within the subfields of a major field (for example, in Electrical Engineering, Magnetics is completely different than Digital Signal Processing).

All of the above applies to an individual publication too ... A publication that gets a lot of citations isn't necessarily the highest impact publication, although this is a pretty accurate measure in many cases.

It would help a lot to have open peer review, before and after (post) publication. You can see Open peer review at work at e.g. the journals Biology Direct (http://www.biologydirect.com/), Atmospheric Chemistry and Physics (http://www.atmospheric-chemistry-and-physics.net/) and other journals from the European Geosciences Union and Copernicus. And recently we saw the launch of PubMed Commons that facilitates post pub reviewing.http://www.ncbi.nlm.nih.gov/pubmedcommons/

Jeroen, I favor open peer reviewing, because it has the potential of improving the whole review process, which many colleagues have troubles with. I'm not quite sure that I understand how open peer reviews would add to measuring impact. Could you give me a jump start on that?

Of course it is no simple counting. But reviews, especially post-pub reviews should give an idea of strong and weak point of a paper and an assessment of its value. That information can inform a researcher of research group evaluation. Article level metrics and Alt Metrics could play a role in the sideline. Researchers could list what they themselves think are their most important contributions/papers and, importantly, also argue why they think so, with reference to the reviews and article level metrics available.

I think Jeroen’s approach can help to decrease only partly the uncertainty of evaluation process with increasing the openness. This cannot help on the other troubles. I have a humble and simple remark: It is difficult to explain that the peers’s interest (downloading of some contributions) does not correspond to the official value (impact factor) of the given papers. One would think that a paper with a high IF should be more interesting for peers than an article without IF. However, this is often not true.

One can ask another question: Does mean citation a real interest and scientific value? Or somebody will only to confirm his/her own mistake? Certainly, citation can be also a technique to increase the citation index of cronies or colleagues. I have heard about a scientific institution where the articles of the colleague should be cited compulsorily by the other fellows.

@Jeroen, the idea of post-pub reviews is a good one and could even be enhanced by up and down voting mechanisms like the ones offered here on RG.

@András, I'm not surprised that papers without IF can be cited more often than those with high IF. After all, IF is a metric for a set of English language journals chosen by ISI. Since it is based on human work, I'm also not surprised that there is impact factoring on every side of the equation: authors' circles citing each other and editors' policies enhancing impact. Besides that, what if the article has been cited in a negative way? The citation counts anyway, but where is the impact?

Could a score system, as it is used here on RG, be part of the solution?

Dear Michael,

Yes, I think the RG score may have some advances: 1. answers send to RG are individual (not collective) performances, 2. the time for reflecting to a reply are briefer so this can explain better the abilities of a participant, 3. the participants must have some courage showing their knowledge, weaknesses and style, this transparency may be risky 4. one works in most cases for nothing but can find people with interesting characters.

@Andras, something about advances of RG. I absolutely agree with your statements. Unfortunately, the prompt or fast reply is only possible if you do not follow too many questions/threads. Everyday presence on RG portal will avoid delayed response! I have followed this thread but it is my very first answer to!

Dear Ljubomir,

We should focus on fewer questions. However, there are too much intersting objects and one waste too much time following many of them. We should select.

Dear All,

I would like you to read this article:

http://www.sciencedirect.com/science/article/pii/S0264410X13014576

The title of this editoral may concern all scientific areas, too.

Dear All,

Please, read this attentively

http://am.ascb.org/dora/

If you want to you can sign the Declaration of Research Assessment.

I want to share my opinion from the point of view of huge gap between Research and Practice. Each time I try to understand the following things:

1. What is the goal of scientific publication?

2. Why does exist a huge gap between Research and Practice?

3. Why Researchers are not interested to reduce this gap?

4. Why Developers don't use the best researches?

5. Why are fewer and fewer Developers participate in the dicsussions on RG? Where is Magel Kenneth and others developers which participated in RG?

6. How to connect Researchers and Developers to create Products for all people?

7. How to connect ResearchGate and CodeProject for the benefit of society? (I want to remind that CodeProject is the huge community for Developers, http://www.codeproject.com/ )

I see only discussions, discussions, discussions ... on RG, but I'd want to see any constructive solutions to reduce this gap. In my opinion, one of ways to connect research and practice is developing Online Tools. More detailed answer can be viewed from my answer on the link https://www.researchgate.net/post/What_do_you_see_as_the_future_of_scientific_publication_and_will_change_be_an_improvement

Gennady, I understand your frustration about specific groups working together on RG. One way is to find researchers here who share your views by directly approaching them to build a pressure group for what you think is important. Or, maybe, for publishing something worthwhile. Don't give up, as many say here to advance science and technology for the better of society and humankind.

And, please, could you give your view on the original question. Is the measure of impact how much it closes the gap you are demanding?

Hi Michael, I think there may be some conflation of what the IF is. It is in fact a Journal Level Metric, not an Article Level Metric. So Professor Clever may publish in a journal with IF 3.0 but it says nothing about his paper or research directly. Also there are disciplinary differences in IF so that 3.0 may be high in social sciences and humanities, low for the sciences.

The answer is probably to gather article level metrics from your publisher and post your article in a repository to gain article level metrics and volunteer them as part of your personal bibliography.

If I were to be very controversial I would say publish in the better OA journals where article level metric are already generated. As they are both public and online they are easy to demonstrate to your peers and colleagues. BW Matt

Matt, your comment is very important by reiterating the meaning of IF and related measures in academics. And, surprise, there would be no big controversy over publishing in a shiny and well reputed OA journal. These won't have IFs, I suppose, but maybe SNIPs and SCImago meters.

PLoS could be an alternative if it weren't that expensive for the majority of researchers outside US, Canada and EU institutes (Australia?).

And (thanks again, Matt) your keyword sparked me to search for ALM. So I found the link below, which maybe one of the answers to the original question.

http://sparc.arl.org/initiatives/article-level-metrics

Hi Michael, yes OA I agree they are expensive. I have been looking into publishing an article in an OA journal. As this is more as a personal experiment to understand the process I will be paying myself. Cost is a limiting factor.

Biomed and PeerJ and probably a number of other publishers offer waivers for countries in this group Low Income Countries, defined by the World Bank [ http://www.biomedcentral.com/about/apcfaq/developingcountries ].

PLOS waive fees from Low Income Countries as well. They also charge a flat fee of 500 USD for Group Two, Developing Countries, which includes Thailand. See here [ http://www.plos.org/publications/publication-fees/ ]. I am not saying this is not a lot of money but it is one third the "First World Cost"! BW Matt

Dear Maxim Kotsemir "What is the criterion of newness? How can we measure the extent of newness? How many new ideas we can detect from the paper? What is “new idea” at all?", are very hard questions to answer in simple sentences, these questions were subject of bold discussions in philosophy of science for many years. As you know better than me, it is enough to take a look at the debates between the believers in Paradigm Shift theory proposed by Thomas Kuhn and believers in other schools of thought like Karl Popper theories. I think if we are going to follow your discussion (which I think we should), the discussion would become too fundamental in definitions of scientific progress and improvement that even we would need a new question topic to focus on it. What do you and other dear colleagues suggest?

Thanks to Maxim we're coming gradually to a more fundamental and critical assessment of numbers as a measure of scientific impact in this valuable discussion so far.

Babak, somewhere in my literature stack is a paper about writing articles. It's suggesting bringing forth one idea per paper (not more), otherwise you'd overload the publication process. So, one paper = one (hopefully) new idea, if we follow the advice.

This reduces (or expands) the original question to 'What is the scientific impact of a new idea?' Does this reasonably address your concerns?

@ Micheal, good point. Of course it does :)

Individual researchers can be assessed by the h index rather than the impact factor.

Hi Dear Michael

I think the impact factor is very important to other index