What is open peer review? Peer review didn't exist before not a so long time ago, and publications were of better quality, without having to wade through all the "not even wrong" papers. Science has become more administrative than scientific, and the scientists act more to preserve their reputation than to be creative.

This question has something out of place, since the format of the publications is bound to change. There is no more the constraint of paper, there is no reason to find again these series of articles from the same author, in different journal, and differing only by some more development in the same vein. There could be "live publication" in the likes of wikis. The HTML format, initially developed for scientific publication, allows to regroup materials from different source in the same document, for example a figure extracted from another article elsewhere in the Web, or even from another author, into a new text. Article need not be self contained if they use hyperlinks.

We have then a choice: keeping peer review, with old fashion and inefficient way of publication, even if n.0, or moving forward, drawing from collective intelligence and trusting the ability of the peers to judge a contribution for and by themselves. In any case, there will be lot of articles, possibly of high quality and very interesting, that will be published avoiding any form of peer review. Science is not a game in which points delivered by some authority have to be earned.

thanks for bringing up this question and for the very interesting links! (double blind) peer review is definitely a very important cornerstone of science and i think it will remain so despite the rise of open access. the conclusion of the nature article is quite remarkable: "Feedback suggests that there is a marked reluctance among researchers to offer open comments." it would be very insightful to conduct a similar experiment nowadays, as the article is already 7 years old. are researchers now more open-minded towards more experimental forms of peer review due to social media, open access, and the open science developments in the last couple of years? i suspect that nothing much has changed, though. many/most authors are probably still quite skeptical towards open peer review.

in my opinion, one of the biggest issues of the current peer review system (and i'm really not an experienced scientist, so my statements are not generalizable or representative in any way) is the lack of transparency and openly available information. this includes points like:

- (average) duration of the peer-review process

- acceptance rate

- level of detail of the reviews one can expect as an author

- number of reviewers assigned

...

many journals do not offer such information. this makes it difficult for less experienced researchers to target the right journal and they have to rely on the - often incomplete and biased - knowledge of peers or senior researchers familiar with the journals' review process. it can lead to frustration, e.g. with the long process or large variance of feedback. a more transparent and standardized documentation could help to improve such issues of journal targeting - and would probably lead to less wasted time and effort for both the reviewers and submitters. the peer review process wouldn't need to be open but journals could think about offering an (open) option for interested authors and reviewers.

furthermore, post-publication metrics, including peer feedback, will become increasingly important. this does not only include citations but also webometrics and altmetrics (shares, likes, comments, views, downloads, comments...), which will serve as post-hoc peer feedback.

two good examples that try to address some of the problems of the current peer review system are "sociological science" (http://www.sociologicalscience.com/journal-information/) and plos one (http://www.plosone.org/). both of them have short review and publication cycles and are open access.

Dear Artur,

last year I submitted a 2.0 dataflow devoted to PR in scientific publication and open-sc-journalism. We gave the name of CircÓs OJ.

Its synthetic scheme is here: https://www.researchgate.net/publication/257701693_Triple-Helix_Complex-Scheme_Open-Mediateque-Interoperation?ev=prf_pub

and brief rational at:

https://www.researchgate.net/publication/257438897_ThePrimate__-HPW_and_NNs_Ecosystem_based_on_DDC_classification_oriented_to_singularities'_peculiarities?ev=pub_cit_inc - 6 : 3.5 (2012)

A prototype (temporary not-) running the mentioned mechanism is at http://the13.theprimate.it, and a preview at http://staff1.iam.theprimate.it/

We are facing shortage of funds to finish/launch it [it uses complex genetic algorithms], for this reason is largely inactive.

The CircÓs's scheme is freely usable.

g

Data Triple-Helix Triplestore Complex-Scheme Open-Mediateque-Interoperation

Conference Paper ThePrimate ® -HPW and NNs Ecosystem based on DDC⁺ classifica...

What is open peer review? Peer review didn't exist before not a so long time ago, and publications were of better quality, without having to wade through all the "not even wrong" papers. Science has become more administrative than scientific, and the scientists act more to preserve their reputation than to be creative.

This question has something out of place, since the format of the publications is bound to change. There is no more the constraint of paper, there is no reason to find again these series of articles from the same author, in different journal, and differing only by some more development in the same vein. There could be "live publication" in the likes of wikis. The HTML format, initially developed for scientific publication, allows to regroup materials from different source in the same document, for example a figure extracted from another article elsewhere in the Web, or even from another author, into a new text. Article need not be self contained if they use hyperlinks.

We have then a choice: keeping peer review, with old fashion and inefficient way of publication, even if n.0, or moving forward, drawing from collective intelligence and trusting the ability of the peers to judge a contribution for and by themselves. In any case, there will be lot of articles, possibly of high quality and very interesting, that will be published avoiding any form of peer review. Science is not a game in which points delivered by some authority have to be earned.

I am not excited by the ideas of open peer review or credits for reviewing papers as substantially improving peer review - so am interested in hearing your thoughts and ideas other possible improvements https://www.researchgate.net/post/How_do_you_think_we_might_we_improve_peer_review_and_the_quality_of_scientific_publications

@Christoph: Not sure about your field, but in mathematics the American Mathematical Society (AMS) publishes once a year in the Notices of the AMS a backlog of math research journals which contains the information about the (selected) math journals roughly of the sort that you would like to know (average time from submission to acceptance and to publication, number of the papers published per year, etc.). Here is e.g. the link to the 2013 backlog: http://www.ams.org/notices/201310/rnoti-p1390.pdf . Perhaps some professional society in your research field collects and publishes similar information on the relevant specialized journals.

@Artur: yes, that's exactly what i meant! i'm not aware of a similar list in my field of study (communication, media, is, sociology). there are only impact factor rankings and lists with other quality criteria but not in terms of the review process. this would be very helpful.

in the list you posted i find it quite worrying that some journals need 25-30 months between submission and publication. but the good thing is that there exists a lot of variance. so, reserachers can choose if they can afford to go for a long review cycle or prefer the faster (but maybe less prestigious?) option.

@Christoph:one reason why long times between submission and publication occur in mathematics is because some papers take very long to referee and some journals choose to not too push the deadlines for reviewers too hard. In the other sciences this is far less common, I think.

To Claude's point, things are changing in significant ways. There are problems with giving a (Nobel) prize to a maximum of 3 people - when many more where involved in the discovery. How science is being performed is changing - so too, should be the means of communication, including how and who is involved in the review.

How do we address communication of ideas and research that:

1) are produced by many individuals

2) are alive, changing quickly - especially with many people involved and changing significantly in the time lapse between being set down as a communication and being published

3) can be better communicated via avenues other than the two-dimensional word-based format of paper or electronic analogs of paper?

I think we have already started a transition to a different way of 'doing' science. The methods of communication need to adjust accordingly (which Claude presents some aspects of).

Will it be 'reward based', such as 'bonus points'? Who gives out the points - an editorial board? Amazon.com style reviewers? Answer.com style points? Do we have the solution yet?

I think we are in for an exciting (and rough) transition.

Multi contributors research can be handled by the same concept of wiki. All the modifications with their date and author(s) are archived, while the final document can be displayed. Of course, that would make the work of the "police of science" more difficult, since it is easy to cheat. But is it really about credit, or about the the advancement of human knowledge?

I think things can be done this way:

1. Submission 2. Initial editorial decision making (if they decide to reject at this stage they must explain the cause and will give the author a chance to defend himself) 3. Review (blind) 4. Author will response 5. Accept or reject (it may take more than one back to back review-defense based on the queries and response from the author) 6. Review report(s) and the author response will be available online. In the final published version of the paper there will be a statement like: "this article was reviewed by X, Y, Z and the reports are available at (URL)".

Michelle Willmers in her paper 'Blurring the boundaries: Expanding conceptions of open access and open education in the African context', examines open access and open education through an (African) scholarly communication lens. http://www.berlin10.org/programme/14-speaker-profiles/52-michelle-wilmers.html

Scholarly communication occurs across many layers and in various types of artifacts—email, conference proceedings, teaching materials (slides, images, video), grey literature, animations, simulations, and so on. Instead of simply focusing on formal research outputs, a more inclusive approach must be followed, working on openness through a single lens rather than through silos of OA. Focusing on the entire “mountain of content” when talking about openness is essential. Complete openness will allow the RESEARCH COMMUNITY to DECIDE ON THE RULES OF THE GAME, and peer reviewing should be included herein, rather than being a preserved silo of 'blind refereeing'. @Riaz Uddin, I therefore tend to agree with your suggestion.

As it happens a colleague of mine and I have explored this very question recently: examined some of the newer peer review models and proposed a role for repositories in this space. Refer: http://www.iiisci.org/journal/sci/Abstract.asp?var=&id=GA195DM13

Can one explain to me what is the protocol of peer review 2.0 exactly ?

Thanks.

The protocol of open peer review ( in http://www.nature.com/nature/peerreview/debate/nature05535.html) , looks much to my "a posteriori" peer reviewing I have develop in Poudres &Grains (see http://www.poudres-et-grains.ecp.fr/spip.php?rubrique1) where readers shall review papers they read (if they have something to tell).

[The rules of Poudres & Grains can be found in http://www.poudres-et-grains.ecp.fr/IMG/pdf/notice-poudres.pdf

and were explained in P&G No 13 - 2 (P. Evesque: Editorial : Why deciding to publish in Poudres & Grains : Towards a more efficient politics of scientific evaluation Translated by Henri-Thierry TOUTOUNJI pages 12-17 , Abstract)]

As a matter of fact, the editor received never some official criticism. Better he tried few times to ask CNRS , universities or French Academy of sciences to perform some offficial evaluation. Each organism rejected this opportunity.

From my practice, I do not feel these papers of P&G to be superfluous. Most of these papers were presented and discussed in official scientific meeting, some of these papers were rejected by normal journals (as Nature, Phys Rev, Europ J. Phys,...); unfortunately the rules do not allo P&G to publish these reports since it is considered as private correspondence.

I discuss these practices in 2 booklets of Testimony (1 & 2) which I gave officially to my lab where one can find them.

I read http://www.nature.com/nature/peerreview/debate/nature05535.html, which explain the protocol used by Nature to test open access peer reveiewing.

For me the real problem of this edition process is not get comments on what is not needed to comment, but to discuss the scientific bases and the understanding of the published papers. So it is normal to expect little number of report.

Please also, I would be gratefull to any comment on this can of edition, how to improve it...

It is fun, while P&G is not refered and not quoted, I know some scientist read some of its papers.

P&G is efficient to transfer scence and to maintain science at its upper level: I am fighting to present P&G as a scientific journal, but academia does not want it.

Well, the drawback with open peer review as of the Nature trial, is the 90% of not even wrong papers that won't get any review at all. That shows that a localized revamping of the organization won't cure its systemic deseases.

@Claude

The answer may be as follows:

1) There is no need to validate a tool if one does not know how to use it. Better a new tool (or a new way of thinking) shall be validate through new use.

2) The fact there is no report on a paper does not mean it is not validate, nor read. It means there is no objection stated at the moment.

3) Are papers published by good journals always true? No, and the more important the journal the more difficult being sure of the paper results. Each reader has always to play science and to do is job, for each paper.

4) I found that some very good papers can hide important information, just to pass referee in the field... This is junk!

Others were rejected (by good journals) because it was known, because clearly and correctly told!!!

Worse others was not published because there were not understood by two referees.

5) Evaluation of good research has to be devoted to science administration. Nowadfays this does not look correct, because when a researcher asks its administration for evaluation, administration counts peer review articles and refuse to play the game of real evaluation (this is true in France at least, but I feel it right in many countries)

6) Take the Thesis by deBrooglie, there was no use before knowing new results, so no way validating the part of right and the part of wrong...

7) Paper publication does not require validation, this is true in economy, in litterature, poesty, philosophy... why not in science.

Readers have to be advised readers always.

Wejust need improving education of readers.

Pierre, your 2) is just the "not even wrong," that means the paper is correct, but who care? Open peer review won't get rid of that useless literature that waste space and time, while it is precisely intended to maximize the quality. That's because of the reward system, not of the review one. The main interest of a paper should be on the reader side, not on the writer's, otherwise it is pure self-gratification.

Claude, that is always the case, even in "a priori" peer reviewed papers:

who grants the papers? only fools, or the reader who has to judge them before using them, or the scientific administration or "scientific" editors.

in France we would said : papers are "sgdg = sans garantie du gouvernement, it means without government granting.

Worse: Chief editors in medicine journals have to do their own job , because peer reviewing is not correct.

Who care that papers appear to be wrong after a while, i.e. see last papers in Poudres & Grains where I demonstrate that most papers published in Phys Rev Lett .... during these 20 last years have missed the crucial point of the boundary effect and boundary excitation, and that this efect plays a role even in the bulk....

Janie, I agree with you, we need to examine all the new possibilities.

But we need to keep in mind the rules for old success: free scientific debate, and clear evaluation about real facts (not biased by internal philosophical lens, nor biased to simplify mananaging and counting);

Joanna, I need reading first your referenceRefer: http://www.iiisci.org/journal/sci/Abstract.asp?var=&id=GA195DM13

Every review system will meet the same illness, a reward system that is based on short term evaluation. The attempt to cast science in an industrial process is bound to fail, because science has its own rules that are incompatible with it.

I believe you are partly right. But science was built up from discussion after reporting results in correspondences, in books or journals or in meeting.

The problem as a sciebtist is to force scientific administration to ensure similar trends and not only to count publications... Because doing so one increase the number of publications without imposing on real goal.

@Pierre Evesque I agree with you, science is much more than counting publications and as @Joanna Richardson and colleagues wrote in the mentioned paper, the current peer review process is not the best, but the best currently in place. So, we should not throw out the baby with the bathwater - we must improve the current system by adding to it, and do it in pace with current new developments. Someone also mentioned looking at a new expanded role for institutional repositories, other than just an archiving facility. I believe that there might be value in this idea.

Janie, and ..., I am not sure it is the best. How can young researcher kow read a good paper. These papers are difficult t read, number of hypotheses are hidden in other publications, these others publications are not always qoted for the real value...

I have examples of these effects.

Further Managers amplify the worth evaluation, because they count publications only.

We see a number of doubling Tripling papers... even when they are wrong.

We are facing too many numbe of publication and refereeing helps increasing this number, because of these publications are accepted, it is not the fault of the writer, but of the referees.

But we know that the probability is not 0 to get two referee saying yes for a wrong paper. As the writer can submit the paper few times indifferent journal a write who submits is certain to be published.

Old publication rules were not allowing this...

Who sees the number of publication incrasing in a posteriori reviewing journal?

Janie, and ..., (I apologize for typing errors)

I am not sure it is the best way. It was the best because edition was expensive.

Now, edition is still more expensive but nobody cares.

Moreover, (i) How can a young researcher know which paper to read ; how can he judge its trueness? Most papers from scientific litterature are difficult to read, numrous hypotheses are hidden, only quoyed through quotatin to other publications; these quotations are not fair in few cases so that these others publications are not always quoted for the real value...

I have personnal examples of these effects, described in my "testimonies 1-3".

(ii) Also Managers amplify the worth evaluation, because they count the number of publications only.

So we see a number of doubling Tripling of papers... even when they are wrong.

It is difficult to qualify a paper as wrong, because the authors do not want.... Hence it is difficult to stop some wrong research....

This is because managers do not want to evaluate...

(iii) Classic Peer review by Referees help destroying correct science: We are facing too many number of publication and refereeing helps increasing this number, since the authors think it is not their fault, but the own of referees, which let their work be edited, even when wrong.

(iv) In the present form of reviewing, the reviewing process is not limitating wongness: since the number N of submissions of the same paper to any Journal is not limitted, we know from statistic laws that the probability Ptot of overpassing the review process is 1 even if the probability of acceptance per refereep is small (p

Dear all,

What means peer reviewing when things as I will explain can exist in the world of research administration:

Pour information de la part de Pierre Evesque, directeur de recherche au CNRS

De l'art de faire soutenir une thèse internationale, en excluant du jury les 2 co-directeurs (advisers)

J'ai co-dirigé avec Mme Hou, professeur au CAS de Pékin la thèse de YanPei Chen sur la physique des gaz granulaires vibrés sur terre et en apesanteur. J'avais programmé une soutenance par vidéo-conférence pour le printemps 2013.

Mais comme j'ai été mis en congé-longue-maladie sans raison et contre mon gré en mai 2013, cette soutenance n'a pu avoir lieu. L'étudiante a simplement passé sa thèse chinoise.

Or cette semaine, le 6 février 2014, une soutenance par vidéo-conférence a été organisée sans que j'aie été officiellement informé. Ayant appris l'information indirectement, j'y suis allé pensant que ma collègue chinoise était en accord avec les autorités pour faire soutenir la thèse. On m'a sommé de sortir, ce que j'ai refusé en disant qu'une soutenance était ouverte au public. Par ailleurs, j'ai mentionné que je n'avais pas été averti, et qu'on avait refusé de me dire ce qui se passait.On a appelé la secrétaire générale de l'ecp accompagnée de ses assesseurs, qui m'a menacé aussi et a continué à dire que je perturbais la thèse!..

Par la suite, je me suis aussi entretenu par mail avec ma collègue M.Hou, cotutrice, et c'est là que j'ai appris le pot aux roses: ma collègue n'était pas là non plus comme membre du jury !!!

J'ai compris pourquoi ma présence pouvait perturber cette soutenance: je me serais vite assuré de notre cohérence de comportement avec ma collègue, et la soutenance devenait très problématique.

Que supposer ? que certains ont besoin que la thèse soit soutenue (pour enrichir une liste de thèses soutenues et justifier des financements, ou pour toute autre raison), mais veulent éliminer le directeur de recherche - qui a le tort de militer pour le respect de la déontologie dans la recherche

Je n'ai rien contre ma thésarde chinoise, YP.Chen, bien au contraire. Elle a toujours très bien travaillé et elle a vu la difficulté d'affirmer des résultats, de les comprendre, d'apprendre à lire et à critiquer des articles scientifiques ; elle a appris l'autonomie...

Il est inadmissible que l'administration, gouvernée par des scientifiques qui ont perdu leur âme et ont pris l'habitude d'évaluer le travail des autres en comptant des publications, sans même savoir si ces publications sont justes ou erronées..., puisse perturber l'avenir d'un jeune talent en la forçant à accepter des procédés inqualifiables.

Merci de diffuser cette information

Pierre Evesque

Documents

1) message que j'ai adressé à la secrétaire générale de l'ECP après l'annulation de la soutenance

Rapport des faits

Excusez-moi de ma "ténacité" de ce matin. Le problème que je rencontre est celui du harcèlement par refus de déontologie; et ce symptôme est une de ces caractéristiques de mon harcèlement.

Faire croire pour cela que je ne peux plus faire de la recherche est un non sens: il suffit d'imposer une déontologie partiellement correcte.

Celle-ci ne peut exister qu'à travers un "tribunal déontologique" indépendant. Comme cela est partiellement fait au CNU.

Pendant la discussion de ce matin, il a été aussi décidé par le directeur de l'Ecole doctorale de l'ECP, présent, que s'il y avait soutenance de thèse de Yanpei hors ma présence, on me ferait une copie de la soutenance, et on m'enverrait copie du texte soutenu.

2) échange de mails avec Mme Hou, co-directrice de la thèse [les adresses mail et les noms propres ont été occultés ]

message de Pierre Evesque (06 février 2014)

Dear Meiying,

>

> Sorry for what happened this morning.

>

> But you see, the administration knew it was wrong: if they could do it,

> they just have to throw me away, asking for the police or for the fireman.

> They ask me to call "le délégué CNRS". They should have to do it

> themselves, if they were right.

>

> So what you see is some administration which tries to obtain wrong

> things, using inadequate ways.

>

> This is the demonstration of what you shall not accept too. Please help

> me,; they will be forced to regularize much faster.

>

> If you do not , and if you follow their bad advices, this will give me

> more job; but I am right; so I cannot let them pass, and will do the

> necessary...

> Up to you.

>

> Remenber, Meiying, we got no good experimental data for a while from

> Airbus, because X did not want to get good illumination. He made us

> wasting lot of time (we discussed about this in Beijing).

>

> Please ask for fast regularisation:

> first I can be in the jury, even now....;

> second all can be stopped at once, because the way they did all is quite

> wrong.

>

> Do not accept a new date.

> Best regards

> Pierre

Réponse de MeiYing Hou (7 février 2014)

Dear Pierre:

For Yanpei's thesis defense, I was not invited either. I sat in out of courtesy.

No one had ever discussed the date with me yet. If they do, I will help.

3) lettre adressée à M. Houssin, directeur de l'AERES [les noms propres ont été supprimés]

Monsieur Houssin,

Vous trouverez ci-joint la réponse de ma collègue chinoise, codirectrice de la thèse de Yanpei, à mon e-mail d'hier faisant le point sur l'ajournement de la soutenance de notre étudiante.

Vous verrez que ma collègue n'a pas été amenée à donner réellement son avis librement, et a été forcée à s'asseoir.

Vous savez, que j'ai voulu assister à ce que j'appelle cet "imbroglio de soutenance de thèse", et que cela a abouti à son ajournement, (et j'espère plus). Je comprends maintenant pourquoi ma présence était insupportable aux autorités ecp présentes (M. G, Mme B, M. D) et non présentes (M. B, M. B).

Je demande pour le moins qu'une enquête soit ouverte sur les conditions mises en place. Il me semble totalement anormal qu'une thèse se passe sous de telles conditions, sans la présence réelle et libre des directeurs de thèse.

Je pense que c'est une faute grave, qui même si elle n'est pas courante, et bien sur si elle est confirmée, relève d'une volonté pour le moins de refus déontologique, et doit être sanctionnée par une dés-habilitation du cursus de 3ème cycle en complète autonomie.

Je constate aussi que cette faute n'aurait pu avoir lieu sans la volonté passive et active du responsable de financement CNES, M. Z. et de mon collègue, G.

Je ne comprends pas comment on peut vouloir dés-éduquer un thésard de cette façon; il y a de la perversité à.cela. Bien sur, ce que dit M.Hou peut être faux, ou mal interprété; cependant, j'ai une longue habitude de son travail et sait qu'elle respecte sa parole.

Au contraire, je connais la duplicité de l'administration ECP, qui essaye systématiquement de forcer à son profit les évènements, par des abus de parole et d'actes; j'ai d'autres exemples à vous donner, si vous le désirez.

bien cordialement

Pierre Evesque

Sorry for talking French . I Will translate it next time

> --

> Pierre Evesque

> DR CNRS en congé de longue maladie pour "raison d'état",

> i.e. en recherche de ses pairs

> ===============

> avant-hier j'avais des camarades, hier des collègues,

> aujourd'hui des concurrents, et demain ?

> ===============

> [email protected]; [email protected];

> tel: 01 43 50 12 22

> ____

> http://www.poudres-et-grains.ecp.fr/spip.php?rubrique1

>

English translation:

How to impose a thesis defense, by excluding from the jury the 2 advisers; a French specificity?

I have been supervising a PhD thesis with Mrs. Hou, professor from the Phys.Dept of CAS in Beijing, on the physics of granular gases vibrated on ground and in weightlessness. I and M.Hou had programmed a defence through videoconference for spring 2013.But as I was put in long-leave-disease without reason and against my will in May 2013, this defence could not take place. The student defended simply its Chinese thesis.

The long-leave-disease was asked by cnrs, to support the will of ECP, because I have been asking for more than five years to support real research deontology... and to evaluate Poudres & Grains papers (the journal at http://www.poudres-et-grains.ecp.fr/spip.php?rubrique1).

That's all. The medical committee tells that iwas feeling harassed due to the will not to evaluate Poudres & Grains papers.

However this week, exactly on February 6th, 2014, I heard that the defence through videoconference was organized without me; I was not officially informed, just by some bias (indirectly). I went to the room to see, thinking that my Chinese colleague was in agreement with the authorities and support the PhD defence of the thesis. One was summoned out to leave, that I refused saying that defense was open to the public. Furthermore, I mentioned that I had not been warned, and was refused to tell me what passait. They called managing director (secrétaire général) of ecp, accompanied by his assistants, who threatened me and also went on to say that I was perturbing the thesis! The defense was adjourned.

Subsequently, I also spoke by email with my colleague M.Hou, the coadviser; so I learned the "pot aux roses": my colleague was not there either as a jury member! !

I understood why my presence could disturb this defense: I would quickly assured me our consistent behavior (with my colleague) and the defense became very problematic!

What can we assume ? Likely few need that the thesis is supported (to enrich a list of theses and justify funding, or for any other reason), but want to eliminate the real advisors - who is wrong because of advocating for the respect of ethics in research.

I have nothing against my Chinese PhD student, YP.Chen; quite the contrary. She has always worked very well and she saw the difficulty of affirming results, of understanding papers of learning how to read and criticize scientific articles; she learned autonomy in research...

It is unacceptable that the research administration, which is governed mainly by scientists who have lost their souls and have become used to evaluate the work of others by counting publications, without even knowing if these publications are right or wrong ... can disrupt the future of young talent by forcing her to accept unspeakable methods.

Thank you to share this information.

Pierre Evesque

I can only translate the documents which I added in the French version.

Just today, ResearchGate launched a new feature: "Open Review". It will be interesting to see how (and if) it's used and whether it will succeed. https://news.researchgate.net/index.php?/archives/187-Peer-review-isnt-working-Introducing-Open-Review.html

Perhaps, an electronic process can be developed to monitor the review process.

Open peer review system

I agree with Dr Christoph Lutz and the links suggested by him give clear understanding of the system.

Open peer review - No gatekeepers

In the proposed system, things would be different when a scientist submits a paper for peer review. Rather than submit it to a journal, they would submit it to some open website, online system, or group of non-affiliated websites that would store it, but not show it publicly yet. It is even technologically possible to make a fully decentralised system to do this, much like how it is done with the TOR network and other P2P networks. Invites could automatically be sent out to researchers that are linked close but not too close to them in the network described above. They could also choose to invite researchers not in their immediate network. Now when this happens, the abstract say could be submitted publicly, but the full paper sent privately to those chosen researchers.  After say three accepted it for review, no other people would need to review it. The reviewers could then offer suggestions etc like usual, but at the end of the process, they would give the paper a rating. This could be a score like 1-5 or just accept/reject.  Their comments and rating would be made public so they would be accountable. (Perhaps reviewers that reject it could stay private but I think it best if not) The paper would then be published for everyone to see on the website (in the appropriate discipline) with its rating and feedback. You can of course play with the numbers and require one, two etc reviews to accept it before it is published. Now one advantage of this is that there is no gatekeeper like the journal editor. So long as someone credible in the field saw value in your work, then it would get published in the peer reviewed section of the site.  The website would be free and open with anyone able to upload papers for review. Ideas that were unpopular could not be suppressed by people that did not like them...

http://www.science20.com/thor_russell/blog/how_improve_peer_review_process-84552

Dear Claude, Dear Ljubomir , Dear All,

It seems to me that scientific literature shall accompany the building of science. So it shall not reproduce the big past errors. Scientific literature shall always convince, not judge as a religion. We (or our science literature) call ourselves “homo sapiens”, and we shall keep on. We cannot accept to be told “homo diabolicus”, (errare humanum est, sed perseverare diabolicum). This is why a debate is beter than a non appropriate decision

And what it is true for scientists, shall also be true for their representatives: administration and journals.

One shall remind that the system published by Copernicus at the end of his life, was already told by “Aristarque de Sammos”, and it has been denied by Aristote & Platon !!!! and lost for 2000 years.

Worst, both systems are maths equivalent (but nobody knows it at that time likely). So it is just a question of point of view (and who knows the real point of view of God, is it really linked to earth , to sun or to some where else). These are the troubles we have to face.

More recently similar problems occurred (see for instance the Nobel price on quasi crystal,…)

The point of view of scientific journals, of administrations have to obey the point of view of correct researcher. That’s all. It shall facilitate discussion, which is contrary to lobbying.

So, I have tried , since the nineties, to evaluate scientific literature (journals) from my own experience and asked them to change. This never happens…. Journals were always supporting that it was not from their owns, but from other authors… It means that journals do not play their part seriously.

Journals have to make effective links in between concurrent teams…. A paper which is told false (and not publishable) by one referee, and told evident (so non publishable) by another shall have the standard “urgent for publication” and open “to discussion”. This will never be the case with the rules used now, whatever the form of peer reviewing.

This shall change.

During these 20-30 years, I have reviewed a number of their errors into a series of “temoignages” to show the different problems, issued from either journals, or administrations… I have transformed a small journal “ Poudres & Grains” used as a link for a meeting occurring every 4 years into a journal with an “a posteriori” reviewing in 1997. Of course the papers published in it can have been reviewed by other groups first before edition. But the decision of publication is linked to the author, not the editor. The editor has just to play the role of transmission of the information; It has also the role of pursuing the evaluation…

I have been asking for a direct evaluation of my works in Poudres & Grains by CNRS, my employer. This never happens. If these papers are wrong, CNRS should have claimed the reality. …

Also I have been working in space research and published these works on granular material in Poudres & Grains. I have used European rockets,, Airbus 0g, Chinese satellite and planning ISS experiments. CNRS wanted never to evaluate these works on granular gas. It has preferred to run a USSR-like medical committee in 2012 to put me out of work. I have no complete access to the documents of comité médical ; and I have the proof that list of these documents is false. Furthermore, my papers in Poudres & Grains show that the model of granular gas used to mimic the granular medium does not obey the second law of thermodynamic. I recall here that Carnot published is “Thermodynamics” principles from his own, without refereeing. I did not win my lawsuit, but I submit my “scientific témoignages” to the court so that they have status of public mail that everybody can use.

If some of you are interested, they can load my “témoignages” at http://defense-pierre-evesque.over-blog.com/2014/11/le-probleme.html and load at the end of the page the pdf:

Evesque Testimonies : (most of the letters are in English)

• Témoignage n°1 : problèmes éditoriaux et de reviewing )
• Témoignage n°2 : sur les actions de reconnaissance de Poudres et grains
• Témoignage n°3 : sur la déontologie et le respect des normes d'évaluation
• Témoignage n°4 : principe d'utilisation des témoignages - De la science à la « réalité » administrative et aux « nouveaux scientifiques de renom »
• Témoignage n°5 : sur les harcèlements subis
• Témoignage n°6 : sur le harcèlement en 2013 jusqu'au Comité Médical
• Témoignage n°7 : (fin 2013) harcèlement lié à un comité médical "à la russe"
• Témoignage n°8 : en cours de procès

So this literature (i.e. my “temoignages”) can be used by everybody as a public discussion on what to avoid, even if the journals do not accept them as such. (see http://defense-pierre-evesque.over-blog.com/2014/11/le-probleme.html, at the end of web page).

Any paper shall be evaluated; some of them are hurting their peers; most of them do not interest their peers who do not know what to do with them. Few of them interest other peers when a peer has already used it…. And most of the cited articles are quoted from the quotation of another paper in the literature (this is not the right way, but it is so).

Since the CNRS decided so I consider myself out from the scientific literature of the homo diabolicus ….

And I do not know with whom I can/may discuss.

I am reading la Recherche (the French magazine) and was interested by “les controverses scientifiques” (special issue of December 2017). What is important in the controversies are not the controversies (because it is normal they are before understanding completely the way the theory applies); the only problems are those from administrations, from a priori judgments, …., which were not treated in this special issue.

In research, one cannot discover (and judge) what is not known. So any open or closed reviewing will be wrong if it is “a priori”. The truth shall come from “a posteriori” discussion.

And the better or the newer the scope the more “a posteriori” the proof.