As a researcher, you have done some research. You have sent your paper to a good journal, and (after a few improvements) the select referees and the editor accepted your paper whose title now appears in your CV.
Must the promotion committee at your university now go and re-evaluate your paper, by calling in a new set of world specialists? No. while idealistic, that is an impossible duplication of work. So, a simple count of your papers is all that is possible. The advantage of such as system is its simplicity and that the evaluation can be done by non-specialists.
All research should be top-quality and it is pointless to split hairs about quality. In practice, few researchers are Nobel prize-winners.
As long as output is in good journals, national research output can easily be gauged by counting the total number of papers that researchers in the country produce.
The number of graduating students supervised to fruition is another important measure of the worth of the researcher at promotion time.
Ian - There are some countries where the process is different from what you have described. In order to be promoted, my colleagues in Taiwan are required to submit one of their published papers to the Ministry or Education and it is part of an overall package that goes out for peer review. So you can say that the paper is reviewed again, as part of the larger promotion package.
Michael W. Marek Thanks, Michael. That is interesting and useful approach in Taiwan. Through a post-publication peer review, the South African National Research Foundation rates top researchers into categories, which can be leveraged for more pay. But I am dubious about the validity of local peers from other specialities being involved in the rating process.
Well I disagree with the mere count of papers. Quality and standard of the Journal should be the criteria. Remember there are plentiful predatory journals that publish papers without recourse to rigorous peer review process.
Quality is not always easy to assess. This is a complex issues - but both are obviously important. The world of publishing is changing and it is important to consider also impact/access.