I would suggest to adopt a standard metrics that is a must to present in each university. The accreditation system for universities have established this criteria, easy to found on the net. This part can be similar accross the universities for the undergraduate levels.

In addition, if not all universities have graudate level, you exclude those universities and only compare the ones which have it. 

you may need to use the following criteria

Ratio between studnets and faculty member 

Diversity in the degrees sources for the faculty members

Faculty memers from other countries

Number of publications in distinguished journals {ERA, ISI..IF, etc..}

International studnets

Availablility of electronic resources/subscriptions to full text journal, WiFi services..etc.

Graduates employability

Facilities at the University campus

International Ranking such as QS and Webometrics...etc.

Hi Mulubrhan,

I function as an assistant vice president for research at my university and we use many of the metrics that you have suggested.  In the United States, two widely used metrics are 1) the annual National Science Foundation Research and Development Expenditures that the universities report and are verified, and 2) the number of Ph.D. students graduated each year.  However, you may not have the systems in place to make those comparison in Ethiopia.  We find that number of publications in refereed professional journals and the number of citations (but not self-citations) from these publications are the next best metrics.  Your use of patents and licenses are also metrics we are using more and more frequently.

I would suggest to adopt a standard metrics that is a must to present in each university. The accreditation system for universities have established this criteria, easy to found on the net. This part can be similar accross the universities for the undergraduate levels.

In addition, if not all universities have graudate level, you exclude those universities and only compare the ones which have it. 

you may need to use the following criteria

Ratio between studnets and faculty member 

Diversity in the degrees sources for the faculty members

Faculty memers from other countries

Number of publications in distinguished journals {ERA, ISI..IF, etc..}

International studnets

Availablility of electronic resources/subscriptions to full text journal, WiFi services..etc.

Graduates employability

Facilities at the University campus

International Ranking such as QS and Webometrics...etc.

Dear Mule

Study by John has classified science into two major groups namely Pure Science and Applied Science as a basis for his study. He has used total publication, average publication per year, average number of scientists and average publication per person per year as parameters of comparison. (for details attached is the paper)

However, because of the fact that one size does not fit all, the author mentioned that these are not comparable estimates. He said this because no two journals are the same and this makes the comparison difficult but you can mention the drawback of the process and use this method for your study.  

As to me, it would have been good if collect more data for instance if the patented work is being implemented/disseminated to the society, number of research grant gained over 5 years (you can use the monetary value for the ease of comparing). KTT (Knowledge and Technology Transfer) and community services, you can confirm this by asking the beneficiary community. You can also see the University community Industry linkage which is give due attention by the Ethiopian government. You need the view of the industries and community on this issue.

It is also important to use index to make the comparison in the presence of variation parameters, you can you yes if present and no if otherwise or else using Likert scale (with highest and lowest value) and make the index accordingly.   

What do you mean by “the evaluation parameters used across many studies vary greatly?”Do you mean that the other studies used different parameters than those you have collected? That depends on the objective of the researchers and you can still compare using the parameters you have already collected.  

Hello Mulubrhan,

additionally i would suggest you include students admission and graduation, student-Lecturer ratio, number of functional facilities, research funding and grants, indexed research publications, subscription to research journals,  webometrics and other international ranking. This should be across universities with same standing.

Best wishes,

Chris.

Dears Christopher Anierobi, Jack S Damico, Dawit Mekonen, Muayyad Ahmad 

Thank you so much for the ideas. I can now see there are a lot of parameters we can use for comparing universities. However, most of the parameters you mentioned like number of PhD students, employability of graduates etc. are more of indicators of academic quality and performance than research. Are these directly related with a research performance. I mean they can indirectly be related to a research excellence and performance in a University, but they do not necessarily measure performance in research. I am still looking for a standard (or frequently used) methodology or parameter or variable developed for measuring research performance. 

There is no such thing as the research performance of a university. Universities don't do research, researchers do. What is currently measured as 'research performance' (and yes, its fashionable) is an aggregate of a university's academics' research performance in their respective fields.

Indicators of research performance are sensitive to fields' research and publication practices (think of the varying role of journal articles in the sciences and humanitities, or of external funding in pure mathematics and experimental sciences). From this follows that any indicator used at the unviersity level is highly sensitive to the composition of the unviersity's research effort. For example, the indicator 'external funding' is quite common as an indicator of research performance in Germany. Applied to unviersities, it is strongly correlated to the unviersity having amedical faculty (because biomedicine attracts a lot of external funding). Whether this says anything about actual performance is not clear.

Thus, the first step would be selecting universities with similar research profiles for comparison. The second step would be using a sensible evaluation procedure. There are no indicators that would not do injustice to some fields. For example, citation-based indicators are not valid measures of research performance in many fields due to the limited coverage of publications in the Web of Science.

For these reasons, if one absolutely has to evaluate universities, peer review (at least in the form of an expert committee looking at the numbers) is the only choice if one wants some validity. See also the Leiden Manifesto on Research Evaluation.

@Jochen Glaser,

Thank you for the interesting insights. Yes, we indeed had some difficulties when we tried for example to compare 'average impact factors' of journals across colleges or departments in our university. It was clear that such measures were dependent more on the discipline than actual effort of the researchers or the units. However, I believe keeping other factors constant, it is still possible to compare units or universities using some of the measures such as citations, impact factors etc. For instance, here in Ethiopia, our universities follow a fairly similar system and research approach and share similarities in terms of the units and disciplines they host. Therefore, I think it could still be safe to compare our universities using some 'objective' measures such as number of patents, Intellectual property rights, publications, impact factors and citations.  Thank you once again for the interesting insights.