Personally, I don't consider publications in journals and participation in conferences to reflect the quality of higher education. It could reflect the quality of research, but not education, and you don't have to be a great researcher to be a great professor (although I know that many would disagree).
If I had to measure the quality of education, I'd look on the following:
I'm sure others would add more indicators, of course.
Personally, I don't consider publications in journals and participation in conferences to reflect the quality of higher education. It could reflect the quality of research, but not education, and you don't have to be a great researcher to be a great professor (although I know that many would disagree).
If I had to measure the quality of education, I'd look on the following:
I'm sure others would add more indicators, of course.
I would add to the preceding answers that the importance given to research versus teaching can vary with the level of studies. In case you are teaching at bachelor level, the teaching quality is most important. But students in master years must be acquainted to recent developments and so it is important that their teachers be true researchers.
in my view the following indicators can be used
Employability of graduates
Impact on society
Connect with industry
Satisfaction of stakeholder
Quality of faculty and quality of students
Contribution of alumni
Well, the very purpose of a university is to create and disseminate knowledge, to equip society with necessary skills and knowledge to solve problems and live a better orderly life. The name university itself tells what its purposes are " places of teachers and scholars" implying to teach what are known and established knowledge and to research and create new knowledge to expand the realm of knowledge that might be needed to solve potent problems.
Therefore measuring the quality of a university has to be seen from these perspectives aside from its handling of the people it hosts, the freedom they have to think free, say and write freely and live better.
Hei Lydia
There is a literature, and you will get lots of hits if you simply google ´Quality in Higher Education´. Having said that, the starting point must be to determine what is defined as a quality indicator. This is a contested issue, but it can be intelligently addressed. I suggest that you begin by writing, ´For present purposes, I shall regard the following factors as indicative of high quality in ... etc. This is necessarily a selective and a partial decision, and here are the grounds for why I make it´.
Good luck. Paul
Please go through our forthcomming paper
https://mpra.ub.uni-muenchen.de/63658/1/MPRA_paper_63658.pdf
Please go through our forthcomming paper
https://mpra.ub.uni-muenchen.de/63658/1/MPRA_paper_63658.pdf
I believe we should consider the following:
Your question is on how to measure quality in higher education but other than research and publication. I find the question important because the latter (research component) is usually given most of the weight in assessing quality of HE, e.g., in global universal rankings.
Jamil Salmi has written a very personal overview of the situation:
Salmi, J. (2014). Global View on Tertiary Education. Retrieved from http://tertiaryeducation.org/2014/07/world-class-universities-or-systems/
If you are up to research in this topic, I think it is important to start with an assumption that there is no such a thing as the ‘golden yardstick’ or perfect assessment from KG IQ to global MBA rankings.
If I were interested in measuring quality of HEIs other than research output, I would look into existing key global university rankings (namely THE, QS Queensland and Shanghai Jiaotong) and tease out all the parameters except the research component. For example, Shanghai Jiaotong gives a lot of importance to high profile alumni.
Students' performance and satisfaction rate may help. Employment records of graduates and staff satisfaction rates may also help.
Dear Lidia, I embed a link to a report by the International Council for open and Distance Education (ICDE) that provides a global overview of quality models suitable for assessing quality in open and online education.
http://www.icde.org/First+global+overview+of+quality+models+in+online+and+open+education.b7C_wRvG43.ips
I can understand and agree with the argument that the expropriation of terms from the business world into the area of public services like education and health are fraught with dangers. That said, seeking to measure outcomes, such as good quality medical care and effective pedagogic and physical landscape design for wheel chair users in colleges, can and do help to improve the experiences of patients and people with disabilities. So we must not shy away from the notion of high quality. Instead, we must make it much more than a business world mantra by turning it into a robust and user-friendly measuring rod. Consider again provision for disabled students at college. It is not uncommon in some countries to have wheel chair users involved in the quality control of wheelchair design, and we need need more of this kind of innovation. It properly confirms the expertise of disabled students and it successfully applies this knowledge in appropriate settings. If we do not measure or ascertain levels of success and simply rely on general impressions, then we do not serve our students well. In addition we risk remaining stuck in what Max Weber described and lamented as the ´realm of the vaguely felt´. Best wishes and good luck Lidia, Paul
Dear Ivo
I agree that education should not be commodified. The commodification of culture in the neo-liberal world is a curse. Your views on the application of Weber´s ideal type are not exhaustive but, if I may say so, a little restrictive. By the way I cited him from The methodology of the social sciences (1949), not from Wiki! I am amazed that you conclude that Weber´s ideal type is unable to distill theory from action. It does precisely that and lots more along the way. l have used the ideal type in an article that compared teacher education in Norway and teacher training in England. This is downloadable on Research Gate, if you wish to consult it and give feedback. Best wishes Paul
I think Paul and Ivo share a lot in common.
"I have nothing against good management of universities, demanding of excellence, public evaluation and so forth" by Ivo is perfectly compatible with usual administrative practice of managerial effectiveness evaluation. The latter is expressed by Paul as "If we do not measure or ascertain levels of success and simply rely on general impressions, then we do not serve our students well."
There are clear tensions here (both at theoretical and practical levels) and they have been discussed critically. I can think of two key ones:
Ginsberg, B. (2011). The fall of the faculty : the rise of the all-administrative university and why it matters. Oxford: Oxford University Press.
Giroux, H. A. (2014). Neoliberalism's war on higher education. Toronto: Between the Lines.
"... I don't consider publications in journals and participation in conferences to reflect the quality of higher education."
I agree. However, in my country (Croatia), to teach at the university, you must first be "a scientist". These degrees go (in my translation) from "research fellow" to "scientific adviser". You must first be a scientist, and only then you can be appointed a teacher (professor). Nobody asks you whether you know *what you teach*. If you have enough published papers, you are good scientist, and then you can become a professor; without this, you cannot. This is not good for education, but that is how it is here.
Research results (that is, publication!) have become the measure of quality for everything; the quality of teaching has been neglected.
Dear Ivo
Many thanks for your insightful reply. Like you, I am wary of measurement madness in the educational sector. Interestingly, in Finland, where adolescents consistently achieve high comparative scores in language, maths and science, students are not assessed through tests until they take public exams. Best wishes, Paul
I believe research and the quality of research papers by faculty staff do matter. But Quality in Higher education does not limot itself to these two indicators. I believe that the entry requiremnts, the enrollment rate, the selection process do reflect the Quality. I should also add the quality of successful students from the institution, how many of them are being recruited by employers. HE institution should keep track of these outgoing students and their recruitment could also be a potentila indicator by relating to how many of the studenst are employable in the market thus including a REAL currency of their programmes.
Un indicador de calidad de la educación superior es la capacidad de producir conocimiento. Por eso un buen profesor será quien, entre otras cosas, enseñe a los alumnos a investigar y producir nuevos conocimientos.
I disagree that quality of research be discounted as a measure of education efficacy. A tremendous amount of effort goes into successful publication in peer reviewed outlets. These experiences typically reveal contemporary, beneficial updates to course materials. Good instructors bring this information into the classroom via discussions, assignments, or course updates. With that said, one cannot rely solely on this a the measure of quality. Classroom observations, student reviews, and reviews of course materials are also important considerations. While student comments sometimes have to be viewed with caution (there are often complaints from students who had trouble in the course unrelated to the instructor, yet they blame them for their woes). The important thing is to not rely on one or two measures but instead to look at the whole picture. Even service can be a measure of quality - do instructors care enough to give back to the students and the institution? Invested inividuals tend to deliver better products all around.
Dear Lidia,
Thank you for the opportunity to discuss this subject. I'm not an expert, but several topics interest me deeply. I do not think the answer is simple, since I agree with many previous ideas. Something that cannot be overlooked is that the work of the teacher or mentor in higher education has a feature handmade. As much as we have new technologies, the relationship between preceptor and pupil in the production of dissertation, these and papers keep almost unaltered. The problem is that nowadays we are evaluated in industrial scales, with so many automatic indexes, and this situation creates conflict between the production method of our work and how it is evaluated.
Another aspect that needs to be considered is that we specialize in research in very limited areas, and sometimes we forget to understand our role in the functioning of higher-education institutions.
Over a period of my career, I was responsible for a Physical Education course, and I tried to map the aspects I thought important for the proper functioning of the course. I have attached a brief translation of this map, which was published in Portuguese in an event in Brazil. I hope it can be useful, and we continued to discuss this subject.
For myself there are two aspects that i consider useful measures
The first around the student is that do they remember you and did what you imparted to them make a difference in their life
The second is around research is that do others want to communicate with you around what it is that you are doing and exploring.
One source that I highly recommend to capture the major aspects of higher education quality is the results section of the Baldrige Excellence Framework for Education (http://www.nist.gov/baldrige/publications/education_criteria.cfm). Category 7 of the Framework has two sections that focus heavily on the major outputs from education. Section 7.1 deals with student learning and the education process. Section 7.2 delves into the broader customer and stakeholder results.
Richland College (http://richlandcollege.edu/opred/) was a Baldrige Award recipient and does quite excellent work at putting a series of KPIs into a comprehensive framework called the Thunion. Although some of that material is private they are open to inquiries and there are some presentations available on the web.
As for journal output, this is often a very poor measure of educational effectiveness. Many schools use the concept of an "A and B list" of journals, which is little more than a way to outsource the screening of faculty content to peer reviewers. Such schools often have poorly defined models of scholarship as well so that publication is the be all and end all rather than understanding the usability of the research and its contribution to student success.
Across the UK, aside from research indicators you outline (which, for us in UK, includes also the 5/6 years cycle Research Excellence Framework scores) the annual National Student Survey feedbacks are used as well as the Times and Guardian (newspaper) ratings for universities and subject disciplines.
The sector also has internal Quality units, at each university, overseeing all courses/programmes/resources/teaching and monitoring these annually. There is also an external agency , the Quality Assurance Agency (QAA), that independently assures the quality
Ivo: I agree that education is not a commodity and thus should not be relegated to consumer views and attitudes. However, I respectfully disagree that measurements of quality are only associated with commodities and products. In fact, services, of which education is one, can and should be associated with quality measures. The quality of an instructor's teaching and of a student's learning can be directly measured through identified outcomes completed through validated assessments. How a student impacts society as a result of her/his education may take a while to measure but given the data that suggests the positive impact of HE one could surmise that at least some aspects of the academy exhibits elements of quality. I think the issue here is that of identification and measurement of desired outcomes e.g. grad rates, placement rates in graduate and professional schools, employment within disciplines, 5 year career promotion rates, propensity for entrepreneurship etc. The issue here is the difficulty in measuring the extent to which HE impacts those outcomes.
It also seems that you have convoluted some ideas to build a case against quality assurance. A disability like blindness should not preclude someone from studying a particular subject, certainly not psychology. I'm not sure what all the consternation was about. In the US, under the Americans with Disabilities Act, a University or college would have to make reasonable accommodation for a qualified student to study in this area. That said, there is no fundamental right to a university or college education, in the US and I imagine in most countries.
I completely agree with your view about commercialization and marketization of HE but disagree with your view of business vocabulary to describe processes associated with the organization and operation of a university. However, that horse seems to have already escaped the barn. I fear that not committing to known and respected organizational theory and its associated vocabulary marginalizes an HEI and places us in a position where we risk endangerment. Businesses have learned that they must adapt or die. Will HE face similar consequences if they hold onto antiquated notions that HE operations differ substantially from business?
Quality in higher education could be measured through the feedback received on employer's rating of the knowledge and technical know-how of the graduates in their workplaces.
Interesting concept, but is a higher education degree more than just knowledge and skills. Are there some intangibles that could be measured? What about those students that are engaged in a liberal arts degree that have the critical thinking and writing skills but may lack some knowledge and technical expertise that they learn on the job.
The comments show that there are many opinions, and in my point of view valid.
Considering that higher education is not only a model, but has several formats (it may be a world-class university or isolated organization) it is necessary to consider different types of measuring things. The use of only one parameter seems problematic.
In Brazil (unfortunately, it is in Portuguese and is too large to translate) there is the National System of Evaluation in Higher Education, made by the government to evaluate students of federal and private institutions (representing 80% of higher education in Brazil).
This system analysis three dimensions of an institution:
- Institutional dimension (institutional development plan, academic policies, management policies, infrastructure)
- Undergraduate courses (infrastructure, course project, teachers and researchers)
- Graduating students (an exam about general knowledge, professional expertise and socioeconomic status)
This evaluation is done through electronic forms (each of these items has several quality indicators with a scale ranging from 1 to 5) and there is the visiting of government officials to check the accuracy of information and teaching conditions. All this happens every three years.
I was director of a Physical Education Course and I pass through this evaluation process. In my opinion, what matters mostly is the consistency and synergy between all the factors involving individuals belonging to the institution.
I managed to find a single document in English that might clarify a little of what I am trying to describe. This document is the GUIDE TO INSTITUTIONAL AUTO-AVALUATION (one topic of the whole process). In our system, all the higher-education institution has to have an inside evaluation committee, and this document describes this aspect.
It is attached..
I hope I have contributed ...
There is a huge need for 'quality' to be clearly defined internationally.
Publications are merely a research output and not a measure of quality, but they do form a concrete and semi-measurable product, and so are taken as such.
The number of tenured faculty versus casual or part time faculty might be an indicator as part time faculty would not be expected to do as much research or take on extra duties.
Quality for me relates to the link between student satisfaction and lecturer safisfaction in terms of learning.
For many years I asked students to tell me the three most positive and the three most negative things of their module. When collated, these turned out to be the same!
Pass/fail rate in examinations doesn't work unless those exams are standardised.
Job uptake is not a good indicator of anything except geography and work opportunities available in the first place.
However ultimately there is a link between quality and commitment of faculty to the job they're doing. If job satisfaction is impaired through destructive change management practices, usually under the guise of improving quality, but actually part of mergers and acquisitions mentality, then quality is eroded.
For me the success of higher education lies in the leadership of alumni who apply all the abilities which they have acquired from their role model professors, not only professional but also in their personal life. So my indicators are:
The indicators include: enrollment rate,retention, performance in exams, transition rate,graduation rate, employability of the graduands, satisfaction of emplyers and supervisors, ability to continue with education after graduation, accountability, competitiveness, ranking status,research, level ofcollaboration, qualification of faculty and other workforce, number of programmes and marketability, leadership,management and administration, safety and health both indoor and outdoor environment and number of hits on google scholar
Output in terms of students performance. I believe that an up and doing educator will produce students with good performance. For me I strongly believe that is the students are not doing well, then the teacher is not teaching well.
I think there are 11 criteria to measure the quality in higher education instutions such as facilities, learners and staff percentage and curriculum taught in these institution.
I recommend you to look for these two compilation papers:
Harvey, L., & Williams, J. (2010a). Fifteen Years of Quality in Higher Education. Quality in Higher Education, 16(1), 3–36. http://doi.org/10.1080/13538322.2010.485722
Harvey, L., & Williams, J. (2010b). Fifteen Years of Quality in Higher Education (Part Two). Quality in Higher Education, 16(2), 81–113. http://doi.org/10.1080/13538322.2010.485722
and for the concepts of quality in teaching and learning in HE it may also help you to read:
Harvey, L., & Green, D. (1993). Defining Quality. Assessment and Evaluation in Higher Education, 18(1), 9–34.
A very heplful feedback of higher education is
- satisfaction at the end of the course and maybe much more powerful
- feedback up to three or five years later.
I like Frank Lipowsky:
Lernen im Beruf – Empirische Befunde zur Wirksamkeit von Lehrerfortbildung (https://www.uni-kassel.de/einrichtungen/fileadmin/datas/einrichtungen/zlb/J2010_-_Lipowsky_Lernen.pdf; its written in Germn). You can transform many of his thesis for higher education.
Total amount of specialists well prepared and working abroad having high funded position:)
Hi Lidia, I think I would agree with Alexander.
Employability and compatibility are important satisfaction factors.
Are our graduates employable in the sense that 85-100% of our graduates are employed soon after graduation and their performance would reflect the kind of education they are given at our University?
Are they internationally compatible in the sense that they can perform or out perform their international counterparts in whatever employment they are engaged in?
Are our graduates trained to the needs of industries or are our programs out of context with industries?
These questions make up the quality of education we provide to our students.
Research and publications are part and partial of academia but falls into the TORs of each individual specialist. Research, seminar presentations and publications helps the expert in a field of expertize to keep abreast with changes and reforms that are going on in and around the world in HE .
I think quality could be measured in terms of student satisfaction, student loyalty and student perception of service quality:
Five elements could be use to measure student perception of service quality as indicated below
Tangibles:
Physical facility, equipment and appearance of personnel
Reliability:
Ability to perform the promised service dependably and accurately
Responsiveness:
Willingness to help customers and provide prompt service
Assurance:
Knowledge and courtesy of employees and their ability to inspire trust and confidence
Empathy:
Caring, individualized attention the firm provides its customers.
I hope this helps
The importance of a Higher Education Institution (HEI) is based on its impact to society. Either the HEI made it part of their goal, or unplanned at all, an impact was created. However, gone were the days that an institution may have the option to be isolated and standalone because the truth is, HEIs must interact with other HEIs to facilitate exchange of experiences.
One cannot discount the role of publication and research because this is the only platform that can be shared by all HEIs being diverse in terms of its cultures and realities. Aside from research and publication, the best indicator for HEIs is when they were able to return the generosity to their immediate society, infusing back to society what it learned during a certain period of incubation. However, whatever impact effort done to the society are worth noting and writing, and sharing to other HEIs to make them learn from your experience.
You see, publication will still be at the end of the line whatever indicator you have in mind. Proof of the tri-focal function of a university that it must perform the pedagogy-research-extension. The three of them looping back incessantly with each other.
Dear Mandru
The importance of quantitative and qualitative indicators (PIs) in performance monitoring and assessing the activities, input, process and output of higher educational institutions is well noticed by everyone through the size of publications offered and the interest of recognized HEIs, The United Nations Educational Scientific and Cultural Organization (UNESCO), World Bank, and Organization of Economic Co-operation and Development (OECD).
In addition, several papers and reports have studied the problem of constructing the performance indicators in different countries.
In the following papers, 150 performance indicators to assess the educational activities including inputs, processes, products, outputs and outcomes of enrollment, teaching, learning, assessment methods, academic and non-academic staff, graduates and research, etc., were proposed for higher education institutions in developing countries:
1- Quantitative and qualitative indicators to assess the performance of higher education institutions. Int. J. of Information and Decision sciences, 2014, Vol.6, No. 4, pp.369-392.
2- Statistical characteristics of performance indicators. Int. J. of Quality and Innovation, 2014, Vol.2, No.3-4, pp. 385-309.
In India we have devised a National Institutional Ranking Framework that is based on 5 broad parameter sets, with about 3-5 sub parameters in each, making a total of about 20 or so, each with its own weight.
The 5 broad sets comprise: Teacher-Learning Resources (since they are pointers to facilitation of the teaching learning process), Research and Extension Activities (This includes publications, citations, extra-mural funding, patents and technology transfers, etc.), Graduation Outcomes, Outreach and Inclusivity and finally, Perception of Pears and Employers.
For more details, you may visit the following link.
https://www.nirfindia.org/
However, this framework falls short of direct measurement of quality of teaching-learning process, since even after a lot of debate it has been difficult to converge on an objectively measurable set of indicators that would serve the purpose. The effort is still on, and if there are some useful ways to do this, it would be nice to hear about them.
In my opinion, the employability and productivity of graduates are the ultimate measures of quality in higher education. If graduates are employable and productive, it means that they are well-educated and possess the right skills needed by the labor market. These attributes can only be acquired in a quality HEI.
But then employability and productivity should include interpersonal skills, values, sustainability competences which many HEIs struggle to address in their programmes.
Hi Lidia
I think that quality in higher education is a complex concept and has an overlap with other concepts such as efficiency and effectiveness. It seems to me the following indicators can be used directly or indirectly (proxy) to measure the quality of higher education:
The employment of graduates, the absorption of financial resources from industry and other sectors of society, the number of foreign students recruited from reputable universities, the amount of international scientific communications, the number of patents, the extent of higher education impacts on the society and culture and policy-making, student’s satisfaction of various aspects of university life, liveliness and dynamics of learning environments. Of course, some of these indicators are context-based and there is not enough data in a comparable way.
I think Internasional Accreditation is development indicators quality in my country, while research publication in scopus include it.
The quality of education can be assessed only when we can formulate its goal - the expanded reproduction of society. Therefore, there is no quality of higher education at all. There are universities that reproduce a local (national) society, and there are universities that solve global problems at the world level. And quantitatively, the quality of higher education can be measured by the demand for studies at specific universities and by financing universities.
Hello Lidia.
Apart from research some quality indicators could be delivery of curriculum in terms of logistics, innovation, completeness. Quality of faculty with respect to content expertise, interpersonal dealings, delivery of explicit curriculum and value-addition is another important indicator.
Student experience during and after University is an important source of information. It covers quality of most processes that form the core functions of a University such as admission protocols, delivery of curriculum, suitability for employment, infrastructure and learning resources. However, depending on one stakeholder would be limiting and a mistake.
Documents such as COLRIM, WFME, ACGME and NAAC Manual (India) provide a wide choice of indicators with varying nuances. The very obvious learning is how context-driven these indicators are.
Dear Mandru
In addition to my answer dated Jan. 26 2018. Recently, three national projects were implemented to measure the quality and performance of HEIs in Oman, which constituted 7 dimensions and 190 indicators. The dimensions and indicators were studied theoretically and empirically, and few papers are published based on these projects. Some of which are:
1- A Performance-based Measurement Model for HEIs: An Integrative Model of Statistical Approach and Performance Measures. Under consideration.
2- A knowledge Management Guidance and Practices for Enhancing the Quality and Performance of Higher Education Institutions: insights from Oman Int. J. of Quality and Innovation, Vol. 4 Nos. ½, pp.99-119.
3- Information Management Model for Intellectual Capital of HEIs in Oman: Theoretical Quantitative Approach and Practical Results. J. of Information and Knowledge Management, Vol. 17, No. 1, pp.1-38.
4- Quality Teaching of Private HEIs: Comparative Study based on Students’ Perceptions and Expectations. Int. J. of Quality and Innovation, Vol. 3, Nos. 2/3/4, pp. 209-228 .
5- HEIs Quality Improvement Through Students and Academic Staff’s Perception: Data Analysis and Robustness of the Results. International Journal for Quality Research, Vol.11, No.2, pp.261-278.
6- Al-Farahidi Performative model to assess HEIs. 4th OQNHE Conference 2017, Muscat, pp.1-15.
7- Students’ Perceptions and Expectations in Quality Teaching of Private HEIs: Comparative Study. 4th OQNHE Conference 2017, Muscat, pp.1-14.
8- Validity and Reliability of Students and Academic Staff’s Surveys to Improve Higher Education. Educational Alternatives, Journal of International Scientific Publications, Vol.14, pp. 242-263 .
9- A Framework for Fostering the Quality of HEIs: Data Collection, Evaluation, Indication and Validation. Int. J. of Quality and Innovation, Vol.3, No.1, pp.42-66 .
10- Standards, benchmarks and qualitative indicators to enhance the institution’s activities and performance: Surveys and data analysis. International Journal of Knowledge-Based Organizations (IJKBO), Vol.5, No.4, pp.38-62 .
11- Performance Measurement in Private HEIs: Performance Indicators, Data Collection and Analysis. Proceedings of the Conference “The Future of Education”, pp. 511-516.
12- Developing and Establishing Statistical Indicators for Fostering the Quality of Omani HEIs. International Journal for Cross-Disciplinary Subjects in Education (IJCDSE), Volume 6, Issue 2, pp. 2142-2151.
13- Quantitative and qualitative indicators to assess the performance of higher education institutions. Int. J. of Information and Decision sciences, Vol.6, No. 4, pp.369-392 .
15- Statistical characteristics of performance indicators. Int. J. of Quality and Innovation. Vol.2, No.3-4, pp. 385-309.
16- Features of performance indicators in quality improvement of HEIs. Proceedings of the Ireland International Conference in Education (IICE), Oct. 26-28, pp. 390-395.
17- Goals and objectives: Statistical techniques and measures for performance improvement of HEIs in Oman. Int. J. of Management in Education. Vol. 8, No. 3, pp.244–264.
Best regards,
Zuhair