Evaluating an MSc Thesis typically involves a presentation from the student, a Q&A session and a thorough review of the Thesis itself. But what are best practices? What if a mark has to be given, how is this mark correctly validated to avoid bias and maintain a consistent marking standards year on year and across disciplines?
Emmanuel... what happens when there is disagreement? How would this be resolved? How do you have examiners validate their assessments which are often opinions rather than quantified evaluations? Opinions can vary greatly based on bias and personal understandings of academic performance standards.
Pani,
1. There are no published common "best practices" in assessment. There is no global Higher Education organization that provides universal guidelines in any aspect of HE. There are standards, but these are often set differentially by a) various international/national HE organizations, b) by scientific societies focusing in education (i.e. for medical education = WFME), c) by individual institutions.
- A list of international HE societies that have various standards:
http://www.herdsa.org.au/international-organisations
- Check this out from Birm Uni as an example of institutional guide:
https://www.birmingham.ac.uk/Documents/university/legal/assess-research-degree-theses.pdf
2. My experience gathered by the UK and Australia follow clear guides on thesis assessment, based on fundamental assessment principles based on Process and Criteria:
- Process: a clear process of who/how/when/what is assessed should be set by the institution/department, and be known to the examinee well in advance. This is usually done in a form of Manual or Guide that is decided, set and used.
- Criteria: in every assessment type/subtype (i.e. written thesis, oral presentation etc) there are specific criteria that are asked to be met, along with different levels of achievement. This is set in a form of a rubric, which also is given to the examinee well in advance.
If the above are completed, no matter what the level of educational quality, then the questions you've asked are answered by default.
3. Answering your questions:
- What if a mark has to be given, how is this mark correctly validated to avoid bias and maintain a consistent marking standards year on year and across disciplines?
= This is a good question. This is partially done by a agreed analytical rubric of assessment. The examiner follows specific set of criteria and level of achievements (i.e. Criteria 1-10, Levels: Proficient, Advanced, Basic, Inadequate). You can name the levels according to your needs. One other important aspect, is the examiner's benchmarking. When the examiners are chosen, a joint session to discuss the Criteria & Levels required by the institution is helpful. Nevertheless, you do NOT want a "homogenisation" of examiners. The reason why you have many examiners (instead of 1) is because different perspectives, different experience and different quality standards by each examiner are needed to reach a mean. So i'd be careful with using the word "consistency" too much. What you mean is quality assessment standardisation, and this is partially met by using a quality rubric.
- What happens when there is disagreement? How would this be resolved? How do you have examiners validate their assessments which are often opinions rather than quantified evaluations? Opinions can vary greatly based on bias and personal understandings of academic performance standards.
= The rubric is used as a guide. The rubric gives an examiner the "avenue" to follow during assessment. The rubric is NOT used as a template for assessing "in the same way". Disagreements by examiners is a usual issue, and in some countries/institutions is even required at some point. However, if two examiners are very-very different in their assessment, then most likely there is a misunderstading in the rubric. If not (i.e. if both examiners understand the rubric and they insist on very different opinion) then usually the Institution has a clause that say something like: if each examiner's assessment results in the student ending up at different par of degree (i.e. first class vs second class), then a third examiner is required to assess and the mean mark is taken.
Rubric info source: https://staff.unimelb.edu.au/__data/assets/pdf_file/0019/2122714/Creating-and-Using-Rubrics_d.pdf
In general, difference of "opinions" are understandable, since these opinions are often set by internal standards, experience, professional status etc. The resolution of these issues should initially be answered in the Manual/Guidelines of the assessment set by the institution, prior to the assessment.
References
- Article:
Article Examining PhD and research masters theses
- Assessment principles source:
http://educationstandards.nsw.edu.au/wps/portal/nesa/k-10/understanding-the-curriculum/assessment/assessment-principles
- Principles of good assessment example:
https://www.coursera.org/learn/university-teaching/lecture/lnUYk/principles-of-good-assessment-practices
- Badges
- Science topic
- Similar topics
- Chemistry