1. At your institution, what methods and processes are used to ensure the quality of each item for both MCQ and CR?
2. Do you pilot these items first before you put them into the actual exam paper?
3. If so, how?
1. External examiner.
2. Advisable to pilot if you have little experience.
3. Draw from previous tests and exams, but introducing many new questions and disguising previous questions. (Minimally change order of answers A-E)
4. My advice it to try to make all MCQ answers appear to be equi-probable.
5. Avoid "All of the above" and "None of the above", MCQ answers, except when you really cannot think of a fifth possible answer.
6. Analyse frequent wrong responses to ensure that it is not due to a poorly worded question.
Thank you for your response.
For (1), can you elaborate further what the external examiners do.
For (2), do you pilot CR items too?
For (5), there is a suggestion that 2 distracters and 1 key is sufficient in MCQ items. What is your take?
Finally, at your university, are each item analysed using IRT before they are considered valid to are printed on final exam papers.
The external examiner is appointed from outside the university to overview the clarity and fairness of the questions and to check the answer memo.
No, if I pick up unforeseen problems (e.g. legitimate synonyms not in the memo), I add to the memo.
No. if all answers do not look (equi) plausible, the students can get more than 20% just by looking for clues.
If IRT is a statistical tool, the answer is No.
- Badges
- Science topic
- Similar topics
- Psychology