Kirkpatrick (07 and before) lists four levels of evaluation, ranging from most-basic-and-pretty-useless to high-level-but-almost-impossible-to-measure (my interpretation not his!). My example, a family doctor goes on a course about latest blood pressure management guidelines

Level 1> Learner satisfaction (eg did they like it)

Level 2> change in learning - knowledge or skill gain (eg do they score better on post-test than pre-test... etc)

Level 3> Change in behaviour in the workplace (do their prescriptions better mirror guidelines since they have been on the course, perhaps 3-6 months later))

Level 4> Outcomes (is blood pressure better controlled/are their fewer heart attacks & strokes etc)

Of course, the higher up you go, the more interference there is from other factors. We managed to demonstrate a small increase in chlamydia testing rates in GP practices where one of the GPs had attended advanced sexual health training compared with those who had not (level 3), which we were pretty proud of (Estcourt et al, 2009), but the stats were very messy.

Kirkpatrick's levels are simple and make a good point, but I find them very limiting. For me evaluating the impact of teaching overlaps largely with concepts of quality:

1. Evaluation is about both “judging quality” and planning improvement, for

the teaching and for yourself as a teacher. (A TQM style approach (Sallis, 2002))

2. When designing teaching, consider how you will evaluate it, so that you can

continue to improve

3. A judgement on the quality of your teaching will rely on *triangulation* of several different approaches.

In workshops that I run, participants tend to generate about 8 different directions to evaluate teaching, with each direction and its sub-methods each having its own pros and cons, limitations. This is why triangulation is so important.

In addition to all this mix, I agree completely with Xavier - but would extend that it not only depends on the purpose of the teaching, but also the purpose of the evaluation. If it is new teaching and you want to improve it, then it will be a different evaluation than if you want to keep the funders happy, and different again if it is well established teaching and you just want to check that there have been no disasters (light touch)

Sorry to be longwinded !

Dason

-KIRKPATRICK, D. L. (2007) The four levels of evaluation : measurement & evaluation, Alexandria, Va., American Society for Training & Development.

-ESTCOURT, C., EVANS, D., DAVIES, J., HUTCHINSON, J., MACQUEEN, R.-A., SHACKLETON, T. & DHAR, J. (2009) Poster presentation: Development and evaluation of STIFCompetencies, a new clinical training program in sexual health for primary care professionals. ISSTDR/BASHH 2009. London.

-SALLIS, E. J. (2002) Total quality management in education, London, Kogan Page.

Info in this image would be interesting.

Thank you for your responses.

Well, it seems that there are two approaches for evaluating the impact of PD. The first approach is following a comprehensive model such as Kirkpatrick's model (@ Dason) and Guskey's model. The second approach is following an evaluation plan that depends on the specific goals of your PD program (@ Emel & Xavier). However, I am not sure which approach would be more convincing for the funding agencies to determine whether the investment in PD yields ‘tangible payoffs ‘ as the budget is being more limited (Guskey, 2002).

Guskey, T. (2002). Does it Make a difference? Evaluating professional development. Educational Leadership. 59(6), 45–51.

Guskey (2000) model works better for CPD within educational context.