What I have seen in the literature is extremely limited. What I want to be able to do is measure not just what is learned in the professional development sessions/courses but how it has an impact on the workplace itself.
Hi Patrick, this is a valid question. I agree with you that the current literature on this topic does not reveal a whole lot, but maybe the answer lies in the practice. I don't know what skills (hard skills, soft skills) you have in mind for your development center, and this might not be applicable to the number and role of your participants. But I facilitated training and longitudinal coaching of 360 degree leadership assessments which work very well to continually assess changes in leader behavior and their subsequent impact on results and people.
I learned that, as long as the parameters are defined precisely (this is probably the most important first step) in form of desirable behaviors, outcomes and measurable results, this can work really well because changes in behaviors can be experimented with and seen quickly by others. The only crux with this approach is to find consultants who are willing to get to know the objectives of the development effort, the culture, desired behaviors, success measures really well, and consequently stick to those during the process from beginning to end. - Hope this helps a little. Good luck! - Andrea
Thanks, Andrea, This is in line with what I am concerned about. The challenge rests in establishing appropriate criteria for improvements in leadership behaviors or other administrative skills. The skills we are responsible for are the so-called soft skills of leadership and managemnt. The other issue is related to logistics. How does one go about measuring such outcomes once the participants return to the workplace? My thoughts are that it would be necessary to gather information from both the participant and those around them (the 360 degree leadership measurement of which you speak.) I just want to find some instruments that both accurately measure such outcomes and are easily administered. It is difficult to get people to respond to requests for feedback as it is, then when the instrument becomes overly complex, it is unlikely that you will get much in the way of feedback. The social scientist in me says that even if I got such an instrument, it might still not measure actual outcomes. Rather, it will measure perceptions of outcomes. The actual outcomes should be improved performance but there are enough other variables that influence this that one would be hard pressed to claim they were a result of the professional development itself.
I would be interested in the kind of instrument you are using, though. Any measurement is better than none at all.
I agree that additional measurements in form of continued written assessments of any kind are not always helpful - they just become an additional tiresome task which consequently looses effectiveness.
Please correct me if I am wrong but I understand what you want to measure is actual changes in people's behaviors, right? (eg more effective communicating, orally and written, during one-on-ones and team meetings etc). First, with the help of a good 360 degree instrument and the subsequent feedback report, the coach can discuss the findings by drilling down on individual behaviors to figure out what behavior would need to change for what reason and in which situation. Together with the participant, the coach can enable this exploration by asking questions to figure out where the problem lies, when it appears, with whom it happens more often and how the person feels when they show ineffective behaviors. Yes, this approach heavily relies on a high degree of self-awareness, trust and openness of the participant but it can also be enabled through effective questioning techniques and a little gentle and good-natured relentlessness on the side of the coach.
Second, once the ineffective behaviors etc have been identified, a strategy can be worked out, but it has to be elicited by the participant to get their full buy in. In subsequent meetings - face-to face - works much better than by phone I realized - the coach and the participant go through their experiences of these changed behaviors in their work place. They discuss how it felt, when they did it/didn't do it, why, with whom and why it succeeded or failed and how others reacted to the changed behaviors. Is it still measuring the participant's perceptions? - in part, but not only. Assuming that participants are not making their success stories up, you can start seeing considerable changes in behaviors because they will tell the coach how it went. You can measure it by their own actual experiences, you can measure it through talking to the people around them, and you will be able to see actual tangible outcomes. However, it is much easier to get a truthful answer to an uncomfortable question face-to-face than in writing. (;-)
It is crucial that the "correct" behaviors are being identified first, and that the participant is willing to learn. However, that is the case always. I am always astonished at the degree of self-awareness (people can always come up with situations, memories, experiences that refer to certain behaviors they get rated high/low on) and their willingness to try new behaviors out if they are the ones who come up with the action plan.
Sorry for the long answer and the lack of measurement tool. But this is an approach I genuinely believe in and know that it works.
Best,
Andrea
It's hard to beat the practical value of the books on assessment centers by George Thornton. Also check the annual conference for papers:
http://www.assessmentcenters.org/articles.asp
The principles of testing and of assessment apply if you want valid results.
Mike
- Badges
- Science topic
- Similar topics
- Psychology