Hi Franziska

As I know there's not any suitable software for your purpose but kindly please note that:

With my best regards,

Amir

Some years ago, colleagues of mine used a combination of all three levels you suggest. See 'BP Refines Leadership' (Brown, Eagar & Lawrence, T+D Magazine, March 2005) However, the measurement instrument was developed specifically for the situation: http://www.csr-bos.com/value/research/BP.pdf

Hi Franziska,

The “measurement instrument” you’re asking for must be tailored to your specific situation. When you prepare a training solution you set before training implementation, the performance goal, which is referred to as “level 3 evaluation” (Behavior) (Kirkpatrick & Kirkpatrick, 2016). In sales, you usually assess level 4 (impact). Whereas level 3 is about the behaviors participants will do in their jobs after training, level 4 is about the results that must be achieve as consequence of those behaviors. Since you’re interested in “post-training behavior on the job” an action plan must be prepared. In the plan, you specify the behaviors that are to be performed and the methods to observe and rate them (check behavioral scales that can be applied to your situation).

Reference: Kirkpatrick, J.D., & Kirkpatrick, W.K. (2016). Kirkpatrick’s four levels of training evaluation. Alexandra, VA: ATD Press.

Thank you very much for your inputs! Jaime, as proposed by you, I will pursue action planning and behavioral scales.

I purposely want to measure Kirkpatrick's level 3 (behavior) and not only level 4 (performance results) as I assume level 3 to be more closely related to the training. I further need a reliable measure that can preferably be evaluated automatically (online), as further individual interventions will depend on this measurement of transfer.

So far, I found some scales for general transfer measurement (Bates et al., 2007; Huang, Ford, & Ryan, 2016 adapted from Tesluk, Farr, Mathieu, & Vance, 1995; Grohmann & Kauffeld, 2013; Turab & Kasimir, 2015). However, I fear that those general scales might be rather susceptible to social desirability.

So, Jaime, I guess you are right! I need to adapt the scales to my specific training content/goals. I like the approach of Kauffeld, Holton III, Bates, and Müller (2008). They used one open-ended item to assess the quantity of transfer (number of training topics applied to the work setting) and one general item to assess the quality of transfer.

I am very grateful for any further ideas or input!

References: 

Bates, R. A., Kauffeld, S. & Holton, E. F. III (2007). Examining the factor structure

and predictive ability of the german-version of the Learning Transfer Systems Inventory. Journal of European Industrial Training, 31 (3), 195-211.

Grohmann, A., & Kauffeld, S. (2013). Evaluating training programs: Development and correlates of the Questionnaire for Professional Training Evaluation. International Journal of Training and Development, 17(2), 135–155.Huang, J. L., Ford, J. K., & Ryan, A. M. (2016). Ignored no more: Within-person variability enables

Huang, J. L., Ford, J. K., & Ryan, A. M. (2016). Ignored no more: Within-person variability enables better understanding of training transfer. Personnel Psychology, 1-40. 

Kauffeld, S., Bates, R., Holton III, E. F., & Müller, A. C. (2008). Das deutsche Lerntransfer-System-Inventar (GLTSI). Personalpsychologie, 1-41.

Turab, G. M., & Casimir, G. (2015). A model of the antecedents of training transfer. International Journal of Training Research, 13(1), 82–95. 

Thanks Franziska for your comments and the interesting references that I’m going to check.

One important aspect of level 3 is that it goes beyond evaluation. It’s basically planning and following-up in order to bridge the gap between learning and performance. Even if you are interested in level 4 (impact) it’s essential to conduct level 3 activities to make sure you have transfer –the prerequisite for results.

Good luck with your project!