Hi Arjun,

Have you considered using a retrospective post, post-then measurement?

See:

Rhodes, H.M., & Martin, A.J. (2014). Behavior Change After Adventure Education Courses: Do Work Colleagues Notice? Journal of Experiential Education, 37(3), 265-284.

Moore, D., & Tananis, C.A. (2009). Measuring change in a short-term educational program using a retrospective pretest design. American Journal of Evaluation, 30(2), 189-202.

Hoogstraten, J. (1982). The retrospective pretest in an educational training context. Journal of Experimental Education, 50, 200-204.

More generally in terms of training evaluation the following might be useful:

Rae, L. (1986). How to Measure Training Effectiveness. Aldershot: Gower.

Ostroff, C. (1991). Training effectiveness measures and scoring schemes: A comparison. Personnel Psychology, 44, 353-374.

Bramley, P. (1996). Evaluating Training Effectiveness (2 ed.). Maidenhead, Berkshire: McGraw-Hill Publishing Company.

Powell, K.S., & Yalcin, S. (2010). Managerial training effectiveness: a meta-analysis 1952–2002. Personnel Review, 39, 227-241.

Kind regards,

Jennie

Hi Arjun,

Have you considered using a retrospective post, post-then measurement?

See:

Rhodes, H.M., & Martin, A.J. (2014). Behavior Change After Adventure Education Courses: Do Work Colleagues Notice? Journal of Experiential Education, 37(3), 265-284.

Moore, D., & Tananis, C.A. (2009). Measuring change in a short-term educational program using a retrospective pretest design. American Journal of Evaluation, 30(2), 189-202.

Hoogstraten, J. (1982). The retrospective pretest in an educational training context. Journal of Experimental Education, 50, 200-204.

More generally in terms of training evaluation the following might be useful:

Rae, L. (1986). How to Measure Training Effectiveness. Aldershot: Gower.

Ostroff, C. (1991). Training effectiveness measures and scoring schemes: A comparison. Personnel Psychology, 44, 353-374.

Bramley, P. (1996). Evaluating Training Effectiveness (2 ed.). Maidenhead, Berkshire: McGraw-Hill Publishing Company.

Powell, K.S., & Yalcin, S. (2010). Managerial training effectiveness: a meta-analysis 1952–2002. Personnel Review, 39, 227-241.

Kind regards,

Jennie

Few ways to check effectiveness of training program if relevant data that could be used for evaluating effectiveness were not collected before and during training may be:

1. How participants took interest in training program?

2. How they asked questions in training program?

3. How they discussed about training program during tea, lunch, etc breaks?

4. On the next day, have they started with yesterdays brief?, Etc.

Hi Subhash,

See if the following articles may give you some hints on how to deal with assessment.

best,

Laura

http://www.thefirefly.org/Firefly/html/whoweare.htm

Data Validation of a rubric to assess innovation competence

Article How to assess innovation competence in services: The case of...

Hi Arjun jee. Namaskar. Congrats for choosing a very interesting topic. I believe the most conventional theoretical foundation to measure training effectiveness has come from the writings of D.L. Kirkpatrick (1983, 1994, 2004). In addition to the list of resources provided by Jennifer Hussey above, you may also find a number of studies I've carried out in Nepalese organisations. Many of my research works are available on ResearchGate also; you may go through them. Good luck.

In my view, the actual measurement of training effectiveness must come after the training itself. But in order to do this, you must first ascertain what the goals of the training were. Why have you decided to provide these training programmes to these people. Secondly, are the recipients students or workers? From your question, I assume they are workers and that makes it simpler. For workers, you only need to compare their performances pre-training to their performances post-training. That brings out in very bold relief the effectiveness of the training delivered.

The softer features of evaluation would probably focus on whether the delegates enjoyed the sessions and their views on the training. While these are important, I think the impact on performance must be supreme.

Thanks Gordon. My problem is evaluation of pre-induction training of about six months to employees (officer level). Training has already been conducted and the participants are working at various capacities. Pre and post-training comparison is not possible. In this case, how can we assess the impact of training on their performance?

I am strongly agree with the answers/discussions of Prof.Gordon Ade-ojo,  Prof.Subhash C. Kundu & Prof. Jennifer Hussey.

Hi Arjuna,

in that case, you could use a combination of participants' perceptions and their actual performances. Find out through a survey from participants where they saw themselves prior to the training and where they think they are now in terms of their work. You could then validate this through an observation of their performance and through evidence from their managers, colleagues etc. You could also administer the questionnaire/survey to their line managers from the onset.

Thanks Gordon.

Hi Dear Arjun Shrestha; Effectiveness of a training program may be adjudged through some statistical tools. If one proforma before training could be answered by participants having questions related to subject on which training is being conducted and the same proforma could be filled after training. And enhancement in knowledge of subject could be a parameter for its effectiveness. Thanks

Arjun, I do agree with Gordon's comments. To those I would add the following suggestion. Consider developing a theory of change model for the training program. While this would normally be done before the program started, doing this retrospectively is ok too. Depending on the aims of the program, have a think about what its immediate outcomes should be for participants. Is it designed to raise awareness, increase knowledge, improve efficiency, or change behaviour, for example? The longer term outcomes might be influenced by other things, but have a think about what can realistically be attributable to the training. Think also beyond the participants to the organisational outcomes. If the training was designed to improve service delivery to others, think about the impact on other end users such as clients.

See Funnell, S., & Rogers, P. (2011). Purposeful Program Theory: Effective use of theories of change and logic models. San Francisco: Jossey Bass.

John gives you good advice. The first question to ask yourself is what was the purpose for the training? It gets done for myriad purposes, often not clear and sometimes overlapping. Some, for example, is done to meet a regulatory requirement, in which case the only measurement needed is to document who got trained, and for how long.  Some is done to build capacity for emergency situations, in which case retention of the skill 9not actual application of it) would be the best measure. But when training is done to improve job performance (the highest and best purpose), then the payoff is changed behavior.  But because job performance is influenced by more than just capability (the immediate outcome of training), you have to address the fact that you are not evaluating "training", but you are evaluating the use made of training. So you'll need to assess who is using training, who is not, when they are using it, what is helping, when they are not, what factors are impeding usage, and so forth. The data from these inquiries are far and away the most actionable and valuable. The goal for evaluation of training should be to help your client improve their capacity for milking performance from learning. Thus, your approach will be most useful when you think of your purpose not as evaluating the training, but rather evaluating how well your client is leveraging the training to get improved job performance. This is in my view the gold standard for measuring training. That's my $0.02. Check this link if you want to read more:

http://viewer.zmags.com/publication/a02cd238#/a02cd238/50

No, the people who evaluate training must have a formation directed to training techniques. In addition, before providing training indicators relating to the field of the training should be built so that the evaluator has a reference and to evaluate training.

A final nopte; I tis all about the impact of your training. In most cases, we have to assume that the primary goal of employers in providing training is to improve performance. Than might well be achieved through a myriad of ways including imporved competence and job satisfaction. All the same, your strategies must be built around the measurement of impact of the training you have delivered.

Mr. Arjun Please develop a clear picture of purpose of your research work in your mind. it will further clear research design for you. Theoretically and practically I belong to this area i.e. training & development. I think you intend to assess learning transfer that can tell you about effectiveness of training transfer. Participants may be newly inducted or experienced would be asked through interview based survey about transfer (application) of skills after training to workplace. Holton LTSI inventory will advance your thought, I hope.

http://www.slideshare.net/edholton/learning-transfer-system-inventory-research.  It provides a good set of diagnostic and evaluative tools.