Training Evaluation Methods

Two Approaches to Evaluation of Training:

Kirkpatrick's Model

And Brinkerhoff's Success Case Method

Nadine Schoenfelder

Student # 230079087

COMM 435 - Organizational Learning, Development and Training

Instructor: Allison Anderson

Due Date: March 27th, 2012

Two Approaches to Evaluation of Training:

Kirkpatrick's Model

And Brinkerhoff's Success Case Method

Evaluation of training is one of the main components of a training program. It can be defined as a systematic analysis of training which focuses on demonstrating whether training objectives have been accomplished in an effective and efficient manner. Because organizations invest substantial amounts of money in training, thorough training evaluation is as important as the design or implementation of training. This paper discusses two evaluations of training methodologies: Kirkpatrick's Model and Brinkerhoff's Success Case Method. It also discusses how they can be applied as well as their advantages and disadvantages. Kirkpatrick's Model was developed decades ago but is still one of the most popular and important evaluation models. It consists of four levels which are successional and increasing in importance as well as complexity. Level 1, reaction, measures training participant's feelings and is often called "a measure of customer satisfaction". Level 2, learning, measures if knowledge is learned, skills are learned and/or increased, and attitudes changed. The third level, behavior, links training to results and measures the transfer of learning to the job. The last level, results, is the most important and challenging part because it measures how much (if any) improvement is due to training tangibly and intangibly.

Brinkerhoff's Success Case Method (SCM) is a newer approach to evaluation of training and is based on the ancient craft of storytelling. It has two major parts: surveys and interviews. In order to conduct a SCM study, organizations first plan the study and create an impact model which determines the optimal outcomes of training. Afterwards, surveys determine the most and least successful training cases which are then interviewed. Last, a final report is produced which should provide the organization with insights if the training objectives were accomplished or not and why or why not.

This paper discusses further the advantages and disadvantages of both, Kirkpatrick's Model and Brinkerhoff's SCM. Although Kirkpatrick's Model is simple and effective, evaluation level 3 and 4 (behavior and results) can be very complex and time-consuming. Yet Brinkerhoff's SCM is quick and simple, it lacks comprehensiveness and consistency. First, in order to understand the purpose of Kirkpatrick's and Brinkerhoff's approach, one must understand what evaluation of training is and why it should be implemented.

Evaluation is commonly defined as the "systematic determination of merit, worth, and significance of something or someone using criteria against a set of standards" (Wikipedia, 2012). Therefore, the evaluation of training can be defined as a systematic analysis of training which focuses on demonstrating whether training objectives have been accomplished in an effective and efficient manner. Although the evaluation of training is listed as the last step in the training process model (Thacker and Blanchard, 2006, p. 6), it actually begins during the development phase of training and is an ongoing process from then on.

According to Thacker and Blanchard (2006), training can be evaluated in terms of outcomes of training and whether the results were worth the cost. Outcomes of training include measuring if the knowledge, skills, and attitudes of training attendees have improved, how well the learning objectives and training objectives were achieved, and how well the learned behavior is transferred to the job (Thacker and Blanchard, 2006) (Tracey, 1992). Belcourt, Wright, and Saks (2000) define evaluation as the "process of data collection used to determine whether training is effective" (p. 213). Determination of the effectiveness of a training program is a paramount and necessary procedure for an organization because it ultimately affects the organization's bottom line (Thacker and Blanchard, 2006). Organizations that invest substantial amounts of money in training programs need to know if those pay off. Nevertheless, organizations using training to cut costs and to increase the productivity of their employees often fail to treat the evaluation of training as a priority (Boverie, Sanchez, and Zondlo, 1994).

The first model of evaluation of training that this paper discusses is Kirkpatrick's Model which perhaps is the best known evaluation methodology (Bates, 2004). In 1959, Donald Kirkpatrick developed the Kirkpatrick Model which is a four-level model for evaluating training programs (Kirkpatrick, 1998). Kirkpatrick's model is a goal-based approach to training evaluation and its purpose is to determine the effectiveness of a training program (Eseryel, 2002) (Kirkpatrick, 1998). The four levels of evaluation of the Kirkpatrick Model are reaction, learning, behavior, and results. According to this model, evaluation should always begin with level 1, reaction. One should then move sequentially through levels 2, 3, and 4 because information from each prior level serves as a base for the next level's evaluation (Kirkpatrick and Kirkpatrick, 2006).

Level 1, reaction, measures how training participants liked the program and how they react to it. Kirkpatrick and Kirkpatrick (2007) called it "a measure of customer satisfaction" because it is important for training's effectiveness that trainees like the program in order to be motivated to learn (p. 24). It does not include a measure of actual learning.

Level 2 of Kirkpatrick's Model is learning which "measures to what extent the (training) program has been effective" in regard to knowledge learned, skills learned and/or increased, and attitudes changed (Kirkpatrick and Kirkpatrick, 2007, p. 79). Learning is defined as "a relatively permanent change in cognition that results from experience and that directly influences behavior" (Thacker and Blanchard, 2006, p. 12). Therefore, measuring learning is important because change in behavior only occurs if learning took place. So if no change in behavior is