Tag: evaluation


Training Evaluation Tools

Training Evaluation Tools

Training evaluations are conducted with the intent of obtaining information on the effectiveness of the training program. Training evaluations are intended to check whether the training event has met its objectives and whether participants are able to implement his or her learning on the job. The results of the evaluation process help training professionals modify future training strategies to meet objectives and implement learning to positively affect job performance.

 

 

Training Evaluation Tools

Training evaluation tools are what is used to collect data on the training programs. They come in a variety of forms and can be divided into categories such as questionnaires, interviews, focus groups, and observations. These methods are often used collectively to achieve a variety of results from different perspectives. Ultimately, several are combined to obtain the best overall assessment of the program.

These methods are designed to determine a learner’s perceptions, experience, and behavior using both qualitative and quantitative data. The information obtained can be used to modify future training sessions.

 

Training Evaluation Tools

Questionnaires

Interviews

Focus groups

Observations

 

Questionnaire

Questionnaires are the most common of the training evaluation tools used to gather information on a program or session. This is because they are an efficient way to gather large amounts of both quantitative and qualitative data about a training course.

Questionnaires are a set of questions delivered to individuals for obtaining useful statistical information on a particular topic. It is an excellent tool for a reaction level assessment from the participants immediately following the training course.

A questionnaire can;

  • Be brief or extensive (typically, the shorter the better)
  • Be paper, email, or online
  • Have closed questions, open-ended questions, scaled questions, or multiple choice questions
  • Be completed immediately after the training or later (days or weeks)

 

Benefits of a Questionnaire

  • Gathers a large amount of data, providing a broad perspective
  • Minimal time to administer
  • Little cost to administer
  • Can easily encompass a large number of participants
  • Can provide a wide scope of information about leaner’s opinions and behaviors
  • Has consistency since every individual is asked the same questions in the same way

 

Disadvantages of a Questionnaire

  • Can have a low response rate
  • Can receive confusing or vague responses
  • Questions may be difficult to understand so respondent may not answer it
  • Difficult to get clarification to responses and investigate issues further
  • They do not allow for any observation
  • Different people understand questions differently

 

Focus Groups

A focus group is a facilitated discussion with a group of individuals who share similar characteristics or common interests intended to collect information or obtain opinions on a specific issue. Focus groups are an excellent method to get more in-depth information on perceptions, insights, attitudes, or experiences.

Focus groups are used to understand participant’s perspective of the training course including instructor engagement, course materials, and application to work duties. The goal of this method is to obtain qualitative data, meaning that the data is descriptive and not necessarily measured numerically.

Ideally, the group discussion is facilitated in a non-threatening environment so participants feel they can speak more freely. Also, a semi-structured approach using open-ended questions to generate discussion is most useful.

 

Benefits of a Focus Group

  • Useful for gathering subjective perspectives
  • Able to receive information from many people at the same time
  • Able to learn about participants experience and perspective through group dialogue
  • Able to ask follow up questions to dive deeper into potential issues
  • Able to further explore a topics, providing a broader understanding of why the participants think or feel a particular way

 

Disadvantages of a Focus Group

  • Can be time-consuming to administer
  • Sessions need a moderator to facilitate
  • Sessions need an assistant to take detailed notes
  • Sessions need approximately six to twelve people who share similar characteristics or common interests
  • Group dynamics can play key role in determining the effectiveness
  • Sessions need to ensure a comfortable environment so participants feel they can speak freely

 

 

Observation

Observation is a method of gathering data by watching an individual’s behavior in their natural setting. It entails simple watching and noting a person’s behavior after a training course to determine if they are able to perform the intended objectives. This method allows many aspects to be captured, including the physical environment, the person’s own behaviors, and his or her interactions with others.

Observation is a fairly inexpensive method for discovering more about learners’ behaviors, but it does have its limitations.

 

Benefits of an Observation

  • Inexpensive method because it only requires an observers time
  • Easy to complete
  • Gathers information on actual behaviors rather than reported behaviors
  • Allows observer to directly see what people do rather than relying on what people say they do
  • Non-verbal language can provide valuable information
  • Able to collect information as the learning is being applied
  • Does not rely on people’s willingness or ability to provide information

 

Disadvantages of an Observation

  • Can be time-consuming since it typically only assesses one person at a time
  • Validity and reliability of the data may be affected since people usually perform better when they know they are being observed
  • Difficult to assess preferences, opinions, or thought process
  • Is open to misinterpretation
  • Does not increase understanding of why a person behaves as they do
  • Can be considered intrusive
  • Susceptible to observer’s bias

 

 

Interviews

Interviews are a data collection method aimed at gathering both facts and opinions. They involve asking open-ended questions where the respondent can answer the questions in as much detail as her or she wishes to. Interviews can provide valuable insight into the behaviors, attitudes, opinions and mindsets.

There are different ways to conduct interviews including face-to-face, by telephone, or through Chat on the computer.

 

Benefits of Interviews

  • Flexible
  • Can provide valuable insight into individual perspectives
  • Can ask follow questions to clarification
  • Can delve deeper into potential issues
  • Can be recorded to provide information for closer review at a later time (i.e. assess body language)

 

Disadvantages of Interviews

  • Can be time-consuming to administer
  • Not the most cost effective since you typically only interview one person at a time
  • No direct evidence a person can perform specific behaviors

 

Please follow and like us:
error

Kirkpatrick’s Level 3 Evaluation

Kirkpatrick’s Level 3 Evaluation

Kirkpatrick’s’ Four Levels of Evaluation model is a series of steps that begins with level one, and moves sequentially through the levels to level four.

Each level provides valuable information to help determine the effectiveness of the overall training program.

 

Kirkpatrick’s model includes four levels of evaluation:

Level 1: Reaction – To what degree did the participants react favorably to the training

Level 2: Learning – To what degree did the participants acquire the intended knowledge, skills, and/or attitudes based on their participation in a training

Level 3: Behavior – To what degree did the participants apply what they learned during training to his/her job

Level 4: Results – To what degree did the targeted objectives/outcomes occur as a result of the training.

Kirkpatrick’s Level 3 Evaluation

Kirkpatick's Level 3 Evaluation

 

Level 3 Evaluation – Behavior

Kirkpatrick’s Level 3 evaluation measures how much participants have changed their behavior as a result of the training they received.

Basically, this level is designed to determine if the newly acquired skills, knowledge, or attitude are being used in the everyday environment of the learner.

At this level, we not only want to find out if the desired behavior change occurred, but if it did not occur, we also want to find out why the change did not occur.

 

Information for Level 3

Information generally sought include:

  1. Did the learners put their learning into effect back on the job?
  2. Are the learners aware that they have changed their behavior?
  3. Was there noticeable and measurable change in the activity and performance of the participants in their job roles?
  4. Are the learners able to teach their new knowledge, skills, or attitudes to other people?
  5. Was the change in behavior sustained over time?
  6. Was the change in behavior supported by others in the organization?

 

Techniques for Obtaining

Techniques for obtaining level three evaluations include:

  1. Observation of the participant while performing job activities.
  2. Interview with the participant to discuss behavior changes in their daily work activities.
  3. 360-degree feedback from others affected by the participant’s performance.
  4. Self-assessment to obtain specific examples of how they implemented the desired change into their daily work activities.

 

Importance of all levels

It is important to note that just because behavior has not changed does not mean that the training was ineffective. This is why it is important to complete the first two levels of Kirkpatrick’s model and to complete them immediately after the training event.

If only the behavior is evaluated, and it is determined that no behavior change has taken place, it may be incorrectly assumed that participants have not learned anything from the training. But, by completing the first two levels, it may show that there was a positive reaction to the training and that the participants learned from the training event.

What may be at issue is the implementation or transfer of the learning to the job activities. This may be because the learner does not receive support, recognition, or reward for their behavior change from their supervisor or manager, or it may be that the learner themselves does not want to apply the new knowledge. Whatever the reason, because the level one and level two evaluations were completed, the focus of the analysis can be at the implementation of the knowledge to job activities and not at the modification of the training event.

 

Management Responsibility

It is also important to note that behavior can only change if conditions in the workplace are favorable. There first has to be a desire to change and support for that change.

Kirkpatrick’s Level 3 evaluation is mainly the responsibility of the management team of the training participants. Working collaboratively with the management team with Change Management techniques will help ensure success.   Also, by demonstrating the benefits to changing behavior to the participant during the training, and by creating methods of reinforcement and follow-up will help ensure success.

 

Strategies for Implementation

Strategies that can be implemented to enhance the transfer of learning include:

  1. Create a learning environment similar to the actual work environment
  2. Provide real world examples and actual experience performing and practicing the behaviors.
  3. Ensure participants understand the benefits behind the desired behaviors
  4. Provide feedback on their performance during the training
  5. Reinforce positive behaviors

In additional to the strategies implemented by the training area/department, the organization can enhance the transfer of learning by creating a culture and climate that support and embraces change as well as provide extrinsic rewards such as recognition, encouragement, praise, increased responsibility, and bonus/pay.

 

Benefits

Benefits of Kirkpatrick’s Level 3 evaluation:

  1. Provides measurement of actual behavior change occurring on the job
  2. Measures more than just a positive reaction or short term learning
  3. It can show gaps between training and on the job performance
  4. It illustrates organization willingness to change
  5. It is objective evidence to support the effectiveness and value of a training program.

 

 

Additional Links

ADDIE Model

Gagnes Nine Events Instruction

Kirkpatricks Four Levels of Evaluation

Types of Evaluations

 

Please follow and like us:
error

Benefits Kirkpatrick’s Evaluation

Benefits Kirkpatrick’s Evaluation Model

Donald Kirkpatrick first published his Four-Level Training Evaluation Model in 1959. To this day, it is still one of the most popular models to evaluate training program. Kirkpatrick’s model evaluates the effectiveness of the training at four different levels with each level building on the previous level(s).

The four levels of evaluation are:

  1. Reaction
  2. Learning
  3. Behavior
  4. Results

 

Four Levels of Evaluation

Kirkpatrick’s model includes four levels or steps of evaluation:

Level 1: Reaction – To what degree did the participants react favorably to the training

Level 2: Learning – To what degree did the participants acquire the intended knowledge, skills, and/or attitudes based on their participation in a training

Level 3: Behavior – To what degree did the participants apply what they learned during training to his/her job

Level 4: Results – To what degree did the targeted objectives/outcomes occur as a result of the training.
Benefits Kirkpatrick's Evaluation

Benefits Kirkpatrick’s Evaluation Model

 

Level 1 – Reaction

This level measures how the participants reacted to the training event.

Benefits of level one evaluation:

  1. It is generally easy and inexpensive to complete
  2. Immediate feedback to training event
  3. It attains a gauge on how the participants felt about the training
  4. Identifies areas that the participant felt were missing from the training
  5. It can provide information on specific aspects of the training
  6. It can provide information that can be used to improve future versions of the training
  7. Provides a simple way to gauge a perceived return on the training investment

 

 

Level 2 – Learning

Level two evaluation measures what the participants have learned as a result of the training.

Benefits of level two evaluation:

  1. Provides opportunity for learner to demonstrate the learning transfer
  2. Quantifies the amount of learning as a result of the training
  3. Provides more objective feedback then level one evaluations
  4. Provides more conclusive evidence of training effectiveness
  5. Identifies gaps between the targeted objectives and actual learning
  6. The assessment information can be used to increase learning in future training programs

 

Level 3 – Behavior

Level three measures how much participants have changed their behavior as a result of the training they received.

Benefits of level three evaluation:

  1. Provides measurement of actual behavior change occurring on the job
  2. Measures more than just a positive reaction or short term learning
  3. It can show gaps between training and on the job performance
  4. It illustrates organization willingness to change
  5. It is objective evidence to support the effectiveness and value of a training program.

 

 

Level 4 – Results

Level four evaluation measures the impact of training and subsequent reinforcement by the organization on business results.

Benefits of level four evaluation:

  1. Measures affect training has to ultimate business results
  2. Illustrates value of training in a monetary value
  3. Ties business objectives and goals to training
  4. Depicts the ultimate goal of the training program

 

Additional Links

ADDIE Model

Gagnes Nine Events Instruction

Kirkpatricks Four Levels of Evaluation

Types of Evaluations

 

Please follow and like us:
error