The Kirkpatrick Model of Training Evaluation

Summary

The video script introduces the Kirkpatrick model, a foundational framework for evaluating training programs, developed in the 1950s. It outlines four levels of evaluation: reaction, learning, behavior, and results, each increasing in difficulty but offering more value. The script emphasizes the importance of planning with business outcomes in mind and suggests an iterative approach to training program design. It also recommends the book 'Kirkpatrick's Four Levels of Training Evaluation' for deeper insights and provides resources for aspiring instructional designers.

Takeaways

• 📚 The Kirkpatrick model is a widely recognized framework for evaluating training programs, developed in the 1950s by Donald Kirkpatrick.
• 🔍 The model consists of four levels of evaluation: reaction, learning, behavior, and results, with each level representing increasing difficulty and value in evaluation.
• 📊 Level one, reaction, focuses on overall satisfaction and engagement, commonly measured through surveys, interviews, and focus groups.
• 🧠 Level two, learning, assesses the acquisition of new knowledge, skills, or attitudes, often through quizzes, tests, interviews, and observations.
• 🏢 Level three, behavior, examines whether the taught knowledge and skills are being applied differently in the workplace, with observation and performance reviews being key methods.
• 💼 Level four, results, evaluates the impact of the training on business outcomes, such as customer satisfaction, sales, and employee retention rates.
• 🛠 The planning phase of a training program should begin with defining the desired business results (level four) and work backwards to plan the intervention.
• 🤔 Important planning questions include identifying the desired business outcomes, determining what behaviors need to change to achieve those outcomes, and what knowledge and skills are required.
• 🔄 The evaluation process is iterative, allowing for adjustments and recalibrations based on ongoing data collection and analysis.
• 📘 For a deeper understanding, 'Kirkpatrick's Four Levels of Training Evaluation' by James and Wendy Kirkpatrick is recommended reading.
• 📈 The ultimate goal of using the Kirkpatrick model is to determine the effectiveness and return on investment of training programs in relation to business success.
• 💬 The script encourages viewers to engage with the content by asking questions and provides additional resources for those interested in instructional design.

Q & A

What is the Kirkpatrick model of evaluation?

The Kirkpatrick model of evaluation is a framework developed by Donald Kirkpatrick in the 1950s for assessing the effectiveness of training programs. It consists of four levels: reaction, learning, behavior, and results.

Why is the Kirkpatrick model considered valuable for training evaluation?

The Kirkpatrick model is valuable because it provides a comprehensive approach to evaluate training programs at different stages, from immediate reactions to long-term business results, offering insights that can help improve program effectiveness.

What does Level One of the Kirkpatrick model focus on?

Level One focuses on the reaction phase, which measures overall satisfaction and engagement of participants with the training experience, typically through surveys, interviews, and focus groups.

How is learning measured at Level Two of the Kirkpatrick model?

At Level Two, learning is measured through quizzes, tests, interviews, focus groups, discussions, observations, and presentations to assess if participants have acquired new knowledge, skills, or attitudes.

What is the purpose of Level Three in the Kirkpatrick model?

Level Three aims to evaluate whether participants are applying the new knowledge, skills, or attitudes in their job behaviors, which can be assessed through surveys, manager observations, work reviews, and key performance indicators (KPIs).

How does Level Four of the Kirkpatrick model differ from the other levels?

Level Four is the results phase, which is concerned with measuring the impact of the training on business outcomes, such as customer satisfaction, sales, and employee retention rates, rather than just learning or behavior changes.

Why is it important to plan for evaluation from the beginning of a training program?

Planning for evaluation from the start ensures that the training program is designed with clear objectives and desired outcomes in mind, allowing for more effective and targeted assessments throughout the program's lifecycle.

What is the recommended approach to planning a training program according to the Kirkpatrick model?

The recommended approach is to start with Level Four, defining the desired business results, and then work backward to determine the behaviors, knowledge, and skills needed, and finally design an engaging intervention to meet these objectives.

How can instructional designers use the Kirkpatrick model to improve their training programs?

Instructional designers can use the Kirkpatrick model to identify areas of strength and weakness in their programs, make data-driven adjustments, and ensure that training leads to measurable improvements in performance and business outcomes.

What book is recommended for those who want to delve deeper into the Kirkpatrick model?

The book 'Kirkpatrick's Four Levels of Training Evaluation' by James and Wendy Kirkpatrick is recommended for a more in-depth understanding of the model and its practical applications.

How can the Kirkpatrick model be used iteratively during a training program?

The Kirkpatrick model can be used iteratively by continuously evaluating and adjusting the training program based on feedback and performance data at each level, ensuring that the program stays aligned with its objectives and produces the desired results.

Outlines

00:00

📚 Introduction to Kirkpatrick's Model

The video script introduces the Kirkpatrick model of evaluation, a widely recognized framework in the field of instructional design and training program evaluation. Developed in the 1950s by Donald Kirkpatrick, the model consists of four levels: reaction, learning, behavior, and results. Each level represents a progressively deeper and more valuable evaluation of training effectiveness, starting with immediate reactions to the training experience and culminating in the assessment of long-term business results. The script emphasizes the importance of understanding each level and suggests that while the first two levels are commonly assessed, the latter two offer significant insights into the true impact of training on business performance.

05:02

📈 Planning and Evaluating Training Programs

This paragraph delves into the planning phase of training programs, highlighting the importance of aligning them with business outcomes from the outset. The script suggests a backward planning approach, starting with the desired business results (level four) and working down to the necessary behaviors (level three), knowledge, and skills (level two), and finally the design of the training intervention (level one). It stresses the iterative nature of evaluation, allowing for adjustments to be made during the training process to ensure it leads to the desired changes in behavior and business outcomes. The script concludes with recommendations for further reading and resources for those interested in instructional design, including a book by James and Wendy Kirkpatrick and a video on acing instructional design interviews.

Keywords

💡Kirkpatrick Model

The Kirkpatrick Model is a framework developed by Donald Kirkpatrick in the 1950s for evaluating training programs. It is the central theme of the video, emphasizing its importance in the field of instructional design. The model consists of four levels of evaluation: reaction, learning, behavior, and results, each becoming progressively more difficult to measure but more valuable for assessing the effectiveness of training.

💡Training Evaluation

Training Evaluation refers to the process of assessing the effectiveness of training programs. In the video, it is tied to the Kirkpatrick Model, which provides a structured approach to measure the impact of training at different levels. The script discusses how to use this model to evaluate not just the immediate reaction to training but also the long-term behavioral changes and business results.

💡Reaction

In the context of the Kirkpatrick Model, 'Reaction' is the first level of evaluation, focusing on the participants' immediate response to the training. It is typically measured through surveys that gauge satisfaction and engagement. The video script mentions how questions about the experience and its relevance to one's job are used to assess this level.

💡Learning

The 'Learning' level of the Kirkpatrick Model is the second level, which assesses whether participants have acquired new knowledge, skills, or attitudes. The video explains that this is often measured through quizzes, tests, or other evaluative methods that demonstrate what has been learned, such as presentations or discussions.

💡Behavior

Behavior refers to the third level of the Kirkpatrick Model, which evaluates whether the training has led to changes in the participants' actions on the job. The video script suggests that while surveys can be used, observation and work reviews are more effective methods to determine if new skills are being applied in the workplace.

💡Results

'Results' is the fourth and most challenging level of the Kirkpatrick Model, which seeks to determine if the training has led to the desired business outcomes. The video emphasizes the importance of connecting training to tangible business metrics like customer satisfaction or sales to measure the true impact of the training.

💡Instructional Designer

An Instructional Designer is a professional who designs and develops educational programs. The video is aimed at individuals looking to become instructional designers, providing insights into how they can use the Kirkpatrick Model to evaluate and improve their training programs.

💡Surveys

Surveys are a common tool for gathering data at the Reaction and Behavior levels of the Kirkpatrick Model. The video script explains that surveys can ask about satisfaction, relevance, and engagement, and can also be used to gather information from managers about changes in employee behavior post-training.

💡KPIs

KPIs, or Key Performance Indicators, are metrics used to evaluate the success of a company or individual in achieving their objectives. In the video, KPIs are mentioned as a way to measure the Results level of the Kirkpatrick Model, such as tracking the use of a new skill like screen sharing in a call center environment.

💡Planning Phase

The 'Planning Phase' is highlighted in the video as a crucial step in the evaluation process. It involves determining the desired business outcomes before designing the training program. The script suggests starting with the Results level of the Kirkpatrick Model during this phase to ensure that the training is aligned with organizational goals.

💡Iterative Process

The video describes the evaluation process as 'Iterative,' meaning it is not a one-time event but rather a continuous cycle of assessment and adjustment. This allows for real-time improvements to the training program based on feedback and results, ensuring that it remains effective and relevant.

Highlights

The Kirkpatrick model of evaluation was developed in the 1950s by Donald Kirkpatrick and has become the most popular model for training evaluation.

The model consists of four levels of training evaluation: reaction, learning, behavior, and results.

Level one, reaction, measures overall satisfaction and engagement through surveys, interviews, and focus groups.

Level two, learning, assesses the acquisition of new knowledge, skills, or attitudes using quizzes, tests, interviews, and observations.

Level three, behavior, evaluates whether employees are performing differently on the job after training.

Level four, results, examines the impact of training on business outcomes and KPIs like customer satisfaction and sales.

Most learning professionals focus on levels one and two, but levels three and four provide more value and insight.

Evaluation planning should begin with defining the desired business results, aligning with level four.

Identifying the behaviors needed to achieve business results is crucial for effective training evaluation.

Understanding what knowledge and skills are required for different behaviors is key to level two evaluation.

Designing an engaging and effective training intervention is essential for knowledge and skill acquisition.

Starting with level four when planning a training program ensures alignment with organizational goals.

The Kirkpatrick model is an iterative process, allowing for adjustments and recalibrations based on evaluation data.

For a deeper understanding of the Kirkpatrick model, 'Kirkpatrick's Four Levels of Training Evaluation' by James and Wendy Kirkpatrick is recommended.

Instructional designers can use the Kirkpatrick model to excel in their field and interviews.

The video provides a comprehensive guide on how to implement the Kirkpatrick model for training evaluation.

Engagement through surveys is a common method to measure reaction, but observation and work reviews can also provide valuable insights.

Evaluating the effectiveness of training programs requires looking beyond just learning outcomes to actual business impact.

Transcripts

if you're trying to become an instructional  designer or evaluate a training program  

it's a great idea to learn more about  the Kirkpatrick model of evaluation

so the Kirkpatrick model of evaluation was  developed all the way back in the 1950s by  

Donald Kirkpatrick himself since then it has  hands down become probably the most popular  

model for training evaluation so again whether  you're just learning about the field or if you  

want to start implementing this model to evaluate  your own programs this is a very very good place  

to start so let's dive into it the Kirkpatrick  model is made up of these four levels of training  

evaluation so as we move down this list so you  know it starts with level one the reaction then  

learning then behavior then results as we work our  way down the levels become increasingly difficult  

to evaluate but also increasingly valuable for us  to conduct those evaluations so we're just going  

to go through each of these levels in order to  make sure you know what each of them consists of  

so the first one is reaction how are  people reacting to the experience  

so what we're really measuring at this point  is the overall satisfaction and engagement  

so you might see questions on how satisfied are  you with this experience would you recommend this  

experience to a friend you know were you engaged  and some of these things can also be measured by  

the like the facilitator or an observer but most  commonly these this is measured via a survey  

so at the end of the course you take a survey  asks you questions about how satisfied you are  

how relevant what you feel what you learned is to  your job and this can also be done in interviews  

and focus groups but again surveys are definitely  the most common way to measure reaction data

so level two this is where we're actually  measuring the learning so are these people  

learning new knowledge new skills or new  attitudes that's the big question for level two  

and the most common ways to measure this are with  quizzes or tests so you know pre-test post-test  

or like little knowledge checks throughout an  experience but also interviews focus groups  

discussions observations and and the list goes on  you know you can have people give presentations  

and you can evaluate how well they do there to  determine whether or not they learn something  

but you get the idea this is where  most learning professionals stop so  

in the field you know instructional designers  in this space of learning and development most  

of the evaluation is let's give people a survey at  the end of the experience to see if they liked it  

and let's give them a test to see if they scored  at least 90 and learned something so those are  

you know that is a way to address level one and  level two but where things start getting really  

valuable and interesting are at level three so this is behavior we wanna know are people  

performing these critical tasks that we identified  are they actually behaving differently on the job  

now that we taught them these new knowledge  skills or attitudes so are they using what  

they learned on the job that's the big question  for level three and again you can use surveys  

for this probably not the most effective method  but you can imagine you know if you're surveying  

people's managers and asking them about how their  employees are performing you might be able to  

get some valuable data about how the employees  are performing differently observation though  

is a common way you know the managers would  be observing the employees to see if they're  

using those new skills or that new knowledge work  reviews again you want to see if these new skills  

were implemented and you can look at KPIs to see  things like you know if you're working at a call  

center and you want to teach people to use screen  sharing on their calls you would look at that  

screen sharing key performance indicator to see  you know are people actually using this new skill  

so again this is this is like the gray area  for a lot of instructional designers that  

it's it's not very common to actually get into  this and see are people performing differently  

but this is very important and valuable  to know because once you know whether or  

not people are performing differently then you  can move into level four which is where you see  

are we producing are we seeing their desired  results from this intervention that we designed  

and developed so are we actually seeing results  for the business so this is where we're looking  

at actual business and sales metrics so we're  not just looking at oh are people screen sharing  

but are people providing a better customer  experience you know are we seeing higher  

customer satisfaction ratings and is that leading  to more sales or a better customer retention rate  

so again this is obviously like gold status what  you want to be measuring when you're using so much  

time and resources and effort to designing these  these big training programs you want to determine  

whether or not they're actually producing a return  for the business and helping the business succeed  

overall not just are people learning something  and they like what they learned because you  

know that's one thing but connecting that to  the business is a completely different piece

so obviously you can't just design a  training program put it out there and  

then start thinking okay now how  am I going to evaluate this thing  

I mean you can probably try piecing that  together but it won't be extremely effective  

so most of the hard work for evaluation comes  up front in the planning phase so let's look at  

how you might plan to use you know how you  might plan to evaluate your learning experience  

so the first question you have to ask yourself is  which results are we trying to achieve here what  

are we trying to do for the business are we trying  to reduce that are we trying to increase that  

customer retention rate are we trying to reduce  the number of you know are we trying to reduce the  

number of employees who quit so increase employee  retention are we just trying to increase sales  

you get the idea and you see this first bullet  the very first question we want to ask ourselves  

which results do we want to achieve for the  business that lines up with level 4 evaluation  

okay so take note of that next we need to ask  what do people need to do differently to achieve  

those results so keyword what do people need to  do differently what behaviors do we want to see  

and you'll see the second question lines up  with level three behavior evaluation so we're  

working our way backwards through these levels  to plan our learning program or our intervention  

so now we might ask okay what knowledge and  skills do these people need to do these things  

differently so again we're moving into level two  and then finally how do we design an attractive  

intervention so that people want to engage with  it and and acquire those knowledge and skills that  

they need so just to jump ahead here you see  we are we're basically you want to start with  

level four when you're planning an experience like  this if we don't start oh how am I going to design  

something that people like oh how will I help  them learn something that's a counterproductive  

way to go about this when we're actually trying  to produce results for an organization we need  

to start with those results in mind and orient all  the rest of our efforts to producing those results  

so this is this is a more useful way to think  about the Kirkpatrick model just from a planning  

perspective again level three and level four not  many people venture into that area but that is  

where you will see you know that's where you'll  get the most value out of your evaluation efforts  

because it will actually help you determine  whether or not your training program or  

intervention was worthwhile in the first place  so again it is always an iterative process so  

if you like evaluation doesn't have to come at  the end of your experience if you're finding  

as you go that what your efforts aren't leading  to these changed behaviors or these results you  

can always adjust at any point recalibrate and you  know use this data to keep you on the right track  

so I hope that this was helpful if you  have any questions drop a comment below  

this book right here Kirkpatrick's  Four Levels of Training Evaluation  

by James and Wendy Kirkpatrick is a great read if  you want to dive deeper into training evaluation  

and how kirkpatrick's model is implemented at  organizations today definitely check out this book  

I'll link that in the description and also if  you are here because you're trying to become  

an instructional designer and you want to use  this info to ace your interview check out my  

full video on becoming an instructional designer  which I will also link in the description  

so thank you again give it a thumbs up if you  liked the video and I will see you in the next one