Does your L&D department exist to make sure everybody has completed their training with a smile? Or does it exist to make people better and the organization more effective? If it’s the latter, you should be measuring the impact your training has on learners and the business.
And in this post, we’ll explain the significance of training metrics and explore the business case for using learning measurement to show L&D's impact on the business.
We’ve covered several business cases for using a learning analytics platform, such as Watershed, to measure content utilization and learner satisfaction. But why are these L&D metrics important?
Well, it’s not a good sign if people dislike or don’t complete your training. And it’s even more concerning if your training isn’t having its intended impact on organizational goals.
In our 2022 Measuring the Business Impact of Learning survey, nearly half of the L&D professionals said they were measured by either learner satisfaction (32%) or content utilization (14%), and a further 15% were not evaluated at all. So are completion and learner satisfaction the purpose of learning?
What is learning impact measurement?
Learning impact measurement analyzes each step of the training content creation process to determine how a piece of content benefits learners in doing their jobs and the organization as a whole.
It also helps you improve inadequate training while creating future content that better aligns with organizational goals.
Why does learning measurement matter?
Joseph Stiglitz, a Nobel prize-winning economist, once said, “If we measure the wrong thing, we will do the wrong thing.” He argues that gross domestic product (GDP) is a poor measure of a country’s success because it leads to bad decisions. And this principle can be true for L&D if you're using the wrong metrics. In other words, what we measure will affect what we do.
For instance, if L&D’s performance metrics relate to course creation and content completion, then L&D teams will be motivated to create as much content as possible—regardless of the business impact of those courses. Or if L&D is measured on learner satisfaction, they’ll set their sights on happy learners rather than organizational goals.
By contrast, if L&D’s performance metrics relate to business performance, L&D teams are motivated to care about business performance. They have the incentive to take the time to explore with business partners what learners need to do differently to perform better and ensure the training leads to the change required.
Learning impact measurement matters because having a business impact matters.
If you work in L&D, I’m sure you share my belief that learning has the potential to improve both people and organizations. But I think we also have to recognize that it has the potential to be a complete waste of time too.
Your organization probably has training in both categories, so measuring its impact helps ensure L&D’s efforts improve people and positively affect the bottom line.
Recommended Reading
How do I know if training is impactful?
For training to have an impact, this chain of steps needs to occur:
- The learner must engage with and complete the content. If they don’t take the time to understand and finish training it is not successful.
- The learner must gain knowledge or develop their skills from the content. If they complete the training but learn nothing, the content is unsuccessful.
- The learner must apply that new knowledge or skills in their work. If a learner doesn’t apply new skills or knowledge, the training is unsuccessful.
- Their application of learning must lead to business results. The training isn’t successful if a learner applies new skills or knowledge to their job, but the results don’t match intended outcomes or business goals.
Learning impact measurement is time consuming. To do it properly means:
- Defining program, performance, and business goals prior to creating learning,
- Investing in impact measurement as additional spending beyond your current L&D budget, and/or
- Producing less content and redeploying the resources and budget into impact measurement.
If your performance metrics relate to completions and learner satisfaction, reducing the amount of content you produce may be daunting.
However, creating and curating a smaller amount of effective training content should result in significant business impacts without wasting time churning out potentially ineffective courses.
Recommended Resources
We've covered measuring learning's impact extensively over the years, including in these resources:
- Learning Evaluation Models. Check out our overviews of Kirkpatrick, Kaufman, Brinkerhoff, Anderson, and Phillips training evaluation methods.
- Watershed's 7 Steps of Learning Evaluation. This series outlines a 7-step process to evaluate the impact of a piece of training.
- How to Design Effective Training for Learning Evaluation & Analytics. Impact measurement starts before you begin designing content. Without attaching business goals to learning, measuring how that learning will achieve those goals is nearly impossible (i.e. What are you trying to achieve, and how do you know if you've accomplished it?). This series explains how to design training with impact measurement in mind.
How do real-world organizations measure learning impact?
Impact measurement is especially critical in medical settings, where mistakes can have significant consequences. For instance, Nebraska Medicine uses Watershed to measure the impact of multiple learning programs and initiatives, such as their workplace hazard audits.
These audits were time consuming for observers, and there was a delay between when someone conducted an audit and when results were available—making it impossible to provide timely feedback to healthcare providers.
By moving this process to xapify’s observational checklist and reporting on the results in Watershed, healthcare providers and their managers can now see results within minutes. Furthermore, compliance has improved, and directors are proactively reviewing results in Watershed to initiate interventions, as shown in the following screenshot.
Recommended Reading
Another healthcare organization used Watershed to evaluate a learning app’s impact on the assessed simulation performance. They discovered that app utilization was so minimal that only a handful of people who completed the assessments had actually used the app. Therefore, the app was not impacting the assessments.
This is a good illustration of why evaluating every step of the chain is essential and why utilization metrics are vital for impact measurement. If learners are not using the learning, it is not having an impact.
And in one last example, industrial supplier Applied wanted to improve the business performance of its service centers by addressing gaps in the financial competencies of service center managers and then providing coaching and training targeted to address those gaps.
In particular, they wanted to support managers who already had good subject knowledge but struggled to apply it.
Their L&D team collected data about business performance metrics and knowledge assessments mapped to those metrics.
As a result, managers and their coaches could quickly identify areas of low KPI performance and compare that information to the relevant knowledge assessment. Coaches then used this information to tailor each manager's coaching and training program to improve their performance.
Applied Webinar
How does Watershed support training impact measurement?
Watershed helps you bring together all your data—ranging from learning and assessment data to job performance and business data—to create a complete chain of evidence—from learning activity to learning acquisition and application to business impact.
Once you’ve collected all your L&D data in Watershed, you can use the Report Builder tool to configure reports and dashboards to visualize your data. Report Builder also helps you ask questions of your data so you can quickly get the answers you need, which may prompt even more questions for exploration.
Watershed can help you understand data trends, the impact of learning on the business, and where you need to fix the chain of evidence.
Making the Case: Why the business needs to measure training impact
If you’re not measuring learning’s impact on job performance, you’re probably not designing learning to impact job performance.
When you measure the business impact of training, you’re working to ensure that the training has the intended business impact. To put it another way, the business goals of the training are the same as the business goals of the impact measurement.
If the business goals are the same, there is also likely to be a lot of overlap between the business case for the training and the business case for the analytics. This overlap means you can use the business case you’ve developed for the training as a starting point when building the business case for the analytics.
That’s because if you’re spending money on creating or curating learning content and paying for learners’ time completing that training, you need to know that investment is worthwhile.
Measuring business impact and holding L&D accountable for that impact can have a transformational effect on the L&D function. You can shift from churning out courses to driving performance improvement, which should lead to better business outcomes.
Recommended Webinar
How can I convince stakeholders of the value?
Your organization probably invests a lot of employee time completing training, which can add up to many paid work hours. So if you’re not measuring business impact, you don’t know if that training is making a positive difference on productivity. You also don’t know how much of that money is simply wasted.
Measuring learning’s impact on the business shows you how to optimize training’s impact and where to remove any scrap learning. And considering all your learners, L&D and the organization as a whole stand to see significant benefits and cost reductions
Also, consider the value of providing feedback on people’s work. If you’re not measuring the business impact of learning, then instructional designers and others on the L&D team are not getting meaningful feedback.
Measuring learning’s impact gives instructional designers valuable insight into what works, helping them enhance their capabilities while also improving the capabilities of your workforce.
Using Watershed and learning analytics to measure the business impact of training equips you with the knowledge to remove bad learning while designing content that addresses the needs of learners and stakeholders.
Understand your stakeholders and how they will benefit from measuring the impact of learning.
| Stakeholders | Pain Points | Benefits |
|---|---|---|
| C-Suite (CLO, CEO, CFO) | Senior leaders want to know how investments in learning impact the business. | Impact measurement informs the C-suite about how investments in learning affect goal achievement. |
| Human Resources | HR wants to reduce time to competency during onboarding. | Impact measurement optimizes onboarding programs, reducing time to competency. |
| Learning Leaders | Learning leaders want to prove that learning has an impact across the organization. | Learning leaders can use impact measurement to show learning’s effectiveness. |
| Instructional Designers | Instructional designers want to improve the impact of their learning content. | Instructional designers can use impact measurement to identify effective learning and inform the design of future learning programs. |
| Compliance | Compliance teams need to prove that related training is effective. | Compliance teams can use impact measurement to identify when compliance training leads to compliant behavior. |
| Line Managers | Line managers want assurance that time spent learning instead of working is worthwhile. | Line managers can use impact measurement to illustrate the real impact of particular training programs on productivity and effectiveness. |
| Learners | Learners want to know that their time spent learning is worthwhile. | L&D can use impact measurement to show learners the positive impacts of a training program. |
Recommended Reading
Next Course: How to use search analytics to offer the right training
Having impactful, relevant learning opportunities is vital in creating a learning culture where learners take responsibility for their development. In such a culture, learners need to be equipped with tools to find the most appropriate and helpful content for themselves.
One of those tools is likely the ability for people to search for content covering topics and skills they need to learn. Searching for learning generates uniquely valuable data because, while most of your L&D data tells you what people are learning, search data tells you what people want to learn.
You can use this information to inform what learning opportunities to provide. And the next post in this series looks at the business case for search analytics, explaining the value of being responsive to learner demand. Be sure to subscribe to our blog above so you don’t miss out!
About the author
As a co-author of xAPI, Andrew has been instrumental in revolutionizing the way we approach data-driven learning design. With his extensive background in instructional design and development, he’s an expert in crafting engaging learning experiences and a master at building robust learning platforms in both corporate and academic environments. Andrew’s journey began with a simple belief: learning should be meaningful, measurable, and, most importantly, enjoyable. This belief has led him to work with some of the industry’s most innovative organizations and thought leaders, helping them unlock the true potential of their learning strategies. Andrew has also shared his insights at conferences and workshops across the globe, empowering others to harness the power of data in their own learning initiatives.
Subscribe to our blog
