Taking Out the Trash: The Business Case for Scrap Learning Analytics

As part of our Building a Business Case for Learning Analytics series, this post outlines the case for using learning analytics to identify and remove scrap learning.

This relatively simple form of learning analytics is a great starting point for organizations to realize substantial cost savings by removing scrap learning from their L&D content licensing. These savings vary, but could easily reach the tens of thousands based on an organization’s size and diversity.

What Is Scrap Learning?

Scrap learning is learning content that fails to improve job performance. This type of learning, combined with unused content, can create noise that prevents learners from benefiting from good content and actively harms their learning efforts. This harmful learning content includes content that:

• has become dated and is no longer relevant or accurate due to changing circumstances, processes, practices, laws, and technologies.
• is old in terms of technology and is no longer compatible with modern devices (e.g. Flash-based eLearning).
• is stylistically old and offputting to learners.
• is ineffective at driving performance improvement.

And the worst thing about this learning content is that you’re probably paying for it—whether that’s license costs for external content libraries or hosting costs for content developed in-house. Not to mention any costs relating to mistakes made due to completing bad training.

It’s also possible that some learning that’s valuable to one part of the organization is not helpful to another. For example, a sales team might find a course on upselling incredibly useful, but a production team would consider that same course as scrap learning. So if you’re paying license fees for everybody to access that training, you’re wasting money. Any strategy to address scrap learning must consider differences in learners across the organization—a one-size-fits-all approach does not work.

Of course, while ascertaining the relevance and ideal audience for some topics, such as upselling content for a sales team, other topics may not be obvious. For example, do you know which departments need access to content on working remotely or using Microsoft Excel? That’s why you need to use learning analytics to identify who accesses what training content and the content’s impact on those learners.

Let’s Cut the Scrap (Learning)

One of our clients wanted to implement Watershed enterprise wide, but some stakeholders were concerned about investing in another platform for their ecosystem. However, when they realized they could use Watershed to identify scrap learning, the conversation shifted once they saw the potential cost savings they could make.

This following example Watershed report shows content viewed less than 50 times in the last year. It includes the number of total views, the number of people who viewed the content, the average time spent, and a timeline of when views occurred for each item. You also can configure reports to look at content viewed during the last month or week or filter to only look at specific parts of the organization.

Another enterprise client uses Watershed to monitor content usage from internal and external libraries.

Since implementing Watershed, they now know which parts of their organization use content and from which vendor. For instance, some content libraries had enough licenses for all their employees, but only a few departments used that content. So they reduced licensing only to cover those few departments and saw considerable savings in annual license costs.

How Can Watershed Help Detect and Prevent Scrap Learning?

Watershed brings together all your data about content access, usage, and completions. This includes data from your LMS, LXP, content libraries, video platforms, and more. And it’s not just data sent via xAPI—it’s all the data.

You can combine this data with HRIS data about your organization’s people and structure, enabling you to report on learning activity by different parts of the organization. And you can set up data connections (as xAPI data sources or CSV) to regularly auto-update the data, so reports are always current.

Once all your data is in place, you can use Watershed reports and dashboards to identify scrap learning and other unhelpful or unused learning content. Specifically, you can report on:

• Unused content. This may be clear-cut scrap learning, but it might include helpful content that learners have been unable to find or don't know exists.
• Content that only some departments use. This may be an opportunity to reduce the number of paid licenses.
• Content that has been started but not completed. This suggests learners want content on this topic, but perhaps did not find the particular content useful.
• Content that learners have started, but not passed. This suggests learners want the content, but it may not be effective at equipping them for the assessment (or there may be a problem with the assessment).
• Content that impacts performance and business outcomes. Content that's being used but not affecting performance is perhaps the worst kind of scrap learning because it's taking up learners' valuable time.

Having identified the scrap learning, you can remove, update, or replace it as appropriate.

^{This example Watershed report shows that while uptake of the Strategic Leadership Academy content has been high among VPs and directors, only 17% of managers have used this content. This report might suggest that providing paid licenses so all managers can access this training is wasted spending.}

Making the Case: Why the Business Needs Scrap Learning Analytics

Using Watershed to identify and remove scrap learning is the textbook quick-win business case for getting stakeholder buy-in. Connecting data sources—such as an LMS or content library—is easy, and setting up reports to identify scrap learning is simple. This means you can start identifying and addressing scrap learning early in your Watershed implementation process.

This business case can provide a clear, significant, and quick return on investment. In particular, if you can cancel unused eLearning content licenses, you'll have a clear, quantified, and immediate cost saving. You also gain access to in-depth L&D insights for improving your learning content's quality and accessibility.

Identifying scrap learning is not just a one-and-done job, however. You need to remain vigilant as learner demand for content changes, existing content ages, and new content is added to your library. And Watershed continues to provide value by helping:

• uncover new scrap content year on year, and
• inform content development and selection to prevent bad training content from being added in the first place.

How Can I Convince Stakeholders of the Value?

A simple mathematical formula may convince stakeholders of Watershed's value in addressing scrap learning content. Specifically, the annual cost of scrap learning is equal to how much you spend on developing, licensing, and hosting learning content that ends up being scrap.

You can estimate this cost by taking your total content development, hosting, and licensing fees and multiplying by the proportion of your content you think might be scrap.

And even if the numbers don’t quite add up, there are less tangible but equally significant benefits of addressing scrap (e.g. Learners save time because they don’t have to sift through bad content, which means they have more time for their daily work.). Remember that this business case is just your initial quick win. There are many other business benefits for using Watershed that can develop over time. For instance:

• Extended enterprise learning analytics—which tells you about content usage beyond your primary user base and learning ecosystem—might take longer to set up because it requires connecting additional data sources.
• Skill data and analytics—which tells you about people's skills in your organization—might be more complex to set up if you haven't already defined all the skills you want to analyze and monitor.

Understand your stakeholders and how they will benefit from extended enterprise learning analytics.

Next Course: Vendor Management Analytics

This post has argued for the value of getting rid of scrap learning content, but what if you have whole platforms or content libraries that aren't delivering value in terms of cost and efficiency?

Of all the systems and content you license, which systems provide significant value and which cost more than they are worth? Do you fully understand the value of platforms when discussing pricing with vendors? And if you had to say goodbye to one of your platforms or content vendors to cut costs, are you confident you would make the right choice?

Our next post sets out the business case for using Watershed to inform vendor management decisions, so you can ensure you have the needed data and insights to answer these questions and more.