Learning Analytics Teams: Are They a Pipe Dream or Reality?

We’ve discovered insights that look at the state of play and are designed to help you on your path, such as:

Organizations that already have analytics capability on their L&D team are three times more likely to recruit for additional learning analytics staff than those that don’t have that capability.

The state of learning analytics: Our approach & methodology

This blog series explores the State of Learning Analytics, combining the results from two recent surveys, which provide insights from more than 1,000 L&D professionals on applying learning analytics in their organizations.

1. Measuring the Business Impact of Learning. Since 2016, Watershed has partnered with LEO Learning to conduct the annual Measuring the Business Impact of Learning (MBIL) survey.
2. Adopting Learning Analytics: Closing the C-Suite/L&D Language Gap. Watershed teamed up with Chief Learning Officer magazine to survey its readers on topics relating to L&D and learning analytics.

What’s a learning analytics team and why do you need one?

A learning analytics teams is dedicated to implementing, managing, and ensuring the quality and consistency of the learning ecosystem and data analytics.

Simply having software in place doesn’t solve the goal of applying analytics to your learning programs. Otherwise, you risk frustration, delays, or even failure. And for any team to be successful, it needs a combination of people who bring various skill sets that complement the whole group. So in this instance, you need the right mix of people to help you “interrogate” the data.

For example, possible roles might include a data and analytics lead, data engineer, data analyst, or data architect.The image below shows an idealized flow of responsibilities and roles in a learning analytics team. We fully acknowledge that in real terms, there is crossover between these roles—one of our clients has just one person carrying out most of these duties.

What does a learning analytics team do?

This will vary by organization capability and learning analytics maturity, but common goals and tasks include:

• Align learning with the business.
• Coordinate on Business KPIs.
• Translate business KPIs into learning design.
• Map and integrate your ecosystem data.
• Ensure data integrity.
• Communicate results with stakeholders.

Survey Results: Do most organizations have a dedicated team or individuals with analytics capability?

Anecdotally, we know that getting a dream team of all the various Learning Analytics roles can feel like wishful thinking, even for larger organizations.

That’s not to say there isn’t usually some sort of capability that L&D can call upon, even if it’s sourced from elsewhere within the business (still useful, but L&D reporting requests will often be treated as a lower priority compared to in-house).

Let’s take a look at what more than 1,000 of you told us in our survey responses:

• The CLO survey found that most organizations (73%) do not have a dedicated learning analytics team. Even in large organizations (20,000+ people), only 41% have a learning analytics team.
• This was also reflected in our MBIL survey that showed 60% of organizations that do not have analytics capability. Of the 40% who do, half of them were either a data analyst or statistician.
• Perhaps unsurprisingly, the MBIL survey also found that having a budget for learning analytics meant you are more than twice as likely to have analytics capability on your team (61% with budget did, versus 30% in L&D functions without budget).

Despite the lack of a dedicated learning analytics team or analytics capability on the L&D team, 68% of organizations are either not expecting changes to their current staffing in relation to analytics or don’t have current plans but are starting to see growing demand.

A similar proportion (72%) gave these responses even when we filter the results to just those organizations that do not have analytics on the L&D team. There is clearly a long way to go before unmet demand sees the required investment in people.

By contrast, only 7% of all organizations said they were actively recruiting learning analytics talent, and that figure is even lower (4%) for organizations that do not already have analytics capability.

For organizations that already have analytics capability on their L&D team, 12% are actively recruiting. To put it another way, organizations that already have analytics capability on their L&D team are three times more likely to recruit for additional learning analytics staff than those that don’t.

Technology + Team = Full Learning Analytics Potential

The findings around people and teams contrast with our findings relating to technology. We'll cover technology in the next post—but for now, all you need to know is that about half of organizations plan to upgrade their technology with a shift toward learning analytics platforms and BI tools.

However, it's essential to implement any new technology with the right people in place to ensure it's used to its highest potential while yielding the best results.

In our experience, the main significant reason an implementation might drag is a lack of time and capacity on the client side to drive the process and the wider learning analytics project. This can be due to a staffing change.

For example, a key champion moves to a position above or outside the L&D team without properly filling their original role. Other times, the team doesn't have anyone dedicated to learning analytics. So when other pressures arise, such as the launch of a new company-wide learning program, they distract from the learning analytics project.

That's why we recommend ensuring you have the right people in place alongside the technology to help ensure a successful and comprehensive learning analytics program. After all, what good is a plane without a pilot?

Recruit for passion and curiosity first, technical skill second

We interviewed several clients about their learning analytics teams and the skills and experience they looked to recruit. And we consistently heard that passion and curiosity are the most critical factors to consider.

People need to be excited about learning analytics and enjoy working with data. They also need to be curious and ask questions, taking time to dig into features of the data that jump out.

Technical analytics skills are also important, but those skills are easier to learn on the job as needed. Without a passion for learning analytics, people are likely to get bored quickly. And without curiosity, they are less likely to notice critical insights that can significantly impact business decisions.

Our clients also recommended hiring people who can focus specifically on learning analytics. It’s possible for other people on the L&D team—such as content development and instructional design—also to have responsibility for learning analytics.

However, 41% of respondents said the biggest challenge of measuring the impact of learning was competing priorities. So having people dedicated to learning analytics helps avoid work stalling when other pressures arrive. A dedicated team makes space for learning analytics—even when the rest of the L&D team is focused on other work.

Ask your technology providers for help

It’s okay if you don't have a complete set of learning analytics capabilities on your team. As we mentioned above, people can learn skills on the job. You can also lean on your vendors for support, especially for technical skills and the initial setup of data source connections with your learning analytics platform.

A great example is Nebraska Medicine's L&D team. They are running several innovative learning analytics projects, including:

• using observation checklists to monitor performance and track hand hygiene compliance, and
• tracking engagement with a complex branching elearning program.

These projects use off-the-shelf tools and support from vendors, including Watershed, for some of the more technical aspects.

Up Next: Tools and technology for learning analytics

People, technology, and budget are crucial for implementing your learning analytics strategy. Now that we've addressed the people-side of things, our next post will look at the technology.

We'll reveal organizations' confidence in their existing technology's ability to use data to support personalized learning. We'll also cover organizations' plans for upgrading their toolkit for learning analytics, which tools organizations currently use, and the tools organizations want to use in the future.