Business Challenge
Catholic Relief Services (CRS) provides aid to people in need during crises and emergencies around the globe. In order to provide the best care, their training helps ensure volunteers and staff members perform well and feel more prepared in emergency situations.
CRS needed the ability to determine the effectiveness of different levels of both in-person and online training and determine if training was helping improve the performance of CRS staff members.
Data Challenge
Most of CRS’ training activities are inherently untrackable—such as simulation, classroom training, self-assessments, and peer assessments—making it nearly impossible to identify learner behaviors and measure preparedness. Furthermore, their data challenges included:
- Learner paths varied from person to person, making it difficult to represent the branching visualization in a user interface.
- Receiving meaningful data at specific points throughout simulations wasn’t possible, as Storyline's xAPI tracking is limited to standard events—such as viewing slides and answering questions—not interactions specific to a simulation story.
Solution
All CRS systems and applications were outfitted with xAPI so Watershed could pull standardized, meaningful data. System integrations included:
- Cornerstone On-Demand courses launched through SCORM Cloud
- Articulate Storyline to provide simulation environments (See Fig. Learner Path Visualization)
- Classroom checklist iPad application used by instructors
- Performance impression checklist with a third-party assesso
- Self-assessment checklist for employees to gauge readiness and comfort level
Watershed’s implementation was designed to capture the appropriate and necessary data—specifically to help show red flags and significant trends.
Additionally, Watershed’s data visualizations helped establish the relationship among training and emergency preparedness. This data will eventually be used to pinpoint how training relates to performance during emergency response.
