Getting Started Using Data-Driven Learning Design

Using Data to Improve Learning Design
Lori Niles-Hofmann @loriniles | www.lori.ca

Lori Niles-Hofmann
Senior learning strategist
20 years L&D experience (banking,
consulting, marketing)
Author: Data-Driven Learning Design
Blog: www.lori.ca

@loriniles www.lori.ca
Diagnose vs. Autopsy

The Learning Autopsy?
▪ Completions
▪ Level 1-4 Evaluation
▪ ROI
▪ Too late
▪ Costly redesign
▪ Reputational risk
▪ Questionable Accuracy
…the room was too cold, the food
was not tasty

In the Classroom

Digital Distant Relationship

Paper to Online News
▪ “If it bleeds, it leads”
▪ Frontpage news cycle: 12-24 hours
▪ One metric: newspapers sold

Digital Changed News Media
▪ Real-time metrics on consumption
▪ Minute-by-minute pivoting
▪ Data revealed insights that challenged
assumptions

Insights

Prediction vs. Reality: November 28th

Data reveals truths that
could challenge what we
presume about our
learners…
…it also explains the
Kardashians

Is Marketing a Bad Word?

Marketing vs. Learning
Both:
▪ Inform their audience with the intent of changing a behaviour
▪ Struggle with short attention spans and competition
▪ Face to face relationship is gone

Two Approaches
Marketing
• Data analysis for trends
• Content designed against
insights
• Monitor and pivot
• Cyclical
Learning
• Performance consulting
• Learning objectives
• Content design and
development
• Evaluation
• Linear

Downloads Drop-offs Time of Day
Likes
Shares
Comments
Trends
Mobile vs. Desktop

“Every drop-off, click, or
share, is a learner shouting
their likes and dislikes.
These actions are the eye-
rolls, smiles, and arms-
crossed from the classroom,
simply in digital format”
Digital Body Language

What If…
…we considered the digital body language
of our audiences to make design decisions?
…we used data like digital marketers do?

I Predict
We will no longer measure
learning by completions,
but by engagement and
sentiment scores

Engagement vs. Completion

Company
Response?
The manager had
completed AHS food
safety training with a
mark of 96 per cent
statement noted Webb had completed AHS
food safety training with a mark of 96 per cent

Data-Driven Learning Design
Respond
Monitor
Discover
Insights

Discover Insights
LMS IT Department Analytics
Most LMS track far
more than most
organisations explore.
Consider:
• Time of day/day of
week most active
• Mobile vs. Laptop
• Search terms
• Incompletes
If your company has an
intranet, inquire about
some of these data
points:
• Top searches
• Failed searches
• Pages most visited
• Time of day/day of
week most active
Simple internet
searches will reveal
loads of data on trends.
Consider:
• YouTube or Vimeo
Analytics
• BuzzFeed
• Marketing blogs

Insights
Strong
preference for
video
Most active
Fridays at noon
Most active
Monday
morning
Strong
preference for
video
High mobile
usage
Text format
preferred

Respond
Data Point Action
Top searches, Failed searches • Promote content aligned
with search preferences
• Determine if you need
content to fill gap from failed
searches
• Use in performance
consulting conversations
Pages most visited • Determine why the pages
are most visited and what it
tells you about what your
learners value
• Promote content on these
pages

Respond
Data Point Action
Time of day/day of week
most active
• Publish content at these
key times for maximum
exposure
Drop off rates • Align length of content to
be lower than attention
threshold
Mobile vs. Laptop • Use this data to make
better decisions about
media choices

Uncomfortable Truth?
The best performance consulting
can be derailed by time, money, and
stakeholder opinion

5 Pieces of Data to Enhance Performance
Consulting
▪ Intranet Search Terms
▪ Time of Day/Week Most Active
▪ Seat Time
▪ Downloads, Views, Likes, and Shares
▪ Mobile vs. Desktop

Story 1: Talking Heads
▪ Stakeholder wanted to be
featured in seven minute
video
▪ Similar videos: ~80% drop-off
rates by 30 seconds
▪ Testimonial style videos by
colleagues had significantly
higher viewing lengths

Story 2: Content Gap
▪ SME and stakeholders want elearning solution on topic X
▪ Feed of intranet searches reveal topic X ranking #152
▪ Decision is made to pivot to build content against top 3
search terms and minor article on topic X

Cautionary Tale
▪ The average life expectancy in
Canada is 81.24 years
▪ Most Canadians like maple
syrup
▪ Therefore, maple syrup
prevents people from living past
82

Only the Beginning
“It is not enough to make learning shorter,
into micro-sized bites, to appease short
attention spans.
The real magic is decoding digital body
language to captivate and engage learners,
extending the impact and depth of learning”

Look at your data before designing
Go beyond the LMS
Be prepared to pivot
Use data points to better manage
SME and stakeholder conversations
Digital Body Language

Final Thought

Getting Started Using Data-Driven Learning Design

More Related Content

What's hot

Similar to Getting Started Using Data-Driven Learning Design

More from David Blake

Recently uploaded

Getting Started Using Data-Driven Learning Design