A Practical Guide to Evaluating eLearning Effectiveness

Part II: Models That Make It Make Sense

Once you’ve asked the right questions, it’s time to anchor your answers in something solid. That’s where evaluation models come in. They help you interpret the story behind the data and structure your approach to measuring success.

While Kirkpatrick might be the household name, it’s far from the only option. Depending on your goals, stakeholders, and the type of training you’re delivering, a different framework might suit you better or add helpful nuance to your existing strategy.

Kirkpatrick’s Four Levels

Kirkpatrick’s model is the OG of training evaluation frameworks. Developed in the 1950s by Dr. Donald Kirkpatrick and still widely used today, it’s popular because it offers a clear, step-by-step approach to measuring training effectiveness across four distinct levels:

1. Reaction – How did learners respond to the training? Did they enjoy it? Was it relevant? This level captures learner satisfaction and engagement, typically through post-course surveys (aka “smile sheets”).
2. Learning – What did learners actually gain from the experience? This measures knowledge or skill acquisition, often using pre/post assessments or quizzes.
3. Behavior – Are learners applying what they learned back on the job? This step gets trickier: it requires time, observation, and often collaboration with managers to identify behavior change in the real world.
4. Results – What’s the broader business impact? Think increased sales, reduced errors, improved safety metrics, higher customer satisfaction, or stronger retention.

The strength of Kirkpatrick’s model is its accessibility. It’s easy to communicate to stakeholders, and it encourages you to think beyond training completion and quiz scores. But here’s the catch: the further up the model you go, the harder (and more resource-intensive) it becomes to collect meaningful data. That’s why many organizations stop at Levels 1 and 2, and that’s also why important change can go unnoticed.

Still, when implemented with intention, the model provides a solid framework for aligning training with performance and organizational outcomes. It also pairs well with more modern models like LTEM or Phillips ROI when you need to zoom in further.

♦️ Use when: You need a structured, stakeholder-friendly way to evaluate training from learner satisfaction all the way up to business value.

Phillips ROI Model

The Phillips ROI Model builds on Kirkpatrick’s framework by taking it one step further: adding a fifth level that calculates the financial return on investment (ROI) of training. Created by Dr. Jack Phillips, this model emphasizes not just outcomes, but the value of those outcomes in dollars and cents.

Here’s how it stacks:

1. Reaction
2. Learning
3. Behavior
4. Results
5. ROI – This level weighs the monetary benefits of training against its costs, including time, tools, and facilitation. It requires careful data collection and often includes isolating training as a variable, using control groups or trend data.

Phillips also encourages evaluation of the reasons behind success or failure, offering a more diagnostic perspective than Kirkpatrick’s.

♦️ Use when: You need to show stakeholders exactly how training impacts the bottom line and justify continued investment in L&D.

LTEM (Learning Transfer Evaluation Model)

Developed by learning scientist Dr. Will Thalheimer, LTEM addresses one of Kirkpatrick’s major gaps: how to meaningfully measure whether learners actually transfer knowledge and skills into action. It outlines eight increasingly meaningful levels of evaluation:

1. Attendance – Tracks whether learners showed up for the training. It’s the most basic metric and confirms exposure, not impact. You can measure this with completion rates or sign-in data.
2. Activity – Measures whether learners actively engaged with the training content. This includes time-on-task, click rates, module progress, and where they might be dropping off.
3. Learner Perceptions – Gauges how learners feel about the training: Was it relevant, useful, enjoyable? Use surveys, feedback forms, or discussion boards to gather both quantity and quality of feedback.
4. Knowledge – Assesses what learners retained through quizzes, knowledge checks, or assessment scores. This is your go-to for measuring basic understanding.
5. Decision-Making Competence – Can learners make smart choices in real-world scenarios? Branching scenarios, simulations, or situational judgment tests help surface their reasoning skills. in realistic contexts? Use scenarios, branching logic, or case-based questions that show nuanced understanding.
6. Task Competence – This is about hands-on execution. Can learners actually do the job? Use demos, performance reviews, or skill-based assessments to evaluate this. Track via practical demos, peer reviews, or hands-on task assessments.
7. Transfer – Checks whether learners are applying what they learned on the job. Look for behavior change through manager feedback, observation, or follow-up interviews. Look for behavior change through manager observations, peer check-ins, or post-training reviews.
8. Transfer Effect – Ties training to business outcomes. Fewer support tickets, increased sales, reduced errors, higher customer satisfaction—this is where you measure the bottom-line impact. Think fewer support tickets, improved sales, higher retention, or better patient outcomes.

Unlike Kirkpatrick, which often stops at “behavior,” LTEM offers a more granular breakdown of what effective transfer looks like and how to measure it with validity. It also separates fluff metrics (like participation) from actual indicators of learning.

♦️ Use when: You’re serious about proving skill application and long-term learning impact, not just engagement or satisfaction.

Success Case Method

Developed by Dr. Robert Brinkerhoff, the Success Case Method (SCM) is part evaluation tool, part storytelling strategy. Rather than evaluating everyone, SCM zeroes in on two groups: the most successful learners and the least successful ones.

The goal is to figure out what made the difference: what systems, supports, or behaviors helped top performers apply their learning, and what held others back. The method involves interviews, case studies, and data-backed narratives to reveal practical, actionable insights.

It’s especially useful when full-scale evaluation isn’t possible, but leadership still needs compelling proof of impact.

♦️ Use when: You need meaningful case studies to show leadership what’s working and where your training strategy needs a boost.

Part III: Tips for Better eLearning Assessments

If you’re serious about evaluating the success of your training program, you can’t skip the assessments. Thoughtfully designed assessments give you more than a score, they offer a pulse check on what’s landing, what’s being retained, and what’s actually making a difference on the job.

Check out the full article on how to strengthen your assessment game and ensure your evaluation efforts are rooted in real learner progress:

Illustration of man shooting a dark into a giant bullseye in the sky (Midjourney, 2025).

Part IV: The Long Game…Continuous Improvement

A strong evaluation strategy isn’t a one-and-done event, it’s an ongoing conversation. Great training programs don’t just measure success once and call it a day. They build a feedback loop that helps teams adapt, improve, and keep pace with real-world change.

Here’s how to stay in it for the long game:

• Set quarterly or biannual evaluation cycles. Regular checkpoints help you catch what’s working (and what’s not) before small issues become big ones.
• Align your learning goals to business KPIs. Don’t just measure learning for learning’s sake. Tie your outcomes to things like productivity, compliance rates, or customer satisfaction.
• Track behavior change over time. Use follow-up surveys, manager check-ins, or observational rubrics to see if training is actually shifting behavior six weeks, or six months, down the line.
• Keep learner feedback alive. Build ongoing feedback mechanisms like pulse checks, feedback prompts at key moments, or informal interviews.
• Revisit your content and design regularly. What worked last year may not work today. Trends change. Workflows change. People change. Be ready to iterate.

Evaluation isn’t a final exam. It’s your GPS. And the more consistently you use it, the more confidently you can steer toward impact.

Final Thoughts: The Real ROI?

When learners grow, your business grows. Full stop. That’s why evaluation matters. Not to chase a perfect score, but to ensure we’re building learning experiences that work, that stick, and that move people forward.

So the next time you hit “Publish” on a course, don’t just celebrate the launch. Plan your check-in points, watch what happens next, and get ready to tweak, remix, and refine. Because that’s where the magic happens.