An excerpt from Adam’s book Customer Education: Why Smart Companies Profit By Making Customers Smarter
I have a pet theory that SaaS businesses often don’t invest in Customer Education departments because “learning” is not a clearly measurable outcome.
You can’t put learning on a scoreboard (well, not in any meaningful sense). You can’t easily award the manager whose team “did the most learning.” There isn’t really a comparable quantifiable metric in Customer Education to deals closed, pipeline generated, or net renewals. This obstacle makes it tougher for you to define and champion the ROI of your Customer Education strategy.
Value metrics for Customer Education
In the past, Education Services teams didn’t have to deal so much with attribution metrics or defining and championing the ROI of their strategy. They were businesses in their own rights, selling training, so they could rely on metrics like revenue, bookings, and margins. Now, Customer Education is more aligned with the practice of Customer Success. It’s less important for many of these businesses to generate high margins on trainings than to ensure that accounts renew over time.
Danielle Campbell, Head of Americas Digital Learning Services at Adobe, notes the shift from Services to Success by saying: “These [traditional] metrics only provide a small window into the true health and value of a training business. The real proof is in the customer behavior. What are the outcomes of a trained customer?”
Christine Souza, whom you may remember from the section of this book on the customer journey, expands on this idea: “We keep talking about how we need to speak the language of the business, but just because you take training activity metrics and start calling them ‘Key Performance Indicators,’ that’s not going to cut it. We need to figure out what aligns meaningfully to the business.”
Cross-Disciplinary Thinking
When you think about the department-level success of Customer Education, I’d encourage you to think of it in a cross-disciplinary way:
Think Customer Success
In customer success, your activity metrics (like number of business reviews delivered, customer engagements) only matter if they deliver outcomes (adoption, renewal, and expansion).
Customer Success generally uses Renewal, Churn, and Customer Lifetime Value (CLTV) as its goal posts, and I believe Customer Education should as well. The main difference is that Customer Education requires infrastructure work, so it’s not as immediately measurable as an individual CSM’s gross churn rate.
Think like Marketing
It’s infamously difficult to measure marketing attribution. Although software is making it easier to measure the attribution that marketing campaigns have in digital channels, how do you really know whether a certain campaign led to a certain amount of sales-qualified leads?
That doesn’t stop Marketing teams from observing macro-level trends and experimenting with different approaches. In Customer Education, it may be even harder to measure the direct business impact, so an experimental approach is also beneficial before you can directly measure ROI.
Think like Product Management
The content development side of Customer Education is actually a lot like Product Management—after all, learning materials at their best are in fact products.
Many Customer Education content teams take pages from the book of Agile product development, planning their capacity for content development in sprints, and measuring themselves on the delivery and quality of content.
Think like Internal Learning and Development
The more sophisticated L&D teams out there have turned away from measuring activity like training hours and number of courses developed. Instead, they try to measure improvements in job performance for their trainees.
With Customer Education, look to the outcomes it generates for customers. Are they more enabled to do their jobs? If so, are they using your product more? Giving you higher CSAT scores? Contacting Support less? You should be able to see and measure a behavior change.
What can we take from these related disciplines? Ultimately, we should aim to find high-level metrics that don’t just describe the activity that we do, but what business outcomes our activity creates. At the same time, we should be able to describe our department health in terms of the products we deliver and the impact that those products have on the customer journey.
In a perfect world, you would be able to measure the exact impact your education programs had on every customer’s health. If you have access to a data science team, then one day you may be able to conduct a regression analysis that finds the exact causal relationship between a customer’s engagement with education materials, and their churn, retention, and expansion.
If you have a wealth of resources or your company’s name rhymes with “moogle,” then, take this approach.
But for the rest of us, we’re going to have to get creative about how we tell this story. An effective way to start is by finding a set of proxy metrics. In the next few sections, I’ll walk through some approaches to measuring the value of Customer Education.
Rollup Metrics
First of all, even though you’ll be doing of plenty of measurement on individual pieces of content, you’ll want a few key “run-the-business” metrics to roll up to the rest of your business. These should be high-level summaries of your operational metrics that can communicate the health of your Customer Education business over time. I will discuss a few of the ones I feel are critical roll up metrics.
Engagement Rate
How is it measured? Customers engaged in education programs divided by total customer base.
What does it communicate? Maybe you’ve surmised by this point that I’m not a huge fan of activity metrics. It doesn’t really matter how much you’re doing if you can’t show the value. But in this case, I do think it’s important to show the rest of your business how often customers are adopting the programs you create, and whether that rate is rising or falling over time.
Helpfulness Index, Satisfaction, and Net Promoter Score
How is it measured?
- Helpfulness Index is the number of upvotes over the total number of votes to a question like “Was this article helpful?”
- Satisfaction is the average or median score for a post-training survey such as “Overall, how would you rate this training?”
- Net Promoter Score asks “On a scale of 0 to 10, how likely are you to promote…” with the percentage of detractors (who rate 0 – 6) subtracted from the percentage of promoters (who rate 9 or 10).
What does it communicate? Earlier I described both Helpfulness Index and Satisfaction as operational metrics. The good news is that once you’re measuring them, you can also use them as rollup metrics to show that the articles and courses you create are well-received by customers. Also, Net Promoter Score is more widely accepted as a leading indicator of customer loyalty.
Education Revenue and Margin
How is it measured? Revenue is the number of dollars you brought in (usually from selling education services or subscriptions), and margin is the percentage of profit you make.
For example, if you sell a course for $2,000 in revenue and it takes $1,500 in development and delivery hours to create, you’ve made 25% margin on that course.
What does it communicate? For Customer Education functions who operate on P&L (Profit and Loss) statements, your revenue and margin communicate in part whether you’ve created ROI for your business.
Many teams operate for net-zero margin—in other words, they aim to pay for themselves instead of generating additional revenue.
Measuring Return on Investment
Your Customer Education function creates Return on Investment (ROI) for your business, but ROI isn’t always where you might think to look for it. If you can’t show ROI, it’s hard to justify future investments in your team.
Part of the struggle for Customer Education teams arises from being classified as a cost center in the business. In other words, you cost money to operate, but you don’t directly add to the company’s profit.
What are your options as a Customer Education leader when you may not be able to invest in the people or software that you need, because you’re a cost center?
- Take your ball and go home
- Start focusing maniacally on revenue-generating training services and subscriptions
- Get better at defining how Customer Education creates ROI
I’m not a big fan of #1, and you probably aren’t either, because then what are you going to do, find a new kickball team?
Many software companies focus on #2, in the form of revenue-generating services. They make strong investments in an Education Services P&L because it’s dramatically easier to define ratios than ROI. Just take each Trainer, assign them a target utilization rate, and hire them (or add training partners) based on your forecasted demand.
So now you have a small army of Trainers who are going on-site for paid trainings, and perhaps a network of training partners to accommodate additional demand.
Of course, you also need people to create your content. There’s a solution for that, too: Either bake the content development cost into the margins of your paid training subscription, or charge a subscription price for your content, to recover your costs of development.
I’m being a little bit audacious here—none of this is easy or quick to build. I won’t fault any company that’s generating direct cost recovery on its training programs, or even healthy service margins. But this is simply not feasible for many early-stage companies, and things can get thorny.
For example, you may find yourself:
- Struggling to get Training Services attached to deals when it’s seen as a minor transaction that should just be discounted or waived anyway.
- Having to produce content (potentially of questionable quality) to keep up with subscription demand.
- Lacking the bandwidth to produce content that helps the rest of your customer base adopt your product, because your content development time is focused on premium offerings and customizing content for individual trainings.
Even if you’re running a strong Education Services business today, there may be a greater way to contribute to your business and your customers’ overall health. Especially in SaaS, where constant adoption is key, a Customer Education team can do much more than simply pay for itself.
Calculating the ROI of Customer Education
Put simply, when you’re an early-stage company, creating direct ROI on your Customer Education efforts is probably less valuable than using your Customer Education efforts to fuel your company’s growth and firm up Customer Lifetime Value.
In many cases, the value of having education content can’t be quantified simply in training revenue generated; it creates additional value by reducing customer friction, increasing customer delight, and differentiating your company in the market through thought leadership.
To use a calculation, ROI is the difference between the benefit and the cost of the training, divided by the cost of the training.
So, for example, let’s say you create a program that will save you $100,000 in headcount costs for Customer Success and Support teams next year. It’s expected to bring in another $20,000 when you monetize it. So now your dollar benefit is $120,000. You pay an Instructional Designer, Graphic Designer, and Trainer $50,000 in total to put the training program together.
So, what do you end up with as an ROI? It’s (120,000-50,000)/50,000. In other words, 70,000/35,000. That’s 200% ROI!
Jack Phillips, whom you may recall from the last chapter from his work on measuring training ROI, admits that “by design, most programs do not warrant the five levels of ROI. Even in facilitator-led training, only 5-10 percent of programs should be measured at Level 5.”
Most training materials by nature won’t be evaluated beyond Level 3, but strategic programs (and your training program as a whole) deserve higher-level measurements.
The example I used above brings the limitations of ROI as a measurement tool to light. It’s hard to say definitively whether an individual program will bring in $120,000 in that example. More likely it’s a collection of programs that achieve that result.
Additionally, a 200% ROI might not be believable to some executives, even if it’s true. It’s often better to talk about ROI in dollar terms than percentages for this reason.
ROI in Hour Buybacks
If your Customer Education strategy is successful, you’ll have influenced many areas of the business. This influence can be quantified.
For example, are you measuring how many potential support cases are being deflected by your education content, thus saving support headcount? You can use this to calculate the support cost savings that came from your education efforts.
The metric for this is “Contact Rate” or “Customer Case Factor.” Contact Rate is defined as the support tickets filed per a certain number of customers—for example, “We receive 500 support tickets a month for every 100 customers.”
In terms of Customer Education, contact rate is a deflection metric—in other words, if your education content works, you should see fewer tickets and you can point to your content’s impact on your Support team. If you can segment this data to show the accounts who have interacted with your education content, all the better.
Chances are that trained customers will either have a lower contact rate overall, or that the types of questions they’re asking are not basic, but more sophisticated questions that show a higher level of engagement with your product.
An ROI Argument for Cost Savings in Support
- Calculate your customer contact rate (the number of tickets submitted monthly divided by the number of customers you have).
- Calculate your Support team’s hourly pay rate.
- Project a percentage decrease in customer contact rate due to increased education efforts.
- Then multiply that percentage decrease in tickets by the average hourly salary it would have taken to solve the tickets.
Voilà! You’ve shown where Customer Education can pay for itself. And this is before you’ve even mentioned that self-service generally increases customer satisfaction.
The other area where you’re buying back time is from CSMs. If you don’t have a Customer Education program, then it’s usually up to CSMs to train customers ad hoc, one-by-one.
Before we even get into the ROI calculation for this, there are some other pitfalls associated with ad hoc CSM training:
- The trainings are usually inconsistent, so there’s no way to know whether a customer was exposed to a certain topic, feature, or technique.
- CSMs often don’t know when a new member of the customer-side team starts, and they can’t always deliver another training each time a new user starts.
- Small customers often aren’t offered CSM-led training altogether.
Once you start developing scalable training, you can use it as a standard part of your initial customer onboarding, as well as progressive user onboarding. This will buy time back from your CSMs.
Calculate it like this:
- Take the number of hours each CSM spends preparing for and delivering an initial customer onboarding training.
- Multiply that by the number of onboarding customers you expect to have in the next year.
- Then calculate the average number of hours CSMs spend either on retraining for basic skills or new user onboarding in existing accounts.
- Add both figures together, and multiply by your CSMs’ average, fully loaded hourly cost.
Some businesses get touchy about the idea of taking facetime away from CSMs. You might bring in this buy back calculation and get the response, “Well, it’s our strategy to have as much customer engagement as possible, so we don’t want to take trainings away from the CSMs.”
If your business isn’t scalable and you only manage a few, large accounts, then that strategy may work. But allow me to offer a few counterpoints:
CSMs Focus on Customer Success
The less time CSMs spend delivering basic training, the more time they have to do deeper discovery with the customer and build stronger relationships. CSMs are best deployed toward strategic issues that deepen the account relationship, not transactional activities like delivering an onboarding slide deck.
Scalable Training & Customer Onboarding
CSM delivery isn’t scalable, so some customers will get left in the lurch if you don’t build scalable training. (And if you just decide to replay a recording of a CSM delivering the training as your “scalable onboarding” strategy, tell me if that moves your adoption numbers at all.)
Enterprise Training
Even large Enterprise customers have an appetite for scalable resources in addition to custom, on-site training. Every business is different, but as businesses become more distributed, the idea that every large customer wants you to come on-site to deliver classroom training is becoming more perception than reality. Scalable resources are key for Train the Trainer programs, where you’ll be rolling out too many teams within a large organization.
Dedicated Training Professionals
You’ll often see a benefit in hiring a dedicated training professional. Typically, the Trainers who get the best engagement during trainings are the ones who know how to facilitate. They can make trainings engaging and interactive, more of a two-way dialogue than a presentation.
You can try instructing all your CSMs to train customers in this manner, but for early-stage businesses where CSMs are already typically underwater chasing renewals and answering frantic customer questions, I’ve rarely seen this work.
Because Customer Education has a unique ability to reduce the hours spent conducting repetitive trainings and answering support tickets, I often refer to it as the “Scale Engine of Customer Success.”
ROI in Behavior Change and Attribution
Pat Durante, President of the Computer Education Management Association (CEdMA) and Senior Director of Technical Enablement at Talend, says, “At the end of the day, no matter how large your training business, [training revenue] numbers are often a rounding error on the overall software revenue and the C-suite is not losing sleep over these numbers.”
Instead of “easy” metrics that show activity, he argues, we should be measuring the impact that our Customer Education efforts have on metrics that the business really cares about—churn, retention, and expansion.
At Optimizely, we used a dashboard to continuously monitor the behavior of customers who did or didn’t use our educational resources.
We would look at the percentage of trained and untrained customers who:
- Logged into our product
- Created new projects
- Achieved statistical significance (the true sign that you were getting value from Optimizely)
- Filed support tickets
Within a given time period, we could look at the difference in behavior for our customers who used educational resources, and those who didn’t. And as you may suspect, trained customers used the product more and filed fewer support tickets (especially basic ones that could have been answered easily).
By showing the numbers side-by-side we were able to show the impact of education on customers.
Training Impact Takeaway:
Measure the percentage of trained vs. untrained customers who use your key product features, or who contact support.
But don’t stop at product adoption and support tickets—look at longer-term business impact, as well. It would stand to reason if that educated customers use the product more; they also renew and expand more often.
We would do the same type of correlation analysis for customer churn, renewal, and expansion, looking at the percentage of:
- Churned customers in the last 90 days who did or didn’t use education
- Renewed customers in the last 90 days who did or didn’t use education
- Expanded customers in the last 90 days who did or didn’t use education
Again, as you might expect, customers who used education made up a higher percentage of renewed and expanded customers, whereas churned customers showed very low engagement with education.
Churn and Renewal Takeaway:
Measure the percentage of churned accounts (or accounts that partially churned, i.e. decreased their contract value) that were trained, vs. the number of renewed or expanded accounts that were trained.
Perhaps at this point, you’re wondering, “How will I tell my executives that these results are actually because of education? How can I prove that this is causation, not correlation?”
If you have enough data to show that education directly caused product adoption, support ticket reduction, time to first value, renewals, or expansion—and you can regress the data to remove all other potential factors—by all means, do that analysis!
Most teams don’t have that rich data or resources to use. In general, you’re not trying to prove that education directly caused every bit of the business impact, but that successful customers were by and large educated customers. If your most successful customers demand high-quality education resources, then you’ve gone a long way toward showing the role education plays in your organization.
One bonus tip
If you’re already measuring the percentage of trained vs. untrained customers, and you know that trained customers renew and expand more, take those stats to your Sales, Customer Success, and Marketing teams. These numbers become good incentive to get more customers into training—or, if you sell training, to attach training to new deals and renewals.
ROI and NPS
Another area where Customer Education can create ROI for your business is by increasing customer satisfaction, which is a leading indicator of renewal. For businesses that prize NPS (Net Promoter Score) as a key success metric, you can show how Customer Education directly increases NPS for customers.
You may be tempted not to release scalable education content because you’re concerned that self-serve education programs will reduce customer touchpoints with your account and Support reps, who make every experience delightful. Tony Hsieh, of the online retailer Zappos, often said that the company’s Customer Service reps were its best marketing channel by creating “Wow” experiences.
Education materials, when built correctly, are a way to broadcast how delightful your company is to work with. Aim to create materials that capture your team’s personality and brand’s voice. Dry, boring manuals are so 1990.
For an example, Wistia, a video hosting company, naturally has a well-produced series of live-action tutorials on effective video production and marketing. Each of their education videos features their team members showing off their personalities, and sometimes getting goofy. When you watch their videos, you don’t just learn a topic—you also get to know the team you’ll be working with, and you see that they’re real humans with personalities.
It’s not just about scaling the delight, however; it’s also about serving customers in the way they want to be served. Many customers would rather have an answer right now. They don’t care how delightful your support experience is—they don’t even want to have to contact support!
Ideally, your education content should be available when and where the customer wants it, and it should be ridiculously easy to find. Often, this means putting well-written education content directly into your product, which in turn should drive traffic to your Knowledge Base and decrease support tickets that trickle down to your agents.
But this is a two-way street: your team should be active in generating scalable content and sharing best practices, so that you can document them and make them discoverable.
About the Author
Adam Avramescu is a Customer Education leader with over 10 years of experience working with software companies. He is author of Customer Education: Why Smart Companies Profit By Making Customers Smarter.
In 2014, he started the Bay Area Customer Education Meetup, which currently has around 800 members. He also co-hosts the podcast CELab: The Customer Education Laboratory, and is active in speaking and writing about the role of Customer Education for modern technology companies.
