New Research: Customer Effort Score Debunked, NPS® Vindicated

CES vs NPS

New Research: Customer Effort Score Debunked, NPS® Vindicated

Get the Business Leaders’ Practical Guide to Implementing Net Promoter

Is the new iPhone better than the latest Android phone? Ask 10 people and you’ll get 10 different answers — all of them based on opinion.

It’s similar with customer feedback metrics. Should you use Net Promoter® or Customer Effort Score or Customer Satisfaction or some other new fad metric?

The opinions vary widely based on the last blog the person has read.

The good news is that an excellent new paper from the University of Groningen sheds some light on which is really better based on empirical research. That’s right, actual science.

What’s nice about this paper is that the researchers don’t seem to have an axe to grind. They don’t work for one of the competing consulting firms that promote this question or that question. Hopefully that makes their research more balanced.

They also look to have done their work very thoroughly, with good sample sizes and methodologies. They controlled for a variety of factors including duration as a customer.

I’m not qualified to assess the statistical approaches they used as they are ahead of my knowledge in this area but have assumed that they are appropriately used.

The Focus of the Research

The paper is aptly titled: “The predictive ability of different customer feedback metrics for retention” and is freely available on line for review.

In it, through a series of surveys and advanced statistics, the authors investigated:

  1. The predictive power of a range of customer feedback metrics
  2. Across industry, firms and individual customers

Summary Of Findings

You can read the whole paper for the details but I’ve pulled a few keys items that I found interesting.

NPS Is an Effective Predictor of Customer Retention

With our findings, we can state that, in contrast with previous research (e.g., Keiningham, et al., 2007b), monitoring NPS does not seem to be wrong in most industries. Our findings indicate that the NPS is an effective predictor of customer retention, though the top-2-box customer satisfaction is slightly better overall.
(emphasis mine)

and:

…”changes in top-2-box customer satisfaction, followed by the official NPS, have the highest impact on customer retention”

Much has been made of the work of Keininhgam, et al. in “proving” NPS is not predictive. This comprehensive research rejects that conclusion.

Customer Effort Score is Not Effective

Yet, by quite a large margin, CES is still the worst-performing CFM. Thus, even for the group for which the CES was designed, it is still outperformed by the other CFMs.

While it may be a great idea, the Customer Effort Score metric is less effective than all of the other metrics analysed.

The authors go on to say:

In terms of the limited overall incremental value of the CES in itself, managers should be reluctant to adopt any metrics that have a past focus and are limited in focus on one specific attribute and/or incident as an overall key performance metric.

Greater Engagement Leads to Greater Retention

“indicating that people who make requests are more likely to remain customers.”

Generally, customers that interact with our brands are more likely to remain customers than those that don’t. The practical implications of this finding are difficult to determine but any way that you can continue to engage with your customers would seem to be a good thing.

Linear Scales Doesn’t Work as Well as Segmented Scales

In general, transforming scales of the CFMs [Customer Feedback Metrics] to capture the proportion of most satisfied customers (as is done with the top-2-box customer satisfaction) or splitting customers up into groups (as is done with the promoters and detractors of the NPS) is preferable to using the full scale of the CFMs.

The link between customer responses and customer retention is not linear and when analysing responses it is necessary to focus on the ends of the scale. The simple customer satisfaction score is not as good an indicator as top-2-box scores. The Net Promoter score is a better indicator that the unscaled response to the “would recommend” question.

Also:

We clearly find that the top-2-box customer satisfaction and the official NPS, which focus on the extremes, outperform the CFMs that use the full scale.

Combing Metrics Can Lift Your Accuracy.

In this case they are not suggesting adding top-2-box customer satisfaction to Net Promoter Score but simply tracking reporting on both indicators.

Combining metrics, especially the CES with the customer satisfaction–related CFMs, results in improved out-of-sample retention predictions. A dashboard of CFMs that measure different dimensions, as indicated in our conceptualization, is preferable to monitoring a single CFM.

Top-2-Box Customer Satisfaction and NPS Tie As the Best Metric to use

The weights reveal a 49.6% certainty that the top-2-box customer satisfaction is the best CFM and a 49.1% certainty that the official NPS is the best CFM; thus, these two CFMs are almost equally likely to be the best CFM, and it is very unlikely that one of the other CFMs is the best.

NPS Is Best for Identifying Risky Customers

The authors kindly provided a handy dandy look up table for which metric to use when.

Groningen
Extract from “The predictive ability of different customer feedback metrics for retention”

In summary the only two metrics you should be focusing on are top-2-box customer satisfaction and Net Promoter Score.

I've created a Business Leaders’ Practical Guide to Implementing Net Promoter.  Download it Here
Creative Commons Creative Commons Attribution 2.0 Generic License