New Research: Customer Effort Score Debunked, NPS® Vindicated


Is the new iPhone better than the latest Android phone? Ask 10 people and you’ll get 10 different answers — all of them based on opinion.


It’s similar with customer feedback metrics. Should you use Net Promoter® or Customer Effort Score or Customer Satisfaction or some other new fad metric?

The opinions vary widely based on the last blog the person has read.

The good news is that an excellent new paper from the University of Groningen sheds some light on which is really better based on empirical research. That’s right, actual science.

What’s nice about this paper is that the researchers don’t seem to have an axe to grind. They don’t work for one of the competing consulting firms that promote this question or that question. Hopefully that makes their research more balanced.

They also look to have done their work very thoroughly, with good sample sizes and methodologies. They controlled for a variety of factors including duration as a customer.

I’m not qualified to assess the statistical approaches they used as they are ahead of my knowledge in this area but have assumed that they are appropriately used.

The Focus of the Research

The paper is aptly titled: “The predictive ability of different customer feedback metrics for retention” and is freely available on line for review.

In it, through a series of surveys and advanced statistics, the authors investigated:

  1. The predictive power of a range of customer feedback metrics
  2. Across industry, firms and individual customers

Summary Of Findings

You can read the whole paper for the details but I’ve pulled a few keys items that I found interesting.

NPS Is an Effective Predictor of Customer Retention

With our findings, we can state that, in contrast with previous research (e.g., Keiningham, et al., 2007b), monitoring NPS does not seem to be wrong in most industries. Our findings indicate that the NPS is an effective predictor of customer retention, though the top-2-box customer satisfaction is slightly better overall.
(emphasis mine)


…”changes in top-2-box customer satisfaction, followed by the official NPS, have the highest impact on customer retention”

Much has been made of the work of Keininhgam, et al. in “proving” NPS is not predictive. This comprehensive research rejects that conclusion.

Customer Effort Score is Not Effective

Yet, by quite a large margin, CES is still the worst-performing CFM. Thus, even for the group for which the CES was designed, it is still outperformed by the other CFMs.

While it may be a great idea, the Customer Effort Score metric is less effective than all of the other metrics analysed.

The authors go on to say:

In terms of the limited overall incremental value of the CES in itself, managers should be reluctant to adopt any metrics that have a past focus and are limited in focus on one specific attribute and/or incident as an overall key performance metric.

Greater Engagement Leads to Greater Retention

“indicating that people who make requests are more likely to remain customers.”

Generally, customers that interact with our brands are more likely to remain customers than those that don’t. The practical implications of this finding are difficult to determine but any way that you can continue to engage with your customers would seem to be a good thing.

Linear Scales Doesn’t Work as Well as Segmented Scales

In general, transforming scales of the CFMs [Customer Feedback Metrics] to capture the proportion of most satisfied customers (as is done with the top-2-box customer satisfaction) or splitting customers up into groups (as is done with the promoters and detractors of the NPS) is preferable to using the full scale of the CFMs.

The link between customer responses and customer retention is not linear and when analysing responses it is necessary to focus on the ends of the scale. The simple customer satisfaction score is not as good an indicator as top-2-box scores. The Net Promoter score is a better indicator that the unscaled response to the “would recommend” question.


We clearly find that the top-2-box customer satisfaction and the official NPS, which focus on the extremes, outperform the CFMs that use the full scale.

Combing Metrics Can Lift Your Accuracy.

In this case they are not suggesting adding top-2-box customer satisfaction to Net Promoter Score but simply tracking reporting on both indicators.

Combining metrics, especially the CES with the customer satisfaction–related CFMs, results in improved out-of-sample retention predictions. A dashboard of CFMs that measure different dimensions, as indicated in our conceptualization, is preferable to monitoring a single CFM.

Top-2-Box Customer Satisfaction and NPS Tie As the Best Metric to use

The weights reveal a 49.6% certainty that the top-2-box customer satisfaction is the best CFM and a 49.1% certainty that the official NPS is the best CFM; thus, these two CFMs are almost equally likely to be the best CFM, and it is very unlikely that one of the other CFMs is the best.

NPS Is Best for Identifying Risky Customers

The authors kindly provided a handy dandy look up table for which metric to use when.


Extract from “The predictive ability of different customer feedback metrics for retention”


In summary the only two metrics you should be focusing on are top-2-box customer satisfaction and Net Promoter Score.


It’s Not the Number Stupid: Net Promoter® is not a Numbers Game

   by  DonkeyHotey
   by  DonkeyHotey

Creative Commons Creative Commons Attribution 2.0 Generic License   by  DonkeyHotey

I’m sure you’ve heard of Bill Clinton’s famous, but unofficial, 1992 campaign slogan: “It’s the economy stupid”. His chief of staff wanted to focus everyone on what was most important in the run up to the election.

That was a great approach but today I’m championing an anti-version of that slogan:

Net Promoter – it’s not the number stupid! [Tweet This]

[Read more…]

Innovation is not just a Product Game; it’s also a Service Game

   by  symphony of love

This week we have a guest post from Janine Scott. Janine is the Regional Manager for Customer Experience with Wolters Kluwer. As someone who works in the business, Janine has some great insights and today she discusses innovation in service businesses.

Take it away Janine…


Quality or Quantity; which strategic position should your company take? [Read more…]

Struggling to Act on Customer Feedback? Try This.

Source: Temkin Group Q2 2012 CX Management Surveys,  © Temkin Group

Less than one in seven companies that collect and use customer feedback are running the process effectively.

Most are missing the key, and only useful, part of the process: taking action to improve their business. Many times though it’s not missing out of choice but because organisations do not know how to drive action in the business with customer feedback data.

Source: Temkin Group Q2 2012 CX Management Surveys,  © Temkin Group

Source: Temkin Group Q2 2012 CX Management Surveys

[Read more…]

The Science of B2B Retention Selling

How to predict which clients you will lose (or will spend more)

Are you confident in your post-sales team’s ability to secure renewal business?

If you have a “land and expand” strategy, do know when you can expand your business in a given client?

We will explore these questions, example cases studies and provide some practical approaches in this informal breakfast session. [Read more…]

Why is More Important than How In Customer Feedback

Why? by BuzzFarmers, on Flickr
Creative Commons Creative Commons Attribution 2.0 Generic License   by  BuzzFarmers 

Why is a very powerful word.

One of the most popular Ted talks of all time, over 19 million views, asks you to start not with the how of your business but the why of your business.

Why asked 5 times is the basis of the incredibly powerful 5 Whys root cause analysis technique.

Young children find out how the world works by asking why a seemingly endless number of times.

So when you are considering investing considerable time and resources in rolling out a new customer feedback or Net Promoter® program you should make sure that why is the first question you ask as well. [Read more…]

The Ultimate List of Net Promoter® Best Practice Tips

Automate your process

Automate your process

Net Promoter was launched in 2003 and since then many, many organisations have used the methodology to drive change in their business. With so much time invested, lots of lessons have been learned.

So you don’t have to re-learn those lessons the hard way I’ve pulled together this massive list of Net Promoter best practice tips.

Note that these tips pretty much apply to any Voice of the Customer or Customer feedback (but not market research) process. So regardless of whether you are using Customer Effort Score, Customer Satisfaction or NPS® you can apply them to your business. [Read more…]

Net Revenue Score is not a new Net Promoter Score®

I’m always open to a debate on the pros and cons of any business approach but it’s better if the person on the other side has done their research. That way we can have a thoughtful debate.

Unfortunately John Greathouse has jumped to a few conclusions when he penned his article in the Wall Street Journal: Startups Should Focus On Their NRS — Net Revenue Score. There are quite a few items that need review including: [Read more…]

Want to build trust with your customers? Recommend a competitor.

Harry Antrim as Mr. R.H. Macy

Amy Scott sent me through her Christmas article and I liked it so much I asked her if I could publish it on the blog. Luckily her answer was yes so everyone can benefit.

Over to you Amy…

Want to build trust with your customers? Recommend a competitor.

When you first see this you must be thinking “Are they crazy why on earth would I want to give business to my competitors?”

But this isn’t as daft as it sounds. [Read more…]

Predicting American Airlines’ Net Promoter Score® Using Twitter


In this recent post by Fonolo American Airlines was hammered for having an outstanding (in a bad way) number of Twitter users complain about being on hold with them.

This got me to thinking:

How good is crowd data at predicting the Net Promoter Score for an organisation?

As it turns out it’s a pretty good indicator so let’s review it in detail. [Read more…]