New Research: Customer Effort Score Debunked, NPS® Vindicated


Is the new iPhone better than the latest Android phone? Ask 10 people and you’ll get 10 different answers — all of them based on opinion.


It’s similar with customer feedback metrics. Should you use Net Promoter® or Customer Effort Score or Customer Satisfaction or some other new fad metric?

The opinions vary widely based on the last blog the person has read.

The good news is that an excellent new paper from the University of Groningen sheds some light on which is really better based on empirical research. That’s right, actual science.

What’s nice about this paper is that the researchers don’t seem to have an axe to grind. They don’t work for one of the competing consulting firms that promote this question or that question. Hopefully that makes their research more balanced.

They also look to have done their work very thoroughly, with good sample sizes and methodologies. They controlled for a variety of factors including duration as a customer.

I’m not qualified to assess the statistical approaches they used as they are ahead of my knowledge in this area but have assumed that they are appropriately used.

The Focus of the Research

The paper is aptly titled: “The predictive ability of different customer feedback metrics for retention” and is freely available on line for review.

In it, through a series of surveys and advanced statistics, the authors investigated:

  1. The predictive power of a range of customer feedback metrics
  2. Across industry, firms and individual customers

Summary Of Findings

You can read the whole paper for the details but I’ve pulled a few keys items that I found interesting.

NPS Is an Effective Predictor of Customer Retention

With our findings, we can state that, in contrast with previous research (e.g., Keiningham, et al., 2007b), monitoring NPS does not seem to be wrong in most industries. Our findings indicate that the NPS is an effective predictor of customer retention, though the top-2-box customer satisfaction is slightly better overall.
(emphasis mine)


…”changes in top-2-box customer satisfaction, followed by the official NPS, have the highest impact on customer retention”

Much has been made of the work of Keininhgam, et al. in “proving” NPS is not predictive. This comprehensive research rejects that conclusion.

Customer Effort Score is Not Effective

Yet, by quite a large margin, CES is still the worst-performing CFM. Thus, even for the group for which the CES was designed, it is still outperformed by the other CFMs.

While it may be a great idea, the Customer Effort Score metric is less effective than all of the other metrics analysed.

The authors go on to say:

In terms of the limited overall incremental value of the CES in itself, managers should be reluctant to adopt any metrics that have a past focus and are limited in focus on one specific attribute and/or incident as an overall key performance metric.

Greater Engagement Leads to Greater Retention

“indicating that people who make requests are more likely to remain customers.”

Generally, customers that interact with our brands are more likely to remain customers than those that don’t. The practical implications of this finding are difficult to determine but any way that you can continue to engage with your customers would seem to be a good thing.

Linear Scales Doesn’t Work as Well as Segmented Scales

In general, transforming scales of the CFMs [Customer Feedback Metrics] to capture the proportion of most satisfied customers (as is done with the top-2-box customer satisfaction) or splitting customers up into groups (as is done with the promoters and detractors of the NPS) is preferable to using the full scale of the CFMs.

The link between customer responses and customer retention is not linear and when analysing responses it is necessary to focus on the ends of the scale. The simple customer satisfaction score is not as good an indicator as top-2-box scores. The Net Promoter score is a better indicator that the unscaled response to the “would recommend” question.


We clearly find that the top-2-box customer satisfaction and the official NPS, which focus on the extremes, outperform the CFMs that use the full scale.

Combing Metrics Can Lift Your Accuracy.

In this case they are not suggesting adding top-2-box customer satisfaction to Net Promoter Score but simply tracking reporting on both indicators.

Combining metrics, especially the CES with the customer satisfaction–related CFMs, results in improved out-of-sample retention predictions. A dashboard of CFMs that measure different dimensions, as indicated in our conceptualization, is preferable to monitoring a single CFM.

Top-2-Box Customer Satisfaction and NPS Tie As the Best Metric to use

The weights reveal a 49.6% certainty that the top-2-box customer satisfaction is the best CFM and a 49.1% certainty that the official NPS is the best CFM; thus, these two CFMs are almost equally likely to be the best CFM, and it is very unlikely that one of the other CFMs is the best.

NPS Is Best for Identifying Risky Customers

The authors kindly provided a handy dandy look up table for which metric to use when.


Extract from “The predictive ability of different customer feedback metrics for retention”


In summary the only two metrics you should be focusing on are top-2-box customer satisfaction and Net Promoter Score.


Predicting American Airlines’ Net Promoter Score® Using Twitter


In this recent post by Fonolo American Airlines was hammered for having an outstanding (in a bad way) number of Twitter users complain about being on hold with them.

This got me to thinking:

How good is crowd data at predicting the Net Promoter Score for an organisation?

As it turns out it’s a pretty good indicator so let’s review it in detail. [Read more…]

Insurance Agent Receives Net Promoter Score® of +98

Over the week we’ve read a very interesting case study on an insurance agent. It’s interesting due to the fact that the company claims to have received a Net Promoter Score of +98 – A score in which we believe is unusually high. Nevertheless, we’ll provide you with some of the details on how they Rising_Arrowreached such a milestone.

The insurance industry is one industry that is not usually known for its high level of customer satisfaction. The company insureon however is changing such a perspective having received a Net Promoter Score of +98. [Read more…]

A German Family-owned Business Strengthens Ties Through Net Promoter®

corporate-governanceA successful German wholesaler drives business through Net Promoter implementation. With an already strong product line, the company implemented Net Promoter to improve customer service and strengthen ties with their customers.

W. & L. Jordan GMBH, with over 50 stores across Europe has been around since 1919 and promotes themselves as a customer centric organisation. The organisation sought the services of CustomerGauge in order to implement the Net Promoter to improve business through a customer feedback program which can be effectively measured. [Read more…]

Digital and Innovation Agency Drives Business with Net Promoter®

Service chart with red marker

Service chart with red markerUK based digital and innovation agency Volume proves there are no right or wrong industries for using the Net Promoter approach. In fact with a score of +60 the organisation is out in front of many of its’ peers:  Amazon’s score is 76 and Apple’s is 71.

Here is a company doing things  differently. Being a small agency with big success in a tough industry wasn’t enough for them. They decided to implement Net Promoter to actively seek out flaws and correct them. [Read more…]

Using Chi-Squared tests on Net Promoter® Data


In a recent post I discussed using the Margin of Error method to determine if the difference between two Net Promoter Scores® is probably real or just statistical noise. A sharp eye reader has identified that you can also use a Chi -Squared test to perform this test. [Read more…]

Case study: How to apologize to your customers when things go badly wrong


When things went pear-shaped over at MozLand, they did so in a big way. The result was lots of unhappy customers, but their response is a case study in how to apologize for problems that seriously affect your customers.

You can read the full text of their apology over on the SEOMoz site but here are the elements of what they did right.

[Read more…]

Case study: nib health fund’s Successful Net Promoter Score® Implementation


Over the last 12 months, Genroe has been privileged to work with nib health funds (one of Australia’s fastest growing health funds) to help them evolve their Net Promoter Score processes. In that time, we helped them implement CustomerGauge, an end to end, integrated, Net Promoter Score data collection, reporting, analysis and action system.

In the past, nib has allowed us to publish insightful information in two blog posts:

Now, the full story behind the successful NPS implementation process has been captured in a new case study: nib health funds: Checking Up On Customer Loyalty.

In this case study, nib offers more insights into the changes they have made using NPS insights: [Read more…]

Positive Customer Experience Drives Loyalty, Revenues


The new Temkin Group research shows what we believe: the customer experience drives customer loyalty. The chart below shows how much a good customer experience can lift your customer loyalty, and how much a bad customer experience can lower it.

Read the whole story: Positive Customer Experience Drives Loyalty, Revenues


Case Study: Beating the Market with Customer Satisfaction

Case Study Chart

If you’re looking to boost customer satisfaction, one of the most promising places to start is customer service. Unfortunately, it’s also a place where long-term goals tend to buckle under short-term financial pressures. Companies try to meet Wall Street’s immediate demands by cutting costs through automation and outsourcing—despite a growing body of research conclusively showing that customers are fed up with lousy service and that increased satisfaction has a positive impact on consumer spending, cash flow, and business performance.

In a groundbreaking 2006 study, University of Michigan business professor Claes Fornell and colleagues showed the relationship between customer satisfaction and financial success by creating a hedge portfolio in which stocks are bought long and sold short in response to changes in the American Customer Satisfaction Index (ACSI). Developed by the University of Michigan’s National Quality Research Center, the ACSI is an indicator of economic success that reflects levels of customer satisfaction with goods and services purchased from about 200 companies in more than 40 industries; it’s based on interviews with more than 65,000 U.S. consumers each year. [Read more…]