Is the new iPhone better than the latest Android phone? Ask 10 people and you’ll get 10 different answers — all of them based on opinion.
The opinions vary widely based on the last blog the person has read.
The good news is that an excellent new paper from the University of Groningen sheds some light on which is really better based on empirical research. That’s right, actual science.
What’s nice about this paper is that the researchers don’t seem to have an axe to grind. They don’t work for one of the competing consulting firms that promote this question or that question. Hopefully that makes their research more balanced.
They also look to have done their work very thoroughly, with good sample sizes and methodologies. They controlled for a variety of factors including duration as a customer.
I’m not qualified to assess the statistical approaches they used as they are ahead of my knowledge in this area but have assumed that they are appropriately used.
The Focus of the Research
The paper is aptly titled: “The predictive ability of different customer feedback metrics for retention” and is freely available on line for review.
In it, through a series of surveys and advanced statistics, the authors investigated:
- The predictive power of a range of customer feedback metrics
- Across industry, firms and individual customers
Summary Of Findings
You can read the whole paper for the details but I’ve pulled a few keys items that I found interesting.
NPS Is an Effective Predictor of Customer Retention
With our findings, we can state that, in contrast with previous research (e.g., Keiningham, et al., 2007b), monitoring NPS does not seem to be wrong in most industries. Our findings indicate that the NPS is an effective predictor of customer retention, though the top-2-box customer satisfaction is slightly better overall.
…”changes in top-2-box customer satisfaction, followed by the official NPS, have the highest impact on customer retention”
Much has been made of the work of Keininhgam, et al. in “proving” NPS is not predictive. This comprehensive research rejects that conclusion.
Customer Effort Score is Not Effective
Yet, by quite a large margin, CES is still the worst-performing CFM. Thus, even for the group for which the CES was designed, it is still outperformed by the other CFMs.
While it may be a great idea, the Customer Effort Score metric is less effective than all of the other metrics analysed.
The authors go on to say:
In terms of the limited overall incremental value of the CES in itself, managers should be reluctant to adopt any metrics that have a past focus and are limited in focus on one specific attribute and/or incident as an overall key performance metric.
Greater Engagement Leads to Greater Retention
“indicating that people who make requests are more likely to remain customers.”
Generally, customers that interact with our brands are more likely to remain customers than those that don’t. The practical implications of this finding are difficult to determine but any way that you can continue to engage with your customers would seem to be a good thing.
Linear Scales Doesn’t Work as Well as Segmented Scales
In general, transforming scales of the CFMs [Customer Feedback Metrics] to capture the proportion of most satisfied customers (as is done with the top-2-box customer satisfaction) or splitting customers up into groups (as is done with the promoters and detractors of the NPS) is preferable to using the full scale of the CFMs.
The link between customer responses and customer retention is not linear and when analysing responses it is necessary to focus on the ends of the scale. The simple customer satisfaction score is not as good an indicator as top-2-box scores. The Net Promoter score is a better indicator that the unscaled response to the “would recommend” question.
We clearly find that the top-2-box customer satisfaction and the official NPS, which focus on the extremes, outperform the CFMs that use the full scale.
Combing Metrics Can Lift Your Accuracy.
In this case they are not suggesting adding top-2-box customer satisfaction to Net Promoter Score but simply tracking reporting on both indicators.
Combining metrics, especially the CES with the customer satisfaction–related CFMs, results in improved out-of-sample retention predictions. A dashboard of CFMs that measure different dimensions, as indicated in our conceptualization, is preferable to monitoring a single CFM.
Top-2-Box Customer Satisfaction and NPS Tie As the Best Metric to use
The weights reveal a 49.6% certainty that the top-2-box customer satisfaction is the best CFM and a 49.1% certainty that the official NPS is the best CFM; thus, these two CFMs are almost equally likely to be the best CFM, and it is very unlikely that one of the other CFMs is the best.
NPS Is Best for Identifying Risky Customers
The authors kindly provided a handy dandy look up table for which metric to use when.
In summary the only two metrics you should be focusing on are top-2-box customer satisfaction and Net Promoter Score.