Predicting American Airlines’ Net Promoter Score® Using Twitter


In this recent post by Fonolo American Airlines was hammered for having an outstanding (in a bad way) number of Twitter users complain about being on hold with them.

This got me to thinking:

How good is crowd data at predicting the Net Promoter Score for an organisation?

As it turns out it’s a pretty good indicator so let’s review it in detail

The chart shows that AA’s complaint rate is not just a bit above the other airlines but more than three times its nearest rival.

When examining the data the first thing that comes to mind is that maybe AA just has many more customers than any of the other airlines: more customers, more customers on hold, more tweets.

So, all things being equal the number of complaints could be expected to be related to the number of customers.

Let’s examine that hypothesis. Here we compare the On Hold hash tag rates with the number of passenger miles that each airline travels. (2012 data as reported by Nations On Line)


Well that shows almost no correlation at all. So, surprisingly, we can eliminate the number people as being a substantial driver of the high hold time complaints.

Of course, another possibility is that AA customers are massively over represented on Twitter relative to the other airlines in the sample. While possible, it seems unlikely.

Having eliminated those two possibilities we can see if perhaps on-hold mentions is a predictor of Net Promoter®. The NPS® data here comes from the Satmetrix  2014 Airline Industry Report.


Hold Time Complaints are a Goodish Predictor of Net Promoter Score.

This is a pretty good correlation. The higher the NPS the lower the number of on hold hashtags. Admittedly R2 = 0.29 is not a terrifically high correlation but considering that there are most likely multiple drivers for NPS and that this is a crowd level data it does seem pretty good.

However, and here is something really interesting, if you remove AA from the picture, the correlation jumps to 0.41. Now most people would admit that 0.41 is a pretty good correlation statistic and indicates that 41% of the variation in NPS can be accounted for by variation in On Hold complaint rate.

Put another way, simply counting on-hold hash tags is a pretty good way of predicting an airline’s NPS.


Maybe AA is an outlier — removing them improves predictive ability

Lastly, and we may be going too far here, if we also remove Southwest airlines from the sample set the R2 rate jumps to 0.95!


Without AA and Southwest the correlation is very high.

So what?

At this point we can draw a few tentative conclusions:

1. Crowd based data could be quite good a predicting important business metrics.

This approach would only really work well for large organisations because it requires a largish number of responses. It wouldn’t, for instance, work for the corner coffee shop. But it is a potentially useful way of ranking the NPS of large organizations.

Incidentally, this is not the first time that Twitter has been found to be good at predicting useful real work statistics so there is some precedence here.

2. When you’re down, a lot of people kick you.

As you can see the number of AA mentions is much higher than their overall NPS would indicate they should get. This could mean that their hold time is really very bad, but everything else is terrific, to balance out their NPS.

But this is seems unlikely and a better explanation might be that when people have a preconceived negative opinion they more quickly perceive service attributes negatively. There is good support for this perspective in this blog post on the halo effect.

So AA customers might be unfairly quick to pull the #onhold trigger.

3. Hold Time is a Pretty Important indicator of Customer Loyalty

Based on this information it would seem that hold time is a pretty good driver of NPS and therefore customer loyalty. This has the ring of truth about it and many organizations have validated this in their business.



Zappos Service is for Zappos, not You


We’ve all heard the stories of famously customer focused organisations where staff have gone wildly above and beyond customer expectations to create raving fans.

At Zappos, one famous (albeit perhaps apocryphal) tale is the story of a customer service person having pizza delivered to customers who were sitting online talking to their contact center. [Read more…]

[Guest Post] Employee NPS: An Early Warning System for Managers


employee-net-promoter-scoreI’ve talked before about Employee NPS (eNPS) as a great way to collect feedback on how your staff feel about the organisation.

However, as I’m not an expert in organisation design it’s been difficult for me to muster the arguments for replacing or even augmenting the existing very long annual employee engagement surveys so often used by organisations.

But recently I was chatting to Beatrice Hofmeyr, who is an expert in organisation design, and she was just as keen about eNPS, and for the same reasons!

Beatrice agreed to discuss using eNPS and how/why it is so effective. [Read more…]

Most People Don’t Understand Sample Size


bigstock-young-business-man-holding-his-53002933You’ve spent weeks working through the numbers to unpick what customers are saying. After checking through the data and analysing a range of root causes, you have created a really practical plan to solve a key customer issue.

The PowerPoint presentation you’ve created nails each of the points you want to make. It starts right up front with the bad news: Net Promoter data for the business group in question.

Before you walked in, you were fully prepared for this meeting, so how is it that 3 minutes in you’re under attack on the very first slide? A senior manager is pointing accusingly at the screen: “I don’t believe that NPS – you don’t have a big enough sample size.” [Read more…]

Surprise: Rob Markey and I agree on Net Promoter® Benchmarking


delight-the-customerAnyone that has been reading this blog for more than a couple of weeks knows that the subject of Net Promoter benchmarking gets me fired up.

In talking to clients and prospects the question of “what’s a good Net Promoter score” almost invariably arises. Many times I have had to choose my words carefully when I tell people don’t waste your time on external Net Promoter® benchmarks.

I’ve also been careful to explain that not everyone agrees with my views on this topic.

One of the people who disagrees with me is Rob Markey (Bain Partner and co-author of The Ultimate Question 2.0). Clearly he knows something about the subject.

Well, recently when chatting to Rob for one of his Net Promoter System podcast interviews he took me to task on my anti-benchmarking views and, surprisingly, we came away in a violent agreement.

He outlined the specific scenarios where Net Promoter benchmarking is useful and I agreed with every one of his points.

[Read more…]

[Guest Post] 4 Insights Into Building a Better Organisational Structure With Customer Feedback


Auto_mechanic_toolsThe only reason we collect customer feedback, including Net Promoter ® is to understand how we can improve the customer experience and lift profits. Often this impacts the organisational structure, but driving change in this area can be difficult.

Beatrice Hofmeyr having identified this issue and is doing something about it. She is currently collecting practical, best practice techniques from real Australian organisations. In today’s guest post Beatrice provides some great early findings from the project.

Please welcome Beatrice Hofmeyr…

[Read more…]

The Practicalities of Giving Frontline Staff Net Promoter Targets


bigstock-Dart-in-bulls-eye-of-dartboard-16555100“If you don’t give us a 9 or 10 on the survey you receive it will mean we have failed”. On the surface it was an odd way to end my check out process at a well known hotel chain but one I suspect that many of us have experienced. It’s called score begging and it’s an indication of poorly set front-line customer satisfaction targets.

One of the critical success factors for NPS or customer feedback success is ensuring that everyone in the organisation has the score in their personal goals. But applying that idea to front line staff is difficult and if done poorly, as you can see, it drives the wrong behaviours.

To me it is clear that you must link NPS/CSAT to performance review outcomes at least as strongly as you link other hard metrics: revenue, average handle time (AHT), etc. If you don’t staff, quite rightly, deduce that NPS is nice but what you really care about is AHT and sales at all costs.

Not having a strong CSAT/NPS goal is at the heart of many issues in the customer experience. If you’ve ever been relentlessly handed off between operators in a contact centre you have felt the effects. You know they have a tight AHT goal and it’s more important to keep their personal AHT down by shuffling you to another operator than to solve your problem. [Read more…]

[Guest Post] Can Technology Delight Customers? Yes and No.

Bob Thompson

bigstock-Social-media-on-smartphone-60749657Like it or not, the world is going digital. Websites were just the beginning. Now we have social media, smartphones and so on. The only conclusion I can reach is that more of our lives will be spent interacting with technology of one kind or another.

The conundrum: People are more likely to create memorable experiences, which is essential to drive genuine loyalty. In CustomerThink’s research, we find that friendly, well-trained, and knowledgeable employees are key factors what customers perceive to be excellent experiences. Only humans can express empathy and solve problems creatively, so it’s not surprising that interactions with people are more memorable.

New technology can be a positive surprise — I still think it’s amazing that I can buy and receive a Kindle e-book from Amazon in less than one minute. But as Jon Picoult of Watermark Consulting notes: “When technology helps fuel a delightful customer experience, it usually doesn’t take much for other firms to copy that technology.”

More typically, automation is used to make customer interactions more efficient, consistent, and less memorable. So if you’re not careful, you can automate your way to a faceless company with no personality whatsoever!

The solution is to understand the three important roles that technology can play in delightful customer experiences. Let’s use the analogy of a great movie, where the audience experience is delivered by the star, supporting actors, and a “key grip” to manage lighting and camera movements. Obviously only the actors are seen on screen, but without great support behind the camera, a film won’t be delightful to watch. [Read more…]

You Unconsciously Resent Your Customers


bigstock-Close-up-portrait-of-a-serious-54544025Recently, while pondering the surge in discussion about customer centricity and customer focus over the last 10 years, I wondered: do employees in most organisations subtly resent their customers?

It has always struck me as just a little odd that commercial enterprises must explicitly say that they are customer focused and create programs and approaches to instil employees with a zeal for serving customers.

After all the only function of a commercial enterprise is to serve customers. It has literally no other reason for existing.

Perhaps I’m being naïve but stating that we are “customer focused” or “we put the customer first” seem just a little absurd in this context. If you are not putting the customer first then who is first?

That truly innovative product that R&D has just developed is worthless without customers to buy it. No matter how great your supply chain management process runs, if it’s not supplying customers it’s just a bunch of idle trucks and conveyer belts.

So why are organisations not more naturally customer focused? Many big (and small) corporations seem to spend most of their time ignoring customers and even actively doing the reverse of what customers really want when it’s customers that they are there to serve.

[Read more…]

[Guest Post] 6 Ways to Improve Your Customer Testimonials Today

6 Ways to Improve Your Testimonials Today Image 3

Today we welcome a guest post by Sam Johnson. Sam is the Founder at Feedback Loop – An Australian Software Company that helps businesses get the best testimonials on their website.

Extending your customer feedback process to also capture testimonials is a relatively simple task. You already know which customers are advocates of your brand from the feedback. Now all you need to do is ask them for a testimonial.

I’ll let Sam take it from here on some great ways to improve your customer testimonials.

Customer testimonials are one of the most powerful forms of marketing yet there are a few common mistakes that you can fix to quickly increase sales on your website.

[Read more…]