Predicting American Airlines’ Net Promoter Score® Using Twitter


In this recent post by Fonolo American Airlines was hammered for having an outstanding (in a bad way) number of Twitter users complain about being on hold with them.

This got me to thinking:

How good is crowd data at predicting the Net Promoter Score for an organisation?

As it turns out it’s a pretty good indicator so let’s review it in detail

The chart shows that AA’s complaint rate is not just a bit above the other airlines but more than three times its nearest rival.

When examining the data the first thing that comes to mind is that maybe AA just has many more customers than any of the other airlines: more customers, more customers on hold, more tweets.

So, all things being equal the number of complaints could be expected to be related to the number of customers.

Let’s examine that hypothesis. Here we compare the On Hold hash tag rates with the number of passenger miles that each airline travels. (2012 data as reported by Nations On Line)


Well that shows almost no correlation at all. So, surprisingly, we can eliminate the number people as being a substantial driver of the high hold time complaints.

Of course, another possibility is that AA customers are massively over represented on Twitter relative to the other airlines in the sample. While possible, it seems unlikely.

Having eliminated those two possibilities we can see if perhaps on-hold mentions is a predictor of Net Promoter®. The NPS® data here comes from the Satmetrix  2014 Airline Industry Report.


Hold Time Complaints are a Goodish Predictor of Net Promoter Score.

This is a pretty good correlation. The higher the NPS the lower the number of on hold hashtags. Admittedly R2 = 0.29 is not a terrifically high correlation but considering that there are most likely multiple drivers for NPS and that this is a crowd level data it does seem pretty good.

However, and here is something really interesting, if you remove AA from the picture, the correlation jumps to 0.41. Now most people would admit that 0.41 is a pretty good correlation statistic and indicates that 41% of the variation in NPS can be accounted for by variation in On Hold complaint rate.

Put another way, simply counting on-hold hash tags is a pretty good way of predicting an airline’s NPS.


Maybe AA is an outlier — removing them improves predictive ability

Lastly, and we may be going too far here, if we also remove Southwest airlines from the sample set the R2 rate jumps to 0.95!


Without AA and Southwest the correlation is very high.

So what?

At this point we can draw a few tentative conclusions:

1. Crowd based data could be quite good a predicting important business metrics.

This approach would only really work well for large organisations because it requires a largish number of responses. It wouldn’t, for instance, work for the corner coffee shop. But it is a potentially useful way of ranking the NPS of large organizations.

Incidentally, this is not the first time that Twitter has been found to be good at predicting useful real work statistics so there is some precedence here.

2. When you’re down, a lot of people kick you.

As you can see the number of AA mentions is much higher than their overall NPS would indicate they should get. This could mean that their hold time is really very bad, but everything else is terrific, to balance out their NPS.

But this is seems unlikely and a better explanation might be that when people have a preconceived negative opinion they more quickly perceive service attributes negatively. There is good support for this perspective in this blog post on the halo effect.

So AA customers might be unfairly quick to pull the #onhold trigger.

3. Hold Time is a Pretty Important indicator of Customer Loyalty

Based on this information it would seem that hold time is a pretty good driver of NPS and therefore customer loyalty. This has the ring of truth about it and many organizations have validated this in their business.



Insurance Agent Receives Net Promoter Score® of +98

Over the week we’ve read a very interesting case study on an insurance agent. It’s interesting due to the fact that the company claims to have received a Net Promoter Score of +98 – A score in which we believe is unusually high. Nevertheless, we’ll provide you with some of the details on how they Rising_Arrowreached such a milestone.

The insurance industry is one industry that is not usually known for its high level of customer satisfaction. The company insureon however is changing such a perspective having received a Net Promoter Score of +98. [Read more…]

A German Family-owned Business Strengthens Ties Through Net Promoter®

corporate-governanceA successful German wholesaler drives business through Net Promoter implementation. With an already strong product line, the company implemented Net Promoter to improve customer service and strengthen ties with their customers.

W. & L. Jordan GMBH, with over 50 stores across Europe has been around since 1919 and promotes themselves as a customer centric organisation. The organisation sought the services of CustomerGauge in order to implement the Net Promoter to improve business through a customer feedback program which can be effectively measured. [Read more…]

Digital and Innovation Agency Drives Business with Net Promoter®

Service chart with red marker

Service chart with red markerUK based digital and innovation agency Volume proves there are no right or wrong industries for using the Net Promoter approach. In fact with a score of +60 the organisation is out in front of many of its’ peers:  Amazon’s score is 76 and Apple’s is 71.

Here is a company doing things  differently. Being a small agency with big success in a tough industry wasn’t enough for them. They decided to implement Net Promoter to actively seek out flaws and correct them. [Read more…]

Using Chi-Squared tests on Net Promoter® Data


In a recent post I discussed using the Margin of Error method to determine if the difference between two Net Promoter Scores® is probably real or just statistical noise. A sharp eye reader has identified that you can also use a Chi -Squared test to perform this test. [Read more…]

Case study: How to apologize to your customers when things go badly wrong


When things went pear-shaped over at MozLand, they did so in a big way. The result was lots of unhappy customers, but their response is a case study in how to apologize for problems that seriously affect your customers.

You can read the full text of their apology over on the SEOMoz site but here are the elements of what they did right.

[Read more…]

Case study: nib health fund’s Successful Net Promoter Score® Implementation


Over the last 12 months, Genroe has been privileged to work with nib health funds (one of Australia’s fastest growing health funds) to help them evolve their Net Promoter Score processes. In that time, we helped them implement CustomerGauge, an end to end, integrated, Net Promoter Score data collection, reporting, analysis and action system.

In the past, nib has allowed us to publish insightful information in two blog posts:

Now, the full story behind the successful NPS implementation process has been captured in a new case study: nib health funds: Checking Up On Customer Loyalty.

In this case study, nib offers more insights into the changes they have made using NPS insights: [Read more…]

Positive Customer Experience Drives Loyalty, Revenues


The new Temkin Group research shows what we believe: the customer experience drives customer loyalty. The chart below shows how much a good customer experience can lift your customer loyalty, and how much a bad customer experience can lower it.

Read the whole story: Positive Customer Experience Drives Loyalty, Revenues

Case Study: Beating the Market with Customer Satisfaction

Case Study Chart

If you’re looking to boost customer satisfaction, one of the most promising places to start is customer service. Unfortunately, it’s also a place where long-term goals tend to buckle under short-term financial pressures. Companies try to meet Wall Street’s immediate demands by cutting costs through automation and outsourcing—despite a growing body of research conclusively showing that customers are fed up with lousy service and that increased satisfaction has a positive impact on consumer spending, cash flow, and business performance.

In a groundbreaking 2006 study, University of Michigan business professor Claes Fornell and colleagues showed the relationship between customer satisfaction and financial success by creating a hedge portfolio in which stocks are bought long and sold short in response to changes in the American Customer Satisfaction Index (ACSI). Developed by the University of Michigan’s National Quality Research Center, the ACSI is an indicator of economic success that reflects levels of customer satisfaction with goods and services purchased from about 200 companies in more than 40 industries; it’s based on interviews with more than 65,000 U.S. consumers each year. [Read more…]

Proof: NPS® is much more sensitive than Customer Satisfaction


Recent analysis of Net Promoter Score® data from one of our clients indicates that NPS is almost three times more sensitive at predicting customer churn than customer satisfaction. In addition, Detractors are 1.5 times more likely to terminate than Promoters.

nib health funds is one of Australia’s leading and fastest growing health funds.  As an organisation, nib has embraced the Net Promoter Score process.  They have integrated transactional measurement of NPS into the Customer Care Centre and other key customer touch points using the CustomerGauge NPS data collection and reporting system (full disclosure: we sell the CustomerGauge system in Asia). [Read more…]