How to calculate NPS® Margin of Error and other Statistics (Excel Download)

Your boss walks in with a chart of the last 12 months of transactional Net Promoter survey results and she’s not happy!

The score went down last month and she wants to know why. Looks like you’ll have to hunt around to find a reason for the change; or will you?

Just because your survey score has gone down, or up, doesn’t mean that there has actually been a change in the overall business NPS. It might just be a fluke of the sample you have collected. The change might be within the Margin of Error.

Most guides stop at “apply the standard 95% formula, done.” That’s what I’ll show you first, it’s the calculation our own spreadsheet uses, because it’s fast and good enough for a monthly check. But after 20+ years of watching teams argue over a two-point wobble, the maths is rarely the hard part. Knowing when a number is worth arguing about at all is. More on that below, plus a more precise formula most guides skip entirely.

In this post we’ll run through how to calculate all of the statistical values you need to understand if the NPS probably has gone down, or up. Or, skip ahead and download my free Excel NPS Statistics spreadsheet and just use that.

Margin of Error for NPS Data

Understanding Margin of Error

When collecting data using a survey, like NPS, what you want to know is the NPS of all your entire customer base, the Population NPS.

The problem is that you are never able to collect a response from every single customer. In reality you make do with a sample; maybe 10%, 20% or whatever of your customers respond.

This means that rather than calculating the Population NPS of all your customers you can only ever estimate it based on the customers who responded: the Sample NPS.

Now, by random chance you might get more responses from very happy or very unhappy customers. This results in the Sample NPS being some amount lower or higher than the true Population score.

The problem is:

How do you know how close your Sample NPS estimate is to the Population NPS?

The answer is Margin of Error.

Using Margin of Error, you can be, say, 95% certain that the Population NPS for all your customers is between your Sample NPS plus the Margin of Error and your Sample NPS minus the Margin of Error.

What’s the difference: Standard Error, Margin of Error and Confidence Interval

If you’ve ever tried to understand the statistical terms around error and confidence intervals it’s easy to become confused. So just quickly I want to summarise the meaning of each for clarity.

Standard Error – is a generic term that applies to many types of sample statistics (mean, standard deviation, correlation, etc). When we talk about Averages, the Standard Error is the Standard Deviation of the sample divided by the square root of the number of responses.

Standard Error of the Mean

Standard Error Visually

Margin of Error – the amount within which you are confident (e.g. 90%, 95%) that the population mean is above or below the sample mean.

Margin of Error Visually

Confidence Interval – this is the range within which you are confident (e.g. 90%, 95%) the population mean lies.

Confidence Interval Visually

How Standard Error, Margin of Error and Confidence Interval are Related

Firstly Margin of Error and Standard Error:

Equation for Margin of Error

Where Z is a value related to level of confidence, 90%, 95% etc and is approximately 1.96 for a 95% level of confidence.

Then

Equation for Confidence Interval

So before you break out the hard hats and wait for the blame game to start you need to determine if the change in score is greater than the Margin of Error, i.e. outside the Confidence Interval.

Method 1: Calculating Margin of Error for Net Promoter®

The problem with Net Promoter is that the statistics that you normally use for survey scores don’t work so well for NPS.

However, there is an approach that you can use to determine if the change is significant. This post “How can I calculate margin of error in a NPS result?” provides a very good and detailed response to the question.

Please note that the above referenced post equates Standard Error and Margin of Error. This is a little unusual and in the rest of this post we have preserved the approach in that post but used the terms Standard Error and Margin of Error in the more common way.

That post is quite technical and if you’re not statistically inclined, reading the post may not help very much. So here I will take you through the process step by step.

First you need to know more than just the score, you need the actual number of Promoters, Detractors and Neutrals in your sample:

• #P is the number of promoters
• #N is the number of Neutrals
• #D is the number of Detractors.

Now we calculate the total respondents:

Count the Number of Respondents

Next calculate a version of NPS that runs from -1 to 1. We don’t multiply by 100 as we normally do. Here is the equation:

Calculate Net Promoter Score

Now determine the Variance of the sample NPS using the discrete random variable approach:

Variance of Net Promoter Score

Now calculate the Standard Error for your sample:

Standard Error of Net Promoter Score

And, as above, our Margin of Error is approximately 1.96 (z value for 95% confidence) times this value so:

95% confidence Margin of Error for Net Promoter Score

Remember, this is the MoE for a -1 to +1 NPS so to get this back to the same range as your normal NPS you need to multiply it by 100.

Method 2: Margin of Error for Net Promoter® with the Adjusted-Wald method

But there are methods of improving the NPS margin of error estimate.

The ±1.96 × Standard Error formula above is a normal approximation borrowed from general survey statistics. Researcher Brendan Rocks tested it against five other candidate methods across a thousand-plus real-world NPS datasets and found a variant called the adjusted-Wald method performs best, both for coverage (how often the interval actually contains the true score) and precision.

MeasuringU’s own follow-up testing on two real datasets found the same thing: adjusted-Wald gave a 90% interval around 40 to 43 NPS points wide, versus 43 to 47 NPS points for the bootstrap and means-based alternatives they compared it to. The reason is structural: NPS is three percentages (Promoters, Neutrals, Detractors) collapsed into one number, not a single proportion, and the plain formula doesn’t account for that.

The issue with the tighter interval is that calculating it is a lot more complex: 11 steps rather than a single formula.

Does that difference matter for a monthly dashboard check? Usually not. Both methods agree on whether a change is real in most real-world samples. It matters more at small sample sizes or when a score sits near the +100 or -100 extreme, exactly the conditions where you’re most tempted to draw a conclusion from thin data.

But if you’re publishing NPS statistics externally, it’s worth knowing the simpler formula isn’t the last word.

Using Net Promoter Score Margin of Error

Add Error Bars to Your Charts

One way to use Margin of Error is to add error bars to your charts. Simply add a couple of rows of values to your chart:

• NPS + MoE, and;
• NPS – MoE

Example NPS Chart including Margin of Error bars

Test if Two Samples Are Different

Another way you can use MoE is to perform the calculation for two different samples and end up with two Standard Errors you can compare.

To compare two such results you need to account for the possibility of error in each. When survey sizes are about the same, the standard error of their difference can be found by a Pythagorean theorem: take the square root of the sum of their squares. [source]

You can now use this information to determine if your score probably (95% confidence, Z value of 1.96) changed between samples:

How to compare two NPS values

If that looks like it’s too much maths you can download our handy dandy NPS Margin of Error Calculator spreadsheet.

All you need to do is enter the number of #Ps, #Ns and #Ds for each sample and it will tell you if the score has really changed and even provide a chart for you.

Net Promoter Sample Size: How Many NPS responses Do You Need?

Exactly how many responses you need to move beyond the NPS Margin of error depends very much on the range of scores you receive and how many responses you have received.

So, let’s do a couple of real-world examples to get a feel for how different volumes of responses relate to the MoE in your NPS analysis.

Each of the examples generates an NPS of 10 but it takes a relatively large number of responses (at least 100) to start to reduce the statistical uncertainty in the score.

You can see that even for larger numbers of responses, say 100, the Margin of Error is quite large in absolute NPS terms.

Problems with using NPS Margin of Error

So, we can use Margin of Error method to determine if the difference between two Net Promoter Scores® is probably real or probably just statistical noise but there are times when it doesn’t work so well.

For instance consider these two sets of Net Promoter results:

Even though the two scores are the same, you can see that there is something qualitatively different between the results. However, looking at the score and even using the Margin of Error calculation implies that the scores are no different.

Using the Chi-Squared Test on NPS Data

Let’s look at the problem (“did my score change”) another way and see if we can solve for that inherent issue in how Margin of Error works.

The Chi-squared test is a well known and simple statistical method used to determine if there is a significant relationship between two categorical variables by comparing observed data to what we would expect if there were no relationship.

The advantage of the Chi-Squared test is that it can be more discerning than margin of error.

The disadvantage is that it can only tell you if the two samples are probably different or probably the same, i.e. there is no estimate of how big that difference is.

Chi-squared can be used to determine if two sets of categorical data are different. Categorical data just means responses that are not numbers, e.g. Promoter or Detractor.

Aside: Interestingly the response from the “would recommend” question is numerical but by transforming the response into Promoters, Neutrals, Detractors the data becomes categorical.

This post is not trying to replicate a statistics course though so for more details on the actual calculation try starting at Wikipedia.

Using the Chi-Squared Test on Net Promoter Score Data

The chi-squared test is sensitive to shifts in the underlying values making up the NPS, not just the NPS score itself. So you can use this test to compare your Net Promoter samples.

Let’s go back and re-test our original data with this approach:

Chi-squared Value = 400.00

Critical Value = 5.99 (for 95% confidence, 2 degrees of freedom)

The Chi-squared value > Critical Value therefore there has been a change.

Once again the maths for this calculation, while not complex, can be a little difficult to follow. So we have updated our free download tool to be a Net Promoter Comparison Tester.

HT: Darren Nicholson for suggesting the use of the Chi-squared test on NPS data.

Which Test is Best for NPS Data: Margin of Error or Chi-squared?

In a sense there is no “best test” – only the test that you feel comfortable using.

In our NPS Statistics Excel download we provide calculations for both tests and we often get asked: “What if the tests disagree”, i.e. one test says no difference and one test says there is a difference.

Because, well, statistics, this can easily happen and your interpretation of the results depends on the risk you’re willing to take.

If you want to be very sure a change has occurred, or has not occurred, you should ensure both tests agree.

If on the other hand you’re less concerned you could take the view that if either test shows a change, or not a change, you go with that interpretation.

There is no right answer to this question, just the one that suits your situation.

Why This Matters Beyond the Spreadsheet

At the sample sizes most transactional NPS programmes run each month, a two or three point movement is usually noise, not signal. We wrote a companion piece on whether your NPS really changed using the same logic, worth reading alongside this one if you’re the one explaining the movement to your boss.

It’s also why we’re wary of organisations that set hard NPS targets and tie them to bonuses. If the score carries a five-to-twenty point margin of error most months, rewarding or penalising someone for a movement inside that band is rewarding noise. Treat NPS as a trend to manage over quarters, not a number to defend every month.

Building your own tracker from scratch instead? Our guide to calculating NPS in Excel or Google Sheets covers the base formula this spreadsheet builds on.