Using Chi-Squared tests on Net Promoter® Data

Using Chi-Squared tests on Net Promoter® Data

Picture of Adam Ramshaw
Adam Ramshaw
Adam Ramshaw has been helping companies to improve their Net Promoter® and Customer Feedback systems for more than 15 years. He is on a mission to stamp out ineffective processes and bad surveys.
Free NPS statistics spreadsheet

In a recent post on NPS data analysis I discussed using the Margin of Error method to determine if the difference between two Net Promoter Scores® is probably real or just statistical noise. A sharp eye reader has identified that you can also use a Chi -Squared test to perform this test.

The advantage of the Chi-Squared test is that it can be more discerning. For instance consider these two set of Net Promoter results:

Sample A

  • Promoters=100
  • Neutrals=0
  • Detractors=100

NPS= 0

Sample B

  • Promoters=0
  • Neutrals=200
  • Detractors=0

NPS= 0

Even though the two scores are the same, you can see that there is something qualitatively different between the results. However, looking at the score and even using the Margin of Error calculation implies that the scores are no different.

Enter the Chi-Squared Test

This statistical test can be used to determine if two sets of categorical data are different. Categorical data just means responses that are not numbers, e.g. Promoter or Detractor.

Aside: Interestingly the response from the “would recommend” question is numerical but by transforming the response into Promoters, Neutrals, Detractors the data becomes categorical.

The test basically works by looking at what we expect to happen and what actually happens, then looking at how similar they are.

This post is not trying to replicate a statistics course though so for more details on the actual calculation try starting at Wikipedia.

Using the Chi-Squared Test

The chi-squared test is sensitive to shifts in the underlying values making up the NPS, not just the NPS score itself.  So you can use this test to compare your Net Promoter samples.

Let’s go back and re-test our original data with this approach:

Chi-squared Value =  400.00
Critical Value =  5.99  (for 95% confidence, 2 degrees of freedom)

The Chi-squared value > Critical Value therefore there has been a change.

Once again that maths for this calculation, while not complex, can be a little difficult to follow. So we have updated our free download tool to be a Net Promoter Comparison Tester.

I would like to thank Darren Nicholson for being that sharp-eyed reader and laying out the Chi-Squared test.

Do you have other approaches that you would like to share with the NPS® community? Leave a comment below.

Download our free NPS statistics spreadsheet and easily calculate margin of  error and test for changes in NPS: Download Now