In a recent post I discussed using the Margin of Error method to determine if the difference between two Net Promoter Scores® is probably real or just statistical noise. A sharp eye reader has identified that you can also use a Chi -Squared test to perform this test.
The advantage of the Chi-Squared test is that it is more discerning. For instance consider these two set of Net Promoter results:
- Sample A: Promoters=100,Neutrals=0,Detractors=100 => NPS= 0
- Sample B: Promoters=0,Neutrals=200,Detractors=0 => NPS= 0
You can see that there is something qualitatively different between these two scores. However, looking at the score and even using the Margin of Error calculation implies that the scores are no different.
Enter the Chi-Squared Test
This statistical test can be used to determine if two sets of categorical data are different. Categorical data just means responses that are not numbers, e.g. Promoter or Detractor.
Aside: Interestingly the response from the “would recommend” question is numerical but by transforming the response into Promoters, Neutrals, Detractors the data becomes categorical.
The test basically works by looking at what we expect to happen and what actually happens, then looking at how similar they are.
This post is not trying to replicate a statistics course though so for more details on the actual calculation try starting at Wikipedia.
Using the Chi-Squared Test
The chi-squared test is sensitive to shifts in the underlying values making up the NPS, not just the NPS score itself. So you can use this test to compare your Net Promoter samples.
Let’s go back and re-test our original data with this approach:
Chi-squared Value = 400.00
Critical Value = 5.99 (for 95% confidence, 2 degrees of freedom)
The Chi-squared value > Critical Value therefore there has been a change.
Once again that maths for this calculation, while not complex, can be a little difficult to follow. So we have updated our free download tool to be a Net Promoter Comparison Tester.
I would like to thank Darren Nicholson for being that sharp-eyed reader and laying out the Chi-Squared test.
Do you have other approaches that you would like to share with the NPS® community? Leave a comment below.