Taguchi Testing: Triggered marketing testing on overdrive

Get the Business Leaders’ Practical Guide to Implementing Net Promoter

If you are already using triggered marketing approaches, then you will want to maximize the value of each contact with your customer.  To do that, you need to test your communications. And if you’re like most people, you’d be happy to improve the conversion on your triggered pieces by a few percentage points. What if there was a way to double or triple the number of conversions?


There is way and the secret is to vastly improve your ability to test e-newsletter elements efficiently and effectively. The way to do that is called Taguchi test design.

As a case study lets consider the sale panel in your on-boarding communication. In trying to maximise conversions there are lots of options that you should test different subject lines, panel on left or right, top or bottom of newsletter, different panel headline, different panel images and different features promoted.

To test these elements traditional direct marketers would perform split mailings. That is, they would create a different test panel for each version of each option. Then send each different test panel to a different test cell before finally comparing results and selecting the best for the final mail-out.

The trouble with this approach is that the number of different test panels needed rapidly gets out of hand. Even for this simple case, if we assume two versions of each option, we need to create 64 (2 x 2 x 2 x 2 x 2 x 2) test panels and have 64 test group cells. Even if you have enough test groups available, the time and effort to coordinate the process is very high.

Taguchi Testing

Enter a new, old, approach. For many years engineers have had similar issues when trying to optimize manufacturing processes. For instance, what is the best combination of speed, pressure and temperature to achieve a specific thickness of metal coming out of a rolling machine? Rather than testing all of the possible combinations, engineers often use an approach called Taguchi design were you only test a few, carefully selected, versions and combinations. Using the results from these tests you can determine the best combination of all the versions of all the factors.

In our example that means rather than needing 64 test groups and test panels you could get away with as few as eight. Where’s the catch I hear you ask – well it has to do with the interaction of options. With just a few test cells you don’t get very much information on how headlines interact with locating the box on the left or the right. In many marketing cases the interactions between options is minimal so it is often not an issue.

On the upside, using Taguchi designs means you can substantially reduce the cost and complexity in testing a variety of marketing initiatives. Suddenly a test that appeared impossible (we just don’t have enough recipients to test 64 groups) becomes quite possible.

Understand which are the key decisions

The other advantage of using this approach is that in the process of determining the optimal mix you also collect information on which options impact have the largest impact on the conversion process. In the next campaign you can then focus your time and effort on getting the few important options correct and not worry about those that have minimal impact on the conversion rate.

Because of the speed and ease with which promotional elements can be changed this approach has many applications the e-marketing world. Taguchi can effectively be applied to triggered marketing communications, web site conversion optimization, banner advertisements, email campaigns, pay per click (PPC – e.g. Google Adwords campaigns) advertising and e-newsletters.

When used effectively the results from this approach can be enormous. For instance one campaign for a major technology vendor resulted in 5.3 times the number of e-newsletter openings and 7.1 times the number of sales as the non-optimized newsletter. Used in a customer survey environment for a consumer product the best combination of options garnered 3.5 times as many responses as the worst combination.

So ask yourself: what would doubling the response rate for your next sales focused e-newsletter mean and think about whether you too should be using Taguchi designs.

I've created a Business Leaders’ Practical Guide to Implementing Net Promoter.  Download it Here
Send this to a friend