After spending 15 minutes on the phone to my bank the nice lady asks me if I would hold on after she hangs up to take a short survey. Sure what’s one more survey to someone who lives them 24/7!
After a couple of prompted button pushes to enter scores I get the chance to provide verbal feedback. I’m not the type to use circumlocution, I like to get straight to the point and I proceed with a relatively detailed account of a recent issue. I’m only half way through when I’m interrupted with a tone and told that the recording has ended.
But I had so much more to say.
Not a great experience.
Post call IVR surveys like this are popping up everywhere with more and more organisations using the approach to gather transactional customer feedback.
The question is not whether they are possible, clearly they are, but whether they are useful. It turns out: much less so than you might think.
In this post I compare post call IVR surveys to email invites for web surveys to see which is more effective. I also provide a checklist of things that you must do if you want to be successful.
Here is a quick table of contents for this longish post.
- What is a Post Call IVR Survey?
- Pros of IVR Surveys
- Cons of IVR Surveys
- Summary of Pros and Cons
- The key requirements for success
What is a Post Call IVR Survey?
First a quick definition of the term:
A post call IVR survey is an audio based survey that is normally used in contact centres. After the customer has interacted with the call centre agent they are transferred (automatically or manually) to a voice promoted survey system. The voice prompts provide the survey questions and the customer enters their response by the telephone keypad and/or by speaking their answers.
Pros of IVR Surveys
Easy to Execute
Post call IVR data collection is generally easy to get up and running. If your IVR vendor supports the survey function you can set up a survey and start collecting data.
Easy to Get Agent Level Scores
If you’re looking to track simple agent level scores for customer satisfaction or Net Promoter Score® then a post call IVR survey is a simple way of doing it.
But there’s pretty good evidence that you should not be giving front line staff score targets because it has a negative impact on culture and overall outcomes.
This type of inbound transferred IVR survey appears to be low cost as there is no initial human interaction and no media costs.
However, when you investigate a little more you realise the the costs to transcribe, code and evaluate the data are much higher than email surveys. You need to compare the total cost, not just the cost to turn the feature on in your IVR.
Needs No Data About the Customers
Forwarding a caller to the survey requires no data to be collected. You don’t need email addresses or even cell/mobile phone numbers to receive the feedback.
For organisations that have poor or limited data in these areas, that can be attractive.
On a side note, wouldn’t it be a better idea to collect and validate the customer’s email address and phone number while they are still on the call in the first place. Then you’ll have this contact information for use in the future for, oh I don’t know, customer service or marketing?
Cons of IVR Surveys
Low Response Rates
It is difficult to get public figures on the response rates for post call IVR surveys. There are lots of data about the completion rate being high but that’s the completion rate for people who agree to be surveyed.
The chart below probably gives a better view of the overall completion rates by technology. As you can see online surveys may have a large number of “no responses” but when customers do respond they respond fully whereas many IVR surveys are only partially completed.
Note also that the online response rates noted here are lower than the Net Promoter survey response rates we typically see of between 10% and 25% of all invites.
Low overall response rates are caused by many things but an important one is that the survey must be done then and there. After the call customers may not want to wait around to complete a 2-5 minute survey.
On the other hand they can do the email survey anytime they like and typically on any device they like.
Low Volumes of Qualitative Feedback
Experienced users indicate around 20% of responses include qualitative or text information. This significantly lower than the rates a well designed internet survey can generate of around 50%.
Only Captures The Telephone Touchpoint
If the only interface you had with your customers was via the phone then post call IVR survey may be perfect but you probably have many more touch points in your business.
How will you measure your website experience, account manager calls, invoicing, delivery, etc? If you want to measure them, at the very least you will need to set up a separate survey process with separate data entry and the associate business issues. Why not simply run all surveys using the same platform?
Another item not to be missed here is the survey bias that different survey methodologies can introduce into the score. Due to the ways that the human brain manages visual and verbal information it is likely that the scores on your IVR and other non-IVR based touch-point surveys cannot be validly compared.
Poor Ability to Use Qualitative Information
The score is just the score and basically not useful for much more than tracking progress.
Qualitative information on the other hand is critical in determining how to drive continuous improvement in your business. While scores are interesting and provide a benchmark to track success it is the qualitative feedback that helps you to understand how to improve.
Post call IVR surveys do not handle this qualitative data very well for the following reasons.
Difficult to Use Captured Feedback
Capturing qualitative information is easy with post call IVR – simply record what the customer says. Unfortunately as a voice recording it is very difficult to use that information.
Having staff listen to each recording is labour intensive and prone to long term apathy. So you need to convert that sound information to text information either by using speech to text software or having staff transcribe the recordings.
At present software systems get it about 70% right. If you have ever received a speech to text SMS message you know that the accuracy of this software is subject to some “variation”. Plus, that still leaves 30% of the feedback from your customers missing.
If your IVR doesn’t have speech to text processing built in this will add significantly to project complexity and costs.
Of course, as soon as you have staff transcribing recordings the apparent cost benefit of post call IVR disappears quickly.
Limited Ability to Tag Qualitative Feedback
If you have only a few pieces of qualitative feedback you can read and act on them. Of course if you are thinking of implementing post call IVR then you will have way more than a few pieces of feedback so that approach is not going to work.
When you get over about 100 pieces of qualitative feedback you will need to start tagging that information so you can sort and analyse it in a productive manner. Tagging can be done in two ways:
- Manual tagging: This is where a person manually tags each piece of feedback based on what is in that feedback. This is a labour intensive process that is subject to human error and quickly gets old for the staff member(s) who have to do it.
- Passive machine tagging: uses software to read the comment and interpret what tags should be applied. This can work quite well but setting up and training the software is not a trivial matter. It is always more expensive and complex than the software vendors will admit pre-sale. In fact coding the data in this way is so difficult, even larger companies simply give up and code the data manually.
Ability to “Game the System”
Some amount of “gaming the system” can be expected in most surveys and post call IVR surveys are no different. Here the gaming can be as simple as not offering customers who are angry or unhappy the option of taking the survey.
Of course, there are a couple of ways around this:
- The survey invitation can be made a mandatory part of the agents closing scripts and emphasized in Quality Assurance reviews.
- The phone system can automatically re-direct all inbound phone calls to the survey at the end of the call. This eliminates the issue by removing the decision on whether to send a caller to the survey system.
In reality both of these approaches are still open to being gamed but the issue can be minimised.
Limited/No ability to initiate Service Recovery contact
Starting the small loop or Service Recovery process is a critical value add in the customer feedback process. This is difficult with post call IVR because they typically lack the alert systems that integrated customer feedback solutions offer.
These integrated solutions enable the organisation to trigger and alert to staff when customers provide low score so they can commence the service recovery process.
Summary of Pros and Cons
As you can see while Post Call IVR surveys can be used to collect data they suffer from a number of issues in enabling the organisation to use the data collected.
If you are considering using this survey mechanism you should consider if the outcomes will be useful as well as cheap and easy.
- Easy to Execute
- Low Cost
- Needs no data about the customers
- Easy to Execute
- Low cost
- Good response rates
- Text data is accurate and immediately available
- Text data is tagged by the customer
- Active customer tagging is possible
- Low response rates
- Low volumes of qualitative feedback
- Poor ability to use qualitative information
- Subject to gaming by operators
- Limited ability to tag feedback
- Inability to track other than telephone Touch points
- Limited/No ability to initiate Service Recovery contact
- Needs customer data, email address
The key requirements for success
Okay, if you’re still keen to turn on that post call survey feature, here’s what you need to do to drive value:
- Have a short survey (this hold true for all feedback surveys)
- Have an active change management process so all staff are aware of the change in process (also true for all feedback surveys)
- Transcribe the recordings so you have access to the text data for analysis. This can be done by people or software but it must be done.
- Tag the data using people or machines: neither comes cheap but it is required.
- Automate the sharing of the data amongst staff. Total transparency of data is one of the key planks of customer feedback success.