After spending 15 minutes on the phone to my bank the nice lady asks me if I would hold on after she hangs up to take a short survey. Sure what’s one more survey to someone who lives them 24/7!
After a couple of prompted button pushes to enter scores I get the chance to provide verbal feedback so I proceed with a relatively detailed account of a recent issue. I’m only half way through when I’m interrupted with a tone and told that the recording has ended.
But I had so much more to say.
Not a great experience.
Post call IVR surveys like this are everywhere with organisations using the approach to gather transactional customer feedback.
The question is not whether they are possible, clearly they are, but whether they are useful.
It turns out: much less so than you might think.
In this post I compare post call IVR surveys to email invites for web surveys to see which is more effective. I also provide a checklist of things that you must do if you want to be successful.
- What is a Post Call IVR Survey?
- Pros of IVR Surveys
- Cons of IVR Surveys
- Low Response Rates
- Low Volumes of Qualitative Feedback
- Only Captures The Telephone Touchpoint
- Bias Introduced by Survey Methodology
- Difficult to Integrate with Existing CRM systems
- Limited Response Options
- Poor Ability to Use Qualitative Information
- Ability to “Game the System”
- Limited/No ability to initiate Service Recovery contact
- Post Call IVR Invite Vs Email Invite Surveys
- Key Requirements for Post Call IVR Survey Success
What is a Post Call IVR Survey?
First a quick definition of the term:
A post call IVR survey is an audio based survey that is normally used in contact centres. After the customer has interacted with the call centre agent they are transferred (automatically or manually) to a voice promoted survey system. The voice prompts provide the survey questions and the customer enters their response by the telephone keypad and/or by speaking their answers.
Pros of IVR Surveys
Easy to Execute
Post call IVR data collection is generally easy to get up and running. If your IVR vendor supports the survey function you can set up a survey and start collecting data.
Easy to Get Agent Level Scores
If you’re looking to track simple agent level scores for customer satisfaction or Net Promoter Score® then a post call IVR survey is a simple way of doing it.
But there’s pretty good evidence that you should not be giving front line staff score targets because it has a negative impact on culture and overall outcomes.
This type of inbound transferred IVR survey appears to be low cost as there is no initial human interaction and no media costs.
However, when you investigate a little more you realise the the costs to transcribe, code and evaluate the data are much higher than email surveys. You need to compare the total cost, not just the cost to turn the feature on in your IVR.
Needs No Data About the Customers
Forwarding a caller to the survey requires no data to be collected. You don’t need email addresses or even cell/mobile phone numbers to receive the feedback.
For organisations that have poor or limited data in these areas, that can be attractive.
On a side note: wouldn’t it be a better idea to collect and validate the customer’s email address and phone number while they are still on the call in the first place? Then you’ll have this contact information for use in the future for, oh I don’t know, customer service or marketing?
Cons of IVR Surveys
Low Response Rates
It is difficult to get public figures on the response rates for post call IVR surveys. There are lots of data about the completion rate being high but that’s the completion rate for people who agree to be surveyed.
The chart below probably gives a better view of the overall completion rates by technology. As you can see online surveys may have a large number of “no responses” but when customers do respond they respond fully whereas many IVR surveys are only partially completed.
Note also that the online response rates noted here are lower than the Net Promoter survey response rates we typically see of between 10% and 25% of all invites.
Low overall response rates are caused by many things but an important one is that the survey must be done then and there. After the call customers may not want to wait around to complete a 2-5 minute survey.
On the other hand they can do the email survey anytime they like and typically on any device they like.
Low Volumes of Qualitative Feedback
Experienced users indicate around 20% of responses include qualitative or text information. This significantly lower than the rates a well designed internet survey can generate of around 50%.
Only Captures The Telephone Touchpoint
If the only interface you had with your customers was via the phone then post call IVR survey may be fine but you probably have many more touch points in your business.
The customer journey typically contains more channels than just the telephone: website, mobile app, etc
So you will also need to measure your website experience, account manager calls, invoicing, delivery, etc?
If you want to measure them, at the very least you will need to set up a separate survey process with separate data entry and the associate business data integration issues .
Bias Introduced by Survey Methodology
Another item not to be missed is the survey bias that different survey methodologies can introduce into the score.
Due to the ways that the human brain manages visual and verbal information it is likely that the scores on your IVR and other non-IVR based touch-point surveys cannot be validly compared.
Difficult to Integrate with Existing CRM systems
To maximise the value of customer feedback it is important to view it in context with all of the other information the company has about that specific customer: their spend, product purchases, support contacts, etc.
Post call IVR systems are able to show and report the feedback in the context of the IVR system (hold time, speed of answer, etc) but often integrating that information back into the overall CRM data is difficult or not possible.
Limited Response Options
While it’s never a good idea to have too many question response options, post call IVR surveys may cause the opposite problem: too few response options.
Because of the way humans process verbal information you can only have a few responses options to any one question.
Which can cause one of two problems:
- Recency bias: if there are too many response options people are more likely to select one of the last options they heard, rather than weigh all options equally.
- Missed options: if not enough options are provided for the question posed, important options may be missed and the respondent is not able to provide an accurate response.
Poor Ability to Use Qualitative Information
The score is just the score and basically not useful for much more than tracking progress.
Qualitative information on the other hand is critical in determining how to drive continuous improvement in your business. While scores are interesting and provide a benchmark to track success it is the qualitative feedback that helps you to understand how to improve.
Post call IVR surveys do not handle this qualitative data very well for the following reasons.
Difficult to Use Captured Feedback
Capturing qualitative information is easy with post call IVR – simply record what the customer says. Unfortunately as a voice recording it is very difficult to use that information.
Having staff listen to each recording is labour intensive and prone to long term apathy. So you need to convert that sound information to text information either by using speech to text software or having staff transcribe the recordings.
At present software systems get it about 70% right. If you have ever received a speech to text SMS message you know that the accuracy of this software is subject to some “variation”. Plus, that still leaves 30% of the feedback from your customers missing.
If your IVR doesn’t have speech to text processing built in this will add significantly to project complexity and costs.
Of course, as soon as you have staff transcribing recordings the apparent cost benefit of post call IVR disappears quickly.
Limited Ability to Tag Qualitative Feedback
If you have only a few pieces of qualitative feedback you can read and act on them. Of course if you are thinking of implementing post call IVR then you will have way more than a few pieces of feedback so that approach is not going to work.
When you get over about 100 pieces of qualitative feedback you will need to start tagging that information so you can sort and analyse it in a productive manner. Tagging can be done in two ways:
- Manual tagging: This is where a person manually tags each piece of feedback based on what is in that feedback. This is a labour intensive process that is subject to human error and quickly gets old for the staff member(s) who have to do it.
- Passive machine tagging: uses software to read the comment and interpret what tags should be applied. This can work quite well but setting up and training the software is not a trivial matter. It is always more expensive and complex than the software vendors will admit pre-sale. In fact coding the data in this way is so difficult, even larger companies simply give up and code the data manually.
Ability to “Game the System”
Some amount of “gaming the system” can be expected in most surveys and post call IVR surveys are no different. Here the gaming can be as simple as not offering customers who are angry or unhappy the option of taking the survey.
Of course, there are a couple of ways around this:
- The survey invitation can be made a mandatory part of the agents closing scripts and emphasized in Quality Assurance reviews.
- The phone system can automatically re-direct all inbound phone calls to the survey at the end of the call. This eliminates the issue by removing the decision on whether to send a caller to the survey system.
In reality both of these approaches are still open to being gamed but the issue can be minimised.
Limited/No ability to initiate Service Recovery contact
Starting the small loop or Service Recovery process is a critical value add in the customer feedback process. This is difficult with post call IVR because they typically lack the alert systems that integrated customer feedback solutions offer.
Post Call IVR Invite Vs Email Invite Surveys
As you can see while Post Call IVR surveys can be used to collect data they suffer from a number of issues in enabling the organisation to use the data collected.
If you are considering using this survey mechanism you should consider if the outcomes will be useful as well as cheap and easy.
Pros of Post Call IVR and Email Invite Surveys
Post Call IVR
- Easy to Execute
- Low Cost
- Needs no data about the customers
- Easy to Execute
- Low cost
- Good response rates
- Text data is accurate and immediately available
- Text data is tagged by the customer
- Active customer tagging is possible
Cons of Post Call IVR and Email Invite Surveys
Post Call IVR
- Low response rates
- Low volumes of qualitative feedback
- Poor ability to use qualitative information
- Subject to gaming by operators
- Limited ability to tag feedback
- Inability to track other than telephone Touch points
- Limited/No ability to initiate Service Recovery contact
- Needs customer data, email address
Key Requirements for Post Call IVR Survey Success
Okay, if you’re still keen to turn on that post call survey feature, here’s what you need to do to drive value:
- Have a short survey (this hold true for all feedback surveys)
- Have an active change management process so all staff are aware of the change in process (also true for all feedback surveys)
- Transcribe the recordings so you have access to the text data for analysis. This can be done by people or software but it must be done.
- Tag the data using people or machines: neither comes cheap but it is required.
- Automate the sharing of the data amongst staff. Total transparency of data is one of the key planks of customer feedback success.