When you want to understand how customers perceive your business, its products and its services, customer surveys have benefits that can be hard to match with any other approach.
Yes, the can be very useful but you should be aware of both the pros and cons of surveys when using them in your business so that you can guard against common issues and leverage the strengths.
In this post I’ll use my 20 years of experience in customer and market research to distil the benefits of using surveys, and problems to be aware of when surveys to understand your customers and market place.
Survey Pros and Cons Overview
- Great for Gathering Qualitative Feedback
- Perfect for Collecting Emotional Feedback
- More Direct than Interpreting Behaviour Data
- More Accurate than Interpreting Behaviour Data
- Able to Collect Comprehensive Data
- Cheaper and Faster Than A/B Testing Every Option
- Easy To Implement
- Fast Data Collection Turn Around
- Able to Collect Large Volumes Of Data Quickly and at Low Cost
- Effective Remote Data Collection
- Surveys Provide Sampled Data not Complete Data
- Survey Fatigue Reducing Response Rates
- Responses are Inherently Not Objective
- Respondent Honestly and Intention can Impact Accuracy
- Unintentional Biases and Effects Can Impact Accuracy
- Poor Survey Design can Lead to Incorrect Conclusions
Survey Advantages in Detail
There are many advantages of surveys and they can provide access to information no other approach can reliably provide.
Great for Gathering Qualitative Feedback
Often the only way to gather important qualitative (text) feedback from your market is by using a questionnaire. Using open ended questions you can gain deeper insights into what your customers think about each aspect of the business.
This qualitive feedback is very good for understanding why the respondent feels a certain way and how they might like you to change to improve your business.
Without understanding the why you are sometimes doomed to random luck to hit on the best approaches.
Perfect for Collecting Emotional Feedback
Similarly, surveys are the best way to collect emotional feedback from your customers.
If you want to know which of your products features and elements excites them and which anger them them then surveys work well to uncover the details of their emotional interaction your company and product.
More Direct than Interpreting Behaviour Data
It is possible to build up an accurate picture of what customers want by watching their actions: clicks, purchases, call times, etc. This is exactly what Google do on a vast scale to determine the most effective ways for people to interact with their applications.
The problem is the technical skills and resources required to design, build and analyses these tests is formidable. Not only do you need to expend resources creating the different version of that you want to test, advanced statistical tools and analysis are needed to interpret the results accurately.
Lastly these methods rely on high volumes of interactions with your customers at each point. If you have smaller numbers of interactions, say you’re a B2B company, then this approach is even more difficult.
Surveys on the other hand can directly ask the questions to which you want to know the answers.
More Accurate than Interpreting Behaviour Data
Even if you have the technical skills required, research has shown customer loyalty models based on customer feedback data are more accurate and effective than models that used on demographic and customer behaviour data.
Able to Collect Comprehensive Data
Surveys can include large numbers of questions allowing a large variety of data to be collected from each respondent.
Cheaper and Faster Than A/B Testing Every Option
Following on from that idea – building and testing many different versions of your product or service to see which sells the best is expensive. While this can be a good approach in a limited way, (see the Lean Startup ) it is also expensive and time consuming.
A quick online customer feedback test can provide many of the same answers quickly and at a much lower cost.
Easy To Implement
Compared to the complex statistical and organizational issues needed to run A/B tests, there are plenty of on-line tools that make the process of implementing a survey relatively straightforward.
Within a few minutes you can design and load a feedback survey then send it your your customers for their response.
Fast Data Collection Turn Around
Another advantage of surveys is that gathering results can be very quick. Typically you will receive 90% of the responses to your survey within 48 hours of sending the invitations.
Able to Collect Large Volumes Of Data Quickly and at Low Cost
Surveys can be delivered via email to large numbers of respondents at a low costs compared to other options.
With on-line surveys the time and cost of data entry is born but the survey respondent. They are also delivering the data in a parallel manner so lots of data can be collected very quickly.
Effective Remote Data Collection
On-line surveys can easily collect data from physically remote locations as easily as it can from local locations.
Survey Disadvantages in Detail
Of course there are also a range of disadvantages of surveys that you must be aware of when using them.
Surveys Provide Sampled Data not Complete Data
Surveys typically operate on a sample size approach where a subset of people in the overall population are invited to respond. Even when everyone in the group is invited to respond, typically only a proportion will do so.
This means that you don’t have data from everyone and introduces the need to perform some statistics to analyze the data effectively.
Those survey statistics don’t have to be complex and they are generally easier than the A/B statistics that you need above. But you will need to consider them.
Survey Fatigue Reducing Response Rates
We’ve all received survey invitations and the trend of companies using customer feedback surveys is up. This means that some level of survey fatigue is setting in with customers.
How much it affects your questionnaire depends on you. If you make it easy for customers to respond and you actually do something with the information then our experience is that the fatigue will be lower.
Responses are Inherently Not Objective
Possibly the biggest problem with surveys is that people are inherently subjective, not objective, when they respond. People are not thermometers, by which I mean they are not able to provide 100% rational responses to your questionnaire.
The fight they had this morning with their spouse will affect the score they give you. How you handled an issue with them a year ago will impact their perception of their call to your support desk today. They can’t objectively prioritize the importance of service attributes because they truly just don’t know exactly what drives their purchase declensions, even if they think they do.
While this might sound like a show-stopper for surveys it isn’t. The data can still be very useful as long as you remember that it is subjective in nature – and remember all their purchasing decisions are also subjective!
Respondent Honestly and Intention can Impact Accuracy
There is not guarantee that people will answer your survey questions honestly with the best intentional. Recent inaccuracies with political polls and the Boaty McBoatface fiasco highlight these issues.
Unintentional Biases and Effects Can Impact Accuracy
It might not even be that people intentionally respond inaccurately. It might be due to biases.
The range and number different types of response biases is large. From Recency and Primacy Effects to the Halo effect and even the medium (email, telephone, face to face) through which you ask questions can subtly change the pattern of feedback your survey generates.
Most of effects can be overcome by careful and thoughtful survey design and as long as you do that you will be fine.
Poor Survey Design can Lead to Incorrect Conclusions
As in every endeavour, if you don’t design and execute your survey properly there can be problems.
There are several ways that you can unintentionally change the responses people provide to the survey.
One common issue is making questions mandatory and not providing a “Not Applicable” or other opt-out style response. Forcing people to respond with they don’t want to will just generate inaccurate data.
Again good customer survey design and execution can eliminate many of these issues.