The Pros and Cons of Surveys That Are Critical to Success

Get the Customer Feedback Data Analyis Spreadsheet

When you want to understand how customers perceive your business, its products and its services, a customer survey is often the first tool you reach for.

And they can be very useful but you should be aware of both the pros and cons of surveys when using them in your business so that you can guard against common issues and leverage the strengths.

The Pros and Cons of Surveys

I’ve packaged the examples in this blog post in an Excel Spreadsheet you can  download and use:Download Now

Pros of Surveys

There are many advantages of surveys and they can provide access to information no other approach can reliably provide.

Great for Gathering Qualitative Feedback

Often the only way to gather important qualitative (text) feedback from your market is by using a questionnaire. Through judicious use of open ended questions you can gain insights into what your customers think about each aspect of the business.

Other approaches, e.g. A/B Testing, can be very good at selecting between two, or more alternatives. However, these techniques only use quantitative data – they don’t collect qualitative information about why a choice is preferred by customers.

Without understanding the why you are sometimes doomed to random luck to hit on the best approaches.

Perfect for Collecting Emotional Feedback

Similarly, surveys are the best way to collect emotional feedback from your customers. If you want to know which of your products features and elements excites them and which anger them them then a survey is your best bet.

More Direct than Interpreting Usage Data

It is possible to build up an accurate picture of what customers want by watching their actions: clicks, purchases, call times, etc. This is exactly what Google do on a vast scale to determine the most effective ways for people to interact with their applications.

The problem is the technical skills and resources required to design, build and analyses these tests is formidable. Not only do you need to expend resources creating the different version of that you want to test, advanced statistical tools and analysis are needed to interpret the results accurately.

Lastly these methods rely on high volumes of interactions with your customers at each point. If you have smaller numbers of interactions, say you’re a B2B company, then this approach is even more difficult.

Surveys on the other hand can directly ask the questions to which you want to know the answers.

Lower Cost / More Practical than building all the alternatives

Following on from that idea – building and testing many different versions of your product or service to see which sells the best is expensive. While this can be a good approach in a limited way, (see the Lean Startup ) it is also expensive and time consuming.

A quick online customer feedback test can provide many of the same answers quickly and at a much lower cost.

Easy to build and implement

Compared to the complex statistical and organizational issues needed to run A/B tests, there are plenty of on-line tools that make the process of implementing a survey relatively straightforward.

Within a few minutes you can design and load a feedback survey then send it your your customers for their response.


Another advantage of surveys is that gathering results can be very quick. Typically you will receive 90% of the responses to your survey within 48 hours of sending the invitations.

Cons of Surveys

Of course there are also a range of disadvantages of surveys that you must be aware of when using them.

Sampled data not full data

Surveys typically operate on a sample size approach where a subset of people in the overall population are invited to respond. Even when everyone in the group is invited to respond, typically only a proportion will do so.

This means that you don’t have data from everyone and introduces the need to perform some statistics to analyze the data effectively.

Those survey statistics don’t have to be complex and they are generally easier than the A/B statistics that you need above. But you will need to consider them.

Customer fatigue

We’ve all received survey invitations and the trend of companies using customer feedback surveys is up. This means that some level of survey fatigue is setting in with customers.

How much it affects your questionnaire depends on you. If you make it easy for customers to respond and you actually do something with the information then our experience is that the fatigue will be lower.

Not Objective Responses

Possibly the biggest problem with surveys is that people are inherently subjective, not objective, when they respond. People are not thermometers, by which I mean they are not able to provide 100% rational responses to your questionnaire.

The fight they had this morning with their spouse will affect the score they give you. How you handled an issue with them a year ago will impact their perception of their call to your support desk today. They can’t objectively prioritize the importance of service attributes because they truly just don’t know exactly what drives their purchase declensions, even if they think they do.

While this might sound like a show-stopper for surveys it isn’t. The data can still be very useful as long as you remember that it is subjective in nature – and remember all their purchasing decisions are also subjective!

Biases and Effects

The range and number different types of response biases is large. From Recency and Primacy Effects to the Halo effect and even the medium (email, telephone, face to face) through which you ask questions can subtly change the pattern of feedback your survey generates.

Most of effects can be overcome by careful and thoughtful survey design and as long as you do that you will be fine.

You Must Build Them Correctly

As in every endeavour, if you don’t design and execute your survey properly there can be problems.

There are several ways that you can unintentionally change the responses people provide to the survey.

One common issue is making questions mandatory and not providing a “Not Applicable” or other opt-out style response. Forcing people to respond with they don’t want to will just generate inaccurate data.

Again good customer survey design and execution can eliminate many of these issues.

I’ve packaged the examples in this blog post in an Excel Spreadsheet you can  download and use:Download Now
cx tribe

Join >5,000 CX Professionals

Genroe uses this information to send you the CX Tribe e-Newsletter. You may unsubscribe at any time. View our Privacy Policy.

Send this to a friend