5 Survey Response Biases you must avoid

5-response-bias-banner-image

What is response bias?

Participants usually don’t give false survey answers on purpose. Most of the time, if they do so, it’s because certain factors influence them to answer that way. These tendencies to respond untruthfully are collectively called response bias.

Reasons for response bias may include the nature of the question or the fact it is framed in a manner that could lead a respondent to answer dishonestly.

For example, respondents may feel a certain pressure to give answers that are socially acceptable.

Basically, any external factor that influences a respondent to answer untruthfully is a response bias. Regardless whether the bias is intentional or accidental, the survey data becomes useless because it is inaccurate.

Weeding out response bias demands a 360-degree analysis of your survey to ensure there is no chance for it to creep in.

 

Why does response bias happen?

There is no definitive theory on why exactly response bias happens. It's usually because of a combination of several factors.

These can include: question phrasing; researcher behavior; the way the experiment is conducted; or the desire of the participant to be a good, candidate and provide socially desirably responses.

Response bias is difficult to detect at first because of its inconsistency. It can lead organizations to derive wrong insights from the data and has the potential to make the entire data set incorrect.

Here's an example to make it clearer:

Do you think it’s okay to drink alcohol frequently?

 

This question lies in the tricky zone. Here’s why:

“Do you think it’s okay to drink alcohol frequently?” is something that society conditions us to say “no” to, even for those who drink heavily. So asking the question this way is unlikely to get a truthful response. Participants tend to answer these types of questions with socially desirable, rather than honest and truthful, answers.

 

The different types of response bias

Here are the various types of response bias a surveyor might come across.

Acquiescence bias

People want to be agreeable in general. If your survey includes agree/disagree type questions, responders are more likely to agree. This tendency to concur is known as acquiescence bias or "yea-saying."

Here's an example to make it clearer:

For instance, let’s assume one of your questions asks, “Are you a social person?” while another asks, “Would you rather read a novel than attend a party?” If the participant agrees to both cases, that’s acquiescence bias. This occurs when the participants always choose to deny or not endorse any of the statements in a survey or a measure in any way.

There are two major theories that suggest why this bias comes up, and, surprisingly, they imply completely opposite things.

  • The first one says that during a survey, the respondent's brain is occupied with pleasing the surveyor, thus resulting in dishonest answers acquiescence bias.

  • The other theory, proposed by American psychologist Lee Cronbach, mentions says that the bias is more related with a participant's active mental processes and has less to do with the motivation to please the researcher. Cronbach suggests that the participant takes time to dig out memories about instances that support a statement and ignores any contradictory information.

However, don't worry: Keeping a well-balanced questionnaire helps to weed out this kind of response bias. Most of the time, keeping half the questions negatively keyed and the other half positively keyed is a good idea to tackle acquiescence bias.

Here's an example to make it clearer:

Q1: Do you eat breakfast everyday?

Q2: Do you have your meals regularly?

In this case, both questions intend to find out if the respondents regularly eat their meals, but are keyed in different ways to keep the survey balanced.

 

Demand characteristics

div class="demand navsec five">

This type of bias only occurs because people are part of a study. It’s a bias that arises in experimental scenarios because knowing they are part of a setting where their behavior is being tested makes respondents change their answers.

There are four broad reasons for demand characteristics bias.

  • The first is that simply knowing they're participating in a survey can make people change their behavior accordingly. Some might try to be "good subjects" for the experiment, while others try their best to provide faulty responses to wreck the experiment. People often like to say they exercise more than they actually do and eat healthy food more often than they actually do just because it makes them look like a better person.

  • Another reason for this bias is only true with face-to-face surveys. Here, behavioral aspects of the surveyor, such as how they greet the person, can cause a deviation in the responses.

  • The third reason this bias happens is the prior experiences of the participants in a similar experiment. For example, if Robert has been in such a study before, and he's told he's being interviewed for a similar study, there is a high chance that the knowledge of the same will influence his answers.

  • The fourth reason has do to with culture. When presented with something like a scale— say a rating scale from 1 to 5—people often seem to be biased to select the most extreme options. This type of bias is called extreme response. . Culture plays a big part in dictating whether a person will select neutral or extreme responses.

 

Habituation bias

Sometimes survey respondents tend to provide similar answers to different questions that happen to be worded similarly, simply because the brain gets too lazy (and this is a perfectly normal thing to happen!).

To conserve energy, our brains have an auto-habituation mode that gets switched on once the thinking part becomes numb. It is our brain’s autopilot mode. As a result, the survey responses start getting very repetitive, and responders start giving out similar answers- this is habituation bias.

Here's an example to make it clearer:

The owner of a store decides to conduct a survey for feedback from customers. He ends up loading the survey with more than 50 questions to understand the minutiae of their shopping experience. This will have the opposite effect as respondents get tired and give out random or repetitive answers to the questions.

Solution:

To avoid this, moderators must limit the number of questions and keep the wording conversational and the questions as short as possible to make for an easy read. When possible, tell respondents the reason for your survey. This could help to motivate them to answer and keep them engaged.

 

Respondent's views on the sponsoring organization's beliefs and core mission can sometimes influence how they answer all questions throughout the survey. This is especially true in a case where the inherent beliefs of the respondent are not in line with the people who are sponsoring your survey. If they do not agree with your core beliefs, they may not choose to answer the survey properly.

For example, a Sugar Association once conducted a study via Duke University researchers found numerous ill health effects of sucralose. This is mainly because the sponsoring company had interest to scare people away from artificial sweeteners. However, research by many organisations including WHO all indicate that sucralose is completely safe.

 

Desirability bias

Desirability bias means a respondent might want to answer questions in a way that favors the study's subject. This is a type of response bias that is the tendency of survey respondents to answer questions in a manner that will be viewed favorably by others. It can take the form of over-reporting "good behavior" or under-reporting "bad," or undesirable behavior.

For example, when you survey people whether they smoke or not, they might be hesitant to give you the right answer straightaway. This is because smoking might not be considered a “desirable” characteristic by most people - so even if they do it in reality, they will try to not admit the same as much as possible.

Reducing response bias is essential for a survey because a survey should deliver people’s true feelings—otherwise your results won’t be accurate. Response bias is particularly tricky since it can lead surveyors to reach a wrong conclusion based off responses they incorrectly think are true.

You don’t necessarily need a data analyst or professional surveyor to make sure you avoid these mistakes though.

At Zoho Survey we've already prepared 200+ fact-checked survey templates for every surveying occasion. We’ve ensured they ask the right questions to get the right data.

Happy bias-free surveying!

Leave a Reply

Your email address will not be published. Required fields are marked

By submitting this form, you agree to the processing of personal data according to our Privacy Policy.

You may also like