WHAT ARE QUESTIONNAIRES?

Questionnaires are a set of written questions designed to gather standardized

information about the opinions, preferences, experiences, intentions, and
behavior of individuals, and can be devised for the purposes of a scientific study.
Traditionally, they have been said to contrast with surveys in the sense that they
do not collect mass data for further analysis, however the terms are largely used
interchangeably these days (and many research studies also use them together).

While questionnaires provide a comparatively cheap, prompt, and efficient means

of obtaining large amounts of information, questionnaire design is a multistage
process that requires attention to a number of aspects at the same time to gather
the information you seek. Why exactly is that?

Depending on the kind of information you aim to acquire, questions need to be

asked in varying degrees of detail and in specific ways.

Given the same topic, it’s rather likely that different researchers will come up
with different questionnaires that vary widely in their choice of questions, a line
of questioning, use of open-ended questions, and length.

Question everything – what makes a good questionnaire?

Basically, well-designed questionnaires are highly structured to allow the same

types of information to be collected from a large number of respondents in the
same way and for data to be analyzed quantitatively.
QUESTIONNAIRE DESIGN - GUIDELINES ON HOW TO
DESIGN A GOOD QUESTIONNAIRE

A good questionnaire should not be too lengthy. Simple English should be used
and the question shouldn’t be difficult to answer. A good questionnaire requires
sensible language, editing, assessment, and redrafting.

Questionnaire Design Process

1. State the information required- This will depend upon the nature of the
problem, the purpose of the study and hypothesis framed. The target
audience must be concentrated on.
2. State the kind of interviewing technique- interviewing method can be
telephone, mails, personal interview or electronic interview. Telephonic
interview can be computer assisted. Personal interview can be conducted
at respondent’s place or at mall or shopping place. Mail interview can take
the form of mail panel. Electronic interview takes place either through
electronic mails or through the internet.
3. Decide the matter/content of individual questions- There are two
deciding factors for this-
 Is the question significant? - Observe contribution of each question.
Does the question contribute for the objective of the study?
 Is there a need for several questions or a single question? - Several
questions are asked in the following cases:

 When there is a need for cross-checking

 When the answers are ambiguous
 When people are hesitant to give correct information.
4. Overcome the respondents’ inability and unwillingness to answer- The
respondents may be unable to answer the questions because of following
reasons-
 The respondent may not be fully informed
 The respondent may not remember
 He may be unable to express or articulate

The respondent may be unwilling to answer due to-

 There may be sensitive information which may cause

embarrassment or harm the respondent’s image.
 The respondent may not be familiar with the genuine purpose
 The question may appear to be irrelevant to the respondent
 The respondent will not be willing to reveal traits like aggressiveness
(For instance - if he is asked “Do you hit your wife, sister”, etc.)

To overcome the respondent’s unwillingness to answer:

 Place the sensitive topics at the end of the questionnaire

 Use the third person technique (For example - Mark needed a job
badly and he used wrong means to get it - Is it right?? Different
people will have different opinions depending upon the situation)
 Categorize the responses rather than asking a specific response
figure (For example - Group for income levels 0-25000, 25000-
50000, 50000 and above)
5. Decide on the structure of the question- Questions can be of two types:
. Structured questions- These specify the set of response
alternatives and the response format. These can be classified into
multiple choice questions (having various response categories),
dichotomous questions (having only 2 response categories such as
“Yes” or “No”) and scales (like Likert scale).
a. Unstructured questions- These are also known as open-ended
question. No alternatives are suggested and the respondents are free
to answer these questions in any way they like.
6. Determine the question language/phrasing- If the questions are poorly
worded, then either the respondents will refuse to answer the question or
they may give incorrect answers. Thus, the words of the question should
be carefully chosen. Ordinary and unambiguous words should be used.
Avoid implicit assumptions, generalizations and implicit alternatives.
Avoid biased questions. Define the issue in terms of who the questionnaire
is being addressed to, what information is required, when is the information
required, why the question is being asked, etc.
7. Properly arrange the questions- To determine the order of the question,
take decisions on aspects like opening questions (simple, interesting
questions should be used as opening questions to gain co-operation and
confidence of respondents), type of information (Basic information relates
to the research issue, classification information relates to social and
demographic characteristics, and identification information relates to
personal information such as name, address, contact number of
respondents), difficult questions (complex, embarrassing, dull and
sensitive questions could be difficult), effect on subsequent questions,
logical sequence, etc.
8. Recognize the form and layout of the questionnaire- This is very
essential for self-administered questionnaire. The questions should be
numbered and pre-coded. The layout should be such that it appears to be
neat and orderly, and not clattered.
9. Reproduce the questionnaire- Paper quality should be good.
Questionnaire should appear to be professional. The required space for the
answers to the question should be sufficient. The font type and size should
be appropriate. Vertical response questions should be used, for example:

Do you use brand X of shampoo?

 Yes
 No
10.Pre-test the questionnaire- The questionnaire should be pre-tested on a
small number of respondents to identify the likely problems and to
eliminate them. Each and every dimension of the questionnaire should be
pre-tested. The sample respondents should be similar to the target
respondents of the survey.
11.Finalize the questionnaire- Check the final draft questionnaire. Ask
yourself how much will the information obtained from each question
contribute to the study. Make sure that irrelevant questions are not asked.
Obtain feedback of the respondents on the questionnaire.
SIX STEPS TO GOOD QUESTIONNAIRE DESIGN:

Before you know how to exactly phrase your questions, you need to wrap your
head around a number of other things first in order to develop a reasonable set of
questions suitable to deliver the information you need.

#1: Identify your research aims and the goal of your questionnaire

What kind of information do you want to gather with your questionnaire? What
is your main objective?

Ideally, there are already existing questionnaires with published results on the
validity and statistical evaluation of all tested questions that you can use (or
borrow a couple of ideas from).

While this is quite common in academic research, it might be rather hard to reuse
existing questionnaires for commercial applications. In this case, you might have
to scan papers and internal reports for key metrics of interest and create a
questionnaire that specifically addresses these aspects.

#2: Define your target respondents

Clearly, you can’t test everyone – it’s rather plausible that there have to be certain
restrictions with respect to the target audience of your questionnaire. The
selection of groups is a key factor for maximizing the outcomes of your study.

To put it another way: You can run multiple questionnaire sessions over a longer
period of time with a single group (longitudinal design), or you can present your
questionnaire once to two or more groups (cross-sectional design).

While the former allows you to analyze how the questionnaire results of the group
change over time, the latter delivers insights into differences among groups.
#3: Develop questions

Smart questions are the cornerstone of every questionnaire. To make them work,
they have to be phrased in a way that prevents any misunderstandings or
ambiguities.

Quite frankly, it’s a lost cause trying to analyze data from a questionnaire where
people have mixed things up, selected incorrect answers or haven’t been able to
read or understand the questions at all.

It makes a significant difference whether you want to hand a questionnaire to

children, adults, or maybe even elderly respondents. It’s important to consider the
cognitive, attentional, and sensory competencies of your target group – handing
out long questionnaires with a huge amount of questions in small letter print and
complicated phrasing might be information (and visual) overload for any
respondent group.

Additionally, remember to avoid jargon or technical language – the text needs to

be fully understood by anyone completing the questionnaire.

#4: Choose your question type

There’s a wide variety in how to phrase questions. In explorative questionnaires,

you will find mainly open questions, where respondents can fill in any answer
(this makes sense whenever you try to gain an understanding of the topics
associated with your research question).

By contrast, quantitative questionnaires primarily include closed-questions,

which have been predefined by the researcher either in form of multiple choice
answers or rating scales (such as the Likert scale).
Here’s one example:

Open question:

“What did you like about the webinar?”

Closed question:

“The webinar was useful.”

[ ] Strongly agree

[ ] Agree

[ ] Cannot decide

[ ] Disagree

[ ] Strongly disagree

As is usually the case, both types of questions have benefits and drawbacks that
are worth considering in order to come up with a solid questionnaire design that
does the trick for you.

Besides open and closed-format questions, there are several other types of
questions that you can use in your questionnaire.
#5: Design question sequence and overall layout

After optimizing each question separately, it is time to improve the overall flow
and layout of the questionnaire.

Are there transitions from one question to the next? Are follow-up questions
placed correctly? Are skip-rules implemented (if needed) so that respondents can
skip questions that do not apply to them?

#6: Run a pilot

This stage is crucial for evaluation and optimization purposes. Any questionnaire
should be handed to a representative sample of your target audience before you
push it to the masses.

During piloting, you can identify issues in readability and understanding, in

phrasing and overall arrangement. Pilot respondents should be monitored and
interviewed closely.

You certainly want to avoid any inappropriate or problematic questions. Also,

keep in mind to evaluate your pilot data statistically to make sure that the analytic
procedures of interest truly can be applied to the data.
BASIC PRINCIPLES FOR DESIGNING A GOOD
QUESTIONNAIRE

Researchers are often very disappointed when the questionnaires or surveys they
use have a very low response rate, and this may even affect the validity of their
research.

It is very important, therefore, that questionnaires are well prepared, well-

constructed & validated in order to enhance the response rate as desired.

Below are the key principles which can help a researcher to draft a good
questionnaire:

The questionnaire flow

A researcher should avoid asking sensitive or difficult questions in the beginning

of the questionnaire. The sensitive question should be posed to respondents
towards the end of the survey so that he can get used of the questioning & the
interviewer has built his rapport with the respondents. Another element which is
critical to the flow of the questioning is to put the related questions together. A
good questionnaire should have a flow where related questions are clubbed
together in meaningful sections. One should avoid asking sensitive questions as
much as possible & if necessary or the research objectives demands, then it should
be well informed to respondents that when sensitive questions will be asked &
should inform about confidentiality of the research that the data will not be
analyzed at individual level. The findings are analyzed at overall level by
clubbing all responses of the respondents participated.
Keep it short & simple

Most of the respondents are not available for a long survey. If the questionnaire
is short & simple, it will have great outcome in terms of better productivity, better
response rate as well as good data. A researcher should avoid using long question
which end up confusing the respondents. Many times, we have noticed that
Researchers in order to make things easy go in length to explain the question to
respondents but this end up in respondents not reading the full question and
moving to next. A short survey of about 15-20 minutes can provide more insights
than an hour long discussion as respondents loses interest in the middle of the
questionnaire & provide response or opinion without much thinking.

Be neutral in questioning

Many researchers, though they tried hard, end up writing questions which
implicitly reconfirms the pre-conceived notions/opinions. A Researcher should
not bias or bring emotional element into questioning. A leading question may end
up being unnoticed by respondents but it may bring heavy skew & biases in the
findings.

Example 1:

 Wrong: Many people have watched, India vs. Pakistan T20 cricket World
cup 2016 than any other match ever watched in last one decade. Have you
watched this match?
 Right: Have you watched the match of “India vs. Pakistan T20 cricket
World cup 2016”?
Example 2:

 Wrong: We all know, India has always beaten Pakistan in any world cup
format, what do you think who will win the upcoming India Vs. Pak match?
 Right: Who will win the upcoming match of “India vs. Pakistan T20
cricket World cup 2016”?

Avoid Double barreled questions

A double-barreled question is a single question in which one question covers

more than one topic, but only allows only one answer. Double-barreled questions
create confusion and leads to inaccuracies in the responses because there is no
indication of which topic in the double-barreled question the respondent is
answering. Researchers should ensure that each question should refer to only one
topic instead of multiple topics.

Example:

 Wrong: How satisfy you are with the performance of Indian Batsman and
Bowlers in World T20 Cricket 2016?
 Right:
 How satisfy you are with the performance of Indian Batsmen in
World T20 Cricket 2016?
 How satisfy you are with the performance of Indian Bowlers in
World T20 Cricket 2016?
Don’t assume respondents are experts

Respondents are not experts unless we are talking to Industry experts. The
Questions & Jargons used in the questionnaire should be easy to interpret. The
researchers should ensure that the research motivates people to respond, their
opinion is critical. There is no right or wrong answers. So a Researcher should
include questions that respondents (even bottom of the pyramids) with no
experience can give their opinions.

Avoid negatives or double negatives

Double negatives are two negative words used in the same sentence. Using two
negatives turns the thought or sentence into a positive one. Double negatives are
not encouraged in English because they are poor grammar and they can be
confusing.

Examples of double negatives: Use agreement scale of 1-5, to respond on these

statements?

 I hardly have no money to buy a new car.

 Customer is not rarely a visitor at the service station.
 We don’t watch movies seldom
 We don’t go to watch Cricket in Stadium seldom since our last bad
experience.
Avoid asking questions which is beyond respondent’s capabilities, cognition
& knowledge

Researchers should keep in mind that people carry limitation in terms of cognition
when asked to recall events of the past. Respondents get discouraged when they
have to answer tough questions in surveys.

Example:

 Wrong: “What did you have for breakfast on last weekend?"

 Right: “I have a list of breakfast items, please look at this list and tell me
which one do you eat on a typical weekend?”

Avoid false or unrealistic scenarios

Many times, Researchers pose unrealistic or hypothetical scenarios to

respondents which many would not relate to or puts respondents in a tough spot.

Example:

 Wrong: Many countries across the globe are facing economic slowdown.
Do you think our country is going through a similar economic slowdown
at the moment? As many respondents may not agree that the recession is
affecting the countries across globe hence could not give the right
response.
 Right: Do you think that our country is going through recession at the
moment?
Avoid using multiple type of scale – Be consistent

The scales should be kept consistent as many respondents get confused if they are
exposed to different kind of scales. If the respondents are shown, for example, a
5-point scale for a series of statements & then in follow up questions, they are
exposed to 3-point scale or 7-point scale, it creates confusion in the mind of
respondents.

On the other side, it is advisable to use a 7-point scale which had been well
established by many statisticians. Respondents simply love more choices.
Statistical reliability of the data increases with the number of scale steps up to
about 7 points, and then it increases more slowly, leveling off around 11 point. If
you have a single item questionnaire, then using 7 or more points is more
beneficial but in case of a multiple item questionnaire, it doesn’t matter how many
points scales are being used.

However, if an organization is using 5-point scale for any tracking study for a
good amount of time, it is not advisable to change the scale as it may have an
impact on the bench-marking.

In short, a researcher should spend good time in preparing a questionnaire by

keeping the key target groups in mind & the survey should be easy to interpret &
respond. A short survey is any day preferred over a lengthy time consuming
survey.
Few examples of different types of questions are given below:

1.
Closed-ended questions
Did you experience cough and cold in the past 6 months? 1. Yes
2. No
Think of the last time you had cough and cold. Did you 1. Yes
consult a doctor? 2. No
Open-ended questions
What is/are the main reason(s) for you to consult a doctor for
your cough and cold? (You can list down more than one
reason)

2. Filtering
Think of the last time you had cough and cold. Did you 1. Yes
consult a doctor? 2. No
If No, please go to Question 14.

3. Sensitive questions
It is common for people to ask for antibiotics when they have 1. Yes
cough and cold. Did you request for an antibiotic from the 2. No
doctor for your cough and cold?

4. Likert scale
We should always consult a doctor for cough and cold. 1. Strongly disagree
(Please circle the answer) 2. Disagree
3. Neutral
4. Agree
5. Strongly agree
5. “Double-barreled” questions
Did you experience sore throat and fever during your cough 1. Yes
and cold? 2. No
Should be:
Did you experience sore throat during your cough and cold? 1. Yes
2. No
Did you experience fever during your cough and cold? 1. Yes
2. No
Or:
Did you experience the following symptoms during your 1. Sore throat
cough and cold? (You can tick more than one box) 2. Cough
3. fever

6. Ambiguous Questions
How often do you visit the doctor for cough and cold? 1. Not at all
2. Not very often
3. Sometimes
4. Quite often
5. Very often
Change to:
How many times did you visit the doctor for cough and cold _________ times in the
in the past 1 year? past 1 year