Research Methodology

Dr. Gangu Naidu Mandala

MA,M.Com, MBA, PhD
Department of Professional Studies
CHRIST (Deemed to be University)
Central Campus, Bengaluru
 Unit – 1
Introduction to Research

10 Hours
Meaning,
Objectives,
Types of research,
significance of research,
research process,
criteria of good research,
Review of Literature,
Identification, selection and Defining research problem.

Senthil Kumar A.
 Research - Meaning

❖Research is - an endeavour - to discover answers - to intellectual and practical

problems - through the application of scientific method.

❖“Research is a systematized effort to gain new knowledge”.

-Redman and Mory.

❖Research is the -systematic process of - collecting and analyzing information

(data) - in order to increase our understanding of the phenomenon - about
which we are concerned or interested.
 Objectives of Research
❖The purpose of research is to discover answers through the application of
scientific procedures.

❖The objectives are:

To gain familiarity with a phenomenon or to achieve new insights into it –

Exploratory or Formulative Research.
To portray accurately the characteristics of a particular individual, situation or
a group – Descriptive Research.
To determine the frequency with which something occurs or with which it is
associated with something else – Diagnostic Research.
To test a hypothesis of a causal relationship between variables –
Hypothesis-Testing Research.
 Characteristics of Research

❖Research is directed towards the solution of a problem.

❖Research is based upon observable experience or empirical evidence.
❖Research demands accurate observation and description.
❖Research involves gathering new data from primary sources or using existing
data for a new purpose.
❖Research activities are characterized by carefully designed procedures.
❖Research requires expertise i.e., skill necessary to carryout investigation, search
the related literature and to understand and analyze the data gathered.
❖Research is objective and logical – applying every possible test to validate the
data collected and conclusions reached.
❖Research involves the quest for answers to unsolved problems.
❖Research requires courage.
❖Research is characterized by patient and unhurried activity.
❖Research is carefully recorded and reported.
 SCIENTIFIC METHOD
‘Science’ refers to the body of systematic and organised
knowledge which makes use of scientific method to acquire
knowledge in a particular field of enquiry.

Scientific method is the systematic collection of data (facts) and

their theoretical treatment through proper observation,
experimentation and interpretation.

Scientific method attempts to achieve a systematic interrelation of

facts by experimentation, observation, and logical arguments from
accepted postulates and a combination of these three in varying
proportions.
 BASIC POSTULATES
IN SCIENTIFIC METHOD

• It relies on empirical evidence.

• It utilizes relevant concepts.
• It is committed to only objective considerations.
• It presupposes ethical neutrality.
• It results into probabilistic predictions.
• The methodology is made known.
• Aims at formulating scientific theories.
 CRITERIA OF A GOOD RESEARCH

• Purpose clearly defined.

• Research process detailed.
• Research design thoroughly planned.
• High ethical standards applied.
• Limitations frankly revealed.
• Adequate analysis for decision maker’s needs.
• Findings presented unambiguously.
• Conclusions justified.
• Researcher’s experience reflected.
 QUALITIES OF A GOOD RESEARCH

• Systematic
• Logical
• Empirical
• Replicable
• Creative
• Use of multiple methods
 NEED FOR RESEARCH

• Exploration
• Describe
• Diagnose
• Hypothesis
• Inductions and deductions
SCOPE / SIGNIFICANCE OF RESEARCH
• RESEARCH FOR DECISION MAKING
• Throws light on risks and uncertainty
• Identify alternative courses of action
• Helps in economic use of resources
• Helps in project identification
• Solves investment problems
• Solves pricing problems
• Solves allocation problems
• Solves decision making issues in HR
• Solves various operational and planning problems of
business and industry
• Provides the basis for all government policies in our
economic system.

• Helps social scientists in studying social relationships

and in seeking answers to various social problems.

• For students, research means a careerism or a way to

attain a high position in the social structure.

• For professionals in research, it may mean a source

of livelihood.
• For philosophers and thinkers, research means the
outlet for new ideas and insights.

• For literary men and women, research means

development of new styles and creative work.

• For analysts and intellectuals, research means

generalizations of new theories.
 PROBLEMS IN RESEARCH
• Not similar to science
• Uncontrollable variables
• Human tendencies
• Time and money
• Lack of computerization
• Lack of scientific training in the methodology of
research
• Insufficient interaction between university research
departments and business establishments
 PROBLEMS IN RESEARCH
• Lack of confidence on the part of business units to
give information
• Lack of code of conduct
• Difficulty of adequate and timely secretarial assistance
• Poor library management and functioning
• Difficulty of timely availability of published data.
• Ignorance
• Research for the sake of research-limited practical
utility though they may use high sounding business
jargon.
ROLE OF RESEARCH IN DECISION-MAKING

• Decision-making is the process of selecting the best

alternative from the available set of alternatives.

• Management is chiefly concerned with decision-making

and its implementation.

• These decisions should be based on appropriate studies,

evaluations and observations.

• Research provides us with knowledge and skills needed to

solve the problems and to meet the challenges of a fast
paced decision-making environment.
According to Herbert A Simon, decision-making
involves three activities:

• Intelligence Activity - scanning the environment

for identifying conditions necessary for the decision.

• Designing Activity - identifying, developing and

analyzing the alternative courses of action.

• Choice Activity - choosing the best course of

action from among the alternatives.
 FACTORS THAT AFFECT MANAGERIAL
DECISIONS
• INTERNAL FACTORS – factors present inside an organisation such
as resources, technology, trade unions, cash flow, manpower etc.

• EXTERNAL FACTORS – factors present outside the organisation

such as government policies, political factors, socio-economic
factors, legal framework, geographic and cultural factors etc.

• QUANTITATIVE FACTORS – factors that can be measured in

quantities such as time, resources, cost factors etc.

• QUALITATIVE FACTORS – factors that cannot be measured in

quantities such as organizational cohesiveness, sense of belonging of
employees, risk of technological change etc.

• UNCERTAINITY FACTORS – factors which cannot be predicted.

 TYPES OF RESEARCH
Descriptive vs Analytical Research

• Descriptive Research is a fact finding investigation which

is aimed at describing the characteristics of individual,
situation or a group (or) describing the state of affairs as it
exists at present.

• Analytical Research is primarily concerned with testing

hypothesis and specifying and interpreting relationships,
by analyzing the facts or information already available.
Applied vs Fundamental Research

• Applied Research or Action Research is carried out to find

solution to a real life problem requiring an action or policy
decision.

• Fundamental Research which is also known as basic or

pure research is undertaken for the sake of knowledge
without any intention to apply it in practice.
• It is undertaken out of intellectual curiosity and is not
necessarily problem-oriented.
Quantitative vs Qualitative Research
• Quantitative Research is employed for measuring the
quantity or amount of a particular phenomena by the use
of statistical analysis.

• Qualitative Research is a non-quantitative type of analysis

which is aimed at finding out the quality of a particular
phenomenon.
Conceptual vs Empirical Research
• Conceptual Research is generally used by philosophers
and thinkers to develop new concepts or to reinterpret
existing ones.

• Empirical Research is a data based research which

depends on experience or observation alone. It is aimed at
coming up with conclusions without due regard for system
and theory.
Some other types of research..
• One-time Research – Research confined to a single time
period.
• Longitudinal Research – Research carried on over several
time periods.
• Diagnostic Research – It is also called clinical research
which aims at identifying the causes of a problem,
frequency with which it occurs and the possible solutions
for it.
• Experimental Research – It is designed to assess the effect
of one particular variable on a phenomenon by keeping the
other variables constant or controlled.
 Some other types of research..

• Exploratory Research – It is the preliminary study of an

unfamiliar problem, about which the researcher has little
or no knowledge. It is aimed to gain familiarity with the
problem, to generate new ideas or to make a precise
formulation of the problem. Hence it is also known as
formulative research.

• Historical Research – It is the study of past records and

other information sources, with a view to find the origin
and development of a phenomenon and to discover the
trends in the past, inorder to understand the present and to
anticipate the future.
Research and Sampling Design
 Unit -2

Research and Sampling Design

Hours: 12
Research Design
• Meaning,

• Need for research design,

• features of a good design,
• Different Research Designs.

Sampling Design:
• Census and sample survey,
• steps in sampling design,

• criteria of selecting a sampling procedure,

• characteristics of a good sample design,
• different types of sampling designs- Probability and Non Probability Sampling Design,

• Sampling v/s Non-Sampling Error;

• Determination of Sample Size.
 Meaning of Research Design
The formidable problem that follows the task of defining the
research problem is the preparation of the design of the research
project, popularly known as the “research design”. Decisions
regarding what, where, when, how much, by what means concerning
an inquiry or a research study constitute a research design.
“A research design is the arrangement of conditions for collection and
analysis of data in a manner that aims to combine relevance to the
research purpose with economy in procedure.”
 As such the design includes an outline of what the researcher will
do from writing the hypothesis and its operational implications to the
final analysis of data. More explicitly, the design decisions happen to
be in respect of:
(i) What is the study about?
(ii) (Why is the study being made?
(iii) Where will the study be carried out?
(iv) What type of data is required?
(v) Where can the required data be found?
(vi) What periods of time will the study include?
(vii) What will be the sample design?
(viii) What techniques of data collection will be used?
(ix) How will the data be analysed?
(x) In what style will the report be prepared?
 Keeping in view the above stated design decisions, one may split the overall research
design into the following parts:
(a) the sampling design which deals with the method of selecting items to be
observed for the given study

(b) the observational design which relates to the conditions under which the
observations are to be made;
(c) the statistical design which concerns with the question of how many items are to
be observed and how the information and data gathered are to be analysed; and
(d) the operational design which deals with the techniques by which the procedures
specified in the sampling, statistical and observational designs can be carried out.
 Need for Research Design
Research design is needed because it facilitates the smooth sailing of the various
research operations, thereby making research as efficient as possible yielding maximal
information with minimal expenditure of effort, time and money. Research design has a
significant impact on the reliability of the results obtained. It thus acts as a firm
foundation for the entire research.
For example, economical and attractive construction of house we need a blueprint
(or what is commonly called the map of the house) well thought out and prepared by an
expert architect, similarly we need a research design or a plan in advance of data
collection and analysis for our research project.
The need for research design is as follows:
• It reduces inaccuracy;
• Helps to get maximum efficiency and reliability;
• Eliminates bias and marginal errors;
• Minimizes wastage of time;
• Helpful for collecting research materials;
• Helpful for testing of hypothesis;
• Gives an idea regarding the type of resources
required in terms of money, manpower, time, and
efforts;
• Provides an overview to other experts;
• Guides the research in the right direction.
 Features of A Good Design
A good design is often characterised by adjectives like flexible, appropriate, efficient, economical
and so on.
Generally, the design which minimises bias and maximises the reliability of the data collected and
analysed is considered a good design.

The design which gives the smallest experimental error is supposed to be the best design in many
investigations. Similarly, a design which yields maximal information and provides an opportunity
for considering many different aspects of a problem is considered most appropriate and efficient
design in respect of many research problems.
Thus, the question of good design is related to the purpose or objective of the research problem
and also with the nature of the problem to be studied. A design may be quite suitable in one case,
but may be found wanting in one respect or the other in the context of some other research
problem. One single design cannot serve the purpose of all types of research problems
 A research design appropriate for a particular research problem,
usually involves the consideration of the following factors:
(i) The means of obtaining information;

(ii) The availability and skills of the researcher and his staff, if any;
(iii) The objective of the problem to be studied;
(iv) The nature of the problem to be studied; and

(v) The availability of time and money for the research work
 Important concepts relating to research design

1. Dependent and Independent Variables

2. Extraneous Variable
3. Control
4. Confounded Relationship
5. Research Hypothesis
6. Experimental and Non-experimental Hypothesis-
Testing Research
7. Experimental and Control Groups
8. Treatments
9. Experiment
10. Experimental Unit(s)
Dependent and Independent Variables
Variable: A concept which can take on different quantitative values. For
example, concepts like weight, height, income etc

Continuous variables – phenomena which can take on quantitatively different

values even in decimal points. For example, age
Discontinuous or Discrete variables – If some variables can be only expressed
in integer values. For example, no. of children.
If one variable depends upon or is a consequence of the other variable, it
is termed as dependent variable, and the variable that is antecedent to the
dependent variable is termed as independent variable. For example, Age →
Height, Smoking → Cancer (Height and Cancer are dependent variables
whereas Age and Smoking are independent variables.
Extraneous variable
Independent variables that are not related to the purpose of the
study, but may affect the dependent variable are termed as
extraneous variables.

Whatever effect is noticed on dependent variable as a result of

extraneous variable(s) is technically described as ‘experimental error’.
A study must always be so designed that the effect upon the
dependent variable is attributed entirely to the independent
variable(s), and not to some extraneous variable or variables
Control
One important characteristic of a good research design is to minimize the
influence or effect of extraneous variable(s).

The technical term ‘control’ is used when we design the study minimizing
the effects of extraneous independent variable(s).
In experimental researches, the term ‘control’ is used to refer to restrain
experimental conditions.
Confounded relationship

When the dependent variable is not free from the influence of

extraneous variable(s), the relationship between the dependent and
independent variables is said to be confounded by an extraneous variable(s).
Research hypothesis
1. When a prediction or a hypothesis relationship is to be tested by
scientific methods, it is termed as research hypothesis.

2. The research hypothesis is a predictive statement that relates an

independent variable to a dependent variable.
3. Usually, a research hypothesis must contain at least, one
independent and one dependent variable.
For example, “e-Learning enhances teaching learning experience”.
Here, the dependent variable is “teaching learning experience”,
whereas “e-Learning” is the independent variable.
Experimental and non-experimental hypothesis- testing research
When the purpose of the research is to test a research
hypothesis, it is termed as hypothesis-testing research.
It can be both of experimental design type or non- experimental
design type.
Research in which the independent variable(s) are manipulated is
termed as ‘experimental hypothesis- testing research’ and a research
in which they are not manipulated is called ‘non-experimental
hypothesis- testing research’
Experimental and control groups
In an experimental hypothesis-testing research when a
group is exposed to usual conditions, it is termed as ‘control
group’, but when a group is exposed to some novel or special
condition, it is termed as ‘experimental group’.
Treatments
The different conditions under which experimental and
control groups are put are usually referred to as ‘treatments’.
Experiment
The process of examining the truth of a statistical hypothesis,
relating to some research problem is known as an experiment.

For example, we can conduct an experiment to examine the

usefulness of a newly developed drug. Experiments can be of two
types viz., absolute experiment and comparative experiment.
Absolute experiment – Determining the impact of a fertilizer on the
yield of a crop.
Comparative experiment – Determining the impact of one fertilizer
as compared to the impact of some other fertilizer.
Experimental unit(s)
The pre-determined plots or blocks, where
different treatments are used, are known as
experimental units.
Such units must be selected (defined) very
carefully
 Research Design
• The research design is the master plan specifying the methods and
procedures for collecting and analyzing the needed information.
• “A research design is the arrangement of conditions for collection and
analysis of data in a manner that aims to combine relevance to the
research purpose with economy in procedure”
• Parts of research design:
• Sampling Design
• Observational Design
• Statistical Design
• Operational Design
• Features of a Good design - factors
• The means of obtaining information
• The availability and skills of the researcher and his staff, if any;
• The objective & Nature of the problem to be studied
• The availability of time and money of the research work.
 Types of Research Design
• Three traditional categories of research design:
• Exploratory
• Descriptive
• Causal
• The choice of the most appropriate design depends largely on the
objectives of the research and how much is known about the problem and
these objectives.
• Basic Research Objectives and Research Design

Research Objective Appropriate Design

•To gain background information, to define terms, to clarify Exploratory
problems and develop hypotheses, to establish research
priorities, to develop questions to be answered

•To describe and measure marketing phenomena at a point Descriptive

in time

•To determine causality, test hypotheses, to make “if-then” Causal

statements, to answer questions
 Research Design: Exploratory Research
• Exploratory research is most commonly unstructured, “informal”
research that is undertaken to gain background information about the
general nature of the research problem.

• Exploratory research is usually conducted when the researcher does not

know much about the problem and needs additional information or
desires new or more recent information.

• Exploratory research is used in a number of situations:

• To gain background information
• To define terms
• To clarify problems and hypotheses
• To establish research priorities

• A variety of methods are available to conduct exploratory research:

• Secondary Data Analysis
• Experience Surveys
• Case Analysis
• Focus Groups
• Projective Techniques
 Research Design: Descriptive Research
• Descriptive research is undertaken to provide answers to questions
of who, what, where, when, and how – but not why.

• Two basic classifications:

• Cross-sectional studies
• Cross-sectional studies measure units from a sample of the
population at only one point in time.
• Sample surveys are cross-sectional studies whose samples are
drawn in such a way as to be representative of a specific population.
• On-line survey research is being used to collect data for
cross-sectional surveys at a faster rate of speed.

• Longitudinal studies
• Longitudinal studies repeatedly draw sample units of a population
over time.
• One method is to draw different units from the same sampling frame.
• A second method is to use a “panel” where the same people are
asked to respond periodically.
• On-line survey research firms recruit panel members to respond to
online queries.
 Research Design: Causal Research
• Causality may be thought of as understanding a phenomenon in
terms of conditional statements of the form “If x, then y.”
• Causal relationships are typically determined by the use of
experiments, but other methods are also used.
Research Design Exploratory Descriptive/Diagnostic
Overall Design Flexible Design Rigid Design
Sampling design Non-probability Probability (random Sampling)
(Purposive/ Judgemental)
Statistical design No pre-planned design for Pre-planned analysis
analysis
Observational design Unstructured instruments Structured or well through out
for collection of data instruments for collection of
data
Operational design No fixed decisions about Advanced decisions about
the operational operational procedures
procedures
 Experiments
• An experiment is defined as manipulating (changing values/situations)
one or more independent variables to see how the dependent
variable(s) is/are affected, while also controlling the affects of additional
extraneous variables.
• Independent variables: those over which the researcher has control
and wishes to manipulate i.e. package size, ad copy, price.
• Dependent variables: those over which the researcher has little to no
direct control, but has a strong interest in testing i.e. sales, profit,
market share.
• Extraneous variables: those that may effect a dependent variable
but are not independent variables.

• Types of Experiments
• Two broad classes:
• Laboratory experiments: those in which the independent variable is
manipulated and measures of the dependent variable are taken in a
contrived, artificial setting for the purpose of controlling the many
possible extraneous variables that may affect the dependent
variable
• Field experiments: those in which the independent variables are
manipulated and measurements of the dependent variable are made
on test units in their natural setting
Basic Principles of Experimental Designs- R.A.Fisher’s
• Principle of Replication
• The experiment should be repeated more than once. Thus, each
treatment is applied in many experimental units instead of one. By doing
so the statistical accuracy of the experiments is increased. The entire
experiment can even be repeated several times for better results.
Conceptually replication does not present any difficulty, but
computationally it does.
• Principle of Randomization
• It provides protection, when we conduct an experiment, against the effect
of extraneous factors by randomization. In other words, this principle
indicates that we should design or plan the experiment in such a way that
the variations caused by extraneous factors can all be combined under
the general heading of “chance”.
• Principle of locus Control
• Under it, the extraneous factor, the known source of variability, is made to
vary deliberately over as wide a range as necessary and this needs to be
done in such a way that the variability it causes can be measured and
hence eliminated from the experimental error.
 Important Experimental Design
• Informal Experimental design
• Before-and-after without control design
• In such a design a single test group or area is selected and the dependent
variable is measured before the introduction of the treatment. The treatment is
then introduced and the dependent variable is measured again after the
treatment has been introduced. The effect of the treatment would be equal to
the level of the phenomenon after the treatment minus the level of the
phenomenon before the treatment.
• After-only with control design
• In this design two groups or area are selected and the treatment is introduced
into the test area only. The dependent variable is then measured in both the
areas at the same time. Treatment impact is assessed by subtracting the value
of the dependent variable in the control area from its value in the test area.
• Before-and-after with control design
• In this design two areas are selected and the DV is measured in both the areas
for an identical time period before the treatment. Then the treatment is
introduced into the test area only, and the dependent variable is measured in
both for an identical time-period after the introduction of the treatment. The
treatment effect is determined by subtracting the change in the DV in the control
area from the change in the DV in test area.
 Formal Experimental Design
• Completely randomized design (C-R)
• Involves only two principles viz., principle of replication and the principle
of randomization of experimental designs. It is the simplest possible
design and its procedures of analysis is also earlier.
• The essential characteristic of the design is that subjects are randomly
assigned to experimental treatments. E.g. A  5; B 5 randomly.
• One way analysis of variance (One way ANOVA)
• Two group simple randomized design

Treatment A
Experimental

Independent
group
Population

Variable
Sample

Random Random
Selected assigned

Treatment B
Control
group
Random Replications Design
Group 1
Population Population

Random Random
Selection Selection

Group1 E
Sample (to Group 1 E Sample ( to conduct
be studied) Group 1 E
treatments)
Group 1 E

Group 1 C
Random Group 1
Group 1
C
C Random
Assignment Group 1 C
Assignment

Treatment A Treatment B

E – Experimental Group
C – Control Group

Independent or Casual Variable

 Randomized block design (R.B. Design)
• It is an improvement over the C-R Design
• The principle of locus control can be applied along with other two
principles.
• Here subjects are divided into groups, known as blocks, such that within
each group the subjects are homogeneous in respect to some selected
variable.
• The number of subjects in a given block would be randomly assigned to
each treatment.
• Two-way ANOVA is used to analysis this R-B design.

Very Low I.Q Low I.Q. Avge I.Q High I.Q Very High I.Q.
Student A Student B Student C Student D Student E

Form 1 82 67 57 71 73
Form 2 90 68 54 70 81
Form 3 86 73 51 69 84
Form 4 93 77 60 65 71
 Latin Square design (L.S. Design)
• It is very frequently used in agricultural research.
• The condition under which agricultural investigations are carried out are
different from those in other studies for nature plays an important role in
agriculture.
• Two way ANOVA is used.
• The treatment in a L-S design are so allocated among the plots that no
treatment occurs more than once in any one column or row.

X1 A B C D E
X2 B C D E A
X3 C D E A B
X4 D E A B C
X5 E A B C D

• If treatments are 10 or more, than each column will be larger in size so

that rows and columns may not be homogeneous.
• This may make the application of the principle of locus control ineffective.
 Factorial Designs
• These are used in experiments where the effects of varying more than one factor are to be
determined.
• They are specially important in several economic and social phenomena where usually a
large number of factors affect a particular problem.
• Simple Factorial designs/Two-factor factorial designs (2 x 2 )
• Consider the effects of varying two factors on the dependent variable.

Experimental Variable
Cell 1 Cell 3
Control variable Treatment A Treatment B
Cell 2 Cell 4
Level 1
Level 2

Complex Factorial designs/ Multi factorial designs

Experimental Variable

Treatment A Treatment B

Control Control Control Control

Variable 2 Variable 2 Variable 2 Variable 2
Level I Level II Level I Level II

Control Level 1 Cell 1 Cell 3 Cell 5 Cell 7

variable 1
Level II Cell 2 Cell 4 Cell 6 Cell 8
 Experimental Design
• An experimental design is a procedure for devising an experimental
setting such that a change in the dependent variable may be solely
attributed to a change in an independent variable.

• Symbols of an experimental design:

• O = measurement of a dependent variable
• X = manipulation, or change, of an independent variable
• R = random assignment of subjects to experimental and control
groups
• E = experimental effect

• Types
• After-Only Design: X O1
• One-Group, Before-After Design: O1 X O2
• Before-After with Control Group:
• Experimental group: O1 X O2
• Control group: O 3 O4
• Where E = (O2 – O1) – (O4 – O3)
 How Valid Are Experiments?
• An experiment is valid if:
• the observed change in the dependent variable is, in fact, due to
the independent variable (internal validity)
• if the results of the experiment apply to the “real world” outside
the experimental setting (external validity)

• Factors affecting Research Design

• Non-availability of sufficient data

• Availability of time
• Availability of resources
• Availability of researcher
 Test Marketing
• Test marketing is the phrase commonly used to indicate an experiment, study, or test that is
conducted in a field setting.
• Two broad classes:
• To test the sales potential for a new product or service
• To test variations in the marketing mix for a product or service
• Types of Test Markets
• Standard test market: one in which the firm tests the product and/or marketing mix variables
through the companies normal distribution channels
• Controlled test markets: ones that are conducted by outside research firms that guarantee
distribution of the product through pre-specified types and numbers of distributors
• Electronic test markets: those in which a panel of consumers has agreed to carry identification
cards that each consumer presents when buying goods and services
• Simulated test markets: those in which a limited amount of data on consumer response to a new
product is fed into a model containing certain assumptions regarding planned marketing programs,
which generates likely sales volume
• Test marketing is used in both consumer markets and industrial or B2B markets as well.
• Lead country test market: test marketing conducted in specific foreign countries that seem good
predictors for an entire continent
• Pros:
• Allows most accurate method of forecasting future sales
• Allows firms the opportunity to pretest marketing mix variables
• Cons:
• Does not yield infallible results
• Are expensive
• Exposes the new product or service to competitors
• Takes time to conduct
Sampling Design:
• Census and sample survey,
• steps in sampling design,
• criteria of selecting a sampling procedure,
• characteristics of a good sample design,
• different types of sampling designs- Probability
and Non Probability Sampling Design,
• Sampling v/s Non-Sampling Error;
• Determination of Sample Size.
 Sample Design
A sample design is a definite plan for
obtaining a sample from a given population. It
refers to the technique or the procedure the
researcher would adopt in selecting items for
the sample.
 Census and sample survey
Census Method
A census method is that process of the statistical
list where all members of a population are analysed.
The population relates to the set of all observations
under concern. For instance, if you want to carry out
a study to find out student’s feedback about the
amenities of your school, then all the students of
your school would form a component of the
‘population’ for your study.
• A statistical investigation in which the data are collected for each and
every element/unit of the population is termed as census method.
• It is also known as ‘complete enumeration’ or ‘100% enumeration’ or
‘complete survey’.

• It is useful when case intensive study is required or the area is limited.

Examples:
1. Demographic data on birth and death rates, literacy, workforce, life
expectancy, size and composition of a population

2. The census of India is conducted every 10 years.

advantages of the census method.
Intensive study

• It provides intensive and in-depth information covering many facets of the

problems.
Example: In a population census, not only the number of persons is counted,
but the information is also collected on various other parameters like the
number of males and females, age, education, marital status, occupational
level, income health conditions, etc.
Results are more accurate and reliable

• Since, in this type of investigation, every item of the universe is taken into
account, the conclusions are more accurate and reliable.
Sample Method
Sample method chooses the different sample entities from the
targeted population. This method involves a statistical analysis of an
already determined number of observations that is derived from a
larger set of population.
Sample methodology can be used of different kinds; these can be
– simple random sampling or systematic sampling, cluster sampling
or stratified sampling, etc. among others.
Sampling method
• The sampling method is the one in which only some of
the representative items of the population are
selected and the data are collected from these.
• Instead of collecting information for and from all the
units of population, we select a sample, i.e., only a few
items of the population.
• Conclusions derived from the small sample are
generalised for the whole population.
 advantages of the sampling method
(1) Economical, i.e., low cost
(2) Quick results
(3) Indispensable or greater scope
(4) Scientific approach
(5) Helpful in checking census results
The Sampling Design Process

Define the Population

Determine the Sampling Frame

Select Sampling Technique(s)

Determine the Sample Size

Execute the Sampling Process

 Steps in Developing a Sampling Plan

• Step 1: Defining the Population:

– Bases for defining the population of interest include:
• Geography
• Demographics
• Use
• Awareness
• Step 2: Choosing a Sampling Frame
– Sampling frame
• List of population elements from which to select units to be
sampled.
 Steps in Developing a Sampling Plan (cont.)

• Step 3: Selecting the Sampling Technique(s):

– Probability samples:
• Samples in which every element of the population has a
known, nonzero probability of selection.
– Non-probability samples:
• Include the selection of specific elements from the population
in a nonrandom manner.
– Sampling error:
• The difference between the sample value and the true value
of the population mean.
 Steps in Developing a Sampling Plan (cont.)

Advantages of probability Disadvantages of probability

samples samples

- The researcher can be sure of - They are more expensive than

obtaining information from a non-probability samples of the
representative cross section of sample size in most cases. The
the rules for selection increase
population of interest. interviewing costs and professional
time must be spent in developing
- Sampling error can be computed. the sample design.
- The survey results are projectable
- Probability samples take more time
to the total population. to design and execute than non-
probability samples.
 Steps in Developing a Sampling Plan (cont.)

Advantages of non-probability Disadvantages of

samples non-probability samples

- Non-probability samples cost less - Sampling error cannot be

than probability samples. This computed.
characteristic of non-probability
samples may have considerable - The researcher does not know the
appeal in those situations where degree to which the sample is
representative of the population
accuracy is not of critical
importance. from which it was drawn.

- The results of non-probability

-Non-probability samples
ordinarily can be conducted more samples cannot and should not be
quickly than probability samples. projected to the total population.
 Steps in Developing a Sampling Plan (cont.)

• Step 4: Determine the Sample Size:

– Once the sampling method has been chosen, the
next step is to determine the appropriate sample
size.
– Developing Operational Procedures:
• Involves determining whether a probability or
non-probability sample is being used.
 Steps in Developing a Sampling Plan (cont.)

• Step 5: Execute the Sampling Process:

– The final step in the sampling process involves
execution of the operational sampling plan
discussed in the previous steps.
– It is important that this step include adequate
checking to make sure that specified procedures
are adhered to.
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE
Two costs are involved in a sampling analysis viz.,
the cost of collecting the data and the cost of an
incorrect inference resulting from the data.
Researcher must keep in view the two causes of
incorrect inferences viz., systematic bias and
sampling error.
A systematic bias results from errors in the
sampling procedures, and it cannot be reduced or
eliminated by increasing the sample size. 14

At best, the causes responsible for these errors can

be detected and corrected.
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE
Usually, a systematic bias is the result of one or
more of the following factors:
 Inappropriate sampling frame
If the sampling frame is inappropriate, i.e., a biased
representation of the universe, it will result in a systematic
bias.
 Defective measuring device
If the measuring device is constantly making errors, it
will result in systematic bias.
In survey work, systematic bias can result if the questionnaire 15
or the interviewer is biased.
Similarly, if the physical measuring device is defective there
will be systematic bias in the data collected through such a
measuring device.
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE

Usually, a systematic bias is the result of one or

• more of the following factors:
 Non-respondents
If we are unable to sample all the individuals initially included
in the sample, there may arise a systematic bias.
The reason is that such a situation the likelihood of establishing
contact or receiving a response from an individual is often
correlated with the measure of what is to be estimated.
 Indeterminacy principle
Individuals act differently when kept under observations than
what they do when kept in non-observed situations. This
indeterminacy principle may also be a cause of a systematic
bias.
• 16
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE

Usually, a systematic bias is the result of one or

more of the following factors:
 Natural bias in the reporting of data
Natural bias of respondents in the reporting of data is often the
cause of a systematic bias in many queries.
There is usually a downward bias in the income data collected
by the government taxation department, whereas we find an
upward bias in the income data collected by some social
organization.
People in general understate their incomes if asked about it for
tax purposes, but they overstate the same if asked for social 17
status or their affluence.
Generally in psychological surveys, people tend to give what
they think is the ‘correct’ answer rather than revealing their true
feelings.
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE
Sampling errors are the random variations in the
sample estimates around the true population
parameters.
Since they occur randomly and are equally likely to
be in either direction, their nature happens to be of
compensatory type and the expected value of such
errors happens to be equal to zero.
Sampling error decreases with the increase in the
size of the sample, and it happens to be of a
smaller magnitude in case of homogeneous 18

population.
Sampling error can be measured for a given
sample design and size.
 CRITERIA OF SELECTING A SAMPLING
PROCEDURE
The measurement of a sampling error is usually called the
‘precision of the sampling plan’.
If we increase the sample size, the precision can be improved.
But increasing the size of the sample has its own limitations
viz., a large sized sample increases the cost of collecting data
and also enhances the systematic bias.
Thus the effective way to increase precision is usually to
select a better sampling design which has a smaller sampling
error for a given sample size at a given cost.
In practice, however, people prefer a less precise design
because it is easier to adopt the same and also because of
the fact that systematic bias can be controlled in a better way 19
in such a design.
In brief, while selecting a sampling procedure, researcher
must ensure that the procedure causes a relatively small
sampling error and helps to control the systematic bias in
a better way.
 CHARACTERISTICS OF A GOOD SAMPLE
DESIGN
Sample design must result in a truly representative
sample.
Sample design must be such which results in a
small sampling error.
Sample design must be viable in the context of
funds available for the research study.
Sample design must be such so that systematic
bias can be controlled in a better way.
20
Sample design should be such that the results of
the sample study can be applied in general, for the
universe with a reasonable level of confidence.
 Characteristics of a Good Sample Design

The following are the characteristic features of a good sample design:

1. The sample design should yield a truly representative sample;

2. The sample design should be such that it results in small sampling error;
3. The sample design should be viable in the context of budgetary constraints of the
research study;
4. The sample design should be such that the systematic bias can be controlled; and 5.
The sample must be such that the results of the sample study would be applicable, in
general, to the universe at a reasonable level of confidence.
 Classification of Sampling Techniques

Sampling Techniques

Non-probability Probability
Sampling Techniques Sampling Techniques

Convenience Judgment Quota Snowball

Sampling Samples Sampling Sampling

Simple random Systematic Stratified Cluster

Sampling Sampling Sampling Sampling
 Probability Sampling Methods

• Simple Random Sampling

– Is considered to be the purest form of probability
sampling. A probability sample is a sample in
which every element of the population has a
known and equal probability of being selected
into the sample.

Sample Size
Probability of Selection =
Population Size
 Procedures for Drawing
Probability Samples
Simple Random
Sampling

1. Select a suitable sampling frame

2. Each element is assigned a number from 1 to N (pop. size)
3. Generate n (sample size) different random numbers between 1
and N
4. The numbers generated denote the elements that should be
included in the sample
 Probability Sampling Methods (cont.)

• Systematic Sampling
– Probability sampling in which the entire
population is numbered, and elements are
drawn using a skip interval.
Population Size
Skip Interval =
Sample Size
 Systematic Sampling

1. Select a suitable sampling frame

2. Each element is assigned a number from 1 to N (pop. size)
3. Determine the sample interval i:i=N/n. If i is a fraction, round to the nearest
integer
4. Select a random number, r, between 1 and i, as explained in simple random
sampling
5. The elements with the following numbers will comprise the systematic random
sample: r, r+i,r+2i,r+3i,r+4i,...,r+(n-1)i
 Probability Sampling Methods (cont.)

• Stratified Samples
– Stratified samples are probability samples that are
distinguished by the following procedural steps:
• First, the original or parent population is divided into
two or more mutually exclusive and exhaustive subsets
(e.g., male and female).
• Second, simple random samples of elements from the
two or more subsets are chosen independently from
each other.
 Stratified
Sampling

1. Select a suitable frame

2. Select the stratification variable(s) and the number of strata, H
3. Divide the entire population into H strata. Based on the classification variable, each
element of the population is assigned to one of the H strata
4. In each stratum, number the elements from 1 to Nh (the pop. size of stratum h)
5. Determine the sample size of each stratum, nh, based on proportionate or
disproportionate stratified sampling, where

6. In each stratum select a simple random sample of size nh

H
nh = n
h=1
 Probability Sampling Methods
(cont.)

• Cluster Samples
– In the case of cluster samples, the sampling
units are selected in groups. There are two
basic steps in cluster sampling:
• First, the population of interest is divided into
mutually exclusive and exhaustive subsets.
• Second, a random sample of the subsets is selected.
 Cluster Sampling

1. Assign a number from 1 to N to each element in the population

2. Divide the population in C clusters of which c will be included in the sample
3. Calculate the sampling interval i, i=N/c (round to nearest integer)
4. Select a random number r between 1 and i, as explained in simple random sampling
5. Identify elements with the following numbers: r,r+i,r+2i,... r+(c-1)i
6. Select the clusters that contain the identified elements
7. Select sampling units within each selected cluster based on SRS or systematic sampling
8. Remove clusters exceeding sampling interval i. Calculate new population size N*, number
of clusters to be selected C*= C-1, and new sampling interval i*.
 Types of Cluster Sampling
Cluster Sampling

One-Step Two-Step Multistage

Approach Approach Approach

Simple Cluster Probability

Sampling Proportionate
to Size Sampling
 Non-probability Sampling Methods

• Convenience Samples
– Non-probability samples used primarily because
they are easy to collect.
• Judgment Samples
– Non-probability samples in which the selection
criteria are based on personal judgment that the
element is representative of the population under
study.
 Non-probability Sampling Methods (cont.)

• Quota Samples
– Non-probability samples in which population
subgroups are classified on the basis of researcher
judgment.
• Snowball Samples
– Non-probability samples in which selection of
additional respondents is based on referrals from
the initial respondents.
Strengths and Weaknesses of Basic Sampling Techniques
Choosing Non-probability vs. Probability Sampling
 Why have to know errors?

• Sample estimate will always be subject to

deviation from parameter.
• 𝑦 is the unbiased estimate𝑁−𝑛
of 𝑌 with Standard
deviation (Standard 𝑆
𝑁𝑛
(inerror)
case of Simple random sampling)
• This error mainly appeared from two sources
– Due to sample
– Other than sample
• It is imprtant to understand common sources
and types of errors so you can avoid them.
 Sampling Errors and Non-sampling
Errors
• The errors involved in collection,
processing and analysis of the data in
survey may be classified

– Sampling error
– Non-sampling error
 Sampling Errors

• Sampling Errors(1): The error which

arises due to only a sample being used to
estimate the population parameter is
termed sampling error or sampling
fluctuation.
• Sampling Errors(2): Sampling error is the
error that arises in a data collection
process as a result of taking a sample
from a population rather than using the
whole population.
 Sources of Sampling Errors
• Population specification error
• Sample frame error
• Selection error
• Non-response
• Sampling errors
 Sampling Errors
• If the sample size 𝑛 is equal to 𝑁 we expect that
sampling error will be zero.
• The decrease in sampling error is inversely proportional
to the square root of the sample size.
Error
Sampling

Sample
Size
 Sampling Errors

• The amount of sampling error decreases with increase

in the samle size but surprisingly it becomes
otherwise in case of non-sampling error.
• The sampling errors are assigned to an estimate
because it is based on a ‘part’ from the ‘whole’ while
non-sampling errors are assinged because there is
departure from the prescribed rules of the survey,
such as survey design, field work, tabulation
andanalysis of data , etc.
 How to remove?
• There is only one way to eliminate this
error. This solution is to eliminate the
concept of sample, and to test the entire
population.
• In most cases this is not possible;
consequently, what a researcher must to
do is to minimize sampling process error.
This can be achieved by a proper and
unbiased probability sampling and by
using a large sample size.
 Non-sampling error

• Non-sampling error can occur at any

one or more stages of a survey i.e.
planning, field work, and tabulation of
survey data
 Non-sampling error

• Besides sampling error, the sample

estimate may be subject to other error
which, grouped toghther, are termed
non-sampling errors.

• Non-sampling error is the error that arises

in a data collection process as a result of
factors other than taking a sample.
 Sources of Non-sampling error

• Non sampling errors broadly grouped

are in three number
– Group A: Errors resulting from
inadequete preparation
(Non-response errors)
– Group B: Errors resulting in the stage of data
collection or taking observation (Response
error).
– Group C: Errors resulting from data
processing (Tabulation errors)
 Sources of Non-sampling error
• Group A: Due to faulty sampling frame,
biased method of selection units,
inadequate schedule.
– Ommision of duplication of units due
to ambiguous definition of locale,
units.
– Inaccurate methods of interview and
schedules
– Difficulties arising due to unawarness on the
part of respondents or faulty methods of
enumeration/ data collection.
 Sources of Non-sampling error

• Group B: These errors refer, the difference between

the individual true value and the corresponding
sample value irrespsective of the reasons for
discrepancy.
• Landholder says 10 hactors
• Cadastral says 11 hactors
• Response error occur…
• Main sources of these errors
– Inadequate supervission and inspection of field
staff
– Inadequate trained and experienced field staff
– Problems involved in data collection and other
 Sources of Non-sampling error

• Group C: These errors can be assinged to a number

of defective methods of editing, coding puncing,
tabulation, etc.
• Main sources
– Inadequate scrutiny of basic data
– Errors in data processing operations such as
editing, coding, punching, listing,verification
etc
– Others errors commited or admited
during publication/ presentation of
results.
 Sources of Non-sampling error

• These are not exhaustive,appear may

large number of errors in a survey
• It is difficult to ensure which of these
errors are admitted, what is the
frequncy of their occurance, and what
are their effects on results?
• Need subtle investigation at every
step of survey.
 Biases and variables errors

• Relation between biases and variables

errors
𝐸(𝑡 − 𝜃)2= 𝑉 𝑡 + (𝐵(𝑡))2
• Total error can be writen as
1
𝑇𝑜𝑡𝑎𝑙 𝑒𝑟𝑟𝑜𝑟
= [𝑉 𝑡 + (𝐵(𝑡))2]2
• It has been seen that sample values
are subject to both sampling and
non-sampling error.
 Biases and variables errors

• 𝜃: true value
• 𝜃𝑝 : Expected survey value
• 𝜃′: estimated parametric value
• Thus the total errorcan be written as
′
𝑡 − 𝜃 = 𝑡 − 𝐸 𝑡 + 𝐸 𝑡 − 𝜃𝑝 + 𝜃𝑝 + [𝜃 − 𝜃]
′
–− 𝜃Sampling error
– Non-sampling error
• Both variable errors and biases can
arise either from sampling or non
sampling operations
 Biases and variables errors

Sampling
Biasses
Biases
Non-sampling
Biases

Sampling
Variances
Variance
s Non-samplin
g
Variances
 Biases and variables errors

• Generalization form of biases and variables

errors 2 2
𝐸(𝑡 − 𝜃) = ( 𝐵𝑖 ) 2
+ 𝑆𝑖 /𝑛𝑖𝑎𝑗
𝑖 𝑖
Where
– 𝐵𝑖 stands for the bias
– 𝑆𝑖 stands for varince
– 𝑛𝑖 stands for sample size
– 𝑎𝑗 Stands for the term for the sampling
design used in the survey
 Error decomposition

Samplin
g
Biases

Non-samplin
g

Variance
Error

Component- II
(A)
Response

Non-response
(B)
Component- I
 Non-response errors

• Non response errors arises due to

variaous causes
– Not-at-home
– Refusal
– Lost schedule
 OUTLIN
E
• Our take home…………….
• What is sample size?
• What is sample size determination?
• How large a sample do I need?
• What are the methods of
determining it?
• What are the factors that affect it?
• Mind my language
• How do you determine it?
• How do you use it?
• A final word………………..
 OUR TAKE HOME
At the end of this presentation, we should be
able to;

 Understand the significance of sample size.

 Determine sample size.
 Understand factors that may affect sample
size
 Use sample size in our research or study.
 WHAT IS SAMPLE SIZE?
 This is the sub-population to be studied in order to
make an inference to a reference population(A
broader population to which the findings from a
study are to be generalized)
 In census, the sample size is equal to the population size.
However, in research, because of time constraint and
budget, a representative sample are normally used.
 The larger the sample size the more accurate the findings
from a study.
  Availability of resources sets the upper limit of the
sample size.
 While the required accuracy sets the lower limit of
sample size
 Therefore, an optimum sample size is an
essential component of any research.

95
96
 WHAT IS SAMPLE SIZE DETERMINATION
 Sample size determination is the mathematical
estimation of the number of subjects/units to be
included in a study.
 When a representative sample is taken from a
population, the finding are generalized to the
population.
 Optimum sample size determination is required
for the following reasons:
4. To allow for appropriate analysis
5. To provide the desired level of accuracy
6. To allow validity of significance test.
 HOW LARGE A SAMPLE DO I NEED?

If the sample is too small:
2. Even a well conducted study may fail to answer it
research question
3. It may fail to detect important effect or associations
4. It may associate this effect or association
imprecisely
 CONVERSELY
• If the sample size is too large:
2. The study will be difficult and costly
3. Time constraint
4. Available cases e.g rare disease.
5. Loss of accuracy.

• Hence, optimum sample size must be

determined before commencement of a study.
MIND MY LANGUAGE
 Random error  Type I(a) error
 Systematic error  Type II (b)
(bias) error
 Precision  Power (1-b)
(reliability)  Effect size
 Accuracy (Validity)  Design effect
 Null hypothesis
 Alternative
hypothesis
Random error: error that occur by chance. Sources are
sample variability, subject to subject differences &
measurement errors. It can be reduce by averaging,
increase sample size, repeating the experiment.
Systematic error: deviations not due to chance alone.
Several factors, e.g patient selection criteria may
contribute. It can be reduce by good study design and
conduct of the experiment.
Precision: the degree to which a variable has the same
value when measured several times. It is a function of
random error.
Accuracy: the degree to which a variable actually
represent the true value. It is function of systematic error.
Null hypothesis: It state that there is no difference
among groups or no association between the
predictor & the outcome variable. This hypothesis
need to be tested.

Alternative hypothesis: It contradict the null

hypothesis. If the alternative hypothesis cannot be
tested directly, it is accepted by exclusion if the test
of significance rejects the null hypothesis. There are
two types; one tail(one-sided) or two
tailed(two-sided)
Type I(a) error: It occurs if an investigator rejects a
null hypothesis that is actually true in the
population. The probability of making (a) error is
called as level of significance & considered as
0.05(5%). It is specified as Za in sample size
computing. Za is a value from standard normal
distribution ≡ a. Sample size is inversely
proportional to type I error.
Type II(b) error: it occur if the investigator fails to
reject a null hypothesis that is actually false in the
population. It is specify in terms of Zb in sample size
computing. Zb is a value from standard normal
distribution ≡b
 Power(1-b): This is the probability that the test will
correctly identify a significant difference, effect or
association in the sample should one exist in the
population. Sample size is directly proportional to the
power of the study. The larger the sample size, the study
will have greater power to detect significance difference,
effect or association.

Effect size: is a measure of the strength of the relationship

between two variables in a population. It is the magnitude of
the effect under the alternative hypothesis. The bigger the
size of the effect in the population, the easier it will be to
find.

105

Design effect: Geographic clustering is generally
used to make the study easier & cheaper to
perform.
The effect on the sample size depends on the
number of clusters & the variance between &
within the cluster.
In practice, this is determined from previous
studies and is expressed as a constant called ‘design
effect’ often between 1.0 &2.0. The sample sizes for
simple random samples are multiplied by the design
effect to obtain the sample size for the cluster
sample.
odds ratio is a measure of effect size, describing
the strength of association or non-independence
between two binary data values.

relative risk (RR) is the risk of an event (or of

developing a disease) relative to exposure. Relative
risk is a ratio of the probability of the event occurring
in the exposed group versus a non-exposed group.
 POWER ANALYSIS
 When the estimated sample size can not be
included in a study, post-hoc power analysis
should be carried out.
 The probability of correctly rejecting the null
hypothesis is equal to 1 – b, which is called power.
The power of a test refers to its ability to detect
what it is looking for.
 the power of a test is our probability of finding what
we are looking for, given its size.
 post-hoc power analysis is done after a study has
108 been carried out to help to explain the results if a
study which did not find any significant effects.
 AT WHAT STAGE CAN SAMPLE SIZE BE
ADDRESSED?
It can be addressed at two stages:
2. Calculate the optimum sample size required during
the planning stage, while designing the study, using
appropriate approach & information on some
parameters.
3. Or through post-hoc power analysis at the
stage of interpretation of the result.
 APPROACH FOR ESTIMATING SAMPLE
SIZE/POWER ANALYSIS

Approaches for estimating sample size and performing
power analysis depend primarily on:
2. The study design &
3. The main outcome measure of the study

There are distinct approaches for calculating sample

size for different study designs & different
outcome measures.
 1. THE STUDY DESIGN
 There are many different approaches for calculating the
sample size for different study designs. Such as case
control design, cohort design, cross sectional studies,
clinical trials, diagnostic test studies etc.
 Within each study design there could be more
sub-designs and the sample size calculation will vary
accordingly.
 Therefore, one must use the correct approach for
computing the sample size appropriate to the study
design & its subtype.
 2.PRIMARY OUTCOME MEASURE

1⁰ outcome measure is usually reflected in the 1⁰

research question of the study & also depend on
the study design.
 For estimating the risk in control study, it will be the
odds ratio, while for cohort study it will be the
relative ratio.
 For case control study, it could be the difference
in means/proportions of exposure in case &
controls, crude/adjusted odds ratio etc.
 Hence, while calculating sample size, one of
these 1⁰outcome measures has to be specified
 statistical inference from the study
results

In addition, there are also different procedure for

calculating sample size for two approaches of drawing
statistical inference from the study result i.e
2. Estimation (Confidence interval approach)
3. Hypothesis testing(Test of significance approach)
A researcher needs to select the appropriate procedure
for computing the sample size & accordingly use the
approach of drawing a statistical inference
subsequently.
NB: Test of significance: Chi-squared, T-test, Z-test,
F-test, P- value
 ADDITIONAL PARAMETERS
Depending upon the approach chosen for calculating the
sample size, one also needs to specify some additional
parameters such as;
 Hypothesis
 Precision
 Type I error
 Type II error
 Power
 Effect size
 Design effect
 PROCEDURE FOR CALCULATING SAMPLE
SIZE.

There are four procedures that could be used for calculating

sample size:
2. Use of formulae
3. Ready made table
4. Nomograms
5. Computer software
 USE OF FORMULAE FOR SAMPLE SIZE
CALCULATION & POWER ANALYSIS
 There are many formulae for calculating sample
size & power in different situations for different
study designs.
 The appropriate sample size for population-based
study is determined largely by 3 factors
3. The estimated prevalence of the variable of interest.
4. The desired level of confidence.
5. The acceptable margin of error.
 
To calculate the minimum sample size required for
accuracy, in estimating proportions, the following
decisions must be taken:
2. Decide on a reasonable estimate of key proportions
(p) to be measured in the study
3. Decide on the degree of accuracy (d) that is desired in the
study.
~1%-5% or 0.01 and 0.05
4. Decide on the confidence level(Z) you want to use.
Usually 95%≡1.96.
5. Determine the size (N) of the population that the
sample is supposed to represent.
6. Decide on the minimum differences you expect to find
statistical significance.
117
 
For population
>10,000.
n=Z2pq/d2
n= desired sample size(when the population>10,000)
Z=standard normal deviate; usually set at 1.96(or a~2), which
correspond to 95% confidence level.
p=proportion in the target population estimated to have a
particular characteristics. If there is no reasonable
estimate, use 50%(i.e 0.5)
q=1-p(proportion in the target population not having the
particular characteristics)
d= degree of accuracy required, usually set at 0.05 level( occasionally
at 2.0)

E.g if the proportion of a target population with
certain characteristics is 0.50, Z statistics is 1.96
& we desire accuracy at 0.05 level, then the
sample size is
n=(1.962)(0.5)(0.5)/0.05
2
n=384.
 If study population is < 10,000

nf=n/1+(n)/(N)

nf= desired sample size, when study population

<10,000 n= desired sample size, when the study
population > 10,000 N= estimate of the population
size

Example, if n were found to be 400 and if the population size were estimated
at 1000, then nf will be calculated as follows

nf= 400/1+400/1000
nf= 400/1.4
nf=286

120
 SAMPLE SIZE FORMULA FOR COMPARISON OF
GROUPS
If we wish to test difference(d) between two sub-samples regarding a
proportion & can assume an equal number of cases(n1=n2=n’) in
two sub- samples, the formula for n’ is

n’=2z2pq/d2

E.g suppose we want to compare an experimental group against a

control group with regards to women using contraception. If we
expect p to be 40 & wish to conclude that an observed difference of
0.10 or more is significant at the
0.05 level, the sample size will be:
n’= 2(1.96)2(0.4)(0.6)/0.12
=184
Thus, 184 experimental subject & another 184 control subjects are
required.
 USE OF READYMADE TABLE FOR SAMPLE
SIZE CALCULATION
 How large a sample of patients should be followed up if
an investigator wishes to estimate the incidence rate of a
disease to within 10% of it’s true value with 95%
confidence?
 The table show that for e=0.10 & confidence level of
95%, a sample size of 385 would be needed.
 This table can be used to calculate the sample size
making the desired changes in the relative precision &
confidence level .e.g if the level of confidence is reduce
to 90%, then the sample size would be 271.
 Such table that give ready made sample sizes are
available for different designs & situation
123
 USE OF NOMOGRAM FOR SAMPLE SIZE
CALCULATION

 For use of nomogram to calculate the sample size,

one needs to specify the study(group 1) & the
control group(group 2). It could be arbitrary or
based on study design; the nomogram will work
either way.
 The researcher should then decide the effect size
that is clinically important to detect. This should
be expressed in terms of % change in the response
rate compared with that of the control group.
 
E.g if 40% of patients treated with standard therapy
are cured and one wants to know whether a new
drug can cure 50%, one is looking for a 25%
increase in cure rate .
(50%-40%/40% = 25% )

125
126
 USE OF COMPUTER SOFTWARE
FOR SAMPLE SIZE CALCULATION
& software
The following POWER can beANALYSIS
used for calculating
sample size & power;
❖ Epi-info
❖ nQuerry
❖ Power & precision
❖ Sample
❖ STATA
❖ SPSS
 Epi-info for sample size determination
 In STATCALC:
 1 Select SAMPLE SIZE & POWER.
 2 Select POPULATION SURVEY.
 3 Enter the size of population (e.g. 15 000).
 4 Enter the expected frequency (an estimate of
the true prevalence, e.g.80% ± your minimum
standard).
 5 Enter the worst acceptable result (e.g. 75%) i.e the
margin of error is 5%
 How to use sample size formulae
Steps:
1st Formulate a research question
2nd Select appropriate study design, primary outcome
measure, statistical significance.
3rd use the appropriate formula to calculate the sample
size.
Finally
 Sample size determination is one of the most
essential component of every research/study.
 The larger the sample size, the higher the degree
accuracy, but this is limit by the availability of
resources.
 It can be determined using formulae, readymade
table, nomogram or computer software.
 8-1

Unit 3 : Data Collection, measurement and scaling

Hours: 10

■Qualitative methods of data collection,

■attitude measurement and scaling-

■types of measurement scale,

■attitude,

■ classification of scales –

■Single Item v/s Multi Item Scales,

■Comparative v/s Non-Comparative scales,

■Continuous Rating Scales;

■Criteria for Good Measurement,

■questionnaire design
 8-2

Basic Measurement Issues

Measurement is the process of assigning numbers

or labels to objects, persons, states, or events in
accordance with specific rules to represent
quantities or qualities of attributes.
We do not measure specific objects, persons, etc.,
we measure attributes or features that define
them.
Ex., What defines the person Brent Wren? What is
a student’s level of education? How customer
oriented is our company?
Overriding Goal: To provide a valid and reliable
description or enumeration of the person, objects,
issue, etc.
 8-3

Measurement Process

1. Define concepts to be measured

2. Define attributes of the concepts
3. Select scale of measurement (data type)
4. Generate Items/Questions
■ Wording
■ Response format
5. Layout and design questionnaire
6. Pretest and refine
 8-4

Basic Measurement Issues

■ Concept or Construct
■ A generalized idea about a class of objects,
attributes, occurrences, or processes
■ Concrete – demographics, traffic patterns,
purchase quantity
■ Abstract – loyalty, personality, satisfaction,
leadership
■ Attribute
■ A single characteristic or fundamental feature of
an object, person, situation, or issue
■ Often measure multiple attributes
 8-5

1) Overview
2) Measurement and Scaling
3) Primary Scales of Measurement
i. Nominal Scale
ii. Ordinal Scale
iii. Interval Scale
iv. Ratio Scale
4) A Comparison of Scaling Techniques
 8-6

Chapter Outline
5) Comparative Scaling Techniques
i. Paired Comparison
ii. Rank Order Scaling
iii. Constant Sum Scaling
iv. Q-Sort and Other Procedures
6) Verbal Protocols
7) International Marketing Research
8) Ethics in Marketing Research
 8-7

Measurement and Scaling

Measurement means assigning numbers or other
symbols to characteristics of objects according to
certain prespecified rules.
■ One-to-one correspondence between the numbers

and the characteristics being measured.

■ The rules for assigning numbers should be

standardized and applied uniformly.

■ Rules must not change over objects or time.
 8-8

Measurement and Scaling

Scaling involves creating a continuum upon which
measured objects are located.

Consider an attitude scale from 1 to 100. Each

respondent is assigned a number from 1 to 100, with
1 = Extremely Unfavorable, and 100 = Extremely
Favorable. Measurement is the actual assignment of
a number from 1 to 100 to each respondent. Scaling
is the process of placing the respondents on a
continuum with respect to their attitude toward
department stores.
 8-9

Primary Scales of Measurement

Scale
Nominal Numbers Finish
Assigned
7 8 3
to Runners

Ordinal Rank Order Finish

of Winners
Third Second First
place place place

Interval Performance
Rating on a 8.2 9.1 9.6
0 to 10 Scale

Ratio Time to 15.2 14.1 13.4

Finish, in
Seconds
 8-10
Primary Scales of Measurement
Nominal Scale

■ The numbers serve only as labels or tags for

identifying and classifying objects.
■ When used for identification, there is a strict
one-to-one correspondence between the numbers
and the objects.
■ The numbers do not reflect the amount of the
characteristic possessed by the objects.
■ The only permissible operation on the numbers in a
nominal scale is counting.
■ Only a limited number of statistics, all of which are
based on frequency counts, are permissible, e.g.,
percentages, and mode.
 8-11

Illustration of Primary Scales of Measurement

Nominal Ordinal Interval Ratio

Scale Scale Scale Scale
Preference Preference
$ spent last
No. Store Rankings
Ratings 3 months
1-7 11-17
1. Lord & Taylor
2. Macy’s
3. Kmart
4. Rich’s
5. J.C. Penney
6. Neiman Marcus
7. Target
8. Saks Fifth Avenue
9. Sears
10.Wal-Mart
 8-12
Primary Scales of Measurement
Ordinal Scale
■ A ranking scale in which numbers are assigned to
objects to indicate the relative extent to which the
objects possess some characteristic.
■ Can determine whether an object has more or less of
a characteristic than some other object, but not how
much more or less.
■ Any series of numbers can be assigned that
preserves the ordered relationships between the
objects.
■ In addition to the counting operation allowable for
nominal scale data, ordinal scales permit the use of
statistics based on centiles, e.g., percentile, quartile,
median.
 8-13
Primary Scales of Measurement
Interval Scale
■ Numerically equal distances on the scale represent
equal values in the characteristic being measured.
■ It permits comparison of the differences between
objects.
■ The location of the zero point is not fixed. Both the
zero point and the units of measurement are
arbitrary.
■ Any positive linear transformation of the form y = a
+ bx will preserve the properties of the scale.
■ It is meaningful to take ratios of scale values.
■ Statistical techniques that may be used include all of
those that can be applied to nominal and ordinal
data, and in addition the arithmetic mean, standard
deviation, and other statistics commonly used in
marketing research.
 8-14
Primary Scales of Measurement
Ratio Scale

■ Possesses all the properties of the nominal, ordinal,

and interval scales.
■ It has an absolute zero point.
■ It is meaningful to compute ratios of scale values.
■ Only proportionate transformations of the form y =
bx, where b is a positive constant, are allowed.
■ All statistical techniques can be applied to ratio data.
 8-15

Primary Scales of Measurement

 8-16

Criteria for Scale Selection

■ Understanding of the questions

■ Discriminatory power of scale descriptors
■ Balanced versus unbalanced scales
■ Forced or non-forced choice scales
■ Desired measure of central tendency and
dispersion
 8-17

Central Tendency and Dispersion

■ Central Tendency ■ Dispersion

■ Mode ■ Frequency distribution

■ Median ■ Range

■ Mean ■ Standard deviations

 8-18

Central Tendency and Dispersion

■ If a nominal scale is used, analysis of raw data can only

be done using modes and frequency distributions
■ If ordinal scales are used, analysis of raw data can be
done using medians and ranges (plus modes and frequency
distributions)
■ If interval or ratio scales are used, analysis of raw data
can be done through the use of sample means and
estimated standard deviations as the sample statistic (plus
the above)
 8-19

A Classification of Scaling Techniques

Scaling Techniques

Comparative Noncomparative
Scales Scales

Paired Rank Constant Q-Sort and Continuous Itemized

Comparison Order Sum Other Rating Scales Rating Scales
Procedures

Semantic Stapel
Likert
Differential
 8-20

A Comparison of Scaling Techniques

■ Comparative scales involve the direct comparison of stimulus
objects. Comparative scale data must be interpreted in relative terms
and have only ordinal or rank order properties.

■ Comparative Rating Scales

■ format that requires a judgment comparing one object, person, or
concept against another on the scale

■ In noncomparative scales, each object is scaled independently of

the others in the stimulus set. The resulting data are generally
assumed to be interval or ratio scaled.

■ Noncomparative Rating Scales

■ format that requires a judgment without reference to another
object, person, or concept
 8-21

A COMPARATIVE RATING SCALE

A COMPARATIVE RATING SCALE ASKS RESPONDENTS TO RATE A

CONCEPT BY COMPARING IT WITH A BENCHMARK.

Please indicate how the amount of authority in your present position

compares with the amount of authority that would be ideal for this
position.

TOO MUCH ABOUT RIGHT TOO LITTLE

 8-22

Relative Advantages of Comparative Scales

■ Small differences between stimulus objects can be
detected.
■ Same known reference points for all respondents.
■ Easily understood and can be applied.
■ Involve fewer theoretical assumptions.
■ Tend to reduce halo or carryover effects from one
judgment to another.
Relative Disadvantages of Comparative Scales
■ Ordinal nature of the data
■ Inability to generalize beyond the stimulus objects
scaled.
 8-23

Comparative Rating Scales

 8-24
Comparative Scaling Techniques
Paired Comparison Scaling

■ A respondent is presented with two objects and

asked to select one according to some criterion.
■ The data obtained are ordinal in nature.
■ Paired comparison scaling is the most widely used
comparative scaling technique.
■ With n brands, [n(n - 1) /2] paired comparisons are
required
■ Under the assumption of transitivity, it is possible to
convert paired comparison data to a rank order.
 8-25
Obtaining Shampoo Preferences
Using Paired Comparisons

Instructions: We are going to present you with ten pairs of

shampoo brands. For each pair, please indicate which one of the
two brands of shampoo you would prefer for personal use.
Recording Form:

a
A 1 in a particular box means that the brand in that column was preferred
over the brand in the corresponding row. A 0 means that the row brand was
preferred over the column brand. bThe number of times a brand was preferred
is obtained by summing the 1s in each column.
 8-26
Comparative Scaling Techniques
Rank Order Scaling

■ Respondents are presented with several objects

simultaneously and asked to order or rank them
according to some criterion.
■ It is possible that the respondent may dislike the
brand ranked 1 in an absolute sense.
■ Furthermore, rank order scaling also results in ordinal
data.
■ Only (n - 1) scaling decisions need be made in rank
order scaling.
 8-27
Preference for Toothpaste Brands
Using Rank Order Scaling

Instructions: Rank the various brands of toothpaste in order

of preference. Begin by picking out the one brand that you like
most and assign it a number 1. Then find the second most
preferred brand and assign it a number 2. Continue this
procedure until you have ranked all the brands of toothpaste
in order of preference. The least preferred brand should be
assigned a rank of 10.
No two brands should receive the same rank number.
The criterion of preference is entirely up to you. There is no
right or wrong answer. Just try to be consistent.
 8-28
Preference for Toothpaste Brands
Using Rank Order Scaling

Form
Brand Rank Order
1. Crest _________
2. Colgate _________
3. Aim _________
4. Gleem _________
5. Macleans _________

6. Ultra Brite _________

7. Close Up _________
8. Pepsodent _________
9. Plus White _________
10. Stripe _________
 8-29
Comparative Scaling Techniques
Constant Sum Scaling

■ Respondents allocate a constant sum of units, such

as 100 points to attributes of a product to reflect
their importance.
■ If an attribute is unimportant, the respondent assigns
it zero points.
■ If an attribute is twice as important as some other
attribute, it receives twice as many points.
■ The sum of all the points is 100. Hence, the name of
the scale.
 8-30
Importance of Bathing Soap Attributes
Using a Constant Sum Scale

Instructions
On the next slide, there are eight attributes of
bathing soaps. Please allocate 100 points among
the attributes so that your allocation reflects the
relative importance you attach to each attribute.
The more points an attribute receives, the more
important the attribute is. If an attribute is not at
all important, assign it zero points. If an attribute is
twice as important as some other attribute, it
should receive twice as many points.
 8-31
Importance of Bathing Soap Attributes
Using a Constant Sum Scale

Form
Average Responses of Three Segments
Attribute Segment I Segment II Segment III
1. Mildness
2. Lather
3. Shrinkage
4. Price
5. Fragrance
6. Packaging
7. Moisturizing
8. Cleaning Power
Sum
Attitude Measurement
ATTITUDE

AN ENDURING DISPOSITION TO
CONSISTENTLY RESPOND IN A
GIVEN MANNER
 8-34

Attitudes as Hypothetical Constructs

● The term hypothetical construct is

used to describe a variable that is
not directly observable, but is
measurable by an indirect means
such as verbal expression or overt
behavior - attitudes are considered
to be such variables.
 8-35

THREE COMPONENTS OF AN
ATTITUDE

■ AFFECTIVE
■ COGNITIVE
■ BEHAVIORAL
AFFECTIVE

THE FEELINGS OR EMOTIONS

TOWARD AN OBJECT
 8-37
COGNITIVE

■ KNOWLEDGE AND BELIEFS

 8-38

BEHAVIORAL
■ PREDISPOSITION TO ACTION
■ INTENTIONS
■ BEHAVIORAL EXPECTATIONS
 8-39
MEASURING ATTITUDES

■ RANKING
■ RATING
■ SORTING
■ CHOICE
 8-40

The Attitude Measuring Process

RANKING - Rank order preference

RATING - Estimates magnitude of a
characteristic
SORTING - Arrange or classify concepts
CHOICE - Selection of preferred
alternative
 8-41

● Ranking tasks require that the

respondent rank order a small
number of objects in overall
performance on the basis of
some characteristic or stimulus.
 8-42

● Rating asks the respondent to

estimate the magnitude of a
characteristic, or quality, that an
object possesses. The respondent’s
position on a scale(s) is where he or
she would rate an object.
 8-43

● Sorting might present the respondent with

several concepts typed on cards and
require that the respondent arrange the
cards into a number of piles or otherwise
classify the concepts.
● Choice between two or more alternatives
is another type of attitude measurement -
it is assumed that the chosen object is
preferred over the other.
 8-44

● Physiological measures of attitudes

provide a means of measuring attitudes
without verbally questioning the
respondent. For example, galvanic skin
responses, blood pressure
measurement, etc.
 8-45

Simple Attitude Scaling

● In its most basic form, attitude scaling
requires that an individual agree with a
statement or respond to a single
question. This type of self-rating scale
merely classifies respondents into one
of two categories.
 8-46

SIMPLIFIED SCALING EXAMPLE

THE PRESIDENT SHOULD RUN FOR RE-ELECTION

_______ AGREE ______ DISAGREE
 8-47

Category Scales
● A category scale is a more sensitive
measure than a scale having only two
response categories - it provides more
information.
 8-48

EXAMPLE OF CATEGORY SCALE

How important were the following in your decision to visit San
Diego? (check one for each item)
VERY SOMEWHAT NOT TOO
IMPORTANT IMPORTANT IMPORTANT

CLIMATE ___________ ___________ ___________

COST OF TRAVEL ___________ ___________ ___________
FAMILY ORIENTED___________ ___________ ___________
EDUCATIONAL
/HISTORICAL ASPECTS _________ ___________ ___________
FAMILIARITY WITH
AREA ___________ ___________ ___________
 8-49

Method of Summated Ratings:

The Likert Scale

● An extremely popular means for measuring

attitudes. Respondents indicate their own
attitudes by checking how strongly they
agree or disagree with statements.
● Response alternatives: “strongly agree”,
“agree”, “uncertain”, “disagree”, and
“strongly disagree”.
 8-50

Likert Scale

A likert scale is
an ordinal scale format
that asks respondents
to indicate the extent to which
they agree or disagree with a series of
mental or behavioral belief
statements about a given object
 8-51
LIKERT SCALE FOR MEASURING
ATTITUDES TOWARD TENNIS
It is more fun to play a tough, competitive
tennis match than to play an easy one.
___Strongly Agree
___Agree
___Not Sure
___Disagree
___Strongly Disagree
 8-52

LIKERT SCALE FOR MEASURING ATTITUDES

TOWARD TENNIS Cont.

There is really no such thing as a tennis

stroke that cannot be mastered.
___Strongly Agree
___Agree
___Not Sure
___Disagree
___Strongly Disagree
 8-53

LIKERT SCALE FOR MEASURING

ATTITUDES TOWARD TENNIS
Cont.

Playing tennis is a great way to exercise.

___Strongly Agree
___Agree
___Not Sure
___Disagree
___Strongly Disagree
 8-54

Semantic Differential Scale

A semantic differential scale is

unique bipolar ordinal scale format
that captures a person’s attitudes
and/or feelings about a given
object
 8-55

Semantic Differential
● A series of seven-point bipolar rating
scales. Bipolar adjectives, such as
“good” and “bad”, anchor both ends (or
poles) of the scale.
 8-56

● A weight is assigned to each position on

the rating scale. Traditionally, scores
are 7, 6, 5, 4, 3, 2, 1, or +3, +2, +1, 0,
-1, -2, -3.
 8-57

SEMANTIC DIFFERENTIAL SCALES

FOR MEASURING ATTITUDES
TOWARD TENNIS
Exciting ___ : ___ : ___ : ___ : ___ : ___ : ___ :
Calm

Interesting ___ : ___ : ___ : ___ : ___ : ___ : ___ :

Dull
Simple ___ : ___ : ___ : ___ : ___ : ___ : ___ Complex

Passive ___ : ___ : ___ : ___ : ___ : ___ : ___

Active
 8-58

Behavioral Intention Scale

A behavioral intention scale is
a special type of rating scale
designed
to capture the likelihood that
people
will demonstrate some type of
predictable behavior intent
toward purchasing an object or
service
in a future time frame
 8-59

Numerical Scales
● Numerical scales have numbers as
response options, rather than “semantic
space’ or verbal descriptions, to identify
categories (response positions).
 8-60

Stapel Scales
● Modern versions of the Staple scale place a
single adjective as a substitute for the
semantic differential when it is difficult to
create pairs of bipolar adjectives.
● The advantage and disadvantages of a
Stapel scale, as well as the results, are very
similar to those for a semantic differential.
However, the stapel scale tends to be
easier to conduct and administer.
 8-61
A Stapel Scale for Measuring a
Store’s Image

Department
Store Name
+3
+2
+1
Wide Selection
-1
-2
-3
 8-62

Select a plus number for words that you think

describe the store accurately. The more
accurately you think the work describes the
store, the larger the plus number you should
choose. Select a minus number for words
you think do not describe the store
accurately. The less accurately you think the
word describes the store, the large the minus
number you should choose, therefore, you
can select any number from +3 for words
that you think are very accurate all the way
to -3 for words that you think are very
inaccurate.
 8-63

Paired Comparisons:

In paired comparisons the respondents are

presented with two objects at a time and
asked to pick the one they prefer. Ranking
objects with respect to one attribute is not
difficult if only a few products are compared,
but as the number of items increases, the
number of comparisons increases
geometrically (n*(n -1)/2). If the number of
comparisons is too great, respondents may
fatigue and no longer carefully discriminate
among them.
 8-64

Rank Order Scales

■ Rank the following in order of preference, 1
being the most preferred and 3 the least:

■ Brand A _________

■ Brand B _________

■ Brand C _________
 8-65

Rank Order Scales

● easy to understand and use, ordinal
scales, comparative data, number
and kinds of alternatives could be a
problem and difficult to know how
far apart.
 8-66

Constant Sum Scale

■ Divide 100 points among each of the
following brands according to your preference
for the brand:

■ Brand A _________

■ Brand B _________

■ Brand C _________
 8-67

Constant Sum Scale

Magnitude of each alternative known

Equal value can be indicated.

Too many can confuse

 8-68

Graphic Rating Scales

● A graphic rating scale presents
respondents with a graphic continuum.
 8-69
Graphic Rating Scale Stressing
Pictorial Visual Communications

3 2 1
Very Very
Good Poor
 8-70

Graphic Rating Scale

 8-71

Graphic Rating Scales

● easy to construct and simple to

use, can discern fine distinctions,
but not very reliable, can be
treated as interval data.
 8-72

Purchase Intent Scale

● How likely are you to buy this product
in the next six months.
Will definitely buy ____
Will probably buy ____
Will probably not buy ____
Will definitely not buy ____
● Can be weighted to get probable
market share.
 8-73

MONADIC RATING SCALE

A MONADIC RATING SCALE ASKS ABOUT A SINGLE CONCEPT

Now that you’ve had your automobile for about 1 year, please tell us
how satisfied you are with its engine power and pickup.

Completely Very Fairly Well Somewhat Very

Satisfied Satisfied Satisfied Dissatisfied Dissatisfied
 8-74

A UNBALANCED SCALE

A UNBALANCED SCALE HAS MORE RESPONSES DISTRIBUTED AT

ONE END OF THE SCALE.

How satisfied are you with the bookstore in the Student Union?

Neither Satisfied Quite Very

Satisfied Nor Dissatisfied Dissatisfied Dissatisfied
Reliability and Validity
 Reliability
When a Measurement Procedure yields
consistent scores when the phenomenon
being measured is not changing.
Degree to which scores are free of
“measurement error”
Consistency of measurement
 VALIDITY
• The extent to which measures indicate what
they are intended to measure.
• The match between the conceptual
definition and the operational definition.
 RELATIONSHIP BETWEEN
RELIABILITY AND VALIDITY

• Necessary but not sufficient

• Reliability is a prerequisite for
measurement validity
• One needs reliability, but it’s not
enough
 Example
• Measuring height with reliable bathroom
scale
• Measuring “aggression” with observer
agreement by observing a kid hitting a Bobo
doll
 Types of Reliability
Measurement

• 1. Stability Reliability

• 2. Equivalence Reliability
 Stability Reliability
• Test-retest
• SAME TEST – DIFFERENT TIMES
• Testing phenomenon at two different
times;
• The degree to which the two
measurements of “Sam Ting,” using same
measure, are related to one another
• Only works if phenomenon is unchanging
 Example of Stability
• Administering same questionnaire at 2
different times
• Re-examining client before deciding on
intervention strategy.
• Running “trial” twice (e.. g. errors in tennis
serving)
 Notes on Stability Reliability
• When ratings are by an observer rather
than the subjects themselves, this is called
Intraobserver Reliability or Intrarater
Reliability.
• Answers about the past are less reliable
when they are very specific, because the
questions may exceed the subjects’ capacity
to remember accurately.
 Equivalence Reliability

1. Inter-item (split ½)
2. Parallel forms [Different types of
measures]
3. Interobserver Agreement
-Is every observer scoring the same ?
 1. Inter-item Reliability

• (Internal consistency): The association of

answers to a set of questions designed to
measure the same concept.
2. Parallel forms of Reliability
• Split ½ (inter-item)
• Different types of measures
• Inter-observer Reliability
– Is everyone measuring the same thing ?
• Different measures – same time
 3.Interobserver Reliability

• Correspondence between measures

made by different observers.
 Types of Validity
• 1. Content Validity
– Face Validity
– Sampling Validity (content validity)
• 2. Empirical Validity
– Concurrent Validity
– Predictive Validity
• 3. Construct Validity
 Face Validity
• confidence gained from careful inspection of
a concept to see if it’s appropriate “on its
face;”
• In our [collective] intersubjective, informed
judgment, have we measured what we want
to measure?
• (N.B. use of good judgment)
• Rosenberg’s self esteem scale (SES)
questions:
 Content validity
• Also called “sampling validity”
• establishes that the measure covers the full
range of the concept’s meaning, i.e., covers
all dimensions of a concept
• N.B depends on “good “ judgment
• EG. Earlier SES scale in class
• Authoritarian personality questions from
Walizer & Wienir
 EMPIRICAL Validity
• Establishes that the results from one
measure match those obtained with a
more direct or already validated measure
of the same phenomenon (the “criterion”)
• Includes
– Concurrent
– Predictive
 Concurrent Validity
• Validity exists when a measure yields scores that
are closely related to scores on a criterion
measured at the same time
• Does the new instrument correlate highly with
an old measure of the same concept that we
assume (judge) to be valid? (use of “good”
judgment)
• EG. Aronson’s doodle measure of achievement
motivation.
• Act vs. SAT
 Predictive Validity
• Exits when a measure is validated by
predicting scores on a criterion measured in
the future
• Are future events which we judge to be a
result of the concept we’re measuring
anticipated [predicted] by the scores we’re
attempting to validate
• Use of “good” judgment
Examples of Predictive Validity
• Bronson screening test for “at risk”
parenting followed up by interviewing and
observing family members and school staff
later
• Sat / ACT scores and later college
“performance” (grades)
• Grades are “judged” to be measured validly
 What’s a Construct?
• Multidimensional concept
– SES
– Industrialization
• Fuzzy concept / hard to define
– Ego strength
– Love
• Concept build out of other concepts
– Force=mass * acceleration
 Construct validity
• : established by showing that a measure is
(1) related to a variety of other measures
as specified in a theory, used when no
clear criterion exists for validation
purposes (2) that the operationalization
has a set of interrelated items and (3) that
the operationalization has not included
separate concepts
 Construct validity
• Check the intercorrelation of items used to
measure construct judged to be valid
• Use theory to predict a relationship and use
a judged to be valid measure of the other
variable then check for relationship
• Demonstrate that your measure isn’t related
to judged to be valid measures of unrelated
concepts
 Convergent Validity
• Convergent validity: achieved when one
measure of a concept is associated with
different types of measures in the same
concept (this relies on the same type of
logic as measurement triangulation)
• Measures intercorrelated
 Discriminant Validity
• Discriminant validity: scores on the
measure to be validated are compared to
scores on measures of different but related
concepts and discriminant validity is
achieved if the measure to be validated is
NOT strongly associated with the
measures of different concepts
• Measure not related to unrelated concepts
 Difference between reliability and
validity
• Reliability: the degree to which a measurement
procedure produces similar outcomes when it is
repeated.
• E.g., gender, birthplace, mother’s
name—should be the same always—

• Validity: tests for determining whether a

measure is measuring the concept that the
researcher thinks is being measured,
• i.e., “Am I measuring what I think I am
measuring”?
SAMPLING RESEARCH
 SAMPLING…
⦿ The process of selecting a number of individuals
for a study in such a way that the individuals
represent the larger group from which they were
selected
⦿ .. the representatives selected for a study
whose characteristics exemplify the larger
group from which they were selected
⦿ Population…
⦿ …the larger group from which individuals are
selected to participate in a study
THE PURPOSE FOR SAMPLING…
⦿ To gather data about the population in order to
make an inference that can be generalized to
the population
THE SAMPLING PROCESS…

POPULATION

INFERENCE

SAMPLE
REGARDING THE SAMPLE…

POPULATION (N)

IS THE SAMPLE
REPRESENTATIVE?
SAMPLE (n)
REGARDING THE INFERENCE…

POPULATION (N)

INFERENCE

IS THE
INFERENCE
SAMPLE (n)
GENERALIZABLE?
STEPS IN SAMPLING...

1. Define population (N) to be sampled

2. Determine sample size (n)
3. Control for bias and error
4. Select sample
1. DEFINE POPULATION TO BE
SAMPLED...

⦿ Identify the group of interest and its

characteristics to which the findings of the
study will be generalized

…called the “target” population

(the ideal selection)
…oftentimes the “accessible” or
“available” population must be
used (the realistic selection)
2. DETERMINE THE SAMPLE SIZE...

⦿ The size of the sample influences both the

representativeness of the sample and the
statistical analysis of the data

…larger samples are more likely

to detect a difference between
different groups
…smaller samples are more likely
not to be representative
RULES OF THUMB FOR DETERMINING
THE SAMPLE SIZE...

1. The larger the population size, the

smaller the percentage of the
population required to get a
representative sample
2. For smaller samples (N ‹ 100), there is
little point in sampling. Survey the
entire population.
3. If the population size is around 500
(give or take 100), 50% should be
sampled.
4. If the population size is around 1500,
20% should be sampled.
5. Beyond a certain point (N = 5000),
the population size is almost
irrelevant and a sample size of 400
may be adequate.
 3. CONTROL FOR SAMPLING BIAS AND
ERROR...
⦿ Be aware of the sources of sampling bias and
identify how to avoid it

● Decide whether the bias is so severe

that the results of the study will be
seriously affected
● In the final report, document
awareness of bias, rationale for
proceeding, and potential effects
4. SELECT THE SAMPLE...
⦿ A process by which the researcher attempts to
ensure that the sample is representative of the
population from which it is to be selected

…requires identifying the sampling

method that will be used
 BASICS OF SAMPLING
Samples offer many benefits:
⦿ A sample is a
⦿ Save costs: Less expensive to study
“part of a whole the sample than the population.
to show what the
⦿ Save time: Less time needed to study
rest is like”.
the sample than the population .
⦿ Sampling helps to
⦿ Accuracy: Since sampling is done
determine the
corresponding with care and studies are conducted
value of the by skilled and qualified interviewers,
population and the results are expected to be
plays a vital role accurate.
in marketing ⦿ Destructive nature of elements: For
research. some elements, sampling is the way to
test, since tests destroy the element
itself.
BASICS OF SAMPLING

Limitations of Sampling Sampling Process

Developin
⦿ Demands more rigid g
control in undertaking Defining a
sample operation. the sampling
population Frame
⦿ Minority and smallness in
number of sub-groups Specifying
often render study to be Sample Determini
suspected. Method ng
⦿ Accuracy level may be Sample
affected when data is Size
subjected to weighing.
⦿ Sample results are good
approximations at best. SELECTING THE SAMPLE
 Sampling: Step 1 Sampling: Step 2
Defining the Universe Establishing the Sampling
Frame
⦿ Universe or population is the
whole mass under study. ⦿ A sample frame is the list of all
⦿ How to define a universe: elements in the population
(such as telephone directories,
◼ What constitutes the units of
electoral registers, club
analysis (HDB apartments)?
membership etc.) from which
◼ What are the sampling units
the samples are drawn.
(HDB apartments occupied in
the last three months)? ⦿ A sample frame which does not
fully represent an intended
◼ What is the specific designation
population will result in frame
of the units to be covered (HDB
error and affect the degree of
in town area)?
reliability of sample result.
◼ What time period does the data
refer to (December 31, 1995)
STEP - 3
DETERMINATION OF SAMPLE SIZE
⦿ Sample size may be determined by using:
◼ Subjective methods (less sophisticated methods)
The rule of thumb approach: eg. 5% of population
Conventional approach: eg. Average of sample sizes of similar
other studies;
Cost basis approach: The number that can be studied with the
available funds;
◼ Statistical formulae (more sophisticated methods)
Confidence interval approach.
STEP 4:
SPECIFYING THE SAMPLING METHOD
⦿ Probability Sampling
◼ Every element in the target population or universe [sampling
frame] has equal probability of being chosen in the sample
for the survey being conducted.
◼ Scientific, operationally convenient and simple in theory.
◼ Results may be generalized.
⦿ Non-Probability Sampling
◼ Every element in the universe [sampling frame] does not
have equal probability of being chosen in the sample.
◼ Operationally convenient and simple in theory.
◼ Results may not be generalized.
 PROBABILITY SAMPLING
Four types of probability sampling

⦿ Appropriate for ⦿ Appropriate for

homogeneous population heterogeneous
◼ Simple random sampling population
Requires the use of a random ◼ Stratified sampling
number table. Use of random number table
◼ Systematic sampling may be necessary
Requires the sample frame ◼ Cluster sampling
only, Use of random number table
No random number table is may be necessary
necessary
 NON-PROBABILITY SAMPLING
⦿ Four types of non-probability sampling
techniques
◼ Very simple types, based on subjective criteria
Convenient sampling
Judgmental sampling
◼ More systematic and formal
Quota sampling
◼ Special type
Snowball Sampling
RANDOM SAMPLING METHODS...

1. Simple random sampling

2. Stratified sampling
3. Cluster sampling
4. Systematic sampling
1. Simple random sampling: the process
of selecting a sample that allows
individual in the defined population to
have an equal and independent
chance of being selected for the
sample
STEPS IN RANDOM SAMPLING...
1. Identify and define the population.
2. Determine the desired sample size.
3. List all members of the population.
4. Assign all individuals on the list a consecutive number from
zero to the required number. Each individual must have the
same number of digits as each other individual.
5. Select an arbitrary number in the table of random numbers.
6. For the selected number, look only at the number of digits
assigned to each population member.
7. If the number corresponds to the number assigned to any of
the individuals in the population, then that individual is
included in the sample.
8. Go to the next number in the column and repeat step #7 until
the desired number of individuals has been selected for the
sample.
advantages…
…easy to conduct
…strategy requires minimum knowledge of the population to be
sampled

disadvantages…
…need names of all population members
…may over- represent or under- estimate sample members
…there is difficulty in reaching all selected in the sample
SIMPLE RANDOM SAMPLING

⦿ Also called random 20

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

sampling
1 37 75 10 49 98 66 03 86 34 80 98 44 22 22 45 83 53 86 23
⦿ Simplest method of 51
probability 2 50 91 56 41 52 82 98 11 57 96 27 10 27 16 35 34 47 01 36
08
sampling 3 99 14 23 50 21 01 03 25 79 07 80 54 55 41 12 15 15 03 68
56
4 70 72 01 00 33 25 19 16 23 58 03 78 47 43 77 88 15 02 55
67
5 18 46 06 49 47 32 58 08 75 29 63 66 89 09 22 35 97 74 30
80

6 65 76 34 11 33 60 95 03 53 72 06 78 28 14 51 78 76 45 26
45
Need to use 7 83 76 95 25 70 60 13 32 52 11 87 38 49 01 82 84 99 02 64
Random 00
8 58 90 07 84 20 98 57 93 36 65 10 71 83 93 42 46 34 61 44
Number Table 01
9 54 74 67 11 15 78 21 96 43 14 11 22 74 17 02 54 51 78 76
76
10 56 81 92 73 40 07 20 05 26 63 57 86 48 51 59 15 46 09 75
64

11 34 99 06 21 22 38 22 32 85 26 37 00 62 27 74 46 02 61 59
81
12 02 26 92 27 95 87 59 38 18 30 95 38 36 78 23 20 19 65 48
2. Stratified sampling: the process of
selecting a sample that allows
identified subgroups in the defined
population to be represented in the
same proportion that they exist in the
population
 STEPS IN STRATIFIED SAMPLING...

1. Identify and define the population.

2. Determine the desired sample size.
3. Identify the variable and subgroups (strata) for which you
want to guarantee appropriate, equal representation.
4. Classify all members of the population as members of one
identified subgroup.
5. Randomly select, using a table of random numbers) an
“appropriate” number of individuals from each of the
subgroups, appropriate meaning an equal number of
individuals
advantages…
…more precise sample
…can be used for both proportions and stratification sampling
…sample represents the desired strata

disadvantages…
…need names of all population members
…there is difficulty in reaching all selected in the sample
…researcher must have names of all populations
3. Cluster sampling: the process of
randomly selecting intact groups,
not individuals, within the defined
population sharing similar
characteristics
CLUSTER SAMPLING

⦿ Is a type of sampling in which clusters or groups of

elements are sampled at the same time.
⦿ Such a procedure is economic, and it retains the
characteristics of probability sampling.
⦿ A two-step-process:
◼ Step 1- Defined population is divided into number of mutually
exclusive and collectively exhaustive subgroups or clusters;
◼ Step 2- Select an independent simple random sample of
clusters.
⦿ One special type of cluster sampling is called area sampling, where
pieces of geographical areas are selected.
STEPS IN CLUSTER SAMPLING…
1. Identify and define the population.
2. Determine the desired sample size.
3. Identify and define a logical cluster.
4. List all clusters (or obtain a list) that make up the population of clusters.
5. Estimate the average number of population members per cluster.
6. Determine the number of clusters needed by dividing the sample size by
the estimated size of a cluster.
7. Randomly select the needed number of clusters by using a table of
random numbers.
8. Include in your study all population members in each selected cluster.
advantages…
…efficient
…researcher doesn’t need names of all population members
…reduces travel to site
…useful for educational research

disadvantages…
…fewer sampling points make it less like that the
sample is representative
4. Systematic sampling: the
process of selecting individuals
within the defined population
from a list by taking every Kth
name.
SYSTEMATIC SAMPLING
STEPS IN SYSTEMATIC SAMPLING...
1. Identify and define the population.
2. Determine the desired sample size.
3. Obtain a list of the population.
4. Determine what K is equal to by dividing the size of the
population by the desired sample size.
5. Start at some random place in the population list. Close you
eyes and point your finger to a name.
6. Starting at that point, take every Kth name on the list until the
desired sample size is reached.
7. If the end of the list is reached before the desired sample is
reached, go back to the top of the list.
advantages…

…sample selection is simple

disadvantages…
…all members of the population do not
have an equal chance of being selected
…the Kth person may be related to a
periodical order in the population list,
producing unrepresentativeness in the
sample
NON-PROBABILITY SAMPLES
⦿ Convenience sampling
◼ Drawn at the convenience of the researcher. Common in exploratory
research. Does not lead to any conclusion.
⦿ Judgmental sampling
◼ Sampling based on some judgment, gut-feelings or experience of the
researcher. Common in commercial marketing research projects. If inference
drawing is not necessary, these samples are quite useful.
⦿ Quota sampling
◼ An extension of judgmental sampling. It is something like a two-stage
judgmental sampling. Quite difficult to draw.
⦿ Snowball sampling
◼ Used in studies involving respondents who are rare to find. To start with, the
researcher compiles a short list of sample units from various sources. Each of
these respondents are contacted to provide names of other probable
respondents.
 SAMPLING VS NON-SAMPLING ERRORS

Sampling Error [SE] Non-sampling Error [NSE]

Very small sample Size

Larger sample size

Still larger sample

Complete census
CHOOSING PROBABILITY VS. NON-PROBABILITY
SAMPLING

Probability Evaluation Criteria Non-probability

sampling sampling
Conclusive Nature of research Exploratory

Larger sampling Relative magnitude Larger non-sampling

errors sampling vs. error
non-sampling error

High Population variability Low

[Heterogeneous] [Homogeneous]

Favorable Statistical Considerations Unfavorable

High Sophistication Needed Low

Relatively Longer Time Relatively shorter

High Budget Needed Low

MISTAKES TO BE CONSCIOUS
OF...

1. Sampling error
2. Sampling bias

…which threaten to render a study’s

findings invalid
Sampling error…
…the chance and random variation in
variables that occurs when any
sample is selected from the
population

…sampling error is to be expected

…to avoid sampling error, a census of
the entire population must be taken
…to control for sampling error,
researchers use various sampling
methods
Sampling bias…
…nonrandom differences, generally the fault
of the researcher, which cause the sample
is over-represent individuals or groups
within the population and which lead to
invalid findings

…sources of sampling bias include the

use of volunteers and available
groups
APPROACHES TO QUANTITATIVE
SAMPLING...

1. Random: allows a procedure

governed by chance to select the
sample; controls for sampling bias
2. Nonrandom (“nonprobability”): does
not have random sampling at any
state of the sample selection;
increases probability of sampling bias
NONRANDOM SAMPLING METHODS...

1. Convenience sampling
2. Purposive sampling
3. Quota sampling
1. Convenience sampling: the process
of including whoever happens to be
available at the time
…called “accidental” or “haphazard”
sampling
disadvantages…
…difficulty in determining how much
of the effect (dependent variable)
results from the cause (independent
variable)
2. Purposive sampling: the process whereby the
researcher selects a sample based on experience
or knowledge of the group to be sampled
…called “judgment” sampling
disadvantages…
…because samples need to be small and many
potential participants are unwilling to undergo the
demands of participation, most qualitative
research samples are purposive
3. Quota sampling: the process whereby
a researcher gathers data from
individuals possessing identified
characteristics and quotas
disadvantages…
…people who are less accessible (more
difficult to contact, more reluctant to
participate) are under-represented
APPROACHES TO QUALITATIVE
SAMPLING...
…qualitative research is characterized by in-depth
inquiry, immersion in a setting, emphasis on
context, concern with participants’ perspectives,
and description of a single setting, not
generalization to many settings
…potential for inaccuracy in the researcher’s
criteria and resulting sample selections
…representativeness is secondary to the quality of
the participants’ ability to provide the desired
information about self and setting
1. Intensity sampling: selecting participants who
permit study of different levels of the research
topic
2. Homogeneous sampling: selecting participants
who are very similar in experience, perspective, or
outlook
3. Criterion sampling: selecting all cases that meet
some pre-defined characteristic
4. Snowball sampling: selecting a few individuals
who can identify other individuals who can identify
still other individuals who might be good
participants for a study
5. Random purposive sampling: with a small sample,
selecting by random means participants who were
purposively selected and are too numerous to
include all in the study
 Sampling Process
Define
Population

Determine Sampling
Frame

Determine Sampling
Procedure

Probability Sampling Non-Probability Sampling

Type of Procedure Type of Procedure
Simple Random Sampling Convenience
Stratified Sampling Judgmental
Cluster Sampling Quota

Determine
Appropriate
Sample Size
Execute Sampling
Design
COPYRIGHT TERRY L.
 Addressing Non-Sampling Error
How do We Get People to Complete Our Surveys
and Provide us with Good Information ??
Non-Respo
nse
Error
Non-Sampli
ng
Response
Error
or
Field
Error
MAJOR CONCLUSION
Sampling Error is Very Minor When
Compared
to Non-Sampling Error
Sampling Error - 5 % of Total Error
Non-Sampling Error - 95% of Total Error
Source: Assael and
COPYRIGHT TERRY L. Keon, JM.
 Sources of Non-Sampling Error
INADEQUATE
SAMPLING
NON- FRAME
COVERAGE
NOT AT HOM
NON- CAN’T LOCAT
SAMPLING CONTACT
NON- EXPLICIT
ERROR RESPONSE SENSITIVE
TOTAL NON- REFUSA TOPIC
OVERBURDENIN
ERROR OBSERVATION LS G
IMPLICIT
LACK OF
NON-SAMPLING FIELD DATA INTERVIEW MOTIVATIO
COLLECTION BIAS N
ERROR

OBSERVATION
ERROR
OFFICE EDITING
PROCESSING CODING
ANALYSES
COPYRIGHT TERRY L.
 Research design – sampling

● Chapter 7 in Babbie & Mouton (2001)

● How we select from an infinite number of
observations we could possibly make
● Why do we sample?
● Size of the population
● Cost of obtaining elements
● Convenience and accessibility of elements
● (in his study on suicide, Durkheim had relatives who held
high governmental positions, who were able to provide
access to French statistical records on suicides)

Research design - sampling 156

 How do we decide what to observe?

● This decision should be a matter of deliberate

choice rather than chance.
● Representativeness – a small sample of
individuals from a population must contain
essentially the same variations that exist in the
population
● BUT: limited to those characteristics that are
relevant to the substantive interests of the
study, not ALL aggregate characteristics

Research design - sampling 157

 Key concepts

● Element
● The unit about which information is collected
● Typically the elements are people
● But look at the section on “unit of analysis”
again: any of them could be elements
(schools, universities, corporations, etc.)

Research design - sampling 158

 Population

● All the potential study elements, as

defined
● Careful specification of the population

Research design - sampling 159

 Study population

● Almost impossible to guarantee that every

element meeting your definition of “the
population” has a chance to be selected
into the sample.
● Thus the “study population” will be
somewhat smaller than “the population”

Research design - sampling 160

 Sampling unit

● Typically the sampling units are the same

as the elements and probably the units of
analysis
● (We are not going to look into more
complex sampling units)

Research design - sampling 161

 Sampling frame

● The actual list of sampling units (or

elements).
● e.g. if you want to study “Students at the
University of Cape Town”, there is a list of
such sampling units (but there are a
number of definition issues to be resolved
here)

Research design - sampling 162

 Sample

● A subset of a population selected to estimate the

behaviour or characteristics of the population
● O/head p. 169
● Importance of sampling properly
● A sample exists to represent its parent
population
● We must know what the actual parent
population is, otherwise we draw false
conclusions
● e.g. if we sample only women, we cannot safely make claims
about men
Research design - sampling 163
 Deciding who to choose

● Basically two sampling strategies

available:
● Probability sampling – each member of the
population has a certain probability to be
selected into the sample
● Non-probability sampling – members selected
not according to logic of probability (or
mathematical rules), but by other means (e.g.
convenience, or access)

Research design - sampling 164

 Non-probability sampling

● Sometimes it is not possible to get the

kind of information about populations
required for probability sampling
● When the sampling frame is not known
● Complicates and limits statistical analyses
● Often well-suited for qualitative research,
where distribution of characteristics is not
important
Research design - sampling 165
 Convenience sample

● Rely on available respondents

● Most convenient method
● Risky; exercise caution

Research design - sampling 166

 Purposive sampling

● Select the sample on the basis of

knowledge of the population: your own
knowledge, or use expert judges to
identify candidates to select

● Typically used for very rare populations,

such as deviant cases

Research design - sampling 167

 Snowball sampling

● Typically used in qualitative research

● When members of a population are
difficult to locate, for covert
sub-populations, non-cooperative groups
● Recruit one respondent, who identifies
others, who identify others,….
● Primarily used for exploratory purposes

Research design - sampling 168

 Quota sampling

● A stratified convenience sampling strategy

● Begins with a table that describes the characteristics of
the target population
● e.g. the composition of postgraduate students at UCT in terms of
faculty, race, and gender
● Then select on a convenience basis, postgraduate
students in the same proportions regarding faculty, race,
and gender than in the population
● Of course, the quota frame (the proportions in the table)
must be accurate
● And biases may be introduced when selecting elements
to study
Research design - sampling 169
 Probability sampling

● Bias introduced in non-probability sampling

● The sample then not representative of the population
● Probability sampling
● Typically produce more representative samples
● Allow us to estimate the accuracy or
representativeness of the sample
● Random sampling is the key – each element has an
equal chance of being selected
● Random sampling offers access to probability theory,
through which we can estimate how representative
our sample is
Research design - sampling 170
 Probability sampling theory

● Sampling distribution – possible samples to be

drawn
● Sampling error
● The degree to which the sample characteristics
approximate the characteristics of the population
● The smaller the sample, the greater the sampling
error
● The larger the sample, the smaller the sampling error
- but only for probability sampling plans

Research design - sampling 171

 Estimating the true mean

● Note how the notion of a sampling

distribution relies on the “Central
Tendency Theorem”, that you will
recognize from the statistics part of the
course
● Standard error and standard deviation
● Inferences
● Confidence levels

Research design - sampling 172

 Populations and sampling frames

● In countries like South Africa, less-than-perfect

conditions regarding sampling frames
● Either the information is not available, or if it is, it is
subject to error (parts of it are missing)
● Remember: we use sampling frames as reflecting the
population; ideally, the population also is the time frame
– but almost never the case
● So problems when there is a disparity between a
population and the sampling frame we use

Research design - sampling 173

 Example

● A good example is using telephone

directories as sampling frames
● Or the difficulties with finding a complete
list of all primary health care facilities in
the country
● And think about the risks in using
municipal service records as a sampling
frame for “the residents of Cape Town”
Research design - sampling 174
 Three kinds of probability sampling
strategies
● 1 Simple random sampling
● Establish a sampling frame (a list, e.g. of all
the company’s customers, or all UCT
students)
● Assign a single number to each element in the
list
● Use random numbers to select the elements

Research design - sampling 175

 2 Systematic sampling

● Usually more efficient than SRS

● Establish a sampling frame
● Select the first element at random
● Then select every nth element in the list, until you have
the required number of respondents
● e.g. with a population of 300, if we want a sample of
10, choose every 30th element
● Keep an eye out for peculiar arrangements in the
sampling frame

Research design - sampling 176

 3 Stratified sampling

● Sampling error reduced by

● 1. using a large sample
● 2. a homogeneous population
● Stratified sampling based on 2.

Research design - sampling 177

 Stratified sampling

● Modifies random sampling and systematic sampling, to

obtain a greater degree of representativeness
● Organize the population into homogeneous subsets, then
sample randomly within each one
● e.g. for university students, stratify according to seniority
and gender
● Stratification ensures equal proportions of people having
the relevant characteristics are selected into your sample
● Depends on what variables are available to stratify on

Research design - sampling 178

Data Collection in Qualitative Research
 Unit 3 : Data Collection, measurement
and scaling
• Qualitative methods of data collection,
• attitude measurement and scaling-
• types of measurement scale,
• attitude,
• classification of scales –
• Single Item v/s Multi Item Scales,
• Comparative v/s Non-Comparative scales,
• Continuous Rating Scales;
• Criteria for Good Measurement,
• questionnaire design
 Qualitative Research
 is an interdisciplinary,
Qualitative research
transdisciplinary, and sometimes counter
field. disciplinary
 It crosses the humanities and the social and physical
sciences. Qualitative research is many things at the
same time.
 It is multi-paradigmatic in focus. Its practitioners are
sensitive to the value of the multi method approach.
 They are committed to the naturalistic perspective, and
to the interpretative understanding of human experience.

 At the same time, the field is inherently political and

shaped by multiple ethical and political positions.
 Qualitative Research (Contd…)

Qualitative Research involves finding out what

people think, and how they feel? or at any rate,
what they say? they think? and how they? And
what they feel about a particular thing?

This kind of information is subjective.

It involves feelings and impressions, rather than

numbers.
 Qualitative Research (Contd…)

 Qualitative research involves the studied use and

collection of a variety of empirical materials - case study,
personal experience, life story, interview, observational,
historical, interactional, and visual texts

 It describes the routine and problematic moments and

meanings in individuals lives.

 Deploy a wide range of interconnected methods, hoping

always to get a better fix on the subject matter at hand.
 Qualitative and Quantitative Research
Contrast
QUALITATIVE QUANTITATIVE
 Multiple realities  Single reality

 Reality is socially constructed  Reality is objective

 Reality is context interrelated  Reality is context free


Holistic  Reductionistic

 Strong philosophical perspective  Strong theoretical base

 Reasoning is inductive Reasoning is deductive and



inductive
Discovery of meaning is the basis

Cause-and-effect relationships are

of knowledge the bases of knowledge
 Develops theory  Tests theory
Qualitative and Quantitative Research Contrasted
(contd…)
QUALITATIVE QUANTITATIVE

 Theory developed during the study Theory developed a priori

 Meaning of the concepts  Measurement of variables


 Process oriented Outcome oriented

 Control unimportant  Control important


 Rich descriptions Precise measurement of variables

Basic element of analysis is words Basic element of analysis is
numbers

Uniqueness  Generalization
 
Trustworthiness of findings Control of error
Research Methods in Qualitative
Steps in Quantitative and Qualitative Studies (contd…)

© 2007 Pearson Education Canada

 Overview of Data Collection Techniques in
Qualitative Research

 Data collection techniques allow us to systematically

collect information about out objects of study (people,

objects, phenomena) and about the settings in which
they occur.

 In the collection of data we have to be systematic. If

data are collected haphazardly, it will be difficult to

answer our research.
Main Activities in Qualitative Research

Qualitative approaches on:

 Literature review

 Explicating researcher’s beliefs

 Role of participants: subject or informant?

 Selection of participants

 Setting for data collection

 Approach to data analysis

 Saturation
 Literature Review

 Conducted after the data have been collected

and analyzed

 Rationale for delaying the literature review:

 To avoid leading the participants in the direction of
what has already been discovered

 Purpose of literature review:

 To show how current findings fit into what is
already known
 Explicating Researcher’s Beliefs

 Bracketing – setting aside one’s biases and

personal views on a topic


Investigator keeps a diary of personal thoughts
and feelings about the topic

 Purpose: the researcher is made aware when

interpretations of the data reflect personal

beliefs rather than those of the participants
 About Subject or Informant

 People being studied are viewed as participant or

informant, not “subject”

 Viewed as active participants in the research

They “inform” the researcher about their culture

 Researcher seeks to understand the participants’ cultural

knowledge

 Hence, requires learning about the participants’

culture through on-going discussion and
involvement with them
 Selection of Study

Participants
Method is called “purposive sampling” Participants must
have first-hand experience with the research topic (e.g.,
homelessness, gang involvement, attending medical
school) and be able to talk about it.

 Researcher establishes clear criteria and rationale

for sample selection (Sampling Methods).

 Goal is not generalization of findings but rich

descriptions of phenomenon by those who have

experienced it.
 Setting for Data Collection

 “Informant-driven” rather than “theory-driven”

 Investigator assumes ignorance of the culture or
experience being studied

Informant teaches the investigator

 Data is collected in the “field” – the natural world where

people live and experience life. Investigator should:

 be nonintrusive

 spend a prolonged time in the field

 Data Analysis (Introduction)

 Researcher involves self in data to bring

order and meaning to vast narrative
 Come to truly understand what the data
are saying?
 Cyclical process – data collection occurs
simultaneously with data analysis
 Analysis begins when data collection begins
 Reading, rereading, intuiting, analyzing,
synthesizing, and reporting on data
 Sometimes called theoretical sampling
(collect data until saturation is reached)
 Saturation

 Refers to a situation in data analysis where

participants’ descriptions become repeated

and then confirm with the previously
collected data.

 An indication that data analysis is complete

 When data analysis is complete, data
collection is terminated.
 Three Qualitative Methods
TABLE COMPARISON OF QUALITATIVE METHODS

METHOD STUDY FOCUS ANALYTIC DISCIPLINES

FOCUS

Ethnography culture/cultural describe a Cultural

group culture/cultural Anthropology
group
Grounded cultural groups Sociology/
generate theory
Theory Symbolic
about a basic
Interaction/
social process Criminology

Phenomenology individual discern the Philosophy/

experience essence of the Psychology/
lived Sociology
 Data Collection Methods in Qualitative
Studies

 Three data collection strategies introduced:

1. Participant observation

2. In-depth interviews

3. Focus group interviews


Qualitative researchers may combine more than
one method
 Data Collection Methods:
1. Participant Observation
 Intensive, usually long term, examination of a social
group, an organization, etc.
 Researcher becomes a participant in the lives
of group members

Observes their behaviour and learns meaning
systems (which are tied to language)

 Most closely associated with Ethnography, as

developed in Classical Anthropology
 Now done in a variety of disciplines
 Steps Involved in
Participant Observation Research

A. Gaining entry into the group

B. Developing and maintaining rapport

C. Developing a method for taking field notes

D. Integrating data collection and data analysis

 Steps in Participant Observation:
(A). Gaining Entry into the Group

 Take into consideration the type of group

 formal organizations require formal entry; involves
letter writing, permission requests, etc.

Informal groups – different strategy needed

 Access may be gained through a gatekeeper (an

individual with special status)

 Want to involve key informants (those who are most

knowledgeable about the group)

 Steps in Participant Observation:
(B). Developing/Maintaining Rapport

 Researcher must work hard to develop and maintain

good relationships in the field


E.g., be sure not to become associated with one
part in a group or an organization

 Researcher could be blamed for problems that arise

in the setting or group.

 Steps in Participant Observation:
C. Strategies for Taking Field Notes

Field notes – integral to participant observation


Include descriptions and interpretations of individuals, interactions,
and events

Distinguish descriptions from interpretations




Record time and location of observations, as well as key information
(weather, events happening and their significance)


Keep theoretical memos – which are the tentative
interpretations emerging and being assessed through further data
collection
 Field Notes (cont’d)

 May not be possible or advisable to take notes while

in the field


Important that they be done as soon after field
observation as possible

 Note-taking is time-consuming because it is integral to

guiding the data collection and continuing the analysis
 Steps in Participant Observation:
D. Integrating Data Collection and Analysis


Organizing field notes into differenttypes of filesfacilitates
data analysis


Master field file – complete journal of field notes; number pages and
include entry dates

 Background, history file – sub file organizing background material


Key character files– sub fileson key players in the group
or organization


Analytic files– sub filesfor differenttypes of observations
or relationships
 Data Collection Methods:
2. In-depth Interviews

 Some studies cannot employ the

participant

observation method.
 In-depth interviews allow participants to describe

their experiences and the meaning of events

taking place in their lives.

 Interviews are flexible and allow for investigation.

 Data Collection Methods
3. Focus Group Interviews
 Interview format, but in a group setting

6-12 participants with common experience


Dates back to the 1940s – used to assess effectiveness of
morale-boosting radio shows

 1970s onward – used by market researchers

 1980s onward – used by academics
 Transcript of discussion is the data

 Plus accompanying notes


Use content analysis or grounded theory approach to analyze
the data
 3. Focus Group Interviews (cont’d)

 Strengths: Open-ended
question
 Spontaneously deal with issues as they
arise
 Cost-effective method of collecting data
 Less time consuming
 Weaknesses:

 One or two participants may dominate


Not done in a natural setting, so little
“observation” to help understand the
experience of the participants
 Advantages and Limitations
 Focus on the whole of the human experience and the

meanings ascribed to them by participants

 They provide the researcher with deep insights

that would not be possible using quantitative

methods

 The major strength of qualitative work is the validity of

the data it produces

 Participants true reality is likely to be reflected

 Major limitation is its perceived lack of objectivity and