RESEARCH Methods Note 3
RESEARCH Methods Note 3
Study population: the population from which we take our sample (often more limited than the
target population, e.g. infants, women of reproductive age, adolescent male, etc.
Sampling design: the scheme for selecting the sampling units from the study population.
Sampling frame: the list of units from which the sample is to be selected.
Probability Sampling
Probability sampling is a technique in which every unit in the population has a chance (non-
zero probability) of being selected in the sample, and this chance can be accurately
determined. All probability sampling have two attributes in common: (1) every unit in the
population has a known non-zero probability of being sampled (i.e., every unit in the
population has equal chance of being selected to participate in the study), and (2) the sampling
procedure involves random selection at some point. The different types of probability
sampling techniques include:
A random sample is one drawn from a population of units in such a way that every member of
the population has the same probability of selection and different units are selected
independently. Random sample plays no favorites, but assigns the same selection probability
to every member of the population. In doing so it assures that the population is fully known
and defined by the investigator. This avoids biases and over- sights, which can lead to faulty
inferences.
We must first draw up a sampling frame listing all the units in the study population, then we
randomly select the required number of units from the list. The selection of units may be done
using random number tables (exercise).
2. Stratified random sampling involves dividing the population to be studied into different
strata of similar social, environmental or health condition. A random selection of study
units is then taken from each stratum.
An advantage of stratified random sampling is that information about the composition of the
population with respect to a number of stratifying variables can be taken into account. If, for
example, the age, sex, and ethnic composition of the population are known, then the sample
can be selected to conform exactly to this composition by using these variables as the basis for
stratification. Proportion is also taken into account.
4. Systematic Sampling: in many situations this is as good as SRS and often more
convenient. Involves a system by which the sample is chosen whereby a particular
sequence is followed, must obtain a sampling frame and calculate the sampling fraction.
For instance, if the sampling frame is 2000 and the sample size is 500, the sampling
proportion is 2000/500 = 4 so 1 in every 4 persons will be selected to get the total of 500
people.
5. Multistage Sampling
Often it is not possible to do a SRS or systematic sample
-if a sampling frame is not available
-if the population is spread out over a wide area
Therefore, many times we need to do the sampling in two or more stages.
Example
For a national survey, make a list of all the states of the federation. Select a random sample of
the first- stage units. In each of these units, take a random sample of the second –stage units
(e.g. LGAs). In each of the LGAs take a random sample of the third-stage units (e.g. districts).
In each of the district select a random sample of the fourth- stage units (villages or towns). In
each of the towns or village select a random sample of the fifth-stage units (Households).
In each stage, it may be necessary to select the units with probability proportional to size
(PPS) that is, the larger units have a greater probability of being selected. To do this we need
a list of all the units in the region where the survey is to take place, together with some
approximate measure of the number of the variable we are interested in.
2. Quota sampling. In this technique, the population is segmented into mutually exclusive
subgroups (just as in stratified sampling), and then a non-random set of observations is
chosen from each subgroup to meet a predefined quota. In proportional quota sampling,
the proportion of respondents in each subgroup should match that of the population. For
instance, if the population of Lagos consists of 70% normal weight, 15% overweight, and
13% obese people, and you wish to understand their diets preferences in a sample of 98
people, you can stand outside a shopping center and ask people their diets preferences. But
you will have to stop asking overweight people when you have 15 responses from that
subgroup (or obese people when you have 13 responses) even as you continue sampling
other normal weigh group, so that the body composition of your sample matches that of
the general Lagos population.
4. Snowball sampling: In snowball sampling, you start by identifying a few respondents that
match the criteria for inclusion in your study, and then ask them to recommend others they
know who also meet your selection criteria. For instance, if you wish to survey public
health physical activity researchers in Nigeria and you know of only one or two such
people, you can start with them and ask them to recommend others who also do physical
activity research in public health. Although this method hardly leads to representative
samples, it may sometimes be the only way to reach hard-to-reach populations or when no
sampling frame is available.
Bias
Bias is an error introduced into an experimental design or study leading to misguided
interpretation of the results obtained from such observations.
Types of Bias
(1) Selection Bias /method selection is not random. Is the observed association due to
the way subjects were selected for the study?
(2) Response Bias/ information bias: Is the observed association due to errors of
measurement or classification of the exposure and/ or the disease?
(3) Observers Bias: This is a bias arising from making wrong observations in the
course of a study especially when the data is solely based on subjective
observation which is not backed by video or voice recording.
(4) Detection/Instrument Bias: This is a bias resulting from faulty instrument which
may have poor sensitivity or detection.
Basically, two main classes of study type may be identified: Descriptive (observational non-
analytic) and Analytical (Intervention /Experimental). By far the vast majority of
epidemiological studies are observational, meaning that data are collected simply ‗to see what
is happening‘, as shown in the classical study of Doll & Hill (1950) on lung cancer and
cigarette smoking. Here nature determines who is exposed to the factor of interest and who is
not exposed. These studies demonstrate association. Association may imply causation, or it
may not. By contrast intervention study is an experiment: that is, things are made to happen.
Intervention studies are considered as the gold standard as far as etiological investigations are
concerned.
Descriptive studies include case/case-series studies, cross sectional studies and ecological
studies. Each design is suitable for different research setting and conditions.
They are distinct from the descriptive design in that they utilize a comparison group that has
been explicitly collected. Their usefulness is largely in the fact that they are used to identify
environmental causes of events or health-related events or states. Analytic study designs in
environmental epidemiology and epidemiology in general are based on the comparison of
individuals who are classified according to exposure and injury/disease status. Here the unit of
analysis is the individual. They fit into two major categories, observational (case-control,
cohort) and experimental. In observational study the researcher evaluates the strength of the
relationship or association between an exposure and health related state of event. The
observed variables are beyond the control or influence of the investigator. On the other hands
in experimental studies some of the participants in the study are deliberately manipulated for
the purpose of studying an intervention effect. However, if the intervention being assessed is
an environmental exposure being investigated for its adverse health effects, it would be
unethical to assign the intervention. Instead a dose effect relationship can be investigated by
categorizing the environmental exposure by time and dose. Nature often produces natural
experiment through peoples‘ behavior and life style. A natural experiment is an unplanned
type of experimental study where the level of exposure to a presumed cause differ from a
population in a way that it is unaffected by extraneous factors so that the situation resembles a
planned experience. John Snow‘s investigation on cholera epidemic in 1854 is a classic
example of natural experiment. The result of the investigation showed that the rate of
infection was 8.4 times greater among those who obtained water from the contaminated
source than those who obtained water from another alternative source.
Perhaps the most important utility of analytical studies is that it enables us to determine the
level of risk associated with developing the disease in the presence of the environmental
factor which enables the establishment of exposure limits and standard. Risk assessment is a
tool to integrate exposure and health effects in order to identify the health hazards in humans.
Various methods are used in the analytic designs. They include experimental, cohort, case-
control and case-crossover. Each of them has its specific application. Experimental study
design examines the relations between intervention and outcome variables in a group of
cohorts that has been followed over time. It may be sometimes referred to as intervention
study. This type of design produces the strongest evidence for causal association, may
produce faster answers to research questions than cohort study and may be the most
appropriate design to answer certain research question. However, it may involve ethical
barriers, outcomes which are too rare, cost and time intensive and high attrition (loss to
follow-up).
Data Collection is an important aspect of any type of research study. Inaccurate data collection
can impact the results of a study and ultimately lead to invalid results. Data collection is the
process of gathering and measuring information on variables of interest, in an established
systematic fashion that enables one to answer stated research questions, test hypotheses, and
evaluate outcomes. The data collection component of research is common to all fields of study
including physical and social sciences, humanities, business, etc. While methods vary by
discipline, the emphasis on ensuring accurate and honest collection remains the same.
Data collection methods for impact evaluation vary along a continuum. At one end of this
continuum are quantitative methods and at the other end of the continuum are Qualitative
methods for data collection.
The selection of the chosen data collection method should be based on the following:
They rely on random sampling and structured data collection instruments that fit diverse
experiences into predetermined response categories. They produce results that are easy to
summarize, compare, and generalize.
Quantitative research is concerned with testing hypotheses derived from theory and/or being able
to estimate the size of a phenomenon of interest. Depending on the research question,
participants may be randomly assigned to different treatments. If this is not feasible, the
researcher may collect data on participant and situational characteristics in order to statistically
control for their influence on the dependent, or outcome, variable. If the intent is to generalize
from the research participants to a larger population, the researcher will employ probability
sampling to select participants.
Typical quantitative data gathering strategies include:
• Experiments/clinical trials.
• Observing and recording well-defined events (e.g., counting the number of patients
waiting in emergency at specified times of the day).
• Obtaining relevant data from management information systems.
• Administering surveys with closed-ended questions (e.g., face-to face and telephone
interviews, questionnaires etc). (http://www.achrn.org/quantitative_methods.htm)
They play an important role in impact evaluation by providing information useful to understand
the processes behind observed results and assess changes in people’s perceptions of their well-
being. Furthermore qualitative methods can beused to improve the quality of survey-based
quantitative evaluations by helping generate evaluation hypothesis; strengthening the design of
survey questionnaires and expanding or clarifying quantitative evaluation findings. These
methods are characterized by the following attributes:
• they tend to be open-ended and have less structured protocols (i.e., researchers may
change the data collection strategy by adding, refining, or dropping techniques or
informants)
• they rely more heavily on iteractive interviews; respondents may be interviewed several
times to follow up on a particular issue, clarify concepts or check the reliability of data
• they use triangulation to increase the credibility of their findings (i.e., researchers rely on
multiple data collection methods to check the authenticity of their results)
• generally their findings are not generalizable to any specific population, rather each case
study produces a single piece of evidence that can be used to seek general patterns among
different studies of the same issue
Regardless of the kinds of data involved, data collection in a qualitative study takes a great deal
of time. The researcher needs to record any potentially useful data thoroughly, accurately, and
systematically, using field notes, sketches, audiotapes, photographs and other suitable means.
The data collection methods must observe the ethical principles of research.
The qualitative methods most commonly used in evaluation can be classified in three broad
categories:
• indepth interview
• observation methods
• document review
The following link provides more information on the above three methods.
http://www.worldbank.org/poverty/impact/methods/qualitative.htm#indepth
1. Read professional journals so as to learn (i) what kind of instruments are being used for
similar studies (ii) their format (iii) their style (iv) how they are used by the writers.
2. Read books that provide a description or an actual copy of various instruments for the
reader.
3. Talk with other researchers who may know of certain tools they have developed for
themselves or who may have used tools developed by others.
4. Combine or adapt one or more tools used by other researchers.
5. Develop his/her own instrument to fulfil a specific need
1. Questionnaire
2. Check list
3. Interview
4. Observation
5. Records
6. Experimental Approach
7. Survey Approach
QUESTIONNAIRE
Paper-pencil-questionnaires can be sent to a large number of people and saves the researcher
time and money.People are more truthful while responding to the questionnaires regarding
controversial issues in particular due to the fact that their responses are anonymous. But they
also have drawbacks .Majority of the people who receive questionnaires don't return them and
those who do might not be representative of the originally selected sample.(Leedy and Ormrod,
2001)
Web based questionnaires: A new and inevitably growing methodology is the use of Internet
based research. This would mean receiving an e-mail on which you would click on an address
that would take you to a secure web-site to fill in a questionnaire. This type of research is often
quicker and less detailed. Some disadvantages of this method include the exclusion of people
who do not have a computer or are unable to access a computer. Also the validity of such
surveys are in question as people might be in a hurry to complete it and so might not give
accurate responses.
Questionnaires often make use of Checklist and rating scales. These devices help simplify and
quantify people's behaviors and attitudes. A checklist is a list of behaviors, characteristics, or
other entities that the researcher is looking for. Either the researcher or survey participant simply
checks whether each item on the list is observed, present or true or vice versa. A rating scale is
more useful when a behavior needs to be evaluated on a continuum. They are also known as
Likert scales. (Leedy and Ormrod, 2001)
Advantages of Questionnaire
Disadvantages of Questionnaire
1. Researcher may read literature about the topic, look through available questionnaire or
obtain help from experts.
2. Open ended questions are preferable than closed ended questions since they reflect
respondent’s attitudes and feelings which are expressed in his own words.
3. If yes – no questions are used, additional information may be gained by leaving space for
respondent’s own ideas.
4. The possibility of a middle ground statement is also important.
5. Every item on the questionnaire should relate to the topic under study.
1. Clarity of Language. It must meet the level of understanding of the respondents in order
to generate needed responses.
2. Singleness of objective. An item must have one and only one answer.
3. One-to-One correspondence. The questionnaire as a whole must correspond with the
objectives of the study.
4. Correct Grammar, Spelling and Construction. The questionnaire must be constructed by
observing grammatically correct sentences, correctly spelled words, coherence in
construction of sentences, etc.
1. Open-Ended Questions: This gives the respondents the ability to respond to the
questions in their own words.
2. Closed-Ended Questions: This allows the subject to choose one of the given
alternatives.
1. Dichotomous questions. This requires the respondents to make a choice between two
responses such as yes/no, male/female, married or unmarried, etc.
Example; Have you travelled outside Nigeria (a) yes (b) No
2. Multiple questions. This offers the respondents more than one choice.
Example; How favourable is it to you to become pregnant at this time? (a) Very
favourable (b) favourable (c) Not sure (d) unfavourable (e) Very unfavourable.
3. Cafeteria Questions. This is a special type of multiple choice question where the
respondents are required to select a response according to their own point of view.
Example
People have different views on ―family planning. Which of the following best represents
your view?
(a)
Family planning is necessary to quality life.
(b)
Family planning is immoral and should be totally banned.
(c)
Family planning has undesirable side effects that suggest need for caution.
(d)
Family planning has beneficial effects that merit its practice. (e) Family planning is
moral and should be practiced.
4. Rank-Order Questions. The respondents are asked to choose a response from the
―most‖ to the ―least‖.
Example; Why must family planning be practiced? Rank your answer from the 1-most
reasonable to 5-least reasonable?
5. Rating Questions. The respondents are asked to judge something along an ordered
dimension. This is typically bipolar in nature with the end points specifying the opposite
extremes of a continuum.
Example: On the scale of 1-5 where 1 means strongly disagree and 5 means strongly
agree, the health centre in Landmark University provides you with the best of health
services.
Scale
CHECK LIST.
Check list are items that comprise several questions on a topic and require the same response
format.
Example: here are some characteristics of birth control devices that are of varying importance to
different people. How important are these in choosing a birth control method?
INTERVIEW
This involves either structured or unstructured verbal communication between the researcher and
the subject during which information is obtained for a study. In Quantitative research (survey
research), interviews are more structured than in Qualitative research.
Types of Interviews
1. Unstructured Interviews.
These are more conversational, very long (sometimes hours) and are conducted in a usual
situation. It allows the researcher flexibility in questioning the subject.
2. Structured Interviews.
These always operate within formal written instrument referred to as interview schedule.
The researcher designed the questions to be asked and their order prior to the interview. The
questions are asked orally either face-to-face or telephone format.
In a structured interview, the researcher asks a standard set of questions and nothing more
(Leedy and Ormrod, 2001)
Face -to -face interviews have a distinct advantage of enabling the researcher to establish
rapport with potential participants and therefor gain their cooperation. These interviews yield
highest response rates in survey research. They also allow the researcher to clarify ambiguous
answers and when appropriate, seek follow-up information. Disadvantages include impractical
when large samples are involved time consuming and expensive.(Leedy and Ormrod, 2001)
Telephone interviews are less time consuming and less expensive and the researcher has ready
access to anyone on the planet who has a telephone. Disadvantages are that the response rate is
not as high as the face-to- face interview but cosiderably higher than the mailed questionnaire.
The sample may be biased to the extent that people without phones are part of the population
about whom the researcher wants to draw inferences.
Advantages of Interview
Disadvantages of Interview
1. It consumes time
2. Biases may result
3. It is usually costly
OBSERVATION
Observation is most commonly used in qualitative research. In observation the researcher usually
has some prior knowledge about the behaviour or occurrence of interest.
Types of Observation:
1. Unstructured Observation. This is a method of collecting research data that has both
opponents and proponents.
2. Structured Observation. This involves preparation of record keeping forms such as
category systems, check lists and rating scales.
Advantages of observation.
Disadvantages of observation.
1. Accurate prediction of a situation or event to be observed is unlikely to occur.
2. Interviewing selected subjects may provide more information, economically than waiting
for the spontaneous occurrence of the situation.
3. The presence of an observer gives the subject a quality normally absent.
4. Observed events are subject to biases.
5. Extensive training is usually needed.
RECORDS
A record refers to all the numbers and statistics that institutions, organizations and people
keep as a record of their activities. Sources of records include:
1. Census data
2. Educational data or records
3. Hospital/Clinic records.
Advantages of Records
Disadvantages of Records
1. All the researcher can have is what has been recorded. If the record is incomplete, there is
no way it can be completed.
2. No one can be sure of the conditions under which the records were collected.
3. There is no assurance of the accuracy of the records.
EXPERIMENTAL APPROACH
This is a powerful design for testing hypothesis of causal relationships among variables. In
experimental approach, the researcher controls the independent variable and watches the effect
on the dependent variable. There are two groups of experimental approach:
1. Treatment/Experimental group
2. Control group.
1. It is difficult to minimize all the variables in which the control and experimental groups
might differ.
2. Causal relationships are difficult if not impossible to establish.
3. The time element may confound the results of experimental research.
4. In an experimental Laboratory setting, it may be difficult to obtain subjects, especially
subjects who are unaware of the experiment.
SURVEY APPROACH
1. Non experimental type in which the researcher investigates the community or a group of
people.
2. Formulative or explorative research
Assignment
1. Prepare a closed ended web based questionnaire for your research project using Google
form and kobocollect tools.
2. Differentiate between data collection, data analysis and data presentation with examples.