Study Design

Dr. Legiran, M.Kes

FK UNSRI 1
Types of Studies

 Descriptive Studies
 Observational Analytic Studies
 Cross Sectional studies
 Case Control studies
 Cohort studies

 Experimental Studies
 Randomized controlled trials
FK UNSRI 2
Hierarchy of Study Types

Descriptive Analytic
•Case report
•Case series
•Survey Observational Experimental
•Cross sectional •Randomized
•Case-control controlled trials
•Cohort studies

FK UNSRI
Strength 3
of evidence for causality between a risk factor and outcome
Descriptive studies
 Descriptive studies are weak because they
make no attempt to link cause and effect
and therefore no causal association can be
determined
 Descriptive studies however, are often the
first step to a well designed epidemiological
study
 They allow the investigator to define a good
hypothesis which can then be tested using
a better design
FK UNSRI 4
Descriptive studies
 Getting a “lay of the land”
 Surveys (NHIS, MCBS)
 “How many men in the U.S. filled Viagra
prescriptions in 2004?”

 Describing a novel phenomena

 Case reports or case series
 Viagra-associated serous macular
detachment.
 Sildenafil-associated nonarteritic anterior
ischemic optic neuropathy.
FK UNSRI 5
 Descriptive studies
 Cannot establish causal relationships
 Still play an important role in describing trends
and generating hypotheses about novel
associations
 The start of HIV/AIDS research
 Squamous cell carcinoma in sexual partner of Kaposi
sarcoma patient. Lancet. 1982 Jan 30;1(8266):286.
 New outbreak of oral tumors, malignancies and infectious
diseases strikes young male homosexuals. CDA J. 1982
Mar;10(3):39-42.
 AIDS in the "gay" areas of San Francisco. Lancet. 1983
Apr 23;1(8330):923-4.
FK UNSRI 6
Analytic Studies
 Attempt to establish a causal link between
a predictor/risk factor and an outcome.

 You are doing an analytic study if you have

any of the following words in your research
question:
 greater than, less than, causes, leads to,
compared with, more likely than, associated
with, related to, similar to, correlated with

FK UNSRI 7
Hierarchy of Study Types

Descriptive Analytic
•Case report
•Case series
•Survey Observational Experimental
•Cross sectional •Randomized
•Case-control controlled trials
•Cohort studies

FK UNSRI
Strength 8
of evidence for causality between a risk factor and outcome
Research Question
Is the regular consumption of Red Bull
associated with improved academic
performance among U.S. medical students?

FK UNSRI 9
 Rationale
 “functional drink” designed for periods of
mental and physical exertion.
 performance, concentration, memory,
reaction time, vigilance, and emotional
balance

 Taurine + glucuronolactone + caffeine

FK UNSRI 10
 Background
 Alford C, Cox H, Wescott R. The effects of red bull energy drink on human
performance and mood. Amino Acids. 2001;21(2):139-50.

 Warburton DM, Bersellini E, Sweeney E. An evaluation of a caffeinated

taurine drink on mood, memory and information processing in healthy
volunteers without caffeine abstinence. Psychopharmacology (Berl). 2001
Nov;158(3):322-8.

 Seidl R, Peyrl A, Nicham R, Hauser E. A taurine and caffeine-containing drink

stimulates cognitive performance and well-being. Amino Acids. 2000;19(3-
4):635-42.

 Horne JA, Reyner LA. Beneficial effects of an "energy drink" given to sleepy
drivers. Amino Acids. 2001;20(1):83-9.

 Kennedy DO, Scholey AB. A glucose-caffeine 'energy drink' ameliorates

subjective and performancedeficits during prolonged cognitive demand.
Appetite. 2004 Jun;42(3):331-3.

FK UNSRI 11
Great idea, but how do you get
started….

 Interesting, novel, and relevant, but…

 You only have 25,000 dollars to start

investigating this question.

 What is feasible?

FK UNSRI 12
 Study Design #1
 Cross-sectional study of UCSF medical students
taking USMLE Step 2

 Questionnaire administered when registering

for USMLE 2
 Primary predictor: self-report of >3 cans Red Bull
per week for the previous year
 Covariates: Age, sex, undergraduate university, place
of birth

 Outcome : Score on USMLE Step 2

FK UNSRI 13
Cross-sectional study: structure

Red Bull consumption

USMLE Score

time

FK UNSRI 14
 Cross-sectional Study:
 Descriptive value:
 How many UCSF medical students drink Red Bull?
 What is the age and sex distribution of UCSF medical
students who drink Red Bull?
 Analytic value:
 Is there an association between regular Red Bull
consumption and test scores among UCSF med students?
 Univariate
 Multivariate (controlling for “confounders”)
 Other cross-sectional surveys:
 AAMC
 California Health Interview Survey (NHIS, CHIS)
 National Health and Nutrition Exam Survey (NHANES)
FK UNSRI 15
 Cross-sectional Study: Pluses

+ Prevalence (not incidence)

+ Fast/Inexpensive - no waiting!

+ No loss to follow up

+ Associations can be studied

FK UNSRI 16
 Measures of association
Risk ratio
Disease (relative risk)

Yes No A
A+B
C
C+D
Yes A B
Risk
Factor
No C D

FK UNSRI 17
Cross-sectional study: minuses
- Cannot determine causality

Red Bull consumption

USMLE Score

time

FK UNSRI 18
Cross-sectional study: minuses
- Cannot determine causality

•ACE inhibitor use and hospitalization rates

among those with heart failure

•Heart failure patients with a documented DNR

status and mortality

time

FK UNSRI 19
Cross-sectional study: minuses

- Cannot determine causality

- Cannot study rare outcomes

FK UNSRI 20
What if you are interested in
the rare outcome?

 The association between regular Red Bull

consumption and…
 A perfect score on the USMLE – Step 2
 Graduating top 1% of the medical school
class
 Acceptance into a highly selective residency

ANSWER: A Case-Control study

FK UNSRI 21
 Study Design #2
 A case-control study
 Cases: 4th year med students accepted to
residency in “highly selective specialty X”.
 Controls: 4th year med students who applied
but were not accepted.
 Predictor: self-reported regular Red Bull
consumption
 Additional covariates (age, sex, medical school,
undergraduate institution)

FK UNSRI 22
 Case control studies
 Investigator works “backward”
(from outcome to predictor)

 Sample chosen on the basis of

outcome (cases), plus comparison
group (controls)

FK UNSRI 23
 Case-control study structure
present

ACTUAL CASES
TARGET CASES
4th year UCSF students
Medical students accepted to
Red Bull consumption who matched in “highly
highly selective residencies
YES selective specialty X”

Red Bull consumption

NO ACTUAL CONTROLS
TARGET CONTROLS
4 th
year students
All unsuccessful who to
applicants failed
highly
to match in “highly selective
selective residency programs
specialty X”

FK UNSRI 24
time
Case control studies
 Determines the strength of the
association between each
predictor variable and the
presence or absence of disease
 Cannot yield estimates of
incidence or prevalence of
disease in the population (why?)
 Odds Ratio is statistics

FK UNSRI 25
Case-control Study: pluses
+ Rare outcome/Long latent period

+ Inexpensive and efficient: may be only

feasible option

+ Establishes association (Odds ratio)

+ Useful for generating hypotheses

(multiple risk factors can be explored)

FK UNSRI 26
Case-control study-minuses
- Causality still difficult to establish

- Selection bias (appropriate controls)

- Caffeine and Pancreatic cancer in the GI clinic

- Recall bias: sampling (retrospective)

- Abortion and risk of breast cancer in Sweden

- Cannot tell about incidence or prevalence

- Studies of diagnostic tests:
- Sensitivity, specificity
- Positive predictive value, negative predictive value

FK UNSRI 27
Measures of association
Disease Sensitivity = A/A+C

Yes No Specificity = D/B+D

Yes A B
PPV = A/A+B
Test
NPV = D/C+D

No C D

FK UNSRI 28
 Case-control - “the house red”
 Rely tampons and toxic shock syndrome:
 High rates of toxic shock syndrome in menstruating women
 Suspected OCPs or meds for PMS
 Cases: 180 women with TSS in 6 geographic areas
 Controls: 180 female friends of these patients and 180
females in the same telephone code
 Tampon associated with TSS (OR = 29!)
 Super absorbency associated with TSS (OR 1.34 per gm
increase in absorbency)
 Led to “RELY” brand tampons being taken off the market.
FK UNSRI 29
 Where are we?
 Preliminary results from our cross-sectional and
case-control study suggest an association
between Red Bull consumption and improved
academic performance among medical students
 What’s missing? - strengthening evidence for a
causal link between Red Bull consumption and
academic performance
 Use results from our previous studies to apply
for funding for a prospective cohort study!
FK UNSRI 30
 Study design #3
 Prospective cohort study of UCSF medical
students Class of 2009
 All entering medical students surveyed
regarding beverage consumption and
variety of other potential covariates
 Survey updated annually to record
changes in Red Bull consumption
 Outcomes: USMLE Step 1 score, USMLE
Step 2 score, match in first choice
residency
FK UNSRI 31
 Cohort studies
• A cohort (follow-up, longitudinal) study is a comparative,
observational study in which subjects are grouped by their
exposure status, i.e., whether or not the subject was
exposed to a suspected risk factor
• The subjects, exposed and unexposed to the risk factor,
are followed forward in time to determine if one or more
new outcomes (diseases) occur
• Subjects should not have outcome variable on entry
• No new subjects allowed in after initial recruitment

• The rates of disease incidence among the exposed and

unexposed groups are determined and compared.

FK UNSRI 32
 Elements of a cohort study
 Selection of sample from population
 Measures predictor variables in sample
 Follow population for period of time
 Measure outcome variable

 Famous cohort studies

 Framingham
 Nurses’ Health Study
 Physicians’ Health Study
 Olmsted County, Minnesota

FK UNSRI 33
Prospective cohort study structure
The present The future

Top USMLE scorers

Everyone else
FK UNSRI 34
time
 Strengths of cohort studies
 Know that predictor variable was present
before outcome variable occurred (some
evidence of causality)

 Directly measure incidence of a disease

outcome

 Can study multiple outcomes of a single

exposure (RR is measure of association)
FK UNSRI 35
 Weaknesses of cohort studies
 Expensive and inefficient for studying rare
outcomes
 HERS vs. WHI

 Often need long follow-up period or a very

large population
 CARDIA

 Loss to follow-up can affect validity of findings

 Framingham
FK UNSRI 36
 Other types of cohort studies
 Retrospective cohort
 Identification of cohort, measurement of
predictor variables, follow-up and
measurement of outcomes have all occurred
in the past
 Much less costly than prospective cohorts
 Investigator has minimal control over study
design

FK UNSRI 37
Other types of cohort studies
 Nested case-control study
 Case-control study embedded in a cohort study
 Controls are drawn randomly from study sample
 Double cohort
 Used to compare two separate cohorts with
different levels of exposure to predictor variable
(e.g., occupational groups)

FK UNSRI 38
 What type of study is this?
 Among individuals with coronary disease, what is the association
between baseline levels of B-type natriuretic peptide and
subsequent risk of heart failure?

 Among individuals presenting to heart failure clinic, what is the

association between self-reported symptoms and risk of
hospitalization for heart failure?

 Using data from HERS (RCT of HRT in women with coronary

disease):
 Determine the risk factors for developing incident heart failure
among women without heart failure at baseline.
 Determine whether HRT is associated with mortality among
women with heart failure.
 Determine genetic markers for development of heart failure
among black women in HERS.

FK UNSRI 39
Hierarchy of Study Types

Descriptive Analytic
•Case report
•Case series
•Survey Observational Experimental
•Cross sectional •Randomized
•Case-control controlled trials
•Cohort studies

FK UNSRI
Strength 40
of evidence for causality between a risk factor and outcome
 What distinguishes observational
studies from experiments?
 Ability to control for confounding
Confounder

Predictor Outcome
Examples:
sex (men are more likely to drink red bull and men are
more likely to match in neurosurgery)
Undergraduate institution (students from northwest school are
more likely to drink red bull and also more likely to score higher on
USMLE)
FK UNSRI 41
But we measured all of the
potential confounders…….
 In a prospective cohort study you can
(maybe) measure all potential known
confounders, but…

 You can’t control for unanticipated or

unmeasured confounders

FK UNSRI 42
 Study design # 4
 Randomized controlled trial of daily Red Bull
consumption among entering UCSF medical
students Class 2009

 Randomized to daily consumption of Red Bull

vs. daily consumption of placebo

 Outcomes: USMLE Step 1 score, USMLE Step

2 score, match in first choice residency

FK UNSRI 43
Randomized controlled trials
 Investigator controls the predictor
variable (intervention or treatment)
 Major advantage over observational
studies is ability to demonstrate
causality
 Randomization controls unmeasured
confounding
 Only for mature research questions

FK UNSRI 44
 Basic Trial Design

Population Treatment Dx No Dx

Sample Randomization

Control
Placebo Dx No Dx

FK UNSRI 45
Steps in a randomized
controlled trial
1. Select participants
 high-risk for outcome (high incidence)
 Likely to benefit and not be harmed
 Likely to adhere
2. Measure baseline variables
3. Randomize
 Eliminates baseline confounding
 Types (simple, stratified, block)

FK UNSRI 46
Steps in a randomized
controlled trial
4. Blinding the intervention
 As important as randomization
 Eliminates
 co intervention
 biased outcome ascertainment
 biased measurement of outcome
5. Follow subjects
 Adherence to protocol
 Lost to follow up
6. Measure outcome
 Clinically important measures
 Adverse events

FK UNSRI 47
What
What is
is Blinding?
Blinding?
 Single
 Single blind
blind -- participants
participants are
are not
not
aware
aware ofof treatment
treatment group
group
 Double
 Double blind
blind -- both
both participants
participants
and
and investigators
investigators unaware
unaware
 Triple
 Triple blind
blind -- various
various meanings
meanings

 persons
persons who
who perform
perform tests
tests
 outcome

outcome adjudicators
adjudicators
 safety

safety monitoring
monitoring group
group

FK UNSRI 48
 Why blind?: Co interventions

 Unintended effective interventions

 participants use other therapy or change
behavior
 study staff, medical providers, family or
friends treat participants differently
 Nondifferential - decreases power
 Differential - causes bias

FK UNSRI 49
Why
Why blind?:
blind?: Biased
Biased Outcome
Outcome
Ascertainment
Ascertainment oror adjudication
adjudication

 If
If group
group assignment
assignment is
is known
known
 participants
participants may
may report
report symptoms
symptoms oror outcomes
outcomes
differently
differently
 physicians
physicians oror investigators
investigators may
may elicit
elicit symptoms
symptoms
or
or outcomes
outcomes differently
differently
 Study
Study staff
staff or
or adjudicators
adjudicators may
may classify
classify similar
similar
events
events differently
differently in
in treatment
treatment groups
groups


 Problematic
Problematic with
with “soft”
“soft” outcomes
outcomes
 investigator
investigator judgement
judgement
 participant
participant reported
reported symptoms,
symptoms, scales
scales

FK UNSRI 50
Analysis of randomized
controlled trial
 Analyzed like cohort study with RR
 Intention to treat analysis
 Most conservative interpretation
 Include all persons assigned to
intervention group (including those who
did not get treatment or dropped out)
 Subgroup analysis
 Groups identified pre-randomization

FK UNSRI 51
High Quality Randomized Trials
 Tamper-proof randomization
 Blinding of participants, study
staff, lab staff, outcome
ascertainment and adjudication
 Adherence to study intervention
and protocol
 Complete follow-up

FK UNSRI 52
Hierarchy of Study Types

Descriptive Analytic
•Case report
•Case series
•Survey Observational Experimental
•Cross sectional •Randomized
•Case-control controlled trials
•Cohort studies

FK UNSRI
Strength 53
of evidence for causality between a risk factor and outcome