1/19/2018

Systematic Review of Literature

What is a Systematic Review?

• “A review that is conducted according to
clearly stated, scientific research methods,
and is designed to minimize biases and errors
inherent to traditional, narrative reviews.”

Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007)

1
 1/19/2018

What is the significance of Systematic

Reviews?

• Systematic Reviews minimize bias. “A systematic review is a more

scientific method of summarizing literature because specific protocols
are used to determine which studies will be included in the review.”

Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD, “Clinical Perspective: A Practical Guide to Meta-
Analysis.” The Journal of Hand Surgery. Vol. 31A No.10 December 2006. p.1671

Why review
• To enable you, the author, to become absolutely clear
on what it is you have chosen to research and study
• To gain an understanding of what has been and is
currently being researched
• To identifying windows of opportunity and helping to
cement your research questions
• For carrying out a critical review of the literature
• It will show a strong awareness and knowledge of your
chosen subject
• You need to build up your credibility, not just for your
work, but for you as an author. Once this is achieved
you can justify your research and explain where it fits
in with other research topics in your subject area.

2
 1/19/2018

The purpose of a literature review is to:

• Place each work in the context of its contribution to the

understanding of the research problem being studied,
• Describe the relationship of each work to the others under
consideration,
• Identify new ways to interpret, and shed light on any gaps in
previous research,
• Resolve conflicts amongst seemingly contradictory previous studies,
• Identify areas of prior scholarship to prevent duplication of effort,
• Point the way in fulfilling a need for additional research, and
• Locate your own research within the context of existing literature.

Why are Systematic Reviews

Necessary?
• “The volume of published material makes it impractical for an individual
researcher to remain up to date on a variety of common conditions. This is
further complicated when individual studies report conflicting conclusions,
a problem that is prevalent when small patient samples and retrospective
designs are used.”

Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research”. PRS Journal. 120/7 (2007) p.1834

3
 1/19/2018

Key Characteristics of Systematic

Reviews
• Clearly stated title and objectives
• Comprehensive strategy to search for relevant studies (unpublished and
published)
• Explicit and justified criteria for the inclusion or exclusion of any study
• Clear presentation of characteristics of each study included and an
analysis of methodological quality
• Comprehensive list of all studies excluded and justification for exclusion
Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature:
Introduction to Meta-analysis”. Power Point Presentation.

Characteristics of Systematic Reviews

(cont.)
• Clear analysis of the results of the eligible studies
– statistical synthesis of data (meta-analysis) if
appropriate and possible;
– or qualitative synthesis

• Structured report of the review clearly stating the aims,

describing the methods and materials and reporting the
results

Linda N. Meurer, MD, MPH Department of Family and Community Medicine. “Systematic Synthesis of the Literature:
Introduction to Meta-analysis”. Power Point Presentation.

4
 1/19/2018

An author of a good Systematic

Review…
• Formulates a Question
• Conducts a Literature Search
• Refines the search by applying predetermined inclusion and
exclusion criteria
• Extracts the appropriate data and assess their quality and
validity
• Synthesizes, interprets, and reports data

Hypothesis
• “A systematic review should be based on
principles of hypothesis testing, and the
hypotheses must be conceived a priori.”
Margaliot, Zvi, Kevin C. Chung. Systematic Reviews: A Primer for Plastic Surgery Research. PRS Journal. 120/7 (2007) p. 1836

5
 1/19/2018

Focus of the Question

• The structured question will determine the
inclusion and exclusion criteria:
– What is the population of interest?
– What are the interventions?
– What are the outcomes of interest?
– What study designs are appropriate?

Inclusion/Exclusion Criteria
• “Once the study question is formalized, the authors must
compose a comprehensive list of inclusion and exclusion
criteria.”
• “To avoid selection bias, inclusion and exclusion criteria
should be agreed upon and formalized before data extraction
and analysis.”

Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1836

6
 1/19/2018

Literature Search
• “A comprehensive and reproducible literature
search is the foundation of a systematic
review.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1837

Literature Search Challenges

• Database Bias - “No single database is likely to contain all published studies on a
given subject.”

• Publication Bias - selective publication of articles that show positive treatment of

effects and statistical significance.

– Hence, it is important to search for unpublished studies through a manual

search of conference proceedings, correspondence with experts, and a search
of clinical trials registries.

Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1837

7
 1/19/2018

Literature Review Challenges (cont.)

• English-language bias - occurs when reviewers

exclude papers published in languages other than English

• Citation bias - occurs when studies with significant or

positive results are referenced in other publications,
compared with studies with inconclusive or negative findings

Data Collection
• “The list of data to be extracted should be
agreed upon a priori consensus during the
design stage of the study.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

8
 1/19/2018

Data Collection (cont.)

• Collected data includes:
– Study characteristics
– Sample demographics
– Outcome data

Data Collection (cont.)

• “It is necessary to design a review-specific
data extraction form, so that the same data
are extracted from each study and missing
data are clearly apparent.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

9
 1/19/2018

Data Collection (cont.)

• “To ensure that data extraction is accurate and
reproducible, it should be performed by at
least two independent readers.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

Quality Assessment
• “The validity of a systematic review ultimately
depends on the scientific method of the
retrieved studies and the reporting of data.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

10
 1/19/2018

Quality Assessment (cont.)

• Randomized Controlled Trials (RCT):
– RCT are considered to be more rigorous than
observational studies
– A review based on well-designed RCT will likely be
more valid and accurate than a review based on
observational studies or case reports
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1839

Data Synthesis
• “Once the data have been extracted and their
quality and validity assessed, the outcomes of
individual studies within a systematic review
may be pooled and presented as summary
outcome or effect”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

11
 1/19/2018

Data Synthesis (cont.)

• The authors summarize heterogeneous data
qualitatively
– “Data that are very conflicting and widely variable
should not, under most circumstances, be
combined numerically.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

Definition of grey literature

“Grey literature is defined as any
literature that is not published in
academic peer reviewed journals and
available through indexed databases
for review”

12
 1/19/2018

Search for grey literature

• Call for information

• Personal and institutional contact

• World-wide web search

Challenges for a representative search

• Problems with call for information
– personal interest/limitations
• Problems with personal and institutional contact
– Staffs changes on NGOs
• World-wide web
– large number and difficulties in filter

13
 1/19/2018

Challenges for a representative search

• Difficult to control bibliographically

• Grey database
– not widely available
– institutional subscribe
– eg SIGLE (System for information on Grey
Literature in Europe)

Different type of grey literature

• Annual reports
• Theses/dissertations
• Conference abstracts/proceedings
• Working papers
• Evaluation reports
• Facts sheets and others

14
 1/19/2018

Searching for representative literature

• Which organisations?
• Which key opinions?
• Which databases?
• Which web sites?

Framework for a representative search

International

International
National
level
National
Level
Local Level Local

15
 1/19/2018

Childhood Malnutrition in Complex emergencies

Initially identified
***
4802

Title/abstract scanned

Included Full report

173 102

Full paper scanned

Non grey Grey

12  8 20

Advantages of including ‘grey’ in a

literature review

• Access to wider variety of information

• Reduces publication bias
• Contain more local information- important
for planning
• Contain process information- important for
policy makers

16
 1/19/2018

Advantages of including ‘grey’ in a

literature review
• Reports not published in peer reviewed
journals (e.g. UN, WHO, HTA)
• Access to new research
• World Wide Web (4-6 Billion web pages)
• Contain valuable and unique information,
which is not found elsewhere

Searching for representative literature

1. Systematic review of
published literature
(databases)
Value of 2.Basic review of grey
literature literature (web)
identified
3. Personal communication
with organisations and
experts identified.
4. Validation of
Phase Phase Phase Phase representaiveness
1 2 3 4

17
 1/19/2018

Literature review framework

Process Published Grey literature
literature
Searching for and 1. Call for information

Standard tools and procedure developed

locating appropriate 2. Personal and institutional
literature contact
3. World-wide web base
search and grey literature
data base search
Critically examining the 1. Assessment of
results and extracting methodological quality
relevant details 2. Data extraction
Synthesising and writing 1. Combination with other
the review evidence
2. Same as peer reviewed

When can data in a systematic

review be synthesized numerically?
• When data are NOT too sparse, of too low quality or too
heterogeneous
– For example: the
patients, interventions and
outcomes in each of the included studies are
sufficiently similar

18
 1/19/2018

Meta-Analysis
• “Meta-analysis is a statistical technique for
combining the results of independent, but
similar, studies to obtain an overall estimate of
treatment effect.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

Meta-Analysis (cont.)
• “While all meta-analyses are based on
systematic review of literature, not all
systematic reviews necessarily include meta-
analysis.”
Margaliot, Zvi, Kevin C. Chung. “Systematic Reviews: A Primer for Plastic Surgery Research.” PRS Journal. 120/7 (2007) p.1840

19
 1/19/2018

Writing a Literature Review

General Guidelines to
Writing a Literature Review
• Introduce the literature review by pointing out
the major research topic that will be discussed
• Identify the broad problem area but don’t be
too global (for example, discussing the history of
education when the topic is on specific instructional strategy)
• Discuss the general importance of your topic
for those in your field

20
 1/19/2018

General Guidelines to
Writing a Literature Review
• Don’t attempt to cover everything written on
your topic
• You will need to pick out the research most
relevant to the topic you are studying
• You will use the studies in your literature
review as “evidence” that your research
question is an important one

General Guidelines to
Writing a Literature Review
• It is important to cover research relevant to
all the variables being studied.
• Research that explains the relationship
between these variables is a top priority.
• You will need to plan how you will structure
your literature review and write from this
plan.

21
 1/19/2018

Organizing Your Literature Review

• Topical Order—organize by main topics or
issues; emphasize the relationship of the
issues to the main “problem”
• Chronological Order—organize the
literature by the dates the research was
published
• Problem-Cause-Solution Order—Organize
the review so that it moves from the
problem to the solution

Organizing Your Literature Review

• General-to-Specific Order—(Also called the
funnel approach) Examine broad-based
research first and then focus on specific
studies that relate to the topic
• Specific-to-General Order—Try to make
discuss specific research studies so
conclusions can be drawn

22
 1/19/2018

Literature Review
• After reviewing the literature, summarize
what has been done, what has not been
done, and what needs to be done
• Remember you are arguing your point of
why your study is important!
• Then pose a formal research question or
state a hypothesis—be sure this is clearly
linked to your literature review

Literature Review
• All sources cited in the literature review
should be listed in the references
• To sum, a literature review should include
introduction, summary and critique of
journal articles, justifications for your
research project and the hypothesis for
your research project

23
 1/19/2018

Common Errors Made in Lit

Reviews
• Review isn’t logically organized
• Review isn’t focused on most important facets
of the study
• Review doesn’t relate literature to the study
• Too few references or outdated references cited
• Review isn’t written in author’s own words
• Review reads like a series of disjointed
summaries
• Review doesn’t argue a point
• Recent references are omitted

Writing the Literature Review

Plagiarism includes (Galvan, pg. 89):
1. Using another writer’s words without proper citation
2. Using another writer’s ideas without proper citation
3. Citing a source but reproducing the exact word without
quotation marks
4. Borrowing the structure of another author’s
phrases/sentences without giving the source
5. Borrowing all or part of another student’s paper
6. Using paper-writing service or having a friend write the
paper

24
 1/19/2018

Questions?

Thank you!

25