Fischler College of Education – Nova Southeastern University

Systematic Review and Meta-Analysis Dissertations

A Brief Overview

In an effort to expand options for doctoral candidates conducting research at the Abraham
S. Fischler School of Education, Systematic Review and Meta-Analysis dissertations are
offered as possible options. These dissertation types were originally developed through
the disciplines of Education and the Social Sciences. However, because they are
particularly suited to resolving differing results from quantitative, random experimental
design studies and are conducted only after defining a strict apriori protocol, these studies
quickly became attached to medicine, nursing, and other clinical specialties where
experimental research is the standard. This overview is to present enough information to
doctoral students so that they can determine if this research selection, evaluation and
reporting paradigm is appropriate for their study.

Meta-Analysis and Systematic Reviews vary in significant ways including the objectives
of the study. The objective of a Meta-Analysis is to resolve conflicting evidence in two
or more quantitative studies where conflicts are already known to exist. It is the
resolution of the conflict that is the objective of the study, rather than an exhaustive
review and synthesis of all significant, published and unpublished research in an area,
which is expected in a Systematic Review. Practitioners in the medical arts and other
clinical specialties seek to resolve these research conflicts due to the potential impact of a
given treatment on the health and well-being of others.

In Education and the Social Sciences, there are often significant qualitative studies.
These studies call for a “meta-synthesis” of the qualitative data. The objectives of a
Systematic Review include both: (a) synthesizing the state of knowledge with regard to
an intervention or set of interventions, their components or models; and (b) discovering
and resolving any conflicting research evidence among studies. A systematic review is a
good choice when the studies include various quantitative and/or qualitative and/or mixed
method studies, as well as when the studies are sets of non-homogenous quantitative
studies. Analysis and synthesis of the research is needed on each approach. Practitioners
and policy makers often want to know “what works” in various settings and with various
target populations.

Analysis techniques differ between Meta-Analysis and Systematic Review studies. If

quantitative studies of the same type are discovered to be in conflict when the Systematic
Review is conducted, then the studies are evaluated using meta-analysis techniques inside
of the Systematic Review. These techniques include varying mathematical treatments
depending on the type of quantitative studies being evaluated. If the studies you are
evaluating are qualitative, then you evaluate them using a “meta-synthesis” technique
inside of the systematic review study. In both cases you evaluate bias and effect
variations due to setting, populations and other critical criteria.

These two dissertation types are neither quick nor easy, are mathematically oriented, are
usually longer than normal dissertations, and require strict adherence to a prescribed
 Page |2

protocol. Some doctoral candidates may select these dissertation types because they do
not have an institution or organization easily available for data collection. In addition,
the candidate rejected quantitative research study types such as evaluation research and
descriptive studies using publicly available data; and, they also rejected various
qualitative studies such as historical, psychological, biographical, or philosophical
dissertations. The doctoral candidate should consider the expectations of Meta-Analysis
and Systematic Review studies before investing both time and financial resources into
such a research effort.

Meta-Analysis and Systematic Reviews have two main detractors in the research
community. One criticism is that some researchers do not establish their protocol for
excluding and including studies to be reviewed prior to evaluating all of the published
and unpublished studies that exist. This is argued to be a source of bias. A second major
criticism is that some researchers attempt to use Meta-Analysis to evaluate non-
homogenous quantitative studies: such as comparing double-blind, randomized
experimental trials with quasi-experimental, descriptive research results. Careful attention
to these design components in the study will limit such criticism.

Interestingly, both Systematic Review and Meta-Analysis dissertations have quantitative

components; however, the research questions are formulated as one would a qualitative
study, with a central question and both issue-based and procedural sub-questions that are
to be answered by the researcher. While these dissertation types are relatively rare in the
education community, they are valuable to practitioners and policymakers.

A final thought: While you may not consider yourself a research “expert”, particularly in
quantitative, statistical analysis; conducting one of these dissertation types will provide
you will life-long professional skills. You learn to assess each study for various sources
of selection and internal bias, as well as become more adept at identifying non-
homogenous studies and aware of sources of error and bias in attempting to generalize
results from differing study designs and data. Each are valuable research skills.

The attached dissertation template was formulated from a variety of scholarly sources,
including dissertations that addressed various clinical, educational and other social
science topics. Major contributions to the template were adapted directly from
Systematic Reviews and Meta-Analysis by Little, Corcoran and Pillai (2008) and from
the “Checklist” reported by the Preferred Reporting Items for Systematic Reviews and
Meta-Analyses (PRISMA) Group (at www.prisma-statement.org). PRISMA is an
incarnation of the QUOROM Group that was previously involved in reporting standards
for these study types.

The following template has similarities to Fischler’s qualitative dissertation template.

However, it can be seen that because other research results and their conflicts are the
problem, and the synthesis of the data and resolution of conflicts are the purposes, each
chapter has its own unique components at make this dissertation type distinctive.
 [Insert Title Here]
For use with Systematic Reviews and Meta-Analysis Studies Only
Please note that this is only a template. These studies vary widely, therefore relevant
headings and subheadings may be added or excluded. Since Systematic Reviews and
Meta-Analysis studies are similar but are distinct kinds of studies from one another, there
will be differences. Differences are addressed in RED typeface.

The title should identify the study as a systematic review, meta-analysis, or both.

by
[Insert Name Here]

An Applied Dissertation Submitted to the

Abraham S. Fischler School of Education
in Partial Fulfillment of the Requirements
for the Degree of Doctor of Education

Nova Southeastern University

[Enter Month and Year Here]
 Approval Page

This applied dissertation was submitted by [Insert Name] under the direction of the
persons listed below. It was submitted to the Abraham S. Fischler School of Education
and approved in partial fulfillment of the requirements for the degree of Doctor of
Education at Nova Southeastern University.

Insert Name and Degree, e.g., Al Smith, EdD

Committee Chair

Insert Name and Degree

Committee Member

Kimberly Durham, PsyD

Dean

ii
 Statement of Original Work

I declare the following:

I have read the Code of Student Conduct and Academic Responsibility as described in the
Student Handbook of Nova Southeastern University. This applied dissertation represents
my original work, except where I have acknowledged the ideas, words, or material of
other authors.

Where another author’s ideas have been presented in this applied dissertation, I have
acknowledged the author’s ideas by citing them in the required style.

Where another author’s words have been presented in this applied dissertation, I have
acknowledged the author’s words by using appropriate quotation devices and citations in
the required style.

I have obtained permission from the author or publisher—in accordance with the required
guidelines—to include any copyrighted material (e.g., tables, figures, survey instruments,
large portions of text) in this applied dissertation manuscript.

___________________________
Name >above the line, type your name<

___________________________
Date >above the line, type the date signed, e.g., March 31, 2013<

iii
 Abstract

[Insert Dissertation Title]. [Insert Name, e.g., Richard Dadier, 2011]: Applied
Dissertation, Nova Southeastern University, Abraham S. Fischler School of Education.
[Insert 4 or 5 ERIC Descriptors, e.g., ERIC Descriptors: Databases, Internet, Media
Selection, Middle Schools, Teacher Education]

The abstract narrative is a structured summary that should include: background,

objectives of the systematic review (or meta-analysis), search strategy and include the
databases used, study selection [inclusion and exclusion] criteria and the search protocol,
participants and interventions evaluated, study appraisal and synthesis methods, main
results, limitations, conclusions, implications of key findings, and systematic review
registration number (if any).

iv
 Table of Contents

Page
Chapter 1: Introduction........................................................................................................1
Xxxxxxxxxxx xx Xxxxxxxxx..................................................................................1
Xxxxxxxx Xxxx Xxxxxx.........................................................................................2
Xxxxxxxx Xxxxx.....................................................................................................4

Chapter 2: Literature Review...............................................................................................6

Xxxxxxx Xxxxxxxxx...............................................................................................7

Xxxxxx Xxxxxxxxxxx...........................................................................................15
Xxxxxx xx xxx Xxxx.............................................................................................24
Xxxxxxxxxxxx xx xxx Xxxxxxx xx xxx Xxxxxxxx.............................................31

Chapter 3: Methodology....................................................................................................38
Xxxxxx...................................................................................................................38
Xxxxxxxx Xxxxxxxx.............................................................................................41
Xxxxxxxxxxx xx Xxxxxxxx..................................................................................43

Chapter 4: Results..............................................................................................................46
Xxxxxxxxxx xxx Xxxxxxxxx xx Xxxxxxxxx.......................................................46
Xxxxxxxxxxx xx Xxxxxxx Xxxxxxxxx................................................................50

Xxxxxx xx Xxxxxx Xxxxxx................................................................................. 53

Chapter 5: Discussion........................................................................................................57
Xxxxxxxx...............................................................................................................58
Xxxxxxxxxx...........................................................................................................64
Xxxxxxxxxxxxxxx.................................................................................................69
Xxxxxxxx xx Xxxx................................................................................................71

References..........................................................................................................................74

Appendices
A Title in Initial Caps and Lower Case—Begin a Second Line Directly Below
the First Line...................................................................................................81
B Title in Initial Caps and Lower Case .............................................................83

Tables
1 Title in Initial Caps and Lower Case..............................................................47
2 Title in Initial Caps and Lower Case..............................................................51

Figures
Title in Initial Caps and Lower Case.....................................................................49

v
 Chapter 1: Introduction

Statement of the Problem

This should include (a) a clear statement that a problem regarding one or more
interventions or intervention strategies or models exist in the research record, (b) evidence
that supports the argument that large numbers of studies and/or conflicting data and
conclusions exist along with the role(s) of the intervention(s) in current practice or policy,
(c) evidence of any trend or change over time that has led to this problem, (d) a description
of the setting in which the problem exists, and (e) probable causes that related to the
problem.

Need for Systematic Review (or Meta-Analysis)

Provide evidence from the literature and experience showing that the problem in the
research record exists and the relevance and perceived justification for conducting the
study. The quantity and quality of existing research data and conclusions should be
discussed within specific context and include assumption, biases, experience, intuitions,
and perceptions related to the belief that inquiry into the research data is important.

Significance of Review
Scholarly Significance – How will the review clarify or resolve differences found
in the research record?
Practical Significance – Who is affected and how will they benefit?

Definition of Terms
Provide complete definitions with regard to the intervention(s) and any included
components and models as well as other terms if necessary. Include as many terms or
variables as needed to clarify terms used in the dissertation.

Definition of Acronyms
Because several studies that are published or unpublished will be evaluated, a large
number of acronyms are often found in the written record. Be sure you identify all
acronyms, including those used and referenced in all extracted studies and tell what each
acronym represents.

Purpose of the Study

The purpose statement should provide a general statement regarding the overall
purpose of the study. Refer to Chapter 4 in Creswell, 4th Edition, 2012 for examples of
qualitative and quantitative purpose statements. Key points to keep in mind when preparing
a purpose statement are as follows:
1. Create a sentence that begins with “The purpose of this study is . .. .” or “The
purposes of this study are….”
2. Clearly identify and define the central concepts and objectives of the study.
3. Identify the critical intervention(s) that are the target of the study.
 2

Chapter 2: Literature Review

Chapter Introduction
Briefly state the purpose of the statement and briefly state what will be presented in
this chapter.

The literature review is a listing and thorough development of the intervention(s) being
addressed in the systematic review [meta-analysis] and may contain the following
elements as appropriate:
(a) historical development and context of all major intervention component, such
as models or types along with their applications and target populations;
(b) discussion of the theoretical or conceptual framework, including learning
theory, upon which the intervention(s) and any models or components is/are based. Since
different models frequently are based on different theoretical frameworks, discuss each;
(c) defining characteristics of the intervention(s) and major components, models
or types;
(d) current practice regarding the various interventions and any models or
components. Include any relevant factors such as typical setting and population served;
(e) detailed study analysis including the numbers and types of quantitative,
qualitative, and mixed methods studies that have been both published and unpublished with
regard to the intervention(s) and any of its components or models;
(f) study discussion of how this research should extend, differ from, or replicate
any past systematic reviews or meta-analyses; and
(g) indicate shortcomings in any prior study, if any, that should be avoided, as well
as strengths to be repeated in conducting this study.

Research Questions
The research questions are formulated to achieve the purpose of the study and
should logically follow from the literature review. Refer to Chapter 4 in Creswell’s (2012)
Educational Research, for formulation of qualitative research questions as well as both
issue-based and procedural sub-questions. The research questions for both systematic
reviews and meta-analysis studies generally follow a qualitative research question format
and adhere to the following guidelines: (a) formation of question or questions based on
theory, previous research found in the literature review, and experience; (b) refer to
participants, interventions, comparisons, outcomes, and study design (PICOS), (c) stated in
the form of a question that cannot be answered with a simple yes/no; (d) answer a “Central
Question” that is the overarching question you explore in the research study; and (e) the
central question is followed by “Sub-questions” that divide the central question into
smaller, specific questions that are used as to guide the evaluation and development of the
data extraction form for the included studies in the review.
As with qualitative studies, sub-questions in these dissertation types consist of both
“issue sub-questions” that refer to issues related to the intervention(s) and “procedural sub-
questions” that are non-issue components of the intervention(s) being extracted and
evaluated. Systematic review and meta-analysis standards require that these questions
address participants, interventions, comparisons, outcomes, and study design (PICOS). See
the PRISMA checklist (Item 4). The PICOS attributes are important because they are the
 3

factors addressed when creating the protocol for screening, and selecting/deselecting
studies for analysis.
Both sub-question types should be framed to support and contribute to the inclusion
and exclusion criteria used in the study selection process detailed in the methodology
section. Also, both of these sub-question types are used to select, modify or develop the
data extraction form(s), for assessing the included studies being reviewed.
 4

Chapter 3: Methodology
Introduction
This section should include a brief overview of the general purpose and outcomes
of the study as well as an overview of what is presented in this chapter.

Introduction
This section should include a brief overview of the general purpose and outcomes
of the study. If an existing protocol is being replicated, this is where it is identified and
where it can be found. If available, provide the existing protocol registration number and if
in either the Cochrane Collaboration or the Campbell Collaboration.

Research Studies Eligibility Criteria

This section lists the characteristics of the studies being reviewed and includes a
complete PICOS description of the thresholds for inclusion, including significance, conduct
of the researchers, peer review, data treatment, and/or other relevant criteria. Justification
for exclusion of studies is detailed and justified in this section. Critical PICOS subsections
include:

Participants. This section should include a complete description of the participants

who would be included in the studies being reviewed. Include the following
elements: (a) the participants defined, consistent with the problem, diagnosis, or
conditions of interest that would qualify them to be included in the studies being
reviewed; and (b) demographic information such as age, gender, ethnicity, setting
or other individual or group characteristics.

Intervention Types. List the types of studies that exist in the studies being
analyzed and provide a brief summary of the intervention, including goals,
theoretical foundations, key techniques or activities, staff qualifications, and
typical frequency, intensity and duration of the intervention(s).

Comparisons. Specify the conditions that will be compared in the treatment

being studied and the nature of the comparisons. State if it in includes
experimental and control/comparison conditions, central comparisons of interest,
restrictions on application of the treatment or dose, frequency of administration,
intensity, or duration of the treatment in question. Contrasts may include a
contrast between one intervention and another. Provide information about
relative effects of the treatments. It may also include contrasts between no-
treatment control groups to provide evidence of absolute effects, including
placebo effects. If the researcher wishes to do both comparisons, each
comparison condition should be kept in a separate analysis.

Outcomes. List the various outcomes measures found in the research studies in
question. These include primary and secondary outcomes, adverse outcomes,
economic data, and/or timing of outcomes assessment due to interventions of
varying duration or follow-up.
 5

Study Design. Certain designs are superior to others for a given question. Since
these studies often address different kinds of questions, the researcher should
identify inclusion criteria that specify the research study design that will be used
to select studies for analysis.
Because most studies in education and the social sciences focus on “what
works”, study designs that are appropriate for intervention effects and causal
inferences should be selected. The literature recommends techniques such as
randomized controlled trials, propensity score matching, Heckman selection
models, differences-in-differences, regression discontinuity, interrupted time-
series, and other appropriate designs. Other designs may be evaluated (e.g., non-
randomized control groups, concurrent control groups, groups of convenience,
pre-test/post-test, etc.), but are more vulnerable to bias and error. For various
reasons, these less rigorous designs may still be worth evaluating in the study.
When including both randomized and non-randomized studies, the Campbell
Collaboration that targets educational and social science reporting expects
researchers to analyze results separately and test the differences with moderator
analysis.

Information Sources
This section fully describes the information sources to be searched and the date last
searched. This includes the databases (a minimum of three) with dates of coverage, contact
with study authors by phone, email or other means to identify additional studies,
conference proceedings, unpublished manuscript sources, and hand-searched journals.

Procedures
This section will be based directly on the research questions (central and sub-
questions). Specifically, this is the “how-to” section of the study and will detail a step-by-
step protocol of how the research record was searched and the methods used in the
selection of studies in the data collection phase resulting in studies extracted for evaluation
and synthesis, based on inclusion and exclusion criteria. (Note: At the proposal stage this
section tells, apriori, exactly how the search will be conducted and how candidate studies
will be tested against the inclusion and exclusion criteria and selected for evaluation and
synthesis. This protocol must be established before the research is conducted. At the final
report stage, this will be reported in past-tense detailing exactly what was done.)

Search Strategies. The intent in a systematic review is the selection of “all”

significant studies, both published and unpublished, that meet the inclusion and
exclusion criteria established in the search protocol. This section should include
search strategies such as the following, as appropriate:
1. Indentify the specific strategy used for electronic searches: bibliographic
databases searched, dates and periods searched, constraints such as by
country, region or language. Either list each database searched in detail in
this section or in a separate table as well as the full step-by-step search
strategy for each in a clear, unambiguous protocol that another researcher
could imitate. Be sure to include a minimum of three relevant databases,
 6

such as ERIC, JSTOR, including advanced search terms and filters, in the
order used, to extract the listing of candidate studies for evaluation.
2. Provide a listing of “gray literature” sources, such as reports, monographs
and conference proceedings and the strategy used for finding these
candidate studies.
3. Provide the titles of any journals that were “hand-searched”.
4. Provide any reference lists that were consulted to find candidate studies.
5. Provide a listing of all Word Wide Web sites searched for candidate
studies, including full URLs for all sites evaluated for candidate study
extraction.
6. Include personal correspondence with fellow researchers or with those
who have conducted potential candidate studies, whether published or
unpublished.
7. Discuss the intended outcome from this type of strategy.
8. Discuss the source of this strategy and why it is appropriate for this study.
9. Identify how the use of this strategy will shape the type of questions
asked, the form of data collection, the steps and data analysis, and the
final narrative.

Study Selection. State the process for study selection. It is recommended that the
selection process is guided by the PRISMA Flow Diagram at the end of this
template. If used, detail the processes in the selection protocol.

Data collection process. This section should include a thorough and exhaustive
description and framework for the data collection, recording and analysis protocol
used to answer the research questions. While writing, keep in mind that the plan
will be flexible. This section should contain the following elements as appropriate:
1. Selection of the studies evaluated: how the selection criteria of inclusion and
exclusion were applied, number of raters involved with regard to the studies for
inclusion, and how disagreements were handled.
2. Data extraction and management: methods used to extract data from the
candidate studies or from investigators in follow-up contact, the data extraction
forms used [see Mental Measurement Yearbook], number of raters involved,
how disagreements were resolved, and methods of processing data to the
extraction form in preparation for data analysis. More about the forms are
detailed in a special section below.
3. Assessment of methodological quality of included studies: methods used,
number or raters, how disagreements were resolved, how results were used in
interpretation of results.
4. Measures of treatment effects: choices of effect size metrics for
a. Dichotomous data (e.g., odds ratio, risk ratio, or risk difference)
b. Continuous data (e.g., weighted mean difference, standardized mean
difference)
c. Time-to-event data if applicable (e.g., hazard rates)
5. Unit of analysis issues: how reviewers handled studies with multiple treatment
or comparison/control groups, crossover trials, or cluster randomized trials.
 7

6. Dealing with missing data on participants or outcomes: attempts to obtain the

missing data from investigators, methods for imputing missing data (if
applicable), intention-to-treat analysis, methods for handling missing statistics
(e.g., means, Standard Deviation).
7. Assessment of heterogeneity: clinical/substantive heterogeneity and statistical
heterogeneity as examples.
8. Assessment of reporting biases: how publication bias and other potential biases
are addressed (e.g., funnel plots, statistical tests, imputation)

Data Items. List and define all variables for which data were sought (e.g., PICOS,
funding sources) and any assumptions and simplifications made.

Risk of Bias. This section should present an assessment of the potential biases
within the selected studies and across multiple studies. This section details how
publication bias, bias in individual studies, bias across studies, and other potential
biases are addressed (e.g., funnel plots, statistical tests, imputation). Discuss the
methods used to assess the risk of bias of individual studies. Indicate whether this
assessment was conducted at the study level or the outcome level and how the
information will be used in any data synthesis in Chapter 4.

Instruments. If an established data extraction form or other instrument was used

then this section will detail each data-collection instrument. The relevant
information pertaining to each instrument should include the source or developers
of the instrument and any other salient information. Discuss modifications and if a
panel of experts were used to validate changes, detail who the experts were and
instrument components added or deleted as a result of the review by the expert
panel.

Quantitative Data Analysis (meta-analysis)

Here you may choose a fixed effects or random effects model.
1. Moderator analysis and investigation of heterogeneity: list of pre-
determined plans for subgroup analysis, meta-regression, etc.
2. Sensitivity analysis: if and how various studies were tested for the
robustness of their conclusions when the study includes some form of
statistical modeling with assumed or predicted data elements.

Qualitative Data Analysis (meta-synthesis)

Here you list the steps involved in conducting a meta-synthesis of qualitative data
extracted from the selected research studies. Describe how the data were organized and
transcribed. Detail the coding procedures and getting qualitative data from the studies to
the data extraction forms. Discuss specific qualitative software you used for your analysis.
Develop a detailed qualitative description. Check the accuracy of your findings and
interpretations and include any of the following: (a) member checking—asking members to
check the accuracy of the account and (b) triangulation—using corroborating evidence.
 8

Limitations of the Study

List the influences that the researcher could not control, including the conditions or
influences that places restrictions on the methodology and conclusions. Any limitations that
might influence the results should be mentioned. Consider all of the following: (a)
incomplete retrieval and analysis of identified research; (b) the nature of self-reporting; (c)
the instruments used; (d) the sample size of the articles extracted for review; (e) all
potential risks for bias; and (f) any time or other constraints. Certain limitations might
mean that the findings cannot be generalized to the larger population. This is especially true
when the extracted studies define a specific population that is very narrowly defined or is
too broad (ex: elderly women).

Delimitations of the Study

Even though the selection criteria are explained in detail in this chapter, here you will
list the choices made by you, the researcher, which set the parameters of the study. This is
the place to explain all of the following: (a) the things that you are not doing (and why you
have chosen not to do them); (b) the studies you will not review (and why not); (c) the
population you are not studying (and why not); (d) the methodological procedures you will
not use (and why you will not use them); and (e) other parameters involving setting,
instrumentation and other variables. Limit your delimitations to the things that a reader
might reasonably expect you to do but that you, for clearly explained reasons, have decided
not to do.
 9

Chapter 4: Results
Chapter Introduction
Give a brief overview statement of the types of studies that were evaluated. Also
give a brief listing of what results this chapter presents (e.g., the study retrieval and
screening process, identification and summary of all studies that met the
inclusion/exclusion criteria, how the studies are categorized, risks of bias, etc.).

Study Retrieval
This major area should use the PRISMA (2009) Flow Diagram at the end of this
template, or other flow diagram if replicating a study, to report the results of the selection
process. Be sure to include the actual number of studies and details of the studies for the
following:
a. identification, including the exact databases and other sources
b. screening methodologies using eligibility and exclusion criteria
c. selected study eligibility criteria (e.g., PICOS, sample size, etc.), giving reasons for
exclusions from the eligible list, and
d. final inclusion numbers and list
Charts reporting included/excluded study details, with their full citations, should be in an
appendix and referenced in this section.

Included Studies Characteristics

For each study in the final selection list, report the characteristics for which data were
extracted to the data extraction form selected. A summary data table that describes each
study should be in an appendix and linked to the summary in this section by study
eligibility criteria (e.g., PICOS, study size, etc.). For every included study, assess each for
risk of bias within the study and present any outcomes assessments that may have been
reported.

Individual Study Results

For each intervention type, model or component being studied, create a separate
study results sub-heading. In each of these sub-headings, for all outcome benefits and
harms evaluated, for each study, present (a) summary data for each intervention group and
(b) effects estimates and confidence intervals. Use a Forest Plot and/or other analysis
technique to assess the sample bias and effect size of each study.

Synthesis of Results
For each meta-analysis of quantitative study types, present the results including an
estimate of average effect using Forest Plots for dichotomous and continuous data, across
studies confidence intervals and measures of consistency. For each meta-synthesis of
qualitative study types, present the results. Any error risk and bias risk that may be
encountered as the various studies should be evaluated (e.g., selective reporting within
studies, publication bias, improper data analysis, improper synthesis, etc.) and reported.

Research Question Results

Based on the research results, answer the central question and all sub-questions.
 10

Chapter 5: Discussion
Introduction
Briefly state the overall purpose of the study including any preconceptions and
ideas as discussed in your introduction. Briefly state how selected studies resulted from an
apriori selection protocol. Tell the reader that this chapter will discuss the implications of
the findings for practitioners, policymakers, and future researchers as well as provide
recommendations for future research.

Major Findings
Summarize each main finding and include the strength of evidence for each main
outcome. This section may include sub-headings that indicate studies that confirm prior
research and scholarship and studies that refute prior research and scholarship. The
findings should be presented in terms of how this will apply to practitioners, policymakers,
and future researchers.

Conclusions
In Meta-Analysis, the goal is to resolve research findings differences and that resolution
should be discussed in detail in this section. In Systematic Reviews, conclusion should be
drawn across study types to help practitioners and policymakers determine “what works” in
a given setting and with a given population.

Recommendations for Future Research

Based on the conclusions and the studies evaluated propose one or more future research
studies that are suggested by the data from this study. Place each of those proposed
projects in terms of possible contributions to practitioners and policymakers.
 11

References
 PRISMA 2009 Flow Diagram

Identification

Records identified through Additional records identified

database searching through other sources
(n = ) (n = )

Records after duplicates removed

Screening

(n = )

Records screened Records excluded

(n = ) (n = )
Eligibility

Full-text articles assessed Full-text articles excluded,

for eligibility with reasons
(n = ) (n = )

Studies included in
qualitative synthesis
(n = )
Included

Studies included in
quantitative synthesis
(meta-analysis)
(n = )
 Reported
Section/topic # Checklist item on page #
TITLE
Title 1 Identify the report as a systematic review, meta-analysis, or both.
ABSTRACT
Structured summary 2 Provide a structured summary including, as applicable: background; objectives; data sources; study eligibility
criteria, participants, and interventions; study appraisal and synthesis methods; results; limitations; conclusions
and implications of key findings; systematic review registration number.
INTRODUCTION
Rationale 3 Describe the rationale for the review in the context of what is already known.
Objectives 4 Provide an explicit statement of questions being addressed with reference to participants, interventions,
comparisons, outcomes, and study design (PICOS).
METHODS
Protocol and 5 Indicate if a review protocol exists, if and where it can be accessed (e.g., Web address), and, if available, provide
registration registration information including registration number.
Eligibility criteria 6 Specify study characteristics (e.g., PICOS, length of follow-up) and report characteristics (e.g., years considered,
language, publication status) used as criteria for eligibility, giving rationale.
Information sources 7 Describe all information sources (e.g., databases with dates of coverage, contact with study authors to identify
additional studies) in the search and date last searched.
Search 8 Present full electronic search strategy for at least one database, including any limits used, such that it could be
repeated.
Study selection 9 State the process for selecting studies (i.e., screening, eligibility, included in systematic review, and, if applicable,
included in the meta-analysis).
Data collection 10 Describe method of data extraction from reports (e.g., piloted forms, independently, in duplicate) and any
process processes for obtaining and confirming data from investigators.
Data items 11 List and define all variables for which data were sought (e.g., PICOS, funding sources) and any assumptions and
simplifications made.
Risk of bias in 12 Describe methods used for assessing risk of bias of individual studies (including specification of whether this was
individual studies done at the study or outcome level), and how this information is to be used in any data synthesis.
Summary measures 13 State the principal summary measures (e.g., risk ratio, difference in means).
Synthesis of results 14 Describe the methods of handling data and combining results of studies, if done, including measures of
consistency (e.g., I2) for each meta-analysis.

Reported
Section/topic # Checklist item on page #
 Risk of bias across 15 Specify any assessment of risk of bias that may affect the cumulative evidence (e.g., publication bias, selective
studies reporting within studies).
Additional analyses 16 Describe methods of additional analyses (e.g., sensitivity or subgroup analyses, meta-regression), if done,
indicating which were pre-specified.
RESULTS
Study selection 17 Give numbers of studies screened, assessed for eligibility, and included in the review, with reasons for exclusions
at each stage, ideally with a flow diagram.
Study characteristics 18 For each study, present characteristics for which data were extracted (e.g., study size, PICOS, follow-up period)
and provide the citations.
Risk of bias within 19 Present data on risk of bias of each study and, if available, any outcome level assessment (see item 12).
studies
Results of individual 20 For all outcomes considered (benefits or harms), present, for each study: (a) simple summary data for each
studies intervention group (b) effect estimates and confidence intervals, ideally with a forest plot.
Synthesis of results 21 Present results of each meta-analysis done, including confidence intervals and measures of consistency.
Risk of bias across 22 Present results of any assessment of risk of bias across studies (see Item 15).
studies
Additional analysis 23 Give results of additional analyses, if done (e.g., sensitivity or subgroup analyses, meta-regression [see Item 16]).

DISCUSSION
Summary of 24 Summarize the main findings including the strength of evidence for each main outcome; consider their relevance
evidence to key groups (e.g., healthcare providers, users, and policy makers).
Limitations 25 Discuss limitations at study and outcome level (e.g., risk of bias), and at review-level (e.g., incomplete retrieval of
identified research, reporting bias).
Conclusions 26 Provide a general interpretation of the results in the context of other evidence, and implications for future
research.
FUNDING
Funding 27 Describe sources of funding for the systematic review and other support (e.g., supply of data); role of funders for
the systematic review.

From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(6):
e1000097. doi:10.1371/journal.pmed1000097 For more information, visit: www.prisma-statement.org.
Page 2 of 2