Schizophrenia Bulletin vol. 46 no. 5 pp.

1072–1085, 2020
doi:10.1093/schbul/sbaa045
Advance Access publication 27 March 2020

What Constitutes Sufficient Evidence for Case Formulation–Driven CBT for

Psychosis? Cumulative Meta-analysis of the Effect on Hallucinations and Delusions

David T. Turner*,1, Simone Burger1, Filip Smit1–3, Lucia R. Valmaggia4, and Mark van der Gaag1,5
1
Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public Health Research Institute, Vrije Universiteit
Amsterdam, Amsterdam, The Netherlands; 2Department of Epidemiology and Biostatistics, Amsterdam Public Health Research
Institute, Amsterdam Medical Centers, Amsterdam, The Netherlands; 3Trimbos Institute, Netherlands Institute of Mental Health,
Utrecht, The Netherlands; 4Department of Psychology, King’s College London, Institute of Psychiatry, Psychology and Neuroscience,
London, UK; 5Parnassia Psychiatric Institute, The Hague, The Netherlands
*To whom correspondence should be addressed; Department of Clinical, Neuro and Developmental Psychology, Amsterdam Public
Health Research Institute, Vrije Universiteit Amsterdam, Van der Boechorststraat 7, 1081 BT Amsterdam, The Netherlands; tel: +31-20-
5985950, fax: +31-20-5988758; e-mail: d.t.turner@vu.nl

Objective: Following 2 decades of research on cognitive hallucinations and delusions demonstrates sufficiency and
behavioral therapy for psychosis (CBTp), it is relevant to stability across comparisons, suggesting limited value of
consider at which point the evidence base is considered new trials evaluating generic CBTp.
sufficient. We completed a cumulative meta-analysis to
assess the sufficiency and stability of the evidence base Key words:  schizophrenia/randomized controlled trials/
for hallucinations and delusions.Method: We updated the psychological intervention/positive symptoms/systematic
systematic search from our previous meta-analytic review review
from August 2013 until December 2019. We identified 20
new randomized controlled trials (RCTs) resulting in in- Introduction
clusion of 35 RCTs comparing CBTp with treatment-as-
usual (TAU) or active controls (AC). We analyzed data It is now approximately 20 years since the evidence base
from participants with psychosis (N = 2407) over 75 con- for cognitive behavioral therapy for psychosis (CBTp)
ventional meta-analytic comparisons. We completed cu- began to accumulate and, as randomized controlled
mulative meta-analyses (including fail-safe ratios) for key trials (RCTs) continue to proliferate, it is relevant to
comparisons. Publication bias, heterogeneity, and risk of consider at which point the evidence base is considered
bias were examined.Results: Cumulative meta-analyses sufficient. Our previous meta-analytic review demon-
demonstrated sufficiency and stability of evidence for strated the efficacy of individually tailored, case formu-
hallucinations and delusions. The fail-safe ratio demon- lation–based CBTp in reducing hallucinations (Hedge’s
strated that the evidence base was sufficient in 2016 for g  =  0.44, P < .005) and delusions (g  =  0.36, P < .05)
hallucinations and 2015 for delusions. In conventional when RCTs were focused on specific symptom reduc-
meta-analyses, CBTp was superior for hallucinations tion.1 These findings were broadly in line with existing
(g = 0.34, P < .01) and delusions (g = 0.37, P < .01) when meta-analytic results for positive symptoms.2,3 We con-
compared with any control. Compared with TAU, CBTp cluded that CBTp was an efficacious intervention for
demonstrated superiority for hallucinations (g  =  0.34, hallucinations and delusions, although the lower mag-
P < .01) and delusions (g  =  0.37, P < .01). Compared nitude of effect for delusions and the absence of a sig-
with AC, CBT was superior for hallucinations (g = 0.34, nificant effect compared with active treatments led us to
P < .01), but not for delusions although this comparison conclude that delusions may be less amenable to change
was underpowered. Sensitivity analyses for case formu- via CBTp than hallucinations.
lation, primary outcome focus, and risk of bias demon- Roughly, 6 years have elapsed since our previous re-
strated increases in effect magnitude for hallucinations.C view. During this time, a number of new RCTs have
onclusions: The evidence base for the effect of CBTp on been published in this research field. These include

© The Author(s) 2020. Published by Oxford University Press on behalf of the Maryland Psychiatric Research Center.
This is an Open Access article distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs licence
(http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial reproduction and distribution of the work, in any medium,
provided the original work is not altered or transformed in any way, and that the work is properly cited. For commercial re-use, please contact journals.
permissions@oup.com
1072
 CBT for Psychosis Cumulative Meta-analysis

trials employing the typical implementation of indi- published reviews alongside our accumulation of newly
vidually case-formulated CBTp in Western mental published trials via automatic update notifications and
health care systems as were prevalent in the former re- expert knowledge via professional networks. We entered
view alongside a range of trials in new settings and/ a relevant range of text variations of the following key
or employing new styles of intervention, eg, culturally search terms via while utilizing Boolean operators, MeSH
adapted CBTp in Pakistan4 or virtual-reality-based terms, and exploded terms and limit setting based on spe-
CBTp.5 There remains well-documented controversy6 cific options within each database: (1) cognitive behavioral
over the effectiveness and implementation of CBTp; therapy, (2) auditory hallucinations OR delusions, and (3)
both the UK National Institute for Health and Care RCTs. Exemplary search strings are included in supplemen-
Excellence7 and the British Psychological Society tary materials.
Understanding Psychosis and Schizophrenia report8 rec-
ommend CBTp, whereas the Cochrane Collaboration Inclusion/Exclusion Criteria
maintain that meta-analytic results are neither clear
nor robust enough to recommend CBTp over standard We included (1) RCTs comparing (2) cognitive behav-
care.9 Recent literature addressing this controversy ar- ioral therapy with (3) treatment-as-usual (TAU) or
gues the importance of attending to methodological an active control condition (eg, supportive counseling
issues including blinding, inclusion criteria and pre- or psychoeducation) for (4) patients diagnosed with
specification of methods.6 schizophrenia-spectrum disorders which (5) assessed
Cumulative meta-analysis is a technique allowing es- hallucinations and/or delusions as post-treatment
timation of both the sufficiency and stability of meta- outcome. Schizophrenia-spectrum disorders included
analytic evidence. This technique was first notably schizophrenia, schizoaffective disorder, delusional dis-
applied to treatment trials for myocardial infarction.10 order, brief psychotic disorder or psychosis not other-
The method has since been applied as a means of sta- wise specified. We included only studies published in
tistically estimating the point at which there is sufficient peer-reviewed journals. Conference abstracts were ex-
evidence to conclude that an intervention is efficacious cluded. We also excluded trials that (1) focused on a
while also estimating the stability of the effect size over primary diagnosis of alcohol or substance use depend-
time.11,12 In light of the further accumulation of trials, ency; (2) included ultra-high risk patients or focused
we concluded that the application of cumulative meta- on prevention of psychosis; and (3) replaced the core
analysis to the CBTp field is warranted. of CBT (ie, identifying and challenging of maladaptive
We first aimed to update our 2014 review to assimi- beliefs) with alternative psychological interventions, eg,
late the new body of research and therefore provide an social skill training or mindfulness. We utilized the def-
up-to-date estimation of the impact of CBTp upon hal- inition of CBTp applied in our previous meta-analytic
lucinations and delusions. We also employed cumulative research.1
meta-analysis to comment on the sufficiency of the ex-
isting evidence base in demonstrating efficacy and the Study Selection
stability of the evidence over time. A secondary objective The PRISMA diagram (figure 1) depicts the study selec-
was to provide a range of sensitivity analyses to allow tion process. Two authors (D.T.  and M.v.d.G.) utilized
more specific estimation of effects under prespecified the Rayyan (rayyan.qcri.org) web application to facilitate
conditions such as individually tailored case formulation, the study selection process. Abstracts were first screened
primary outcome focus, blinded RCTs, and RCTs with for duplicates then relevance before a sample of full text
minimal risk of bias. PDFs were checked against the inclusion and exclusion
criteria. Conflicts in inclusion were resolved via discus-
Methods sion. We attempted to contact the authors of one RCT
due to PSYRATS subscales being unavailable in the man-
We provide a systematic review including both conven- uscript but received no response.14
tional and cumulative meta-analyses based on PRISMA
guidelines.13 A protocol for this review was registered at
the Open Science Framework (https://osf.io/nwxbz/). Data Extraction
Two authors (D.T. and S.B.) independently completed the
data extraction for new trials included since 2013. The
Systematic Search data from trials included in the 2013 review were also
Our previous meta-analytic review in this area completed checked for consistency by both authors, and any incon-
a systematic search on August 3, 2013.1 We repeated the sistencies were investigated and corrected. Spreadsheets
systematic search from this date until December 11, 2019 utilized in the previous meta-analyses were adapted and
across the same 3 databases included in 2013 (PubMed, updated for use in the current review. We contacted one
Embase, and PsychInfo). We considered reference lists of author for unavailable data although on closer inspection
1073
D. T. Turner et al

305 references identified by literature search

2013-2018:
PubMed: 130
PsychInfo: 74
Embase: 101 Articles identified from automatic
notifications and expert knowledge

After removal of duplicates: 259

abstracts
184 articles excluded after
screening title and abstracts

75 publications retrieved for full-

text PDF screening Excluded: 55

Non-relevant outcome measures (12)

Non-relevant intervention (20)
Secondary analyses of RCTs (3)
Non-relevant patient population (3)
20 new CBTp articles included from Non-relevant study design e.g. quasi-
2013-2018 systematic search experimental studies (7)
Duplicates (4)
Conference abstract only (6)

15 CBTp RCTs included from previous

meta-analysis (van der Gaag et al, 2014)

35 CBTp RCTs included in 49 meta-

analytic comparisons in current
review

Fig. 1.  Flowchart of inclusion of studies.

of the manuscript, the intervention in this trial did not in broader subcategories such as positive symptoms (eg,
meet inclusion criteria. Data were extracted on study the PANSS). In instances where 2 hallucinations or 2
characteristics (year of publication, country, sample delusions scales were reported, data from both were ex-
characteristics, format [individual or group], duration, tracted, and an average pooled effect size was calculated.
application of case formulation, primary vs. secondary All scales included were continuous outcomes.
focus and intervention style) and post-treatment outcome
data. Risk of Bias Assessment
To account for risk of bias among the included RCTs, we
Outcome Measures applied an adapted version of the Cochrane Risk of Bias
Although a considerable proportion of meta-analytic tool. The final 2 items of the tool (selective outcome re-
research on cognitive behavioral therapy for psychosis porting and other sources of bias) were omitted due to
(CBTp) has focused on its effect in reducing the posi- limited evidence regarding their impact on validity for
tive or negative symptoms of psychosis,2,3,15 there has meta-analytic comparisons.19 Utilization of the 4 key
been less focus on the more specific, discrete outcomes areas of bias (namely sequence generation, allocation con-
of hallucinations and delusions. It has been suggested cealment, blinding of assessors, and incomplete outcome
that diagnostically based tools such as the Positive and data) provided the opportunity for clear sensitivity ana-
Negative Syndromes Scale (PANSS)16 provide less com- lyses as applied in previous meta-analytic reviews.2,20 Risk
prehensive measurement of psychotic symptomatology of bias was assessed independently by 2 authors (D.T. and
than symptom-specific outcome measures such as the M.v.d.G.). Conflicts were resolved via discussion. Risk of
Psychotic Symptoms Rating Scales (PSYRATS).17,18 bias items were rated low risk (0) or high risk (1), contrib-
Our primary outcomes were therefore hallucinations uting to a total score of 0–4 for each RCT. Items that were
and delusions. We extracted all outcome measures that unclear in the published manuscripts were rated conserv-
reported hallucinations or delusions as independent atively as high risk. Due to an alternative method of risk
scales or subscales. We did not include outcome meas- of bias assessment being employed in the earlier review,
ures that subsumed items on hallucinations or delusions risk of bias was assessed over the whole sample of RCTs.

1074
 CBT for Psychosis Cumulative Meta-analysis

Meta-analyses Rosenthal’s22 standard to estimate the sufficiency of the

Our strategy for analysis was to move gradually from in- evidence base as each RCT was added. The fail-safe ratio
clusive comparisons to more exclusive sensitivity analyses provides an estimate of sufficiency of evidence when the
for a number of criteria based on (1) relevant study char- ratio surpasses a value of 1.0.
acteristics and (2) risk of bias. These sensitivity analyses
were designed to provide information relevant to our Publication Bias, Heterogeneity, and Power
aforementioned research objectives, namely our focus We utilized the Q statistic and the I2 statistic to assess het-
on individually tailored, case formulation–driven CBT erogeneity. We examined publication bias to estimate the
with hallucinations and delusions as primary outcome. potential impact of unpublished RCTs. We applied power
We therefore first analyzed all eligible RCTs each for hal- calculations to estimate the number of RCTs required
lucinations and delusions. We then completed sensitivity for adequate power in each comparison.23 Full informa-
analyses examining TAU only, active controls only, case tion for these procedures is provided in supplementary
formulation only, and primary outcomes only. When materials.
study availability allowed sufficient number of RCTs for
comparison, we also included smaller categories including Results
group CBT only, secondary outcomes only, self-help CBT
only, and virtual-reality CBT (VR-CBT) only. We note Study Selection
that the minimum number of RCTs required for adequate After the automated removal of duplicates, the updated
meta-analytic comparisons is suggested as approximately search resulted in 305 new citations being retrieved for ab-
5.21 Comparisons we reported in this section, which fell stract screening, of which 184 were excluded and 75 full-text
below this 5 RCTs, were therefore provided only for indic- PDFs were retrieved. Following careful matching of exclu-
ative information regarding current best estimates. When sion and inclusion criteria, 20 new studies published since the
possible, based on RCT availability, we also performed previous meta-analysis were included meaning a total of 35
sensitivity analyses including only RCTs with low risk of RCTs were included in this meta-analytic review. The total
bias (one of more items scored on the risk of bias tool) amount of participants measured at post-treatment was
and no known risk of bias (no items scored on the risk of N = 2407, which included 1205 patients who received CBT
bias tool). and 1202 patients who received TAU or an active control.
All meta-analytic comparisons were completed using We analyzed their data over 75 meta-analytic comparisons.
the Comprehensive Meta-analysis (CMA) version 3.3.070 Selected study characteristics are provided in table  1.
computer software package. CMA provides an aggre- Twenty-eight RCTs (80%) applied individually tailored
gated effect size estimating the pooled mean difference case formulation, whereas 9 studies (26%) did not. Thirty-
between treatment and control groups at post-treatment one RCTs (89%) targeted hallucinations and/or delusions
using Hedge’s g, which is an estimate of the standard- as primary outcome, whereas 6 (17%) targeted these out-
ized mean difference between study groups. Hedge’s g is comes as secondary. Twenty-nine RCTs utilized TAU as
recognized as providing a more accurate effect estimation the comparison condition, 6 RCTs compared CBT with
in small samples than alternative methods for continuous active controls or other psychological interventions,
measures such as Cohen’s d. We utilized the .05 alpha level whereas 2 RCTs included both. Active control treatments
for all comparisons with 95% confidence intervals (CIs) included supportive counseling,24–29 psychoeducation,30
provided. We also employed a random-effects model in befriending,17 and virtual-reality exposure.31 Only one
all comparisons due to the expectation of between-study RCT explicitly excluded participants taking antipsychotic
variance. medication from the CBTp treatment group,32 therefore
indicating that CBTp was broadly provided as adjunctive
Cumulative Meta-analysis to standard care.
Risk of bias varied among RCTs although the majority
To assess the sufficiency and stability of meta-analytic
(24 RCTs; 67%) achieved the best possible risk of bias
evidence for CBTp for hallucinations and delusions, we
score on the adapted Cochrane tool. A  further 8 RCTs
completed cumulative meta-analyses for each outcome.
(23%) scored one risk of bias item, whereas 2 RCTs (6%)
The cumulative meta-analysis function in CMA was
scored 2 items, 2 RCTs (6%) scored 3 items, and 1 RCT
utilized. RCTs were listed by year of publication, and a
(3%) scored 4 items, indicating the highest possible score
pooled effect size in Hedge’s g was calculated for the point
on the tool. Risk of bias assessment scores are provided
at which each new study was chronologically added to the
in tabular form in supplementary materials.
evidence base. The cumulative forest plots (supplemen-
tary figures  5–9) provide a visual representation of the
stability of evidence as the RCT evidence based has ac- Effect of CBT on Hallucinations
cumulated. We also followed Muellerleile and Mullen’s11 Table 2 provides an overview of all meta-analytic results
recommendations for calculating the fail-safe ratio from of the effect of CBTp on hallucinations. Supplementary
1075
 Table 1.  Selected Study Characteristics of CBTp RCTs for Hallucinations and Delusions

1076
Experimental Condition Control Condition
Selected
Duration Male Control Con- Bias Risk Outcome
Author Year Format Intervention CBT Format CBTp Age Sex Format trol Age Male Sex Country CF 0–4 Measure
D. T. Turner et al

N Mean (SD) % N Mean (SD) %

Lewis et al 29 2002 Indiv 15–20 h in CBT 101 29.1 71% [1] SC [1] 106 [1] 27.2 [1] 71% UK Y 0 PSYRATS
5 wk [2] TAU [2] 102 [2] 27.0 [2] 68%
Durham et al24 2003 Indiv 9 mo CBT 22 36.0 (10.0) 68% [1] SC [1] 23 [1] 37.0 [1] 65% UK Y 1 PSYRATS
[2] TAU [2] 21 (11.2) [2] 71%
[2] 36.0
(10.2)
Trower et al33 2004 Indiv 6 mo CT CH 18 36.6 (10.3) 56% TAU 20 35.1 (10.4) 70% UK Y 1 PSYRATS
Cather et al30 2005 Indiv 16 weekly fCBT 15 40.4 (12.0) 57% PE 13 40.4 (12.0) 57% USA Y 1 PSYRATS
sessions
Wykes et al34 2005 Group 10 wk CBT 45 39.7 (10.8) 53% TAU 40 39.7 (10.1) 65% UK N PSYRATS
Valmaggia et 2005 Indiv 6 mo CBT 35 35.5 (10.8) 77% SC 23 35.5 (11.4) 61% NL Y 0 PSYRATS
al25
McLeod et al35 2007 Group 8 weekly ses- CBT 10 n.a. n.a. TAU 10 n.a. n.a. UK N 3 PSYRATS
sions
O’Connor et 2007 Indiv 24 weekly CBT 12 40 (9.4) 45% SC 12 36.8 (13.5) 67% CAN Y 3 MADS
al26 sessions
Garety et al36 2008 Indiv 20 in 9 mo CBT 60 H 39.1 (10.3) 71% TAU 60 H 37.1 (10.9) 72% UK Y 0 PSYRATS
85 D 85 D
Penn et al27 2009 Group 12 wk CBT 32 41.7 (11.8) 53% SC 33 39.6 (15.7) 49% USA N 0 PSYRATS
Haddock et 2009 Indiv 17 sessions CBT 38 35.7 (12.5) 86% SC 39 33.9 (9.7) 86% UK Y 0 PSYRATS
al17
Peters et al27 2010 Indiv 6 mo CBT 36 34.0 (9.8) 72% TAU 38 39.6 (10.2) 53% UK Y 2 BAVQ-R
Foster et al37 2010 Indiv 4 wk CBT 9 40.0 (10.0) 58% TAU 11 39.1 (9.2) 58% UK N 2 PSYRATS
Lincoln et al38 2012 Indiv 29 sessions CBT 40 33.2 (10.4) 55% TAU 40 33.1 (10.9) 58% GER Y 0 PDI
Krakvik et al39 2013 Indiv 6 mo 20 ses- CBT 23 35.3 (8.9) 65% TAU 22 37.5 (11.2) 64% NOR Y 1 PSYRATS
sions
Rathod et al40 2013 Indiv 16 weekly CA-CBT 17 31.4 (12.4) 63% TAU 18 35.6 (10.7) 59% UK Y 0 CPRS DHS
sessions
Leff et al41 2013 Indiv 6 weekly ses- Avatar CT 14 n.a. n.a TAU 12 n.a. n.a UK Y 1 PSYRATS
sions
Morrison et 2014 Indiv 9 mo CBT 37 33.0 (13.1) 46% TAU 37 29.7 (12.0) 59% UK Y 0 PSYRATS
al32
Birchwood 2014 Indiv 9 mo CT CH 98 38.8 (12.2) 62% TAU 99 35.9 (11.9) 53% UK Y 0 PSYRATS
et al42
Freeman et 2014 Indiv 6 sessions CBT confi- 15 41.9 (11.5) 73% TAU 15 41.5 (13.1) 60% UK Y 0 PSYRATS
al43 dence
Tarrier et al44 2014 Indiv 24 sessions CBT suicide 25 32.6 (11.7) n.a. TAU 24 37.3 (14.2) n.a. UK Y 0 PSYRATS
Freeman et 2015 Indiv 8 sessions CBT sleep 24 39·6 (11.6) 67% TAU 26 42·2 (13.5) 69% UK N 0 PSYRATS
al45
Naeem et al46 2015 Indiv 16 weekly CBT 53 42.0 (11.6) 17% TAU 49 38.6 (12.0) 13% CAN Y 0 PSYRATS
sessions
Habib et al47 2015 Indiv 10–16 ses- CBT 21 33.5 (10.5) 44% TAU 21 30.2 (6.7) 56% PAK Y 0 PSYRATS
sions
Freeman et 2015 Indiv 8 weekly ses- CBT-W 73 40.9 (10.5) 58% TAU 77 42.1 (12.2) 57% UK Y 0 GPTS and
al45 sions PSYRATS
Waller et al48 2015 Indiv 6 wk CBT-TW 20 39.1 (10.5) 75% TAU 11 43.0 (10.7) 64% UK Y 0 DC, DD,
and DP
 Table 1. Continued

Experimental Condition Control Condition

Selected
Duration Male Control Con- Bias Risk Outcome
Author Year Format Intervention CBT Format CBTp Age Sex Format trol Age Male Sex Country CF 0–4 Measure

N Mean (SD) % N Mean (SD) %

Naeem et al49 2016 Indiv 12–16 ses- CBT-GSH 18 42.0 (11.5) 44% TAU 15 38.6 (12.0) 60% CAN Y 0 PSYRATS
sions
Freeman et 2016 Indiv 1 session VR-CBT 15 42.1 (13.4) 67% Exposure 15 40.6 (14.4) 67% UK Y 0 PSYRATS
al31
Hayward et 2017 Indiv 16 weekly Relating 14 41 (n.p) 43% TAU 15 43 (n.p.) 67% UK Y 0 PSYRATS
al50 sessions therapy
Hazell et al51 2017 Indiv 8 sessions CBT-GSH 14 39.1 (10.2) 29% WL 14 45.9 (13.5) 50% UK N 0 HPSVQ
Gottlieb et 2017 Indiv 10 skill eCBT 19 43.8 (13.2) 47% TAU 18 40.3 (11.7) 78% USA N 1 PSYRATS
al 52 modules
Pot-Kolder 2018 Indiv 16 sessions VR-CBT 58 36.5 (10) 69% TAU 58 39.5 (10) 72% NL Y 0 ESM
et al5
Morrison et 2018 Indiv 9 mo CBT 242 42.2 (10.7) 73% TAU 245 42.8 (10.4) 71% UK Y 0 PSYRATS
al53
Husain et al4 2017 Indiv 12 weekly CBT 18 34.1 (9.55) 78% TAU 18 30.5 (8.15) 55.6% PAK Y 0 PSYRATS
sessions
Craig et al28 2018 Indiv 12 weekly Avatar CT 75 42.5 (10.7) 76% SC 75 42.9 (11.2) 60% UK Y 0 PSYRATS
sessions
Wong et al54 2019 Group 7 weekly CBT 25 30.6 (10.6) 24% PE 23 35.1 (12.9) 48% HK N 1 PSYRATS,
session + BAVQ
booster

Note: BAVQ-R, Beliefs About Voices Questionnaire-Revised; CA-CBT, culturally adapted CBT; CAN, Canada; CBT, cognitive behavioral therapy; CBT-GSH, cognitive-
behavioral guided self-help; CBT-I, cognitive behavioral therapy for insomnia; CBT-TW, “Thinking Well” cognitive-behavioral therapy; CBT-W, cognitive-behavioral therapy
for worry; CH, command hallucinations; CPRS, Comprehensive Psychopathological Rating Scale; CT, cognitive therapy; D, delusions; DC, delusional conviction; DD, delu-
sional distress; DHS, delusions and hallucinations scale; DP, delusional preoccupation; eCBT, online cognitive-behavioral therapy; ESM, experience sampling method; fCBT,
functional cognitive behavioral therapy; GER, Germany; GPTS, Green et al Paranoid Thoughts Scale; H, hallucinations; HK, Hong Kong; HPSVQ, Hamilton Program for
Schizophrenia Voices Questionnaire; KOR, SK, South Korea; MADS, Maudsley Assessment of Delusions Scale; n.a., not applicable; n.p., not provided; NL, Netherlands;
NOR, Norway; PAK, Pakistan; PDI, Peters et al Delusion Inventory; PE, psychoeducation; PSYRATS, Psychotic Symptoms Rating Scales; SC, supportive counseling; TAU,
treatment-as-usual; VR-CBT, virtual-reality-based cognitive behavioral therapy; WL, waiting list. BAVQ comparisons included only resistance, omnipotence, and malevolence
subscales.

1077
CBT for Psychosis Cumulative Meta-analysis
D. T. Turner et al

Table 2.  Effect Sizes of CBTp for Auditory Hallucinations

N g 95% CI Z Q-Value I2 (%)

Main comparison with all eligible RCTs

  Any risk of bias score included 28 0.34** 0.20, 0.49 4.63 52.83** 49
  High risk of bias (>1)a 26 0.34** 0.19, 0.49 4.43 51.12** 51
  Lowest risk of bias (0)b 19 0.40** 0.22, 0.58 4.40 41.49** 59
CBTp vs TAU
  Any risk of bias score included 22 0.35** 0.18, 0.52 4.00 45.94** 54
  High risk of bias (>1)a 20 0.34** 0.17, 0.52 3.77 44.24** 58
  Lowest risk of bias (0)b 14 0.41** 0.19, 0.63 3.65 36.90** 65
CBTp vs active intervention
  Any risk of bias score included 8 0.34** 0.15, 0.53 3.58 7.03 0
  High risk of bias (>1)a 8 0.34** 0.15, 0.53 3.58 7.03 0
  Lowest risk of bias (0)b 5 0.42** 0.20, 0.64 3.70 4.15 4
CBTp with hallucinations as primary outcomec
  Any risk of bias score included 23 0.40** 0.24, 0.56 4.90 40.42* 46
  High risk of bias (>1)a 21 0.40** 0.23, 0.57 4.66 38.84** 49
  Lowest risk of bias (0)b 14 0.51** 0.32, 0.70 5.22 25.67* 49
CBTp with individualized case formulationd
  Any risk of bias score included 21 0.41** 0.25, 0.57 5.03 39.86* 50
  High risk of bias (>1)a 20 0.42** 0.26, 0.59 5.02 39.44* 52
  Lowest risk of bias (0)b 15 0.45** 0.25, 0.65 4.47 39.98** 62
CBTp with individualized CF + primary outcomec,d
  Any risk of bias score included 16 0.51** 0.34, 0.68 5.99 23.15* 35
  High risk of bias (>1)a 15 0.53** 0.36, 0.70 6.01 22.24 37
  Lowest risk of bias (0)b 11 0.59** 0.39, 0.80 5.73 18.64* 46
Blinded RCTs onlye
  All eligible CBTp RCTs 24 0.36** 0.20, 0.51 4.48 49.10** 53
  Case formulation only 19 0.43** 0.26, 0.61 4.95 39.17** 54
  Case formulation + primary outcomec,d 14 0.55** 0.37, 0.73 5.99 21.36 39
Additional analyses
  Group CBTp 4 0.11 −0.18, 0.41 0.76 3.15 5
  Hallucinations as secondary outcome 5 0.05 −0.15, 0.24 0.46 3.87 0
  Virtual-reality CBTp 2 0.56** 0.22, 0.89 3.27 0.75 0
  Self-help CBTp 3 0.47 −0.42, 1.37 1.03 9.26** 78
After removal of 2 outliers
  Any risk of bias score included 26 0.27** 0.15, 0.40 4.37 34.35 27
  High risk of bias (>1)a 24 0.27** 0.14, 0.39 4.16 32.52 29
  Lowest risk of bias (0)b 16 0.31** 0.16, 0.46 4.05 24.21 39
  Case formulation only 19 0.32** 0.19, 0.45 4.91 23.01 22
  Case formulation + primary outcomec,d 14 0.41** 0.29, 0.54 6.56 9.64 0
  Case formulation, primary outcome + RoBa–c 9 0.44** 0.31, 0.58 6.46 6.11 0
Excluding RCTs with high ratio nonschizophrenia spectrum
  Any risk of bias score included 26 0.32** 0.17, 0.47 4.23 49.46** 50
  Lowest risk of bias (0)b 16 0.38** 0.19, 0.57 3.92 38.78** 61
  Case formulation + primary outcomec,d 16 0.47** 0.30, 0.64 5.28 24.99 40
  Case formulation, primary outcome + RoBa–c 10 0.58** 0.37, 0.79 5.35 17.58* 49

Note: All comparisons were using random model. Risk of bias scores refer to assessment using adapted version of the Cochrane Risk of
Bias tool (0–4). CBTp, cognitive behavioral therapy for psychosis; CF, case formulation; CI, confidence interval; g, Hedges’s g; n/a, not
applicable; RCT, randomized controlled trial; RoB, risk of bias; TAU, treatment-as-usual.
Sensitivity analysis exclusions were as follows:
a
Risk of bias score greater than 1 excluded: McLeod et al (2007); Peters et al (2010).
b
Risk of bias score greater than 0 excluded: Cather et al (2005); Durham et al (2003); Gottlieb et al (2017); Krakvik et al (2013); Leff et al
(2013); McLeod et al (2007). Peters et al (2010); Trouwer et al (2004); Wykes et al (2005).
c
Hallucinations as primary outcome only: Birchwood et al (2014); Garety et al (2008); Haddock et al (2009); Tarrier et al (2014); Trouwer
et al (2004).
d
Case formulation only: Freeman et al (2015); Gottlieb et al (2017); Hazell et al (2017); McLeod et al (2007); Penn et al (2009); Wykes
et al (2005).
e
Nonblinded RCTs excluded: Krakvik et al (2013); McLeod et al (2007); Peters et al (2010).
*P < .05. **P < .01.

1078
 CBT for Psychosis Cumulative Meta-analysis

figures 3 and 4 provide a forest plot with all eligible RCTs present with the least risky RCTs (g  =  0.32, P < .01).
included. When analyzing this broad sample of all 28 el- When comparing CBT with active controls, CBT did not
igible RCTs for hallucinations, results demonstrated su- demonstrate significant superiority when including all el-
periority of CBT over controls (g = 0.34, P < .01). When igible RCTs (g = 0.23, P = .16) and when including only
including only RCTs with the lowest possible risk of the lowest risk RCTs (g = 0.30, P = .28).
bias scores (n = 19), we observed a marginal but statis- A similar pattern was present when including only
tically nonsignificant increase in the magnitude of effect RCTs with delusions as the primary outcome target; the
(g  =  0.40, P < .01). Similarly, when including all RCTs magnitude of the significant effect in favor of CBT was
comparing CBT against TAU, there was a significant ef- highest when all eligible RCTs were included (g  =  0.38,
fect favoring CBT (g = 0.35, P < .01), which had a mar- P < .01) and when including only the lowest risk RCTs
ginal, nonsignificant increase when including only those (g = 0.34, P < .01). When including only RCTs with in-
RCTs with the lowest assessed risk (n = 14; g = 0.41, P dividually tailored case formulation, the effect size was
< .01). The same pattern was observed when comparing consistent for the all eligible RCT comparison (g = 0.37,
CBT with active controls; CBT demonstrated superiority P < .01) and when including only the lowest risk RCTs
when all eligible RCTs were included (g = 0.34, P < .01), (g = 0.37, P < .01).
while including only the lowest risk RCTs resulted in a When only blinded trials were included, CBT was su-
small and statistically nonsignificant increase in effect perior to any control (g = 0.31, P < .01), which was con-
magnitude (n = 5; g = 0.42, P < .01). sistent when limiting to case formulation RCTs (g = 0.35,
We observed the same pattern when including only P < .01) and RCTs with case formulation and delusions
RCTs with hallucinations as the primary outcome target. as primary outcome (g = 0.34, P < .01).
When including all such RCTs, CBT demonstrated supe- A similar pattern was observed in the most stringent
riority over control (g = 0.40, P < .02) and increased when comparison, which included only RCTs applying indi-
including only the lowest risk RCTs (n = 14; g = 0.51, P vidualized case formulation with delusions as the primary
< .02). Similarly, when analyzing the impact of CBT with outcome target. The effect favoring CBT was of highest
individually tailored case formulation vs controls we ob- magnitude when all eligible RCTs were included (g = 0.38,
served a significant effect when all eligible RCTs were in- P < .01), whereas the effect was marginally lower when ex-
cluded (g = 0.41, P < .01), and when including only the cluding RCTs with a high risk of bias (g = 0.37, P < .01)
lowest risk RCTs (n = 15; g = 0.45, p < .01). and RCTs with the lowest risk of bias (g = 0.37, P < .01).
When including only blinded RCTs, CBT was superior
to any control (g = 0.36, P < .01), when including only Heterogeneity
blinded case formulation RCTs (n  =  19; g  =  0.43, P <
.01) and when limiting to RCTs, which applied blinded There was a significant degree of heterogeneity present in
case formulation and hallucinations as primary outcome the majority of comparisons. For hallucinations, the de-
(n = 14; g = 0.55, P < .01). gree of significant heterogeneity ranged from 37% to 65%,
When performing the most stringent comparison— indicating the existence of heterogeneity primarily within
namely including only RCTs, which utilized case formu- the moderate range across comparisons. Heterogeneity
lation alongside targeting hallucinations as the primary was lower in comparisons including RCTs, which utilized
outcome—we again observed the same pattern of individualized case formulation and also targeted hallu-
increasing magnitude with bias reduction. The effect sizes cinations as primary outcome focus. Heterogeneity in the
demonstrated superiority for CBT when including all eli- delusions comparisons was overall higher, ranging from
gible RCTs (g = 0.51, P < .01) and the lowest bias risk RCTs 39% to 75% and therefore indicating moderate to high
(n = 11; g = 0.59, P < .05; see supplementary figure 5). heterogeneity. The sensitivity analyses for case formula-
tion and primary outcome in the delusions category did
not display a pattern of lower heterogeneity.
Effect of CBT on Delusions
Table 3 provides the results from all meta-analytic com-
parisons of CBT for delusions, whereas supplementary Publication Bias
figure 4 provides a forest plot for all eligible RCTs. When The examination of funnel plots identified the possibility
including all eligible RCTs, CBT demonstrated superi- of unpublished negative studies across both symptom
ority over controls (g = 0.37, P < .01). There was a mar- domains. For hallucinations, when all eligible RCTs
ginal, nonsignificant reduction in the magnitude of this were included, there was an estimation that 4 unpub-
effect when including only the lowest risk RCTs (g = 0.34, lished negative trials may exist. Duval and Tweedie’s54
P < .01). When including only comparisons against trim and fill procedure provided an adjusted effect size
TAU, CBT demonstrated superiority against TAU when by removing 5 RCTs. This procedure reduced the magni-
including all eligible RCTs (g  =  0.36, P < .01), while a tude of the effect favoring CBT, but the effect remained
similar pattern of a small reduction of magnitude was significant (g  =  0.24, 95% CI: 0.15–0.33). Egger’s55 test

1079
D. T. Turner et al

Table 3.  Effect Sizes of CBTp for Delusions

N g 95% CI Z Q-Value I2 (%)

Main comparison with all eligible RCTs

  Any risk of bias score included 27 0.37** 0.23, 0.52 4.95 54.54** 53
  High risk of bias (>1)a 25 0.36** 0.20, 0.10 4.64 53.34** 55
  Lowest risk of bias (0)b 18 0.34** 0.17, 0.50 4.02 39.33** 57
CBTp vs TAU
  Any risk of bias score included 22 0.36** 0.20, 0.52 4.34 48.96** 57
  High risk of bias (>1)a 21 0.35** 0.18, 0.51 4.15 46.85** 57
  Lowest risk of bias (0)b 16 0.32** 0.15, 0.49 3.64 34.42** 56
CBTp vs active intervention
  Any risk of bias score included 7 0.23 −0.19, 0.55 1.41 12.51 52
  High risk of bias (>1)a 6 0.20 −0.45, 0.55 1.14 11.76 57
  Lowest risk of bias (0)b 3 0.30 −0.25, 0.85 1.07 7.85* 75
CBTp with delusions as primary outcomec
  Any risk of bias score included 23 0.38** 0.22, 0.54 4.56 47.94** 54
  High risk of bias (>1)a 21 0.36** 0.19, 0.53 4.21 45.83** 56
  Lowest risk of bias (0)b 14 0.34** 0.15, 0.52 3.51 32.01** 59
CBTp with individualized case formulationd
  Any risk of bias score included 21 0.37** 0.20, 0.54 4.33 49.43** 60
  High risk of bias (>1)a 20 0.37** 0.20, 0.54 4.19 49.10** 61
  Lowest risk of bias (0)b 16 0.37** 0.19, 0.55 4.00 38.09** 61
CBTp with individualized CF + primary outcomec,d
  Any risk of bias score included 17 0.38** 0.19, 0.57 3.86 41.93** 62
  High risk of bias (>1)a 16 0.37** 0.18, 0.57 3.70 41.60** 64
  Lowest risk of bias (0)b 12 0.37** 0.67, 0.58 3.48 30.68** 64
Blinded RCTs onlye
  All eligible CBTp RCTs 22 0.31** 0.16, 0.47 3.96 46.77** 55
  Case formulation only 19 0.35** 0.17, 0.52 3.90 45.44** 60
  Case formulation + primary outcomec,d 15 0.34** 0.14, 0.54 3.37 38.11** 63
Additional analyses
  Group CBTp 2 0.35 −0.02, 0.72 1.84 0.74 0
  Delusions as secondary outcome 4 0.36 −0.06, 0.78 1.69 7,15 58
  Virtual-reality CBTp 2 0.56** 0.24, 0.89 3.36 0.86 0
After removal of 1 outlier
  Any risk of bias score included 26 0.32** 0.19, 0.46 4.71 41.30* 39
  High risk of bias (>1)a 24 0.31** 0.17, 0.44 4.41 38.72* 41
  Lowest risk of bias (0)b 17 0.26** 0.13, 0.40 3.81 23.96 33
  Case formulation only 20 0.31** 0.16, 0.47 4.07 34.90* 46
  Case formulation + primary outcomec,d 16 0.31** 0.14, 0.498 3.59 27.72* 46
  Case formulation, primary outcome + RoBa–c 11 0.28** 0.11, 0.45 3.26 15.99 37

Note: All comparisons were using random model. Risk of bias scores refer to assessment using adapted version of the Cochrane Risk of
Bias tool (0–4). CBTp, cognitive behavioral therapy for psychosis; CF, case formulation; CI, confidence interval; g, Hedges’s g; n/a, not
applicable; RCT, randomized controlled trial; RoB, risk of bias; TAU, treatment-as-usual.
Sensitivity analysis exclusions were as follows:
a
Risk of bias score greater than 1 excluded: Foster et al (2010); O’Connor et al (2007).
b
Risk of bias score greater than 0 excluded: Cather et al (2005); Durham et al (2003); Freeman et al (2016); Gottlieb et al (2017); Krakvik
et al (2013); Foster et al (2010); O’Connor et al (2017); Waller et al (2015).
c
Hallucinations as primary outcome only: Freeman et al (2014); Garety et al (2008); Haddock et al (2009); Tarrier et al (2014).
d
Case formulation only: Foster et al (2010); Freeman et al (2015); Gottlieb et al (2017); Penn et al (2009); Waller et al (2015).
e
Nonblinded RCTs excluded: Foster et al (2010); Freeman et al (2016); Krakvik et al (2013); O’Connor et al (2007); Waller et al (2015).
*P < .05. **P < .01.

of the intercept was not significant, whereas the classic was significant on this comparison, whereas the classic
fail-safe N estimate that 291 unpublished studies would fail-safe N suggested that it would require 335 missing
have to exist to bring the P-value above the alpha level of studies to bring the P-value to above the .05 alpha level.
.05. For delusions, the funnel plot estimated the existence
of 8 unpublished trials. The trim and fill procedure pro-
vided an adjusted effect removing 7 RCTs, which again Post Hoc Investigation of Outliers
remained significant although had reduced magnitude We identified significant heterogeneity across a high
(g = 0.18, 95% CI: 0.09–0.26). Egger’s test of the intercept proportion of comparisons for auditory hallucinations

1080
 CBT for Psychosis Cumulative Meta-analysis

that was not observed in the previous review. We there- presented in table 2 and show only marginal changes to
fore examined forest plots to identify primary studies effect sizes.
as potential outliers contributing to high heterogeneity.
Examination of supplementary figure 3 suggested that Post Hoc Case Formulation Head-to-Head Comparison
the trial by Habib et  al47 was a significant outlier be-
cause its 95% CI did not overlap with that of the pooled We completed a direct comparison of the RCTs including
effect size. The effect from one RCT by Naeem et al49 CBTp case formulation vs those without. In the delusions
was also identified as a potential outlier. We therefore analysis, CBTp demonstrated significant effects of sim-
assessed heterogeneity when excluding both outliers ilar magnitude for case formulation trials (g = 0.38, P <
in an exploratory sensitivity analysis. Excluding both .01) and noncase formulation trials (g = 0.35, P < .05). In
RCTs reduced the heterogeneity in the comparison in- the hallucinations analysis, CBTp demonstrated a signif-
cluding all eligible RCTs below the alpha .05 level to icant effect for case formulation trials (g = 0.40, P < .5),
I = 27% (Q = 33.35, P = .10). Heterogeneity was grad- but not for noncase formulation trials (g = 0.10, P = .51).
ually reduced in subsequent sensitivity analyses and
was observed as 0% in the most stringent and homoge- Cumulative Meta-analysis
nous group of RCTs (case formulation, hallucinations Figure 2 depicts the cumulative forest plots for both the
as primary outcome, and minimal bias risk). We also CBTp for hallucinations and delusions comparisons
investigated the possible impact of the outliers on the when including all eligible RCTs. This figure demon-
magnitude of effects. We observed nonsignificant re- strates the stability of the effect size over time. Table  4
duction in the effect magnitude across categories al- (Supplementary materials) provides fail-safe ratio cal-
though the pattern of marginally increasing magnitude culations for all 4 cumulative meta-analyses; namely the
following stricter sensitivity analyses was maintained. main analysis comparisons for both hallucinations and
Results from outlier exclusion for hallucinations are delusions when including all eligible RCTs alongside the
reported in table 2. most stringent sensitivity analysis when including only
Similar examination of CIs in supplementary figure 4 RCTs that scored zero on the risk of bias assessment,
identified the effect size from the Naeem et al46 trial in de- utilized individualized case formulation and had primary
lusions as an outlier. We therefore completed the same set outcome focus. More extensive figures for all cumulative
of sensitivity analyses when excluding this RCT. Results meta-analyses including all relevant data are available in
demonstrated that heterogeneity was broadly reduced; in supplementary figures 6–9.
some comparisons to the extent that heterogeneity was For hallucinations, the 1.0 level of the fail-safe ratio
no longer significant. There were also marginal and sta- demonstrating sufficiency was surpassed in 2016, which
tistically insignificant reductions in the effect size. Results was consistent in the sensitivity analysis. For delusions,
from outlier exclusion for delusions are reported in table 3. the 1.0 level was surpassed in 2015 for the main analysis
and in 2017 for the sensitivity analysis. Cumulative forest
Post Hoc Sensitivity Analyses plots for each of the remaining 3 comparisons are in-
The length of treatment varied considerably between cluded in supplementary materials and demonstrate sta-
RCTs; the shortest treatment was a single session of bility of the effect size.
VR-CBTp (Freeman et  al, 2016), whereas the longest
CBTp treatments lasted 9 months. To investigate the im- Discussion
pact of this variation, we completed 2 further post hoc
analyses: first, a sensitivity analysis excluding the shortest Cumulative Meta-analysis: Sufficient and Stable
treatment in the delusions comparison and second a This cumulative meta-analysis allowed us to demonstrate
meta-regression investigating the impact of treatment that the existing evidence base for the effect of CBTp on
length on effects for both hallucinations and delusions. hallucinations and delusions is both statistically stable
The results of the sensitivity analyses are reported in ta- and sufficient according to Muellerleile and Mullen’s11
bles  2 and 3. Removal of the shortest RCT resulted in guidelines. A  notable demonstration of the stability of
only marginal changes to effect sizes. The meta-analysis the evidence base is that the addition of a large trial with
showed that the number of CBTp session participants a null finding53 had only a marginal impact on the ef-
received did not have a significant impact on the effect fect size (g  =  0.358 to g  =  0.351 for hallucinations and
for hallucinations (P = .88) or delusions (P = .63). These g = 0.383 to g = 0.363 for delusions). The evidence base
findings were consistent when controlling for risk of bias. for hallucinations has been sufficient since 2016, after
We also conducted a sensitivity analyses when removing which another 6 RCTs were added. Similarly, our review
2 RCTs with a higher proportion of participants with suggests sufficiency of evidence for delusions from 2015
psychosis diagnosed with nonschizophrenia-spectrum after which point 6 RCTs have also contributed data.
disorders.38,39 Results of these sensitivity analyses are Our findings suggest that further RCTs repeatedly testing

1081
D. T. Turner et al

Fig. 2.  Cumulative meta-analysis forest plots for CBTp for (a) hallucinations and (b) delusions, all eligible RCTs.

CBTp are unlikely to have a significant impact on the with a higher risk of bias. Effect sizes in delusions com-
magnitude or significance of treatment effects or to alter parisons remained in the region of g = 0.32–0.38 for all
our conclusions in any substantive way, although we note main comparisons with the exception of the nonsignificant
that in conventional meta-analysis CBTp did not dem- comparisons against active treatments. It should be noted
onstrate superiority for delusions compared with active that this category was comparatively underpowered and
controls in the context of low power. that the sensitivity analysis that included only RCTs with
the lowest bias risk provided a significant effect of a sim-
ilar magnitude (g = 0.3, P < .05). Despite the finding that
Conventional Meta-analysis CBTp was not superior to active control treatments for
The conventional meta-analytic comparisons in this re- delusions, because CBTp for delusions was demonstrated
view provided broadly similar results to our earlier review1 as meta-analytically effective overall, while the active
despite adding 19 RCTs published during the 6  years control conditions have no meta-analytical evidence, we
elapsed since the previous systematic search. There were suggest that CBTp for delusions continues to be recom-
however notable differences in some comparisons. For mended until evidence for other treatments emerges.
hallucinations, when including only RCTs utilizing both Our head-to-head comparison of case formulation–
case formulation and primary outcome focus, the effect driven CBTp compared with that without also suggests that
size increased to g = 0.6 when controlling for risk of bias. case formulation–driven CBTp is more effective in reducing
However, after removing 2 outliers, this effect shrank to hallucinations, whereas no difference was evident in the
g = 0.44, which is consistent with our 2014 review. We ob- effects for delusions. We note that there were significantly
served a broadly consistent pattern across comparisons more RCTs in the case formulation arm and therefore lower
for hallucinations; when risk of bias was minimized and power in the noncase formulation arm, although the lower
when including only case formulation and primary out- effect magnitude for nonformulation-based CBTp for hallu-
come focus, the magnitude of effects increased margin- cinations is still indicative of potential inferiority. We note a
ally but not significantly. Effects remained in the range of recent secondary analysis57 of one RCT included in our re-
g = 0.3 to g = 0.6. The facility to examine risk of bias in view,32 which failed to find a significant effect of case formu-
this specific form of sensitivity analysis was not included lation on outcome. This study also reported a nonsignificant
in the previous review; therefore, this finding, alongside trend of poorer treatment outcome for case formulation
the broad consistency of results in the hallucinations do- participants. Our findings are on a meta-analytic level in-
main, further suggests robust evidence of the impact of dicative that case formulation is more beneficial for hallu-
targeted, formulation-driven CBTp for hallucinations. cinations, although definitive comment awaits more RCTs
The effects of CBTp on delusions were of similar mag- becoming available in the noncase formulation arm. Because
nitude to those for auditory hallucinations when including many novel CBTp applications adhere less to the traditional
all eligible trials, although did not display the pattern of formulation-based treatment approach, further pooling and
marginally increasing magnitude when excluding RCTs comparison of this developing dichotomy is warranted.
1082
 CBT for Psychosis Cumulative Meta-analysis

Limitations with the hobbyist pursuit of postage stamps. There is how-

A notable limitation in this meta-analytic review was sig- ever the possibility that individual-participant data meta-
nificant heterogeneity across a high proportion of the analysis techniques may be applied by combining the original
comparisons. Significant heterogeneity was present only databases of CBTp RCTs to provide more precise estimation
in comparisons for delusions in the previous 2014 re- of effects and the examination of moderating variables (eg,
view; no hallucinations comparison in the original review demographic or clinical characteristics) on specific halluci-
demonstrated significant heterogeneity. Post hoc investi- nation and delusion outcomes. Due to the identification of
gation established that heterogeneity introduced to the potential publication bias, we also encourage any researchers
hallucinations comparisons was largely attributable to 2 contributing to the “file drawer problem” to publish any rel-
outliers, one of which adapted CBTp for application in evant trials, which are not yet available in the public sphere
other cultural settings,47 while the other applied group- for meta-analytic comparison. Future research may also
based self-help CBTp.49 Similarly, another RCT of cul- focus further on the intricacies of the relationship between
turally adapted CBTp contributed to heterogeneity in the CBTp and antipsychotics; despite the demonstrated suffi-
delusions comparisons.58 Our earlier review conceptual- ciency of evidence for CBTp, it remains to date investigated
ized case formulation–driven CBTp RCTs as “apples” in primarily as an adjunctive treatment.61 Finally, although
comparison to “oranges”; a broader and more inclusive interesting findings such as the pattern of increasing effect
sample of RCTs applying CBTp principles in alternative magnitude when primary outcome focus or case formulation
style. We may therefore consider the newer, less homog- are applied, definitive comment on the effectiveness of spe-
enous CBTp trials and interventions again as such “or- cific CBTp components awaits detailed dismantling studies.
anges.” The development of such novel approaches and There may therefore be opportunity to apply the developing
application across wider settings is of importance in the factorial design principles of intervention optimization re-
CBTp field; therefore, we expect further such heteroge- search to the psychosis field.62
neity in future reviews. We also acknowledge that a number
of comparisons in our review—namely those examining Conclusions
novel interventions and those comparing CBTp to active This meta-analytic review further demonstrates the effi-
interventions—were underpowered. Low power there- cacy of CBTp for auditory hallucinations and delusions
fore means that there exists potential for Type 2 error in and suggests that the evidence base is now both sufficient
missing effects that do exist. We also acknowledge the and stable. The robust performance of the effect on hallu-
limitation of our narrow focus relying only on pre–post cinations in sensitivity analyses supports the notion that
change, meaning that we cannot report on enduring ef- CBTp is particularly effective in this domain, whereas
fects at longer-term follow-up. Our focus on the specific heterogeneity and potential publication bias are issues
hallucination and delusion outcomes also meant that that should be carefully examined in future reviews as
other important outcomes such as relapse, functioning, further research becomes available for inclusion.
or level of distress were not considered, while focusing
on schizophrenia-spectrum diagnoses also excludes many
experiencing psychosis as a symptom of other diagnoses Supplementary Material
such as bipolar disorder and substance use disorders.59,60
Supplementary data are found at Schizophrenia Bulletin
online.
Future Research
Our cumulative meta-analysis suggests that there is little value
in researchers repeatedly testing conventional, formulation- Funding
driven CBTp in further RCTs; because the evidence base has None.
demonstrated sufficiency, resources may better be directed
toward novel approaches. The question of whether CBTp
“works” is no longer central, while previous disputes appear Acknowledgments
to have been settled.6 Further development of RCTs exam- We acknowledge Professor Pim Cuijpers for supporting
ining novel approaches such as culturally adapted CBTp this project. The authors have declared that there are no
and VR-CBTp will allow clearer conclusion on their efficacy conflicts of interest in relation to the subject of this study.
via increased power in meta-analysis including only these
interventions. We also note that RCTs examining novel ap-
proaches typically provide briefer interventions, although References
our post hoc analysis did not suggest a significant impact
1. van der Gaag M, Valmaggia LR, Smit F. The effects of in-
of treatment duration on outcome. Our results also suggest dividually tailored formulation-based cognitive behavioural
that there may be limited value in “collecting” further con- therapy in auditory hallucinations and delusions: a meta-
ventional meta-analyses, which Murray58 notably compared analysis. Schizophr Res. 2014;156(1):30–37.

1083
D. T. Turner et al

2. Turner  DT, van  der  Gaag  M, Karyotaki  E, Cuijpers  P. 18. Steel C, Garety PA, Freeman D, et al. The multidimensional
Psychological interventions for psychosis: a meta-analysis measurement of the positive symptoms of psychosis. Int J
of comparative outcome studies. Am J Psychiatry. Methods Psychiatr Res. 2007;16(2):88–96.
2014;171(5):523–538. 19. Higgins  JPT. Measuring inconsistency in meta-analyses.
3. Jauhar  S, McKenna  PJ, Radua  J, Fung  E, Salvador  R, BMJ. 2003;327(7414):557–560.
Laws  KR. Cognitive-behavioural therapy for the symp- 20. Turner  DT, McGlanaghy  E, Cuijpers  P, van  der  Gaag  M,
toms of schizophrenia: systematic review and meta-analysis Karyotaki  E, MacBeth  A. A meta-analysis of social skills
with examination of potential bias. Br J Psychiatry. training and related interventions for psychosis. Schizophr
2014;204(1):20–29. Bull. 2018;44(3):475–491.
4. Husain MO, Chaudhry IB, Mehmood N, et al. Pilot random- 21. Jackson  D, Turner  R. Power analysis for random-effects
ised controlled trial of culturally adapted cognitive behavior meta-analysis. Res Synth Methods. 2017;8(3):290–302.
therapy for psychosis (CaCBTp) in Pakistan. BMC Health
Serv Res. 2017;17(1):808. 22. Rosenthal R. The file drawer problem and tolerance for null
results. Psychol Bull. 1979;86(3):638–641.
5. Pot-Kolder RMCA, Geraets CNW, Veling W, et al. Virtual-
reality-based cognitive behavioural therapy versus waiting list 23. Cuijpers  P. Meta-analyses in Mental Health Research – A
control for paranoid ideation and social avoidance in patients Practical Guide. Vol. 15; Amsterdam, Netherlands: Pim
with psychotic disorders: a single-blind randomised con- Cuijpers Uitgeverij; 2016. doi:10.1016/j.clinthera.2009.11.030.
trolled trial. Lancet Psychiatry. 2018;5(3):217–226. 24. Durham RC, Guthrie A, Morton RV, et al. Tayside-Fife clin-
6. McKenna  P, Leucht  S, Jauhar  S, Laws  K, Bighelli  I. The ical trial of cognitive-behavioural therapy formedication-
controversy about cognitive behavioural therapy for schizo- resistant psychotic symptoms: results to 3-month follow-up.
phrenia. World Psychiatry. 2019;18(2):235–236. Br J Psychiatry. 1992;182:303–312.
7. Kuipers  E, Yesufu-Udechuku  A, Taylor  C, Kendall  T. 25. Valmaggia  LR, van  der  Gaag  M, Tarrier  N, Pijnenborg  M,
Management of psychosis and schizophrenia in adults: sum- Slooff  CJ. Cognitive-behavioural therapy for refractory
mary of updated NICE guidance. BMJ. 2014;348:g1173. psychotic symptoms of schizophrenia resistant to atypical
antipsychotic medication. Randomised controlled trial. Br J
8. Cooke A. Understanding Psychosis and Schizophrenia: Why
Psychiatry. 2005;186:324–330.
People Sometimes Hear Voices, Believe Things That Others
Find Strange, or Appear out of Touch With Reality… 26. Connor  KO, Stip  E, Pelissier  M, et  al. Treating delusional
and What Can Help. London, UK: British Psychological disorder: a comparison of attention placebo control. Revue.
Society; 2017. 2007;52(3):182–190.
9. Jones C, Hacker D, Xia J, et al. Cognitive behavioural therapy 27. Penn DL, Meyer PS, Evans E, Wirth RJ, Cai K, Burchinal M.
plus standard care versus standard care for people with schizo- A randomized controlled trial of group cognitive-behavioral
phrenia. Cochrane Database Syst Rev. 2018;2018(12):1–227. therapy vs. enhanced supportive therapy for auditory hallu-
doi:10.1002/14651858.CD007964.pub2 cinations. Schizophr Res. 2009;109(1–3):52–59.
10. Lau  J, Antman  EM, Jimenez-Silva  J, Kupelnick  B, 28. Craig TK, Rus-Calafell M, Ward T, et al. AVATAR therapy
Mosteller  F, Chalmers  TC. Cumulative meta-analysis of for auditory verbal hallucinations in people with psychosis: a
therapeutic trials for myocardial infarction. N Engl J Med. single-blind, randomised controlled trial. Lancet Psychiatry.
1992;327(4):248–254. 2018;5(1):31–40.
11. Muellerleile  P, Mullen  B. Sufficiency and stability of evi- 29. Lewis S, Tarrier N, Haddock G, et al. Randomised controlled
dence for public health interventions using cumulative meta- trial of cognitive-behavioural therapy in early schizophrenia:
analysis. Am J Public Health. 2006;96(3):515–522. acute-phase outcomes. Br J Psychiatry. 2002;181(suppl.
43):s91–s97.
12. Love R, Adams J, van Sluijs EMF, Foster C, Humphreys D.
A cumulative meta-analysis of the effects of individual phys- 30. Cather C, Penn D, Otto MW, Yovel I, Mueser KT, Goff DC. A
ical activity interventions targeting healthy adults. Obes Rev. pilot study of functional cognitive behavioral therapy (fCBT)
2018;19(8):1164–1172. for schizophrenia. Schizophr Res. 2005;74(2–3):201–209.
13. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA state- 31. Freeman D, Bradley J, Antley A, et al. Virtual reality in the
ment for reporting systematic reviews and meta-analyses of treatment of persecutory delusions: randomised controlled
studies that evaluate health care interventions: explanation experimental study testing how to reduce delusional convic-
and elaboration. J Clin Epidemiol. 2009;62(10):e1–e34. tion. Br J Psychiatry. 2016;209(1):62–67.
14. Chadwick P, Strauss C, Jones AM, et al. Group mindfulness- 32. Morrison AP, Turkington D, Pyle M, et al. Cognitive therapy
based intervention for distressing voices: a pragmatic random- for people with schizophrenia spectrum disorders not taking
ised controlled trial. Schizophr Res. 2016;175(1–3):168–173. antipsychotic medication: a randomised controlled trial.
15. Zimmermann G, Favrod J, Trieu VH, Pomini V. The effect of Schizophr Res. 2014;153:S75.
cognitive behavioral treatment on the positive symptoms of 33. Trower  P, Birchwood  M, Meaden  A, Byrne  S, Nelson  A,
schizophrenia spectrum disorders: a meta-analysis. Schizophr Ross K. Cognitive therapy for command hallucinations: ran-
Res. 2005;77(1):1–9. domised controlled trial. Br J Psychiatry. 2004;184:312–320.
16. Kay  SR, Fiszbein  A, Opler  LA. The Positive and Negative 34. Surguladze S, Fannon D, Wykes T, et al. What are the effects
Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull. of group cognitive behaviour therapy for voices? A random-
1987;13(2):261–276. ised control trial. Schizophr Res. 2005;77(2–3):201–210.
17. Haddock G, Barrowclough C, Shaw JJ, Dunn G, Novaco RW, 35. Mcleod  T, Morris  M, Birchwood  M, Dovey  A. Work with
Tarrier  N. Cognitive-behavioural therapy v.  social ac- voice hearers. Part 1. Br J Nurs. 2007;16(4):248–252.
tivity therapy for people with psychosis and a history of 36. Garety PA, Fowler DG, Freeman D, Bebbington P, Dunn G,
violence: randomised controlled trial. Br J Psychiatry. Kuipers E. Cognitive–behavioural therapy and family inter-
2009;194(2):152–157. vention for relapse prevention and symptom reduction in

1084
 CBT for Psychosis Cumulative Meta-analysis

psychosis: randomised controlled trial. Br J Psychiatry. 49. Naeem  F, Johal  R, McKenna  C, et  al. Cognitive behavior
2008;192(6):412–423. therapy for psychosis based guided self-help (CBTp-GSH)
37. Foster C, Startup H, Potts L, Freeman D. A randomised con- delivered by frontline mental health professionals: results of
trolled trial of a worry intervention for individuals with per- a feasibility study. Schizophr Res. 2016;173(1–2):69–74.
sistent persecutory delusions. J Behav Ther Exp Psychiatry. 50. Hayward  M, Jones  AM, Bogen-Johnston  L, Thomas  N,
2010;41(1):45–51. Strauss  C. Relating therapy for distressing auditory hallu-
38. Lincoln TM, Ziegler M, Mehl S, et al. Moving from efficacy cinations: a pilot randomized controlled trial. Schizophr Res.
to effectiveness in cognitive behavioral therapy for psychosis: 2017;183:137–142.
a randomized clinical practice trial. J Consult Clin Psychol. 51. Hazell CM, Hayward M, Cavanagh K, Jones AM, Strauss C.
2012;80(4):674–686. Guided self-help cognitive-behaviour Intervention for VoicEs
39. Kråkvik  B, Gråwe  RW, Hagen  R, Stiles  TC. Cognitive be- (GiVE): results from a pilot randomised controlled trial in a
haviour therapy for psychotic symptoms: a randomized transdiagnostic sample. Schizophr Res. 2018;195:441–447.
controlled effectiveness trial. Behav Cogn Psychother. 52. Gottlieb  JD, Gidugu  V, Maru  M, et  al. Randomized con-
2013;41(5):511–524. trolled trial of an internet cognitive behavioral skills-based
40. Rathod  S, Phiri  P, Harris  S, et  al. Cognitive behaviour program for auditory hallucinations in persons with psych-
therapy for psychosis can be adapted for minority ethnic osis. Psychiatr Rehabil J. 2017;40(3):283–292.
groups: a randomised controlled trial. Schizophr Res. 53. Morrison  AP, Pyle  M, Gumley  A, et  al.; FOCUS Trial
2013;143(2–3):319–326. Group. Cognitive behavioural therapy in clozapine-resistant
41. Leff  J, Williams  G, Huckvale  MA, Arbuthnot  M, Leff  AP. schizophrenia (FOCUS): an assessor-blinded, randomised
Computer-assisted therapy for medication-resistant audi- controlled trial. Lancet Psychiatry. 2018;5(8):633–643.
tory hallucinations: proof-of-concept study. Br J Psychiatry. 54. Wong  AWS, Ting  KT, Chen  EYH. Group cognitive be-
2013;202:428–433. havioural therapy for Chinese patients with psychotic dis-
42. Birchwood M, Michail M, Meaden A, et al. Cognitive behav- order: a feasibility controlled study. Asian J Psychiatr.
iour therapy to prevent harmful compliance with command 2019;39:157–164.
hallucinations (COMMAND): a randomised controlled trial. 55. Duval S, Tweedie R. Trim and fill: a simple funnel-plot-based
Lancet Psychiatry. 2014;1(1):23–33. method of testing and adjusting for publication bias in meta-
43. Freeman D, Pugh K, Dunn G, et al. An early phase II ran- analysis. Biometrics. 2000;56(2):455–463.
domised controlled trial testing the effect on persecutory de- 56. Egger  M, Smith  GD. Bias in meta-analysis detected by a
lusions of using CBT to reduce negative cognitions about simple, graphical test measures of funnel plot asymmetry.
the self: the potential benefits of enhancing self confidence. BMJ. 2011;315(7109):1–21.
Schizophr Res. 2014;160(1–3):186–192. 57. Spencer  HM, McMenamin  M, Emsley  R, et  al. Cognitive
44. Tarrier  N, Kelly  J, Maqsood  S, et  al. The cognitive behav- behavioral therapy for antipsychotic-free schizophrenia
ioural prevention of suicide in psychosis: a clinical trial. spectrum disorders: does therapy dose influence outcome?
Schizophr Res. 2014;156(2–3):204–210. Schizophr Res. 2018;202:385–386.
45. Freeman  D, Waite  F, Startup  H, et  al. Efficacy of cogni- 58. Murray  RM. On collecting meta-analyses of schizo-
tive behavioural therapy for sleep improvement in patients phrenia and postage stamps. Psychol Med.
with persistent delusions and hallucinations (BEST): a pro- 2014;44(16):3407–3408.
spective, assessor-blind, randomised controlled pilot trial. 59. Keck PE Jr, McElroy SL, Havens JR, et al. Psychosis in bi-
Lancet Psychiatry. 2015;2(11):975–983. polar disorder: phenomenology and impact on morbidity and
46. Naeem F, Saeed S, Irfan M, et al. Brief culturally adapted CBT course of illness. Compr Psychiatry. 2003;44(4):263–269.
for psychosis (CaCBTp): a randomized controlled trial from 60. Voce  A, Calabria  B, Burns  R, Castle  D, McKetin  R. A
a low income country. Schizophr Res. 2015;164(1–3):143–148. systematic review of the symptom profile and course of
47. Habib N, Dawood S, Kingdon D, Naeem F. Preliminary evalu- methamphetamine-associated psychosis substance use and
ation of culturally adapted CBT for psychosis (CA-CBTp): misuse. Subst Use Misuse. 2019;54(4):549–559.
findings from developing culturally-sensitive CBT project 61. Bighelli I, Salanti G, Huhn M, et al. Psychological interven-
(DCCP). Behav Cogn Psychother. 2015;43(2):200–208. tions to reduce positive symptoms in schizophrenia: system-
48. Waller H, Emsley R, Freeman D, et al. Thinking well: a ran- atic review and network meta-analysis. World Psychiatry.
domised controlled feasibility study of a new CBT therapy 2018;17(3):316–329.
targeting reasoning biases in people with distressing per- 62. Collins  L. Optimization of Behavioral, Biobehavioral, and
secutory delusional beliefs. J Behav Ther Exp Psychiatry. Biomedical Interventions: The Multiphase Optimisation
2015;48:82–89. Strategy (MOST). New York, NY: Springer; 2018.

1085