Incredible Years Parent

Incredible
Years
Pilot Study
Evaluation Report
Published April 2013
Ministry of Social Development
PO Box 1556
Wellington 6140
New Zealand
Telephone: +64 4 916 3300
Facsimile: +64 4 918 0099
Email: info@msd.govt.nz
Web: www.msd.govt.nz
ISBN: 978-0-478-33567-5 (print)
ISBN: 978-0-478-33568-2 (online)
Incredible
Years
Pilot Study
Evaluation Report
Fiona Sturrock and Dorian Gray

Centre for Research and Evaluation
Ministry of Social Development
In collaboration with
Academic Advisors:
Dr John Church
Professor David Fergusson
Associate Professor John Horwood
Professor Richie Poulton
and
Government Agencies:
Dr Sarah Hayward
John McGovern APRIL 2013
APRIL 2013
Contents
List of tables/ List of figures..................................................................................................................................................................... VI
Acknowledgements............................................................................................................................................................................................vII
Executive summary.................................................................................................................................................................................................. 1
Chapter 1: Introduction.....................................................................................................................................................................................4
1.1 Background to the Incredible Years Pilot Study....................................................................................................4
1.2 Childhood conduct problems...........................................................................................................................................5
1.3 Early intervention.......................................................................................................................................................................6
1.4 Parent training programmes..............................................................................................................................................6
1.5 Cost effectiveness.......................................................................................................................................................................7
1.6 The New Zealand context.....................................................................................................................................................7
1.7 The report structure..................................................................................................................................................................8
Chapter 2: The Incredible Years Parenting programme...................................................................................................9
2.1 An overview of the Incredible Years Parenting programme........................................................................9
2.2 Evaluations of the Incredible Years Parenting programme....................................................................... 11
Chapter 3: The New Zealand Incredible Years Pilot Study.............................................................................................. 13
3.1 Research design........................................................................................................................................................................ 13
3.2 Fieldwork issues........................................................................................................................................................................ 19
3.3 Statistical analyses.................................................................................................................................................................. 20
Chapter 4: Recruitment, retention and participant profiles................................................................................. 22
4.1 The referral process to Special Education Incredible Years programmes........................................ 22
4.2 Research participant recruitment and retention.............................................................................................. 23
4.3 Participant profiles.................................................................................................................................................................. 24
Chapter 5: Findings related to child behaviour, parenting practice and relationships............... 27
5.1 Child behaviour......................................................................................................................................................................... 28
5.2 Parenting practices and relationships...................................................................................................................... 30
5.3 Teacher pre/post measures of child behaviour.................................................................................................... 36
Chapter 6: Other outcomes.............................................................................................................................................................................. 39
6.1 Outcomes for health and stressful life events..................................................................................................... 39
6.2 Outcomes for children in clinical and sub-clinical groups.............................................................................. 42
6.3 Outcomes for phase 1 and phase 2............................................................................................................................ 46
6.4 Outcomes for the three sites.............................................................................................................................................. 48
Chapter 7: Programme outcomes for Maori and non-Maori...................................................................................... 50
7.1 The Incredible Years Pilot Study.................................................................................................................................... 50
7.2 The kaupapa Māori study.................................................................................................................................................. 58
7.3 Incredible Years for whānau Māori.............................................................................................................................. 59
Chapter 8: Single Case Studies....................................................................................................................................................................... 60
8.1 Parent Reports............................................................................................................................................................................. 60
iv
Chapter 9: Programme fidelity, Group Leader feedback and parent satisfaction................................ 68
9.1 Programme fidelity: Unified protocols..................................................................................................................... 68
9.2 Group Leader feedback....................................................................................................................................................... 69
9.3 Parent satisfaction with the Incredible Years Parenting programme................................................. 69
Chapter 10: Policy implications................................................................................................................................................................... 72
10.1 The efficacy of the Incredible Years Parenting programme in New Zealand............................. 73
10.2 Extensions of the Incredible Years Pilot Study..................................................................................................... 73
10.3 The selection of Incredible Years Parenting programme client families.............................................. 74
10.4 The effects of context........................................................................................................................................................ 74
10.5 Māori/non-Māori differences....................................................................................................................................... 75
10.6 Participants who do not respond to the Incredible Years Parenting programme....................... 75
10.7 Site differences....................................................................................................................................................................... 75
10.8 The need for an effectiveness study........................................................................................................................ 76
10.9 Organisational implications.......................................................................................................................................... 76
10.10 Summary of Evaluation Advisory Group recommendations.............................................................. 77
Chapter 11: Summary and conclusions.............................................................................................................................................. 78
1.1 Summary of findings related to research questions...................................................................................... 78
11.2 Conclusions............................................................................................................................................................................... 80
11.3 Future research....................................................................................................................................................................... 81
Appendix 1. The Incredible Years Pilot Study Evaluation Advisory Group...................................................... 82
Appendix 2. Source instruments for the Incredible Years Pilot Study.............................................................. 83
Appendix 3. Statistical methods.................................................................................................................................................................. 87
REFERENCEs......................................................................................................................................................................................................................... 91
v
List of Tables
Table 1. A sample of RCT studies investigating the efficacy of IYP programmes,
with post-treatment child behaviour effect sizes (ES)........................................................................................... 11
with post-treatment parenting practices effect sizes (ES).................................................................................. 12
Table 3. Topics covered in the Family Interview............................................................................................................................ 17
Table 4. Child behaviours: Mean parent reports before, during and after IYP programme........................... 29
Table 5. Child behaviours: Mean parent reports before IYP programme and at follow-up........................... 30
Table 6. Parenting practices: Mean parent reports before, during and after IYP programme..................... 31
Table 7. Parenting practices: Mean parent reports before IYP programme and at follow-up...................... 32
Table 8. Parental conflict and relationships: Mean parent reports before,
during and after IYP programme.......................................................................................................................................... 34
Table 9. Parental conflict and relationships: Mean parent reports before
IYP programme and at follow-up......................................................................................................................................... 35
Table 10. Child behaviours: Mean teacher reports before and after IYP programme........................................... 36
Table 11. Child behaviours: Mean teacher reports before IYP programme and at follow-up......................... 37
Table 12. Child behaviours: Mean teacher reports before and after IYP programme
and at follow-up, by referral source.................................................................................................................................... 37
Table 13. Child behaviour: Effect sizes of mean teacher reports of change from baseline
to post-course and follow-up, by referral source...................................................................................................... 38
Table 14. Health and stressful life events: Mean parent reports before,
during and after IYP programme.......................................................................................................................................... 40
Table 15. Health and stressful life events: Mean parent reports before
IYP programme and at follow-up......................................................................................................................................... 41
Table 16. Child behaviours after IYP programme: Effect sizes of mean parent
reports of change, by sub-clinical and clinical ECBI Intensity.......................................................................... 42
Table 17. Child behaviours at follow-up: Effect sizes of mean parent reports of change,
by sub-clinical and clinical ECBI Intensity....................................................................................................................... 43
Table 18. Child behaviours after IYP programme: Effect sizes of mean parent
reports of change, by sub-clinical and clinical Social Competence Scale (SCS)................................. 44
by sub-clinical and clinical Social Competence Scale (SCS).............................................................................. 45
Table 20. Child behaviours after IYP programme: Effect sizes of mean parent reports
of change, by study phase......................................................................................................................................................... 46
by study phase................................................................................................................................................................................... 47
of change, by study site............................................................................................................................................................... 48
Table 23. Child behaviours at follow-up: Mean parent reports, by study site............................................................. 49
Table 24. Child behaviours after IYP programme: Effect sizes of mean parent reports of
change, by Māori and non-Māori children........................................................................................................................ 51
by Māori and non-Māori children........................................................................................................................................ 52
Table 26. Parenting practices after IYP programme: Effect sizes of mean parent reports
of change, by Māori and non-Māori children.................................................................................................................. 53
Table 27. Parenting practices at follow-up: Effect sizes of mean parent reports of change,
by Māori and non-Māori children........................................................................................................................................... 54
vi
Table 28. Conflict and relationships after IYP programme: Effect sizes of mean parent
reports of change, by Māori and non-Māori children............................................................................................ 55
Table 29. Conflict and relationships at follow-up: Mean parent reports of change,
by Māori and non-Māori children........................................................................................................................................... 56
Table 30. Summary of the multivariate analyses for Māori and non-Māori children at IYP
programme completion and follow-up........................................................................................................................... 57
Table 31. Satisfaction with course organisation: opinions before and after the course by ethnicity................ 70
Table 32. Satisfaction with the course overall at course completion by ethnicity.................................................. 71
Table 33. Effect sizes for child behaviour and parenting practices reported in
Larsson et al (2009) and the Incredible Years Pilot Study.................................................................................... 80
Table 34. Bonferroni adjustments applied........................................................................................................................................... 89
Table 35. Summary of analysis methods used.................................................................................................................................. 90
List of Figures
Figure 1. Incredible Years Parenting programme pyramid...................................................................................................... 10
Figure 2. The research design....................................................................................................................................................................... 16
Figure 3. Incredible Years Pilot Study participant recruitment and retention............................................................ 23
Figure 4. Example of no behaviour change (Child 101)............................................................................................................. 62
Figure 5. Example of behaviour change towards improvement (Child 311).............................................................. 62
Figure 6. Predicted positive behaviour scores for each child and for the IYP programme overall..................... 63
Figure 7. Predicted negative behaviour scores for each child and for the IYP programme overall................... 64
Figure 8. Predicted combined behaviour scores for each child and for the IYP programme overall............... 65
Figure 9. Predicted positive behaviour scores for each child through the programme
to the follow-up................................................................................................................................................................................ 66
Figure 10. Predicted negative behaviour scores for each child through the programme
to the follow-up................................................................................................................................................................................ 66
Figure 11. Predicted combined behaviour scores for each child through the programme to
the follow-up....................................................................................................................................................................................... 67
vii
Acknowledgements
The Incredible Years Pilot Study was a substantial Associate Professor John Horwood of the
cross-agency study conducted over 2 years. The Christchurch Health and Development Study
Ministry of Education Special Education delivered provided essential statistical advice and review.
the Incredible Years Parenting courses that were The Group Leaders of the Incredible Years
evaluated in the study. The Ministry of Health Parenting courses and their Supervisors
provided the operational funding for the study. co-operated with the evaluation team,
The evaluation team is within the Ministry of supplying data additional to their regular
Social Development. administrative requirements and answering
We thank the following for their contribution to endless questions.
the Incredible Years Pilot Study: Christina Smits helped with the administration
The Incredible Years Evaluation Advisory of the Incredible Years Pilot Study and the
Group provided ongoing advice and editing of this report.
support for the study: Dr John Church, The hard work and dedication of Robyn
Professor David Fergusson, Professor Deighton, Kathryn Handley and Trish Ross, the
Richie Poulton, Dr Sarah Hayward, Brian Interviewers in the three sites, resulted in quality
Coffey, John McGovern and Ross Mackay. data and high retention rates for the study.
In particular: Professor Fergusson provided We are indebted to the 166 families who
invaluable advice and guidance throughout agreed to participate in the study, almost all
the study; Dr Church designed, oversaw of whom remained engaged throughout
and reported on the Direct Observations the four interviews.
trial; and John McGovern was the liaison
between Special Education and the
evaluation team.
Fiona Sturrock
Dorian Gray
viii
Executive summary
Introduction The main study used a repeated measures
design in which assessments of the research
A growing body of research provides evidence participants were made at baseline, mid-
of the prevalence of childhood conduct programme, post-programme and 6 months
problems and the long-term negative following programme completion.
consequences that result. It also identifies
Family Interviews were supplemented by
parent management training such as the
a postal Teacher Questionnaire which was
Incredible Years Parenting (IYP) programme as
administered at baseline, post-programme and
an effective evidence-based treatment.
the 6-month follow-up.
As part of the Drivers of Crime work programme
The Family Interview incorporated items from a
the Ministries of Education, Health and Social
number of previously validated instruments to
Development established a pilot study of the
measure outcome variables describing changes
IYP programme to assess the effectiveness of
over the course of the study. These changes
this programme in reducing conduct problems
related to child behaviour, parenting practices,
in a New Zealand context.
family relationships and other relevant
The project was influenced by the outcomes such as health, parental depression
recommendations of the Government Advisory and stressful life events.
Group on Conduct Problems, the Ministry of
The study also included Single Case Studies
Education Positive Behaviour for Learning
with a sample of the participants using weekly
strategy and the desire to develop a new
Parent Reports.
collaborative model to evaluate government-
funded programmes.
Key findings
The higher rates of conduct problems in
Māori children mean that, to reduce conduct There was clear evidence of child behaviour
problems in New Zealand, it is critical parenting change, with effect sizes as expected on the
interventions are effective for and acceptable to basis of the international literature. Effect size, as
Māori parents. measured by Cohen’s d, provides an indication
The IYP programme is based on social learning of the extent of reported change, whereby an
theory. It is an intensive programme consisting effect size of d=.20 is considered small, an effect
of weekly 2-hour sessions delivered over 12–18 size of d=.50 is medium and an effect size of
weeks by two trained Group Leaders to a group d=.80 is large.
of up to 16 participants. There was clear evidence of parenting behaviour
The Incredible Years Pilot Study was a change, with effect sizes as expected on the basis
substantial 2-year, multiple-informant study. of the international literature.
It included mixed measurement methods, The benefits of the IYP training were broadly
single case studies and a 6-month similar for Māori and non-Māori families.
follow-up. The study was designed to Nevertheless the evidence suggests the need
understand the profile of the families referred for further work on maximising gains for Māori
to the IYP programme, to ensure programme families, particularly in the maintenance of
fidelity, to measure both programme behaviour change.
effectiveness and parent satisfaction, and to Both Māori and non-Māori parents
assess programme responsiveness to Māori. expressed high to moderate satisfaction
A total of 166 primary caregivers of children with the programme.
aged 3–8 years attending an IYP programme at These results suggest the IYP programme can
three sites (Bay of Plenty, Canterbury and Mid- be successfully implemented in New Zealand
Central) participated in the study. and retain its general level of effectiveness.
1
The findings provide a series of performance Before-and-after comparisons of scores on the
benchmarks against which the future roll out of Social Development Scale completed by teachers
the IYP programme in New Zealand can produced modest effect sizes (d=.17
be assessed. at post-course, d=.29 at follow-up).
Children with scores in the clinical range on
Detailed findings the pre-course Eyberg Child Behaviour Inventory
(ECBI) Scaled Intensity measure improved to
Seventy-two percent of parents visited by
a greater extent at post-course on all child
Group Leaders to enrol in the IYP programme
behaviour measures except Social Competence
participated in the research. The recruitment rate
than children with scores below the clinical
rose to 85 percent when taken as a proportion of
cut-off. Significant differences in reported Conduct
parents who consented to be contacted for the
Disorder (CD) and Oppositional Defiant Disorder
research at the time of enrolment. Retention in
(ODD) remained at follow-up, whereas differences
the study was 98 percent of parents for whom
in Attention Deficit Hyperactivity Disorder (ADHD),
there were baseline data.
Self-control and Anxiety/Withdrawal were no
The study reached the target group, ie children longer evident, perhaps reflecting a continued
aged 3–8 years, a third of whom identified improvement for children in the sub-clinical group.
as Māori.
The greater improvement reported for children
The programme was delivered with fidelity to the
in the clinical range on the pre-course Social
stipulated IYP programme delivery, processes and
Competence Scale (SCS) on measures of CD and
techniques.
ODD at post-course were sustained at follow-up.
Baseline and post-course comparisons of Self-control and Social Competence had also
outcome measures for families enrolled in the improved to a greater extent for children in the SCS
IYP programme produced effect size estimates clinical group than for children in the sub-clinical
for change in child behaviour, parenting practice group 6 months following course completion.
and family relationships consistent with previous
Small to medium improvements in
IYP programme evaluations using randomised
caregiver stress and anxiety were reported at
control trials.
post-course and follow-up, as were fewer child
Children and parents receiving IYP training
visits to health services and fewer stressful life
showed significant improvements in the
events. Parents also reported a decrease in
following areas: child behaviour median effect
depressive symptoms at follow-up.
size of d=.65 (ranging from d=.51 to d=.96),
parenting practices median effect size of d=.54 Parent ratings in both the Family Interview
(ranging from d=.26 to d=.83) and relationships and the in-course Client Satisfaction
median effect size of d=.48 (ranging from Questionnaire showed high to moderate
d=.21 to d=.60). satisfaction with the IYP programme.
The improvements evident at the completion Although improvements were evident at all
of the IYP course were mostly sustained at the three sites, larger effect sizes were found in
6-month follow-up. Significant improvements Mid-Central than in the Bay of Plenty
were maintained in the following areas: child or Canterbury.
behaviour median d=.71 (range d=.56 to d=1.0), The evidence suggests the IYP programme
parenting practices median d=.52 (range d=.25 to was both effective for and accepted by Māori
d=.79) and relationships median d=.43 families. However, at post-course and
(range d=.15 to d=.59). follow-up, parents of Māori children reported
Effect sizes for parenting practices and family slightly smaller improvements in child behaviour
relationships were typically lower than effect sizes and slightly smaller reductions in negative
for child behaviour. This indicated small changes parenting practices on the individual outcome
in parenting practice may produce substantial measures than did parents of non-Māori
improvements in child behaviour. children. These differences were not significant.
2
On the combined outcome measures at The present consortium structure for evaluating
programme completion the differences programmes targeted at childhood conduct
between Māori and non-Māori children were problems should be maintained.
not significant. However, at follow-up a small
but statistically significant difference between
responses by parents of Māori and non-Māori
Recommended future research
children was detected for the overall child A longer-term follow-up of the children and
behaviour outcome measure (p=.025). This parents in the present trial to examine the
suggests there may be a particular challenge in extent to which IYP programme benefits
maintaining the benefits of the IYP programme are sustained and the extent to which these
for Māori families. benefits may dissipate over time.
The data from the weekly Parent Report Single A national effectiveness study based on the
Case Studies replicated the improvement in Incredible Years National Register data held
child behaviour evident in the main study for by the Ministry of Education to investigate
the majority of children. It also highlighted the whether or not the effect sizes for the changes
variability in child behaviour change. in the ECBI and SCS measures obtained in the
Incredible Years Pilot Study can be replicated
on national aggregated data and for individual
Evaluation Advisory Group policy courses at all sites.
recommendations The performance benchmarks reported here
The Incredible Years Pilot Study should be provide measures against which the future roll out
viewed as providing adequate evidence of the of IYP programmes should be judged. Consistent
efficacy of the IYP programme in New Zealand. and rigorous quality control monitoring of
Investments should be made into the provider outcomes will ensure the programme is
longer-term follow-up of the families effective and the funding is well spent.
participating in this evaluation, with further Further investigation is required to understand
assessments being made at 1 year and 2 years the circumstances that produced site variation
post-treatment. in outcomes to inform best practice principles
Consideration should be given to providing and to ensure quality control.
parent management training programmes, The suggestions that the benefits of IYP training
such as Triple P Level 4, that are less intensive may be slightly larger for non-Māori than for
than the IYP programme for client families with Māori families and that there appears to be a
children whose behaviour scores on the ECBI particular challenge for Māori in maintaining
scales place them in the sub-clinical range. behaviour change point to the need for
Consideration should be given to developing further research to examine Māori/non-Māori
parallel research to evaluate the Incredible Years differences in greater detail.
Teacher programme to determine the benefits An evaluation of Ngā Tau Mīharo ō Aotearoa,
of this programme for behaviours in the the Incredible Years resources developed by
classroom and for behaviours at home. the Werry Centre for Māori Group Leaders, is
Further investment is needed to increase the required to ascertain whether or not differences
efficacy of the IYP programme for Māori families. in outcomes for Māori and non-Māori families
There is a need to develop further programmes are evident in courses using these resources.
and interventions to meet the needs of families The evaluation of the Incredible Years Teacher
whose children do not show sufficient behaviour programme is needed to assess a school-based
change at the end of the IYP programme. programme.
Investment should be made into a national Further research is required to compare the
effectiveness study using the Incredible Years cost benefit of providing the IYP programme
National Register data held by the Ministry against the cost benefit of existing parenting
of Education. programme.
3
Chapter 1:
Introduction
1.1 Background to the

Incredible Years Pilot Study
This report describes an evaluation of the
Incredible Years Parenting (IYP) Programme
in New Zealand. IYP is a parent management
training programme developed for parents of
children with conduct problems. The effective
treatment and management of conduct problems
are a high priority for New Zealand’s health,
education, justice and welfare sectors and form
part of a broader cross-government priority work
programme to address the Drivers of Crime.
The Ministries of Education, Health and Social
Development established a pilot study of the IYP
programme under the Drivers of Crime Conduct
Problems work stream to assess the programme
in the New Zealand context. The results of the
evaluation of this pilot will inform the ongoing
development of services across government for
conduct problems and guide decisions about
whether or not to invest significant funding in
rolling out the IYP programme on a population
basis in New Zealand. If the programme is proved
to be effective, this investment has the potential
to make a significant difference in reducing the
rates of conduct problems and the long-term
social distress and high levels of fiscal costs that
conduct problems incur.
4
1.2 Childhood conduct
problems
The Incredible Years Pilot Study evolved from the A growing body of research evidence, both in
following influences: New Zealand and overseas, has built a clear picture
The Advisory Group on Conduct Problems of the prevalence, consequences and treatment of
(AGCP) was set up in 2007 in response to a childhood conduct problems:
growing awareness of the research findings Conduct problems are considered to be one of
on the adverse outcomes associated with the most commonly occurring mental health
childhood conduct problems. The AGCP issues among children and adolescents. Based
was charged with the responsibility of on multiple-informant data, conduct problems
identifying the issues raised by childhood are estimated to afflict up to 10 percent of
conduct problems and of recommending children in New Zealand (Fergusson et al 2011)
effective programmes and policies for the and internationally (Scott 2007). Māori children
prevention, treatment and management of have higher rates (15% to 20%) of conduct
these conditions. problems than non-Māori (Ministry of Social
The advice provided by the AGCP formed Development 2009a).
the foundations of the second influence on Conduct problems during early and middle
the Incredible Years Pilot Study, the Positive childhood are associated with negative outcomes
Behaviour for Learning (PB4L) strategy in adolescence and adulthood, including
developed by the Ministry of Education. This antisocial behaviour, mental health difficulties,
strategy includes the implementation of three suicidal behaviours, substance abuse, teenage
evidence-based programmes for addressing pregnancy, inter-partner violence and physical
childhood conduct problems, one being the health (Fergusson et al 2011; Lindsay et al 2011).
IYP programme. The difficulties and stresses caused by conduct
The third element was the desire to develop problems affect both the young people and
a new model to evaluate government-funded those associated with them such as parents,
programmes. This collaborative model teachers and peers (Kazdin 2007). A recent review
was based on a consortium approach that of the evidence on conduct problems by the
brought together the skills of several groups: AGCP (Ministry of Social Development 2009a)
i) the service providers who delivered the concluded there is probably no other commonly
programme; ii) the government researchers occurring childhood and adolescent condition
who evaluated the programme; iii) academic that has such far-reaching consequences for
advisors with expertise in research design and individual health, development and wellbeing.
analysis; and iv) Māori researchers. Moreover, the wide range of negative outcomes
associated with conduct problems has high social
and fiscal costs (Fergusson et al 2005a; Feinstein
and Sabates 2006; Odgers et al 2007).
Research into effective interventions for the
prevention, treatment and management of
conduct problems is the subject of a number of
reviews and meta-analyses that have identified
effective programmes (Brestan and Eyberg 1998;
Kazdin 2007; Ministry of Social Development
2009b). Among these are parent management
training programmes based on social learning
theory, such as the IYP programmes, Triple P and
programmes developed by the Oregon Social
Learning Centre.
5
1.3 Early intervention 1.4 Parent training
Early intervention is recognised as a crucial element programmes
in reducing the onset of behaviour problems that Given the importance of family and parenting
often start in early childhood. A body of robust
characteristics in the development of preschool
evidence shows early intervention can have a
children, parenting programmes are generally
significant impact on child development and later
viewed as an essential component of early
life outcomes (Fergusson et al 2005a; Karoly et al
intervention. Parent-based intervention strategies
2005; Odgers et al 2007). O’Neill (2009) notes that
focus on enhancing parental skills in the
early childhood interventions, particularly at the
expectation these enhancements will translate
family level, have proven effective in overcoming
into improved child outcomes. For example,
long-term disadvantage.
ineffective parenting and poor disciplinary
The enduring influence of parenting during the practices are major determinants of conduct
early years in a child’s life is demonstrated in a problems (Scott 2007). The parenting behaviours
longitudinal study that followed a sample of that constitute risk factors for the development
American children from birth to 12 years of age of conduct problems are the main targets of
examining how parenting in the infant, toddler interventions that seek to improve parenting
and preschool years might influence children when skills and thus to improve outcomes for children
they are older (Belsky et al 2007). The quality of (Dadds et al 2003).
parenting experienced across the first 4.5 years of
life consistently predicted many aspects of children’s Parent management training is one of the most
functioning even after many alternative sources successful approaches to addressing conduct
of influence were accounted for, such as family problems in early and middle childhood,
economic and educational resources. Children who particularly for children aged 3–7 years. There is
experienced parenting that was warm, sensitive, strong evidence from rigorous efficacy trials that
cognitively stimulating and not intrusive or over parenting programmes can improve parenting
controlling early in life showed better cognitive skills and reduce children’s behavioural difficulties
functioning, academic achievement and social (Lindsay et al 2011). A range of manualised,
adjustment when in middle primary school. The well validated and widely used programmes
opposite was true when children experienced care are available. These programmes offer options
that could not be characterised this way. for delivering parent behaviour management
training, from universal programmes to highly
Recognition that the early years lay the foundation
intensive programmes, provided in a group or
for future development has led to investment
individually (Fergusson et al 2011).
in evidence-based prevention and treatment
programmes for young children and their families Parenting programmes that focus on teaching
(Gluckman 2011). These early intervention specific parenting skills and/or changing parents’
programmes seek to mitigate risks for vulnerable attitudes and beliefs can be effective at improving
children by improving parental capabilities, parenting and can lead to improved child
addressing risk factors and enriching children’s behaviour (Hendricks and Balakrishnan 2005).
experiences. In the New Zealand Families Typically, these programmes aim to strengthen
Commission review of early interventions, Gray parent-child relationships and bonding, to
(2011) points out that programmes and services strengthen parents’ interpersonal skills and
providing support early have stronger effects than support networks and to promote effective limit
those that intervene later. In the case of antisocial setting, non-punitive discipline and systematic
behaviour, conduct problems that first become behaviour plans. Programme strategies include
evident early in life are particularly difficult to increasing parents’ empathy towards their
correct if left untreated until adolescence. Moreover, children and increasing parents’ knowledge of
remedial programmes in the adolescent and young child developmental needs.
adult years cost much more to produce the same
level of attainment in adulthood (Heckman 2006).
6
1.5 Cost effectiveness 1.6 The New Zealand context
Childhood conduct problems are a major Because of the unique socio-demographic and
predictor of lifetime resource use which results cultural profile of the New Zealand population it
in substantial costs in the education, health, is important to ensure a treatment for conduct
justice and welfare sectors. A wider uptake of problems, such as parent management training, is
evidence-based interventions is likely to lead to effective in the New Zealand context. The higher
considerable economic benefits in the short term, rates of conduct problems in Māori children mean
and probably even more in the long term (Romeo that, to reduce conduct problems in New Zealand,
et al 2006). Heckman (2006) maintains that, given it is critical parenting interventions are effective for
the dynamic process of life cycle skill formation in and acceptable to Māori. This imperative highlights
which early inputs strongly affect the productivity the need for Māori to engage in programmes that
of later inputs, the economic return from early build resilience and improve outcomes for whānau.
intervention is higher and the return from later This is more likely to be achieved if there is a range
intervention is lower. of intervention options available for Māori, including
kaupapa Māori interventions, Māori-enhanced
Research shows the return from well-implemented interventions, and generic interventions.
and well-evaluated prevention, intervention, and
Ongoing concerns are expressed about the suitability
treatment programmes for conduct problems
for, and acceptability to, Māori of programmes
is often very good, with programmes returning
based on western science methodologies (Altena
several times their costs as a result of reduced
and Herewini 2009; Cargo 2008; Durie 2004,
rates of crime, imprisonment and associated costs
2006). Although some evidence-based parenting
(Fergusson et al 2011). O’Neill’s (2009) cost-benefit
programmes, such as the IYP programme, have been
analysis of the IYP programme suggests the
culturally enhanced to reflect local Māori tikanga (eg
long-term rate of return from parenting
Werry Centre 2012), these should be distinguished
programmes is likely to be relatively high, while
from kaupapa Māori interventions, which are
Scott (2007) estimated the longer term return from designed, developed and delivered by Māori.
IYP training to be 10 times higher than its cost.
MacFarlane (2011) notes that, while the literature
Although there is no guarantee cost-benefit is growing on the development of kaupapa Māori
analyses conducted overseas will apply in research, less consideration has been given to the
the New Zealand context, there is a universal ways in which western science and kaupapa Māori
consensus in the literature that a long-term research can be combined to produce consensual
investment strategy is likely to be highly decisions about programme effectiveness. He sets
cost-effective, providing the investment is out a conceptual model that attempts to integrate
made in well-founded and well-implemented western science and kaupapa Māori models of
evidence-based programmes (Fergusson programme development and evaluation. The
et al 2011). analogy is a braided river (he awa whiria) in which
there are two main streams, representing western
science and kaupapa Māori models, which are
interconnected by minor tributaries with the two
streams reaching a point of convergence.
MacFarlane’s model acknowledges the western
science and kaupapa Māori streams as distinctive
approaches to the development and evaluation
of programmes but permits knowledge and
evaluation methodologies from each to inform and
to be applied to the development of programmes
in the other. Furthermore, the model assumes the
determination of programmes as being effective will
rely on an acceptance of evidence from both streams.
7
1.7 The report structure
This report is structured as follows:
Chapter 2 gives an overview of the IYP
programme and a brief summary of existing
evaluations.
Chapter 3 provides a description of the
research design.
Chapter 4 describes recruitment and
retention of the study participants and
presents the participant profile.
Chapter 5 summarises the findings related to
child behaviour and parenting practice and
relationships.
other outcomes.
Māori and non-Māori families.
Single Case Studies.
Chapter 9 addresses programme fidelity,
Group Leader feedback and parent
satisfaction.
Chapter 10 discusses the policy implications
of the findings and presents Evaluation
Advisory Group (EAG)1 recommendations.
Chapter 11 provides a summary of findings
and a recommended future research
programme.
A Technical Appendix to this report (Gray 2013)
contains detailed descriptions of the study
instruments and statistical analyses. This report is
available on request from the author.
1 See Appendix 1 for a list of the members of the EAG.
8
Chapter 2:
The Incredible Years
Parenting programme
2.1 An overview of the

Incredible Years Parenting
programme
The Incredible Years Parent, Teacher and Child
Training series has been developed over the
last 30 years at the University of Washington
by Carolyn Webster-Stratton and her associates
(Webster-Stratton 1994). It has been implemented
widely within the United States, the United
Kingdom, Canada, Ireland, Norway, Denmark,
Sweden, Australia and New Zealand. The
Incredible Years series received a proven rating
for IYP BASIC from the RAND Corporation
(RAND 2007), has been endorsed in a number of
jurisdictions (MSD 2007) and has been identified
as one of 11 Blueprint interventions by the
Centre for Violence Prevention at the University
of Colorado, having satisfied stringent scientific
criteria (Jones et al 2007).
The Incredible Years series consists of a
number of comprehensive, multi-faceted and
developmentally-based training programmes: five
for parents, two for children and one for teachers.
The programmes are specifically designed to
promote emotional and social competence in
young children and to prevent, reduce and treat
behavioural and emotional problems.
9
The IYP programme is based on social learning praise, motivating children through rewards,
theory, particularly the coercion hypothesis effective limit setting, and effective ways of
of reinforcement developing and maintaining handling misbehaviour. Parent competences are
deviant behaviour and the notions of modelling developed in areas such as communication,
and self-efficacy. It is an intensive programme, limit-setting, problem-solving and anger
with two trained Group Leaders delivering a management. Parents are also encouraged to
2–2.5 hour session each week, typically to a group develop support networks. The programme
of up to 16 participants. Each programme lasts for covers all the basic aspects of parenting as well as
12–18 weeks. The programme uses a collaborative offering help to individual parents who find their
approach between Group Leaders and parents, child’s behaviour challenging.
including group discussions of video material
A central metaphor used in the IYP programme is
showing both appropriate and inappropriate
a pyramid of parenting principles (see Figure 1).
parent-child interactions. The IYP preschool
The pyramid is used to depict a hierarchy of
programme is for parents of children aged
parental strategies, some of which are the basic
3–5 years; the school age IYP programme is
building blocks to help children develop to their
targeted at parents of children aged 6–12 years.
potential and should be used very frequently
The IYP training aims to increase positive and (eg play, attention and involvement), and others
nurturing parenting styles, to decrease negative which are also important for optimal child
or harsh parenting, and to promote children’s development but should be used less frequently
social-emotional development. It covers a range (eg consequences for aggression).
of parenting strategies, including: play, effective
Figure 1. Incredible Years Parenting

programme pyramid
(sourced from www.incredibleyears.com)
10
2.2 Evaluations of the Incredible
Years Parenting programme
There is a strong evidence base for the effectiveness Table 1 summarises a sample of RCT studies
of the IYP programme in enhancing parenting that report significant effect sizes for changes
competencies and in reducing disruptive behaviours in child behaviour at post-treatment following
in children, and these gains have been maintained the parent’s attendance at an IYP programme.
at long-term follow-up. Webster-Stratton and Effect size2, as measured by Cohen’s ‘d’, provides
Reid (2010) reviewed the numerous published an indication of the extent of reported change,
randomised control group trials (RCT) conducted whereby an effect size of d=.20 is considered
by Webster-Stratton and her colleagues that small, an effect size of d=.50 is medium and an
have demonstrated the effectiveness of the IYP effect size of d=.80 is large. Two studies reported
programme for young children with conduct a moderate effect size and three studies reported
problems (eg Webster-Stratton and Hammond large effects for child behaviour change. Effect
1997). These findings have been replicated in sizes for measures of parenting practices are
RCTs by independent investigators in England presented in Table 2 for those studies that
(Gardner et al 2006; Scott et al 2001), Wales reported these outcomes. Here, the effect size for
(Hutchings et al 2007; Jones et al 2007), Ireland Positive Parenting was small, while changes in the
(O’Neill 2009) and Norway (Larsson et al 2009). other parenting practices ranged from medium
Studies using a matched control group rather to large. The children in these studies were in the
than a RCT (eg Kelleher and McGilloway 2006; clinical range for conduct problems.
Posthumus et al 2011) have also demonstrated
the effectiveness of IYP programmes in reducing
childhood conduct problems.
Table 1. A sample of RCT studies investigating the efficacy of IYP programmes, with
post-treatment child behaviour effect sizes (ES)
Child age
Study N (years) Country Measure ES (d)
Gardner et al (2006) 76 2 to 9 England ECBI d=.55

Hutchings et al (2007) 153 3 to 5 Wales ECBI d=.89
Larsson et al (2009) 127 4 to 8 Norway ECBI d=.65
Scott et al (2001) 141 3 to 8 England PR d=.89
Webster-Stratton and Hammond (1997) 97 4 to 8 US CBCL d=.89*
ECBI = Eyberg Child Behaviour Inventory, Intensity Score

CBCL = Child Behaviour Checklist
PR = Parent Report of Conduct Problems
*Effect size for Webster-Stratton and Hammond (1997) retrieved December 2012 from http://www.incredibleyears.com/library
2 Effect sizes are explained in Chapter 3, Section 3.3 and in Appendix 3.
11
with post-treatment parenting practices effect sizes (ES)
Child age
Study N (years) Country Design Measure ES (d)
Gardner et al (2006) 76 2 to 9 England RCT

Positive Parenting Observation d=.38
Negative Parenting Observation d=.74
Parent Competencies AOPS d=.65
Hutchings et al (2007) 153 3 to 5 Wales RCT

Positive Parenting Observation d=.57
Parent Competencies AOPST d=.95
Larsson et al (2009) 127 4 to 8 Norway RCT

Positive Parenting PPI d=1.44
Harsh Discipline PPI d=.61
Inconsistent Discipline PPI d=1.05
AOPS = Arnold-O’Leary Parenting Scale Total

PPI = Parenting Practices Interview
There remains the question of whether reductions in problem behaviour and increases
evidence-based programmes developed in child social competence following the
outside of New Zealand are effective within programme, and that client satisfaction with
a New Zealand context and, in particular, are the programme was high. Similar findings were
effective with young Māori or Pacific people obtained for Māori and non-Māori parents. A
(Smith 1999). These debates highlight the second study, based on a qualitative evaluation
importance of a thorough evaluation, including of the responses of 10 Māori parents attending a
of cultural appropriateness, of programmes marae-based version of the IYP courses (Altena
such as the IYP programme in New Zealand and Herewini 2009), also reported that Māori
before they are widely implemented (Fergusson enrolled in the IYP programme showed high rates
et al 2011). Further, as Aos et al (2006) notes, of retention and satisfaction with the programme.
programme efficacy must be demonstrated to
While these findings were promising, a more
ensure public funding is being well spent.
searching evaluation of the IYP programme in
A preliminary evaluation of IYP training in New Zealand and its applicability to Māori parents
New Zealand (Fergusson et al 2009) found that was required.
parents attending the IYP courses reported
12
Chapter 3:
The New Zealand
Incredible Years Pilot Study
On the recommendation of the AGCP, the

Ministries of Education, Health and Social
Development, with advice from academic experts,
established the Incredible Years Pilot Study to
assess the efficacy of the IYP programme in the
New Zealand context. The Incredible Years EAG
advising the study consisted of representatives
of these Ministries, plus academic experts in the
fields of conduct problems and research. The
results of the evaluation detailed in this report will
inform the further development of existing IYP
programmes and guide the decisions about the
investment of significant funding to roll out the
IYP programme on a population basis in
New Zealand.
3.1 Research design

To inform the future planning and delivery of
the IYP programme, an evaluation must provide
agencies with information on the profile of
families referred to IYP training, demonstrate
programme fidelity, measure both programme
effectiveness and parent satisfaction, and assess
programme responsiveness to Māori. To this end
the Incredible Years Pilot Study was designed as a
substantial 2-year, multiple-informant study that
included mixed measurement methods, single
case studies and a 6-month follow-up.
The study received ethical approval from the
Multi-region Ethics Committee, which reviews
national and multi-regional studies (ethics
reference MEC/10/08/075).
13
Research questions 2. Sustainability of gains: To what extent were any
benefits of IYP sustained over time?
The design of the Incredible Years Pilot Study
was based around the following series of This question was addressed by extending the
research questions relating to the delivery of the longitudinal, teacher and single case study
programme in New Zealand. design described above to include follow-up
data at 6 or 7 months after course completion.
1. Programme efficacy: To what extent was there
evidence to suggest that the IYP programme was 3. Benefits for Māori: To what extent were the
effective in a New Zealand context? programme outcomes for Māori similar to or
different from the outcomes for non-Māori?
This question was examined using three
research approaches: The extent to which programmes such as IYP
that are based on western science methodology
a. The first approach was a before-and-
are effective for Māori has been the subject of
after longitudinal design that compared
substantial debate (eg Altena and Herewini
the outcomes of children and families
2009). The current consensus is that IYP training
receiving the IYP programme at: baseline,
is effective for Māori providing the programme
mid-programme and programme
is delivered in a culturally appropriate way (eg
completion. The main dependent variables
Berryman et al 2009). To address these issues in
were: parent reports of child behaviours;
the Incredible Years Pilot Study, analyses were
parent reports of parenting behaviours;
conducted comparing:
measures of family functioning and
parental mental health. Consistent with a. The outcomes for Māori and non-Māori
previous research it was hypothesised families on the measures of IYP programme
there would be improvements in these effectiveness described in 1a and 1b, and
outcomes with parent participation in the b. Measures of parent satisfaction gathered
IYP programme. at programme entry and at programme
b. The second approach involved collecting completion.
teacher report information on child 4. Programme entry measures: To what extent do
behaviour at: baseline, mid-programme the outcomes of the IYP programme vary with
and programme completion. the child’s behavioural adjustment at baseline?
c. The third approach was a series of Special Education policy is to accept referrals
single case studies in which child from all sources without screening. They
behaviour outcomes were assessed at operate on the basis of IYP programme
weekly intervals over the course of the provision within a non-stigmatising
programme. The main purpose of this community setting. However, this policy is
component of the research was to examine not empirically based. To test this policy it
the various trajectories followed by was decided to examine the extent to which
children whose parents were enrolled in IYP programme outcomes as assessed in the
the IYP programme. design described in 1a and 1b varied with the
It was proposed to supplement these three child’s scores on the ECBI at baseline.
tests of programme efficacy with data from a
direct observation study using a single case
design but, for reasons discussed later, this
approach proved to be non-informative.
14
5. Between site differences: To what extent do the Participants
outcomes of IYP vary with the site at which the
All parents enrolled in these IYP courses at the three
programme was delivered?
sites were invited to take part in the evaluation;
As described below, the study was conducted
no parents were excluded from the research. The
at three Special Education sites. Comparisons
minimum sample size was set at 150 parents, 50 at
of the outcomes at the three sites were
each site. A total of 166 parents (Bay of Plenty=56,
investigated to examine the extent of between
Canterbury=57 and Mid-Central=53) agreed to
site variability in IYP programme outcomes.
participate and completed the baseline interview.
6. Client satisfaction: To what extent do the parents Written consent was obtained at each stage of the
who received the IYP training see the programme
study. Participants received a grocery voucher for
as: a) helpful; b) appropriately organised;
each completed interview as reimbursement for
c) effective in dealing with their child’s
their time.
behavioural problems; d) culturally appropriate?
To examine these issues parent satisfaction Preliminary research
data were gathered at the baseline interview,
the mid-course interview, the end-of-course Preliminary research assessing client uptake and
interview and the 6-month follow-up. programme fidelity in the delivery of IYP training
in Special Education courses was carried out from
The research-designated IYP June to December 2010. Eleven courses across
programmes four sites took part: Bay of Plenty (3), Waikato (3),
Central (3), and Canterbury (2). Programme fidelity
The IYP programmes evaluated in this study were documentation indicated the IYP programmes
delivered by the Ministry of Education Special
were delivered as specified by the Incredible Years
Education in 2011 at three sites: Bay of Plenty,
manual. Although issues with the administrative
Canterbury and Mid-Central. Four 18-week courses
data on the Special Education Incredible Years
were offered at each site in two phases, two in
National Register meant historic client uptake
the first half of the year (Phase 1) and two in the
data could not be ascertained, the problems were
second half (Phase 2), yielding 12 courses in all.
resolved so that reliable monitoring data were
Parents were referred to IYP training because they
available for the main study.
had a child aged 3–8 years with conduct problems.
Special Education did not make any changes to
their usual practice in organising and delivering the
Research design
research-designated courses. The IYP programmes The main study was a repeated measures design
evaluated in the Incredible Years Pilot Study were in which all research participants were interviewed
run in the same manner as Special Education IYP four times: at baseline before the IYP course began,
programmes not included in the research. mid-course, post-course and 6-month follow-up.
The research was conducted following a 3–5 Family Interviews were supplemented by a postal
year period in which Special Education was Teacher Questionnaire administered at baseline,
building capacity and capability for delivering post-course and follow-up. Data for the Family
IYP programmes throughout New Zealand. This Interviews and Teacher Reports from the two
entailed ensuring fidelity, procuring resources, phases of the study were combined.3 The study also
building Group Leader skills and national data included Single Case Studies using weekly Parent
collection processes to evaluate outcomes and to Reports in Phase 1 and a trial of Direct Observations
establish ongoing monitoring and quality control in Phase 2. Figure 2 contains a diagram of the
as part of normal service delivery. research design.
3 Analyses of outcomes by phase are reported in Chapter 6, Section 6.3.
15
Figure 2. The research design
Phase 1: Jan to June 2011 Phase 2: JULY to DEC 2011
Main Study Single Case Main Study Single Case

Canterbury, Mid-Central, & Bay of Plenty All sites Canterbury, Mid-Central, & Bay of Plenty Canterbury
Family Interviews Teachers Parent Reports Family Interviews Teachers Direct Obs
75 + parents 25 parents 75 + parents 8 parents
1 start of course
Interview 1 Report 1 Interview 1 Report 1
2 2
3
4 4
5
Interview 2 6 Interview 2 6
7
8 8
9
10 10
11
12 12
13
14 14
15
16 16
17
18 END of course 18
six-month follow-up
1
2
3
4
Family Interviews
Trained Interviewers employed for the Incredible Interview incorporated items from a number of
Years Pilot Study conducted the four home- previously validated measures to assess changes
based interviews with the primary caregivers. in outcome variables over the course of the study.
Each interview lasted approximately 1–1.5 hours Appendix 2 describes the source instruments
and covered a range of topics relating to child that comprised the Family Interview; a Technical
behaviour, parenting practices and relationships, Appendix (Gray 2013) provides more details
and the family context (Table 3). The Family about the instruments.
16
Table 3. Topics covered in the Family Interview
Sections Topic
A Family composition
B Child schooling
C to J Child behaviour (aggression, defiance, hyperactivity, self-control, anxiety/withdrawal, friendships, social skills)
K Child health
L & M Parenting practices, dealing with misbehaviour, child management (primary caregiver)
N Parenting practices, dealing with misbehaviour, child management (partner)
O Partners and partner relationships
P Primary carer’s health/depression
Q Stressful life events
R Standard of living
Family problems
Work, education, living circumstances
Ethnicity and cultural participation
S Expectations of/satisfaction with Incredible Years
Note there was a process in place if any questions in the interview caused distress to the participants, or if
the participants wanted help with an issue uncovered by the interview. Neither circumstance eventuated.
Teacher Questionnaires Single Case Studies

Childcare, preschool or class teachers of the Parent Reports: Family Interviews were
children whose parents gave permission were supplemented in Phase 1 at all three sites by
asked to complete a standardised behavioural Single Case Studies using weekly Parent Reports.
assessment for that child. The assessment was Participants for this component of the study
a modified version of the Canterbury Social were randomly selected from those enrolled in
Development Scale (Church et al 2006). The the research programmes. A total of 25 primary
questionnaires were posted to teachers at the caregivers completed weekly telephone Parent
beginning of the IYP course and again at course Reports, one for each of the 18 weeks of the IYP
completion and the 6-month follow-up. Two course, and again for 4 weeks at the 6-month
forms were used: one with wording appropriate follow-up. The Interviewers went through
for preschoolers and one with wording a checklist of 23 positive and 26 negative
appropriate for primary school children. behaviours that may have occurred in the
previous 24 hours.
17
Direct Observations: The measures described Administrative data
thus far are based on parent reports of
Parent referrals, uptake and retention were
behaviour. An attempt was made to supplement
monitored using a Recruitment Form designed for
the self-report data with direct observations
the study and Special Education’s administrative
of parents and their children interacting. The
records captured in the Incredible Years
intention was to trial the Direct Observations
National Register.
methodology in participants’ homes in
Canterbury in Phase 1 of the study, and to Special Education maintains a National Register
conduct these Single Case Studies at all three for each IYP course containing the demographic
sites in Phase 2. Due to delays, including the details of the parents and the pre-course ECBI and
Christchurch earthquake, the Phase 1 trial was Social Competency scores for the children. The
not possible. Instead, the methodology and Group Leaders recorded a variety of information
parent compliance were tested in a clinic setting on the Incredible Years National Register,
in Phase 2 in Christchurch only. Because of the including: how many, and which, sessions the
small sample of parents participating in this parent attended; whether she/he withdrew and
component of the study, and the fact it did at what point in the programme and the reason;
not achieve what was intended, the findings whether the Group Leaders did a make-up session
are not included in this report. Church (2012a) for parents unable to attend a particular session;
summarises the Direct Observations and and any additional home visits made to support
discusses lessons learned from the trial. parents or any support parents were receiving to
attend via a contribution to childcare or transport.
Programme fidelity
The Unified Protocol developed by Special
Māori in-depth study
Education assessed the fidelity of the IYP A separate study examined Māori perspectives
programme delivery from the point of parent of the IYP programme using a kaupapa Māori
recruitment through to the end of the delivery approach that drew upon kaupapa Māori research
of the treatment. The following aspects were methodology. The researchers obtained the
documented: recruitment, reducing barriers, views of four Māori Group Leaders and four
programme delivery (number of sessions, Māori parents who had been involved with the
order of activities, materials used, methods IYP programmes that were part of the evaluation
deployed, Group Leader training, and cultural to assess whether or not this mainstream
enhancements), client retention, compliance, programme met their needs. Māori researchers
quality assurance strategies, and follow-up. Group contracted by the Ministry of Education carried
Leaders completed the Unified Protocol checklists out this study (Berryman et al 2012).
at three points throughout the set-up and
delivery of the course: pre-programme, delivery
and final. Supervisors and Group Leaders also
provided reports of their IYP programme debriefs.
18
3.2 Fieldwork issues
Programme scheduling
The scheduling of two rounds of 18-week IYP The impact of these delays, if any, on the
research-designated courses in 1 year caused estimates of treatment gains is not known. They
some logistical problems for the providers and do, however, reflect the settings in which the
the researchers, particularly in Phase 2 in the programmes are usually delivered.
second half of the year. For the most part Interviewers reported little
The tight 2-week turnaround between Phase 1 trouble in scheduling appointments and
and Phase 2, which also coincided with school completing the interviews, although some
holidays, meant there was not the 4–6 week second and third appointments were necessary.
lead-in for home visits in Phase 2. This resulted in The shorter lead-in for the courses in Phase 2
some delayed parent referrals to the Interviewers. meant some of the baseline interviews were more
The tight timeframe also had an impact on the difficult to arrange than those in Phase 1. Most
scheduling of the interviews, notably the third of the interviews took between 1–1.5 hours to
interview for Phase 1 and the baseline interview complete. No incidents were reported.
for Phase 2. With precedence given to the latter,
there were delays in completing the former. This Evaluation team location
meant some interviews were completed several The evaluation team was located in Wellington,
weeks after the course had finished. The Phase 2 some distance from the three research sites. This
courses finished at the start of the summer school physical separation compounded the complexity
holidays in mid-December, a difficult time to of managing the fieldwork. A local project
engage parents in the research. To be consistent manager in each site would have been beneficial.
with the timing of interviews in Phase 1, these
Phase 2 end-of-course interviews were deferred
to mid-January.
The earthquake in Christchurch in February 2011
meant the courses in Phase 1 were delayed 1
month and it was not possible to complete all of
the baseline interviews before the IYP programme
began. Finding suitable venues was also an issue.
The disruption resulting from the earthquake
created considerable problems both for the
Group Leaders organising the IYP courses and
for the parents attending. However, operating
under these trying conditions has added to
the ecological validity of the study findings for
Canterbury.
19
3.3 Statistical analyses
The main analyses are based on examining estimate is the standardised difference between
the outcomes of the IYP programme for the means or proportions, expressed as Cohen’s ‘d’.
parents and their children. These outcomes (Cohen 1977). Cohen suggests that an effect
are determined by comparing the measures of size of d=.20 is small, an effect size of d=.50 is
child behaviour, parenting practice and other medium, and an effect size of d=.80 is large.
measures observed at baseline with the same In real-world terms, the relationship between
measures observed again at mid-course, course smoking and lung cancer is large, whereas the
completion, and at a 6-month follow-up. All data relationship between life stress and depression
are based on self-reports provided by parents and is small. These interpretations are arbitrary
teachers. Data from the two phases of the Study but provide an indication of the extent of the
were combined.4 The analyses were carried out behaviour change. A positive effect size (d>0)
on an intention-to-treat research design, which indicates improved behaviours while a negative
includes parents who had attended the parenting effect size (d<0) indicates worsened behaviours.
programme and those who had not or who had
dropped out. Testing for interactions between
Any changes in the average scores of measures sub-group pairs
at each observation time were statistically tested The programme outcomes or effects as
under the hypothesis there was a straight-line or demonstrated by behaviour change over time
linear trend of change in the scores from baseline, may be different between sub-groups of parents
through mid-course, to course completion. The or children. Tables of programme outcomes by
maintenance or sustainability of behavioural key dichotomous sub-groups (or pairs) were
change was also tested under the hypothesis examined with the estimated effect sizes for each
there was a statistical difference in the mean sub-group pair. These tables contain statistical
scores observed between baseline and at the tests for interactions based on a 2x2 factorial
6-month follow-up. A probability value (or analysis, which compares the average scores at
p-value) is provided by the statistical tests for the two observations in time (eg baseline and
each measure of behaviour. If the p-value is small course completion) by the two sub-group pairs of
then the probability there was no linear trend interest (eg Māori and non-Māori children).
of change or no difference in the mean scores is
small. As a standard rule, p-values smaller than the The 2x2 factorial analyses are performed using
α=.05 level were accepted as evidence of a linear Analysis of Variance models, which provide
trend of change in average scores. p-values representing the probability there was
no interaction between the time of observation
Effect sizes and the sub-groups. P-values smaller than the
α=.05 level suggest there was an interaction,
In addition to the statistical test of changes in which essentially suggests the two sub-groups of
behaviour through the course, the size of those parents or children had different effect sizes.
changes is also examined. It is the size of the
outcome that is of key interest. The size of the
changes in behaviour demonstrates the size of the
programme outcomes, which is expressed using
‘effect size’ estimations. The effect size
4 Analyses of outcomes by phase are reported in Chapter 6, Section 6.3.
20
Adjustments to multiple statistical tests
Each programme outcome and outcome
sustainability table examines behaviours with
more than one measure. For instance, Child
Behaviour was assessed with eight measures and
a statistical test of significance for each measure.
Multiple significance testing increases the
chance of false-positive conclusions where stated
significant differences in average scores occurred
by chance. To minimise the chance of false-
positive conclusions, the α=.05 level for p-values
was adjusted using a Bonferroni adjustment
where the α=.05 level was divided by the number
of tests made in each table. For example, α=.05
level was divided by eight for the Child Behaviour
tables, which means only p-values smaller than
.006 were considered significant.
The outcomes by sub-group tables contain
multiple statistical tests for interactions for
each individual measure. Each table was further
supported by a 2x2 factorial analysis where all
the measures in a table were analysed as one
multivariate. These tests were done using a
Multivariate Analysis of Variance, which provided
a p-value that represents the probability there
was no interaction between the observation
times on the overall measure and the two
sub-group pairs.
Profile tables
The differences between parent and child profiles
discussed in this report were verified using
chi-square statistics on contingency tables where
p-values were less than α=.05.
Details of the statistical models used

The statistical models used in this study are
described in Appendix 3, with more details
available in the Technical Appendix (Gray 2013).
21
Chapter 4:
Recruitment, retention
and participant profiles
4.1 The referral process

to Special Education
Incredible Years
programmes
In the Bay of Plenty the local district health
board (DHB) handled the wait-list for the IYP
programmes delivered by all providers in the
region. Parents were referred to the DHB, who
then sent letters to everyone on the wait-list
when courses became available. The DHB
forwarded a list to Special Education of only those
parents interested in an IYP course delivered by
Special Education. In Mid-Central and Canterbury
the wait-lists did not go through the DHB, but
went directly to Special Education.
Special Education IYP Group Leaders contacted
parents from the wait-list in chronological order
when an IYP programme became available. They
aimed to recruit 20 parents for each course to
allow for drop-outs. Interested parents were visited
This chapter describes the referral process to at home. At this time the programme and time
Special Education IYP courses, the recruitment commitments were explained, the concerns for
and retention of parents to the research and the her/his child discussed and any barriers identified
characteristics of the children, primary caregivers (eg transport or childcare). If, after this, the parent
and households of the participant families. intended to enrol in the IYP programme she/he
completed the consent forms and the pre-course
measures. For the IYP programmes included in
the Incredible Years Pilot Study, the research was
explained and the consent to be contacted by the
Interviewer was sought.
22
4.2 Research participant
recruitment and retention
Following the home visit, Group Leaders Figure 3. Incredible Years Pilot Study
provided the names and contact details of those participant recruitment and
parents who gave consent to be contacted
retention
for the research to the Interviewers who then
arranged an appointment to conduct the baseline
interview. Figure 3 contains a flow diagram
accounting for all parents the Group Leaders IN Home-visited OUT
visited through to course enrolment and research for enrolment
participation. Research participation rates were N=229
calculated both as a proportion of families
home-visited (Invitation to Treat) and as Intended to enrol Did not enrol
a proportion of families enrolled in an IYP N=195 N=34
programme (Intention to Treat). Group Leaders
completed a Recruitment Form that documented
Not referred to study
the number of families contacted for the IYP N=5
course, those who did not enrol in the course, and
those who did not consent to be contacted for
Study consent not given
the research. N=18
The Group Leaders obtained parental consent to Study consent given
be contacted for the research from 75 percent of N=172
those home-visited to enrol in the IYP programme Withdrew
N=6
(Invitation to Treat) and 88 percent of those who
Interview #1
actually did enrol at the time of the home visit N=166
(Intention to Treat). Seventy-two percent of the
Invitation to Treat parents and 85 percent of the
Intention to Treat parents completed the baseline Interview #2
interview. Of these 166 parents, 162 (98%) N=166
remained in the research through the follow-up Withdrew
N=2
interview. Recruitment and retention rates were
Interview #3
similar at all three sites.
N=164
Withdrew
N=2
Interview #4
N=162
23
4.3 Participant profiles
Parent loss from the Incredible Years Pilot Study The participant profiles presented are as at the
up to the point of IYP course enrolment was baseline interview for the 166 children and
relatively high (25%), whereas the loss was very parents participating in the Incredible Years Pilot
low (4%) after enrolment. Of the 57 parents the Study. Similar client profiles have been reported
Group Leaders visited who did not take part in in other Incredible Years research (eg Hutchings
the research, most (34) decided not to enrol in the et al 2007, Larsson et al 2009; Scott et al 2001;
programme while a further five were not offered Webster-Stratton and Hammond 1997).
the research due to an administrative error. Most
The profile of participants shows the research
of the parents who had expressed interest in
reached the desired target group: the majority of
the IYP programme when contacted but did not
the children were aged 3–8 years and one-third
enrol stated that their family circumstances had
were identified as Māori by their primary caregivers.
changed or that they were no longer interested.
A comparison of the participants in Phase 1 and
Eighteen of the parents intended to enrol and
Phase 2 indicates that splitting the study into two
were invited to participate in the research but
phases did not introduce bias into the sample as
declined to be involved with the study. Nearly all
the only significant difference between the two
of these parents gave no reason for not taking
groups was with the occurrence of one stressful life
part in the study although a few felt it would be
event. Most of the significant differences found in
too intrusive or they were too busy.
the profiles of the parents at each of the three sites
While the high retention rate of the 166 parents in were as expected, such as the highest proportion
the study indicates good internal validity for the of Māori parents being located in the
Incredible Years Pilot Study, the extent to which Mid-Central region.
the findings apply to the 229 parents contacted
for the IYP programme is not known. This raises The children
issues about external validity. Data were not
Almost all (94%) of the children in the study
available to assess whether or not the study
were within the target age range of the IYP
sample was representative of all parents referred
programme, with 40 percent aged 3–4 years
for IYP training, although it is not unreasonable to
and 54 percent aged 5–8 years. Just over a third
assume it would be similar.
(36%) were identified by their primary caregivers
as Māori, and three-quarters (73%) were male.
The distribution of ethnicity varied between the
sites, with the smallest proportion (19%) of Māori
children in Canterbury, over one-third (38%) in
the Bay of Plenty, and more than half (53%) in
Mid-Central.
Only one of the children in the study had
not attended preschool and very few (8%)
had been refused entry to preschool or had
been stood down or suspended from primary
school. Kindergartens, play centres, and general
childcare facilities accounted for most (84%) of
the preschools attended. There were differences
in the types of preschools attended by site,
with Canterbury having the highest proportion
attending mainstream preschools/schools (95%),
followed by the Bay of Plenty (81%) and Mid-
Central having the lowest proportion (77%).
24
One in five (20%) of the children had contact with Just over half (54%) of the parents had no
Special Education within the 3 months before qualifications or school qualifications only, while
the baseline interview and slightly more (23%) 31 percent had a certificate or diploma and
had seen a psychologist. A higher proportion 12 percent had a degree. The highest level of
of the children in the Bay of Plenty had seen a education differed by site, with the Bay of Plenty
psychologist (39%) than in either Canterbury having proportionally more educated primary
(18%) or Mid-Central (11%). About a third (37%) caregivers and Canterbury having the fewest.
of the children had been to health professionals Canterbury had twice as many primary caregivers
such as a GP, paediatrician or public health nurse. with no education qualifications (49%) as the Bay
A minority had visited specialist health services of Plenty (25%) and Mid-Central (25%), whereas
such as an audiologist (13%), a speech/language the Bay of Plenty had the highest proportion of
therapist (13%) or an optometrist (10%). Specialist degree-qualified primary caregivers (20%)
mental health services (10%), social workers compared with Mid-Central (9%) and
(11%) and Barnardos (11%) were also mentioned. Canterbury (7%).
Significantly fewer participants used these
A quarter (24%) of the study parents reported
services in Canterbury compared with those at
eight or nine of the nine depression symptoms
the Bay of Plenty and Mid-Central sites.
measured, and a further 22 percent reported six
or seven of the nine symptoms. A third of primary
The Primary caregivers caregivers (36%) showed signs of mid to high
The great majority of primary caregivers were levels of stress, and 13 percent showed medium
women (95%) and the natural mother of the to high levels of anxiety.
participant child (92%). About half (55%) of
the primary caregivers were currently married
or partnered and the others were either single
(33%), separated or divorced (10%) or living apart
from a partner (2%). Two-thirds (64%) were aged
25–39 years, one in 10 (11%) were aged 19–24
years and a quarter (25%) were older than 40.
However, a majority (60%) of the caregivers had
their first child when they were younger than
24 years of age.
Nearly a quarter (23%) of primary caregivers
identified themselves as Māori. Mid-Central had
the highest proportion of Māori caregivers (40%),
with the Bay of Plenty next (20%) and Canterbury
with the lowest proportion (12%).
25
The households
Where the primary caregiver had a resident Four in five (82%) of the households experienced
partner, the data for the respondent and at least one stressful life event in the 3 months
the partner were combined to describe the before the baseline interview. Two-thirds (67%)
household. reported three or more such events. Serious
illness, accident or death affected almost half
Two-thirds of the participants (66%) lived in
(45%) of the households, serious financial
households of four or more people, including
problems were experienced by 39 percent, almost
siblings. About a quarter (23%) of the children
a third (31%) reported prolonged arguments
had no siblings, while 40 percent had one sibling
within their family/extended family, and a quarter
and 21 percent had two. Private rental (54%)
(25%) of the primary caregivers experienced
was the most common type of accommodation,
prolonged arguments with their partners.
followed by mortgaged housing (30%) and state
housing (10%). Households in Canterbury reported significantly
fewer stressful life events than did those in the
About a third (36%) of the caregivers were single
Bay of Plenty or Mid-Central. Fifty-one percent of
and unemployed, with a further 30 percent being
Canterbury households reported at least three
part of a one-income couple. Employment was
stressful life events compared to Mid-Central
the main source of household income for over
households (72%) and Bay of Plenty households
half (56%) of the participants and the Domestic
(79%). Arguments in the family were more
Purposes Benefit provided the income for a third
common in the first phase than in the second
(34%). The majority of households in the study
phase of the study, with 42 percent of households
lived in areas at the medium to high (poverty)
reporting arguments in Phase 1 compared to
end of the New Zealand Deprivation Index, which
19 percent in Phase 2. This was the only
is based on current census data. Eighty percent
significant difference between the Phase 1
were in quintiles 3 (25%), 4 (26%) and 5 (29%)
and Phase 2 samples.
of the Index, where 5 is the most deprived. The
three sites differed slightly in terms of the Index
with Mid-Central having the highest proportion
of participants living in quintile 5 (42%). In
comparison, the Bay of Plenty had 26 percent in
quintile 5 and Canterbury had 20 percent.
26
Chapter 5:
Findings related to child
behaviour, parenting
practice and relationships
Central to the Incredible Years Pilot Study was
the assessment of change in child behaviour and
parenting practice following the IYP training. This
chapter presents the findings of these before-and-
after comparisons. The comparisons were made
on all study participants irrespective of the extent
to which they had participated in IYP courses. The
child behaviour and parenting practice measures
were based on self-reported data collected in the
Family Interview administered before, during and
after the IPY course and again 6 months later.
Family Interview data made up the core of the
research.5 The outcomes of the IYP programme
were determined by comparing the first
(baseline) interview scores with the scores of
the third (post-course) interviews. The second
(mid-course) interview was used in the test
for a linear trend. The trend p-value provides
the evidence of a linear trend of change in the
mean scores throughout the duration of the
programme. Data from the fourth (follow-up)
interview assessed whether or not any changes
evident at course completion were sustained
over the medium term. All changes referred to
in Table 3 onwards are changes in mean parent
or teacher reports.
5 Data re-entry checks produced an infinitesimal

error rate across all items tested.
27
5.1 Child behaviour
Pre-test/post-test comparisons: Social Competence: This dimension was based

Child behaviour on a sum of 26 items describing the extent to
which the child showed helpful, empathetic,
The instruments6 respectful, diligent or likeable behaviours
In the Family Interview, parents were questioned (α=.91).
about their child’s behaviour during the past 4
weeks using 111 items based on four recognised The findings
scales: the Eyberg Child Behaviour Inventory As shown in Table 4, parents reported significant
(Eyberg and Ross 1978; Eyberg 1980), the Strengths improvement in their children’s behaviour on all
and Difficulties Questionnaire (Goodman 1997), the six measures in the Family Interview following
Incredible Years Social Competence Scale (Corrigan the completion of the IYP programme. All the
2002) and items from the 5-year evaluation of Early behaviour scores show a linear trend of improving
Start (Fergusson et al 2005b). For consistency, all change through the programme (p<.001). The
scale items were rated on a 3-point rating scale: not effect sizes of these changes were medium to
at all, somewhat and a great deal. large. The greatest improvement was evident for
Oppositional Defiant Disorder (d=.96) and
Confirmatory factor analysis of this item set
Self-control (d=.96), followed by Social
(Horwood et al 2011) showed these items
Competence (d=.68) and Conduct Disorder
measured six correlated dimensions of child
(d=.62). ADHD (d=.55) and Anxiety/Withdrawal
behaviour7. These dimensions were:
(d=.51) showed moderate improvement.
Conduct Disorder: This dimension was based
on a sum of 18 items describing the extent Significant, large improvements were also
to which the child displayed aggressive and reported in the ECBI Intensity (d=1.0) and Social
antisocial behaviours. The reliability of the Competency (d=.87) scales administered by the
scale assessed by Cronbach’s alpha (α) was .87. Group Leaders before and after the IYP course.
The international RCT studies quoted in Chapter 2
Defiance: This dimension was based on a sum
(Gardner et al 2006; Hutchings et al 2007;
of 15 items describing the extent to which
Larsson et al 2009; Scott et al 2001;
the child showed oppositional, defiant or
Webster-Stratton and Hammond 1997) all
dishonest behaviours (α=.89).
reported post-treatment effect sizes for child
ADHD: This dimension was based on a sum behaviour change in this range.
of 16 items describing the extent to which
the child showed hyperactive, impulsive or Moreover, the proportion of children scoring
inattentive behaviours (α=.92). above the clinical cut-off of 60 on the ECBI
Intensity Scale dropped from 75 percent
Self-control: This dimension was based on a
pre-course to 19 percent post-course.
sum of 15 items describing the extent to which
Twenty-five percent of the children were
the child showed self-regulatory, flexible or
below the clinical cut-off of 17 on the Social
compliant behaviours (α=.87).
Competency Scale pre-course compared with
Anxiety/Withdrawal: This dimension was based 58 percent post-course. Larsson et al (2009)
on a sum of 16 items describing the extent to also reported significantly fewer children in
which the child showed anxious, withdrawn or the clinical range after the IYP course.
shy behaviours (α=.78).
6 See Appendix 2 for descriptions of source instruments.

7 Five of the 111 items were discarded as they did not belong obviously to any of the subscales.
28
Table 4. Child behaviours: Mean parent reports before, during and after IYP programme
Baseline Mid-course Post-course Trend Post-course effect

mean (std) mean (std) mean (std) p6 size d (95% CI)
Child behaviours1
Conduct Disorder (CD) 28.8 (6.9) 25.6 (6.3) 24.5 (5.6) <.001 .62 (.40 to .84)
ODD2 35.8 (7.3) 31.4 (7.5) 28.8 (6.8) <.001 .96 (.73 to 1.2)
ADHD3 34.2 (8.5) 31.2 (8.3) 29.6 (8.5) <.001 .55 (.33 to .77)
Self-control 35.9 (7.8) 40.4 (8.0) 43.3 (8.2) <.001 .96 (.73 to 1.2)
Anxiety/Withdrawal 27.4 (5.8) 25.4 (5.6) 24.5 (5.6) <.001 .51 (.29 to .73)
Social Competence 54.6 (10.4) 58.5 (10.7) 61.5 (10.2) <.001 .67 (.45 to .90)
In-course behavioural assessments

ECBI Intensity4 65.1 (10.2) 54.8 (9.5) <.001 1.0 (.72 to 1.3)
Social Competence Scale 5
17.1 (7.9) 23.9 (8.5) <.001 .87 (.59 to 1.2)
1
Sample sizes are N=166, 165, 163 (baseline, mid-course, post-course).
2
Oppositional Defiant Disorder.
3
Attention Deficit Hyperactivity Disorder.
4
Sample sizes are N=151, 108 (baseline, post-course).
5
Sample sizes are N=150, 107 (baseline, post-course).
6
Trends with p-values <.006 are considered significant allowing for the multiple statistical tests of significance (Bonferroni adjustment
to α=.05 for eight tests).
Child behaviour outcomes at follow-up All the behaviour scores in Table 5 showed a
Parents were interviewed again about half a year linear trend of improving change (p<.001) with
after course completion, which is approximately the estimated effect sizes ranging from medium
1 year after the baseline interview before the to large (d=.56 to d=1.0, median d=.71). Parents
start of the IYP programme. The results from reported their children’s improved behaviours
the follow-up interviews were compared to the following the IYP programme were sustained
baseline interviews to measure the medium-term 6 months after programme completion. The ECBI
sustainability of any behaviour change evident at Intensity scores and the Social Competence Scale
post-course. were obtained by Group Leaders before the IYP
course and at completion only. These measures
were not collected at follow-up.
29
Table 5. Child behaviours: Mean parent reports before IYP programme and at follow-up
Baseline Follow-up Trend Follow-up effect size

mean (std) mean (std) p4 d (95% CI)
Child behaviours1
Conduct Disorder (CD) 28.8 (6.9) 24.1 (6.0) <.001 .68 (.46 to .91)
ODD 2
35.8 (7.3) 28.6 (7.6) <.001 .98 (.75 to 1.2)
ADHD 3
34.2 (8.5) 29.0 (8.3) <.001 .61 (.39 to .84)
Self-control 35.9 (7.8) 43.9 (8.7) <.001 1.0 (.81 to 1.3)
Anxiety/Withdrawal 27.4 (5.8) 24.2 (5.6) <.001 .56 (.34 to .78)
Social Competence 54.6 (10.4) 62.1 (10.5) <.001 .73 (.50 to .95)
1
Sample sizes are N=166, 162 (baseline, follow-up).
2
3
4
to α=.05 for six tests).
5.2 Parenting practices and

relationships
Pre-test/post-test comparisons: disorder. It loads onto five subscales: parental

Parenting practice supervision, positive parenting, corporal
punishment, parental involvement and
The instruments8 inconsistent discipline (α=.44 to .79, median
The parenting practices measures in the α=.53). An evaluation with a community sample
Family Interview consisted of two recognised of Australian children aged 4–9 years (Dadds et al
instruments: the Alabama Parenting 2003) showed good internal consistency, validity
Questionnaire (Shelton et al 1996) and the and test-retest reliability for the measure.
Arnold-O’Leary Parenting Scale (Arnold et al The Arnold-O’Leary Parenting Scale is a
1993). Intact instruments were used, but the scale 30-item inventory of parenting competencies
items were all rated on a consistent 3-point scale: that measures dysfunctional and/or ineffective
never, sometimes and often for the Alabama; parenting practices of parents with younger
never, less than once a month and once a week or children. The scale yields an overall score and
more for the Arnold-O’Leary. three revised subscale scores of dysfunctional
The Alabama Parenting Questionnaire is a 42-item strategies used by parents tackling problem
scale designed to tap the parenting dimensions behaviour (α=.42 to .74, median α=.65). ‘Laxness’
that are risk factors associated with child conduct refers to insufficient monitoring of the child
30
and her/his behaviour, allowing rules to go the programme (p<.001). The sizes of the
unenforced or providing positive reinforcement improvements were medium or medium/large
for misbehaviour. ‘Over-reactivity’ refers to displays (d=.46 to d=.71), with the exception of a small
of anger, meanness or irritability. ‘Hostility’ refers improvement for Poor supervision (d=.26). The
to the use of verbal or physical force. The scale has overall median effect size was d=.54 (excluding
adequate internal consistency and has been found the Total Scale). The Total Scale (Arnold-O’Leary
to have good test-retest reliability. Parenting Scale), which includes items additional
to those loading on the specific factors, produced
The findings a large effect size of d=.83. The parents in
According to parent reports of their parenting the Hutchings et al (2007) and Larsson et al
practices, all eight measures obtained in the (2009) studies also reported the reduced use
Family Interviews showed an improvement of harsh and inconsistent discipline as well
after the parents had completed the IYP course as the increased use of positive parenting
compared to their reported practices before the strategies, while Gardner et al (2006) reported a
course (Table 6). All parenting factors showed reduction in observed negative parenting and an
significant linear trends of change throughout improvement in observed positive parenting.
Table 6. Parenting practices: Mean parent reports before, during and after IYP programme
Parenting measures1 Baseline Mid-course Post-course Trend Post-course effect

Parenting practices2
Poor supervision 6.6 (1.3) 6.3 (0.8) 6.3 (0.9) <.001 .26 (.04 to .47)
Positive parenting 13.7 (1.5) 14.3 (1.0) 14.4 (0.9) <.001 .46 (.24 to .68)
Corporal punishment 3.7 (0.8) 3.4 (0.7) 3.2 (0.6) <.001 .59 (.37 to .81)
Parental involvement 11.8 (2.0) 12.4 (1.8) 12.9 (1.7) <.001 .53 (.31 to .75)
Inconsistent discipline 7.4 (2.2) 6.6 (1.9) 6.2 (2.0) <.001 .54 (.32 to .76)
Dealing with misbehaviour3

Lax discipline 8.2 (2.1) 7.5 (2.0) 7.1 (1.9) <.001 .53 (.31 to .75)
Over-reactive discipline 8.7 (2.2) 7.7 (1.6) 7.1 (1.7) <.001 .71 (.49 to .94)
Hostile discipline 4.4 (1.1) 3.8 (1.0) 3.7 (0.9) <.001 .68 (.46 to .91)
Total scale4
1.6 (0.3) 1.5 (0.3) 1.4 (0.3) <.001 .83 (.60 to 1.1)
1
Sample sizes are N=166, 166, 164 (baseline, mid-course, post-course).
2
From the Alabama Parenting Questionnaire.
3
From the Arnold-O’Leary Parenting Scale.
4
The Total Scale includes all the questions from the Arnold-O’Leary Parenting Scale including additional questions that are not part of
the Lax, Over-reactive, or Hostile discipline factors, and the Total Scale scores are standardised to an average score between 1 and 3.
5
Trends with p-values <.005 are considered significant allowing for the multiple statistical tests of significance
(Bonferroni adjustment to α=.05 for nine tests).
31
Parenting practice outcomes at follow-up were small to medium (d=.33 to d=.68) with Poor
The follow-up effect sizes indicated the improved supervision remaining small (d=.25). The overall
parenting practices evident at programme median effect size was d=.54, excluding the Total
completion were sustained 6 months later. As Scale. The Total Arnold-O’Leary Parenting Scale
shown in Table 7, the significant linear trends of maintained a large effect size of d=.79 at the
change in parenting practices remained at the fourth interview.
follow-up interviews (p<.001). The effect sizes
Table 7. Parenting practices: Mean parent reports before IYP programme and at follow-up
Parenting measures1 Baseline Follow-up Trend Follow-up effect size

Parenting practices2
Poor supervision 6.6 (1.3) 6.3 (0.8) <.001 .25 (.04 to .47)
Positive parenting 13.7 (1.5) 14.2 (1.1) <.001 .33 (.11 to .54)
Corporal punishment 3.7 (0.8) 3.3 (0.6) <.001 .48 (.26 to .70)
Parental involvement 11.8 (2.0) 12.8 (1.8) <.001 .47 (.25 to .69)
Inconsistent discipline 7.4 (2.2) 6.2 (1.9) <.001 .55 (.33 to .77)
Dealing with misbehaviour3

Lax discipline 8.2 (2.1) 7.1 (2.0) <.001 .52 (.29 to .74)
Over-reactive discipline 8.7 (2.2) 7.2 (1.6) <.001 .68 (.46 to .90)
Hostile discipline 4.4 (1.1) 3.7 (1.0) <.001 .60 (.38 to .82)
Total scale4
1.6 (0.3) 1.4 (0.3) <.001 .79 (.56 to 1.0)
1
Sample sizes are N=166 baseline, 162 follow-up.
2
From the Alabama Parenting Questionnaire.
3
From the Arnold-O’Leary Parenting Scale.
4
the Lax, Over-reactive, or Hostile discipline factors, and the Total Scale scores are standardised to an average score between 1 and 3.
5
to α=.05 for nine tests).
32
Pre-test/post-test comparisons: The Partner Attachment Scale measures the
Conflict and relationships quality of the relationship between parents.
The items used in the Incredible Years Pilot
The instruments9 Study were based on a selected series of items
from Braiker and Kelley (1979) as used in the
The revised Straus Parent/Child and Partner
Christchurch Health and Development Study
Conflict Tactics Scales (Straus et al 1996; Straus et
(CHDS) 21-Year Interview, and included 12 items
al 1998), the Partner Attachment Scale (Braiker
producing reliability coefficients of α=.80 to .93,
and Kelley 1979) and the Dadds Parent Problem
median α=.88.
Checklist (Dadds and Powell 1991) measured
parents’ relationships with their children and
The findings
partners. The intact instruments were used with
scale items all rated on a consistent 3-point scale: Table 8 summarises the findings on a range of
never, less than once a week and once a week inter-partner and parent/child relationships.
or more (Parent/Child Conflict Tactics); never, Verbal aggression and Physical assault, for both
sometimes and often (Partner Conflict Tactics); not the primary caregivers and their partners, showed
at all, somewhat and a great deal (Parent Problem a linear trend of change (p<.001) with medium
Checklist); doesn’t apply, somewhat applies and effect sizes (d=.46 to d=.60, median d=.50).
definitely applies (Partner Attachment Scale). Reported Child-rearing disagreement between
parents reduced (p<.001) with a medium
The Conflict Tactics Scale Parent/Child (CTSPC) estimated effect size of d=.49 but there was no
and the Revised Conflict Tactics Scale 2 (CTS2) discernible difference in the quality of the parents’
assess violence within families and intimate relationship before and after the IYP course. There
relationships. The 24 CTSPC items related to the was a small reduction reported in violence from
parents’ use of verbal aggression and physical the partner towards the primary caregiver (d=.27)
assault in their relationships with their children but no change in violence from the primary
(α=.28 to .73, median α=.50). The CTS2 focused caregiver towards the partner.
on violence between the parents (α=.67 to .87,
median α=.82) across 42 items.
The Parent Problem Checklist (PPC) was
developed as a measure of inter-parental conflict,
especially as it relates to the parents’ ability to
co-operate and to act as a team in performing the
executive parenting functions within the family.
It contains 16 items measuring the presence or
absence of parental disagreement over rules and
discipline for child misbehaviour, the occurrence
of open conflict over child-rearing issues and
whether or not parents undermine each other’s
relationships with the children (α=.86 to .93,
median α=.89). The PPC is a unidimensional
measure with moderately high internal
consistency and high test-retest reliability.
33
Table 8. Parental conflict and relationships: Mean parent reports before, during and after
IYP programme
Relationship types Baseline Mid-course Post-course Trend Post-course effect

Primary caregiver and child1
Verbal aggression 8.4 (2.3) 7.3 (2.1) 7.0 (2.0) <.001 .60 (.38 to .83)
Physical assault 11.9 (1.2) 11.5 (1.1) 11.3 (0.8) <.001 .50 (.28 to .72)
Other parent2 and child

Verbal aggression 7.5 (2.3) 6.6 (1.8) 6.4 (1.5) <.001 .50 (.20 to .81)
Physical assault 11.8 (1.3) 11.5 (1.1) 11.2 (0.7) <.001 .46 (.15 to .77)
Inter-parental violence3
Violence to partner 22.5 (2.3) 22.1 (2.4) 21.9 (1.4) .182* .23 (-.07 to .54)
Violence from partner 22.6 (2.5) 22.0 (1.7) 21.9 (1.5) .003 .27 (-.03 to .58)
Inter-parental relationship4
Child-rearing disagreement 25.0 (6.8) 22.3 (5.8) 21.6 (5.1) <.001 .49 (.18 to .80)
Relationship quality 31.5 (4.5) 32.0 (4.1) 32.5 (3.4) .108* .21 (-.09 to .51)
1
Scores are based on the Parent/Child Conflict Tactics Scale (M. Straus); for the primary caregiver sample sizes were N=166, 165, 163
(baseline, mid-course, post-course).
2
For the other parent and inter-parental factors (where applicable) samples sizes were N=91, 87, 84 (baseline, mid-course, post-course).
3
Inter-parental violence is from the Revised Conflict Tactics Scale (CTS2, M. Straus) as found in the CHDS 21-Year Interviews.
4
Child-rearing disagreement is from the Parent Problem Checklist (M. Dadds), and Relationship quality is from the CHDS 21-Year Interviews.
5
* No evidence of a linear trend of change.
34
Conflict and relationships outcomes at outcomes at post-programme. In addition,
follow-up caregivers reported a reduction in their
As shown in Table 9, the reported changes in partners’ violence towards them (median d=.32,
conflict and relationships at follow-up (d=.40 p<.001) not evident at programme completion.
to d=.59, median d=.47) were similar to the Relationship quality remained unchanged.
Table 9. Parental conflict and relationships: Mean parent reports before IYP programme
and at follow-up
Relationship types Baseline Follow-up Trend Follow-up effect size

Verbal aggression 8.4 (2.3) 7.1 (2.0) <.001 .59 (.37 to .82)
Physical assault 11.9 (1.2) 11.4 (0.8) <.001 .46 (.24 to .68)

Verbal aggression 7.5 (2.3) 6.4 (1.7) <.001 .48 (.18 to .78)
Physical assault 11.8 (1.3) 11.3 (0.8) <.001 .40 (.11 to .70)
Violence to partner 22.5 (2.3) 21.8 (1.9) <.001 .29 (-.01 to .58)
Violence from partner 22.6 (2.5) 21.7 (1.6) <.001 .35 (.05 to .64)
Child-rearing disagreement 25.0 (6.8) 21.5 (6.3) <.001 .51 (.21 to .80)
Relationship quality 31.5 (4.5) 32.2 (4.0) .230* .15 (-.14 to .44)
1
Scores are based on the Parent/Child Conflict Tactics Scale (M. Straus); for the primary caregiver sample sizes were N=166, 162
(baseline, follow-up).
2
For the other parent and inter-parental factors (where applicable) samples sizes were N=91, 90 (baseline, follow-up).
3
Inter-parental violence is from the Revised Conflict Tactics Scale (CTS2, M. Straus) as found in the CHDS 21-Year Interviews.
4
Child-rearing disagreement is from the Parent Problem Checklist (M. Dadds), and Relationship quality is from the CHDS 21-Year Interviews.
5
* No evidence of a trend of improvement or change.
35
5.3 Teacher10 pre/post measures
of child behaviour
The behaviour of children displaying conduct Teachers of the study children reported on
problems may be setting-specific, such as at home 30 specific positive and negative behaviours
or at school. Webster-Stratton (2012) noted that in the school, preschool, or care centre setting
only about half of the children with behaviour before the parents attended the IYP programme.
problems at home also display these behaviours This was repeated at post-programme and
in the school environment. To assess whether or follow-up for the children for whom there was
not any improvements in child behaviour had teacher baseline data. Lower scores indicated
generalised from home to school following the IYP better behaviour.
training, behaviour in the classroom or early child
centre was assessed using the Canterbury Social Outcomes post-course and follow-up in
Development Scale11 (Church et al 2006). This is a the school setting
30-item rating scale which consists of brief
descriptions of 15 antisocial behaviours and Table 10 shows teachers reported small but
15 positive social behaviours which are likely to significant improvements in child behaviours, on
occur in the preschool/school environment (α=.96). average, in the school setting at post-course (71.2)
The original scale was adapted by reducing the compared to the baseline score (75.7), albeit with
5-point rating scale to a 3-point rating scale. a very modest effect size of d=.17 (p=.008). The
follow-up score of 67.9, with the small effect size
Packets containing information about the Incredible of d=.29 (p=.005), demonstrated the behaviour
Years Pilot Study, a copy of the Social Development change was sustained and slightly improved
Scale and a self-addressed pre-paid envelope for (Table 11). The effect sizes on teacher reports of
returns were couriered to the preschool/school behaviour change were much smaller than those
principals of the 160 study children whose parents based on parent reports.
had given consent for the researchers to contact
their children’s teachers or childcare staff. Principals
were asked to pass the packets on to the relevant
teachers. After two telephone follow-ups, the return
rate was 86 percent.
Table 10. Child behaviours: Mean teacher reports before and after IYP programme
Baseline Post-course Trend Post-course effect size

mean (std) mean (std) p d (95% CI)
Teacher reports1
Total scores 75.7 (27.1) 71.2 (25.5) .008 .17 (-.07 to .40)
Sample sizes are N=150 baseline, 139 post-course.

1
10 The term ‘teacher’ refers to preschool/school teacher or childcare worker; ‘school’ refers to preschool/school or
childcare facility.
36
Table 11. Child behaviours: Mean teacher reports before IYP programme and at follow-up

mean (std) mean (std) p d (95% CI)
Teacher reports1
Total scores 75.7 (27.1) 67.9 (25.4) .005 .29 (.04 to .54)
Sample sizes are N=150 baseline, 124 follow-up.

1
Referrals from schools those who were school-referred and other, the
children in the former group appeared to engage
It is possible that referrals to IYP training from a
in higher rates of misbehaviour, on average,
school-based source (early childhood education
than children in the latter. This difference was
teacher, Resource Teacher Behaviour and
significant (p=.001 and p<.001) and occurred at
Learning, classroom teacher, or Special Education)
each stage of the study (Table 12).
indicated poorer child behaviour in the school
setting. When teacher reports were grouped into
Table 12. Child behaviours: Mean teacher reports before and after IYP programme and at
follow-up, by referral source
Non-school Difference between Test1 for no difference

referred School referred referral source between referral source2
mean (std) mean (std) mean (std) p
Teacher reports
Baseline scores3 67.3 (25.6) 87.2 (25.8) 19.9 (25.7) <.001
Post-course scores 4
64.4 (23.7) 80.0 (26.6) 15.6 (24.8) .001
Follow-up scores 5
61.4 (22.8) 78.6 (26.6) 17.2 (24.2) <.001
1
Satterthwaite test.
2
Sample sizes were slightly smaller for this analysis because referral source was not available for all participants; unclassified were N=13
baseline, 12 post-course, 9 follow-up.
3
Baseline sample sizes are N=84, 53 (non-school referred, school referred).
4
Post-course sample sizes are N=79, 48 (non-school referred, school referred).
5
Follow-up sample sizes are N=74, 41 (non-school referred, school referred).
37
Although teachers reported higher levels of school-referred children (d=.28) were small and
misbehaviour for children referred by a school consistent (p=.242). The follow-up stage also
source compared with those referred by another showed small effects for non-school referred
source, improved behaviour was reported for children (d=.23) and school referred children
both groups of children following their parents’ (d=.33), which were consistent (p=.746). Regardless
attendance at the IYP programme. As seen in Table of whether the children were school-referred or not,
13, the effect sizes at post-programme for the improvement in behaviour was broadly similar.
non-school referred children (d=.11) and for
Table 13. Child behaviour: Effect sizes of mean teacher reports of change from baseline
to post-course and follow-up, by referral source
Non-school Behaviour change test

referred1 School referred2 between source of referral
d (95% CI) d (95% CI) p
Child behaviours
Baseline to post-course .11 (-.20 to .43) .28 (-.13 to .68) .242
Baseline to follow-up .23 (-.09 to .56) .33 (-.10 to .77) .746
1
Sample sizes for children who were referred to the IYP programme by non-school sources are N=84 baseline, 79 post-course,
74 follow-up.
2
Sample sizes for children who were referred to the IYP programme by a school are N=53, 48, 41 (baseline, post-course, follow-up).
Sample sizes for children not classifiable are N=13 baseline, 12 post-course, 9 follow-up.
The lower effect size on the teacher measure the IYP courses may be beneficial in reducing
compared with the parent measure may suggest behaviour problems in school, they need to be used
the major benefit of the IYP programme was in in conjunction with school-based programmes
the home, not in the school. However, given the for maximum return in the school environment.
informant and the measuring instrument differed Webster-Stratton et al (2004) reported that adding
as well as the setting, it is not possible to conclude Teacher Training to Parent Training or Child Training
that the different effect sizes can be accounted improved treatment outcomes in terms of teacher
for by a change in setting. Nevertheless, it is likely behaviour management in the classroom and in
that, although parent management training like reports of behaviour problems.
38
Chapter 6:
Other outcomes
The previous chapter presented the findings

relating to the child behaviour, parenting
practice and relationship outcomes for families
who attended the IYP programme. This chapter
examines other relevant outcomes.
In addition to the outcomes already reported,
parent management training might be expected
to have broader benefits for the family in terms
of improved health or a decrease in stressful life
events. Also, it is possible that the presenting level
of behaviour problems exhibited by the child
might affect the degree of behaviour change
observed. Because the study was conducted over
two phases at three sites, it was necessary to
examine outcomes by phase and site to assess the
possibility of bias in the study sample.
6.1 Outcomes for health and

stressful life events
Parental depression, stress and anxiety and poor
child health are known correlates of conduct
problems (Odgers et al 2007), while a family’s
exposure to stressful and adverse life events may
exacerbate existing problems.
An inventory of child health and child-related
services derived from the Early Start Field Trial
5-Year Follow-up questionnaire (Fergusson et al
2005b) was used to assess child health (α=.48 to
.52, median α=.51). The Composite International
Diagnostic Interview version of the Diagnostic
and Statistical Manual of Mental Disorders (World
Health Organisation 1993) assessed depression
symptoms (α=.87 to .89, median α=.89) and
the anxiety and stress scales of the Depression
39
Anxiety Stress Scale (Lovibond and Lovibond post-course because the 12-month recall period
1995) measured anxiety and stress (α=.63 to .91, for the measure was outside the post-course
median α=.86). The occurrence of stressful life timeframe. Parental reports of decreased
events was measured using items from the Early stress were found in other studies (eg Larsson
Start Field Trial 5-Year Follow-up questionnaire et al 2009).
(Fergusson et al 2005b), based on Holmes
There was a suggestion of a trend towards a
and Rahe (1967).
decrease in the occurrence of stressful life events
Table 14 presents parents’ reported changes in (p<.001) post-course, albeit with a small effect
life and health factors following their attendance size of d=.25. There was also evidence the children
at the IYP programme. Stress decreased (p<.001) in the Incredible Years Pilot Study had fewer
with a small effect size of d=.30. Anxiety showed visits to health and other child-related agencies
no change, but the initial scores were very low. at programme completion (p=.007). Again, the
Depression symptoms were not measured at effect size of this improvement was small (d=.21).
Table 14. Health and stressful life events: Mean parent reports before, during and after
IYP programme
Baseline Mid-course Post-course Trend Post-course effect size

mean (std) mean (std) mean (std) p5 d (95% CI)
Measures of child health
Health and services1 1.7 (1.7) 1.5 (1.6) 1.4 (1.5) .007 .21 (-.01 to .42)
Measures of parental health2

Stress 10.8 (3.6) 10.3 (3.8) 9.7 (3.5) <.001 .30 (.08 to .52)
Anxiety 7.4 (1.9) 7.2 (2.2) 7.0 (1.8) .065* .19 (-.03 to .40)
Depression symptoms3 4.7 (3.0) NA NA
Stressful life events4

Score of life events 6.9 (7.1) 5.4 (5.9) 5.2 (5.7) <.001 .25 (.03 to .47)
1
Average number of visits to health agencies and/or child-related services in a 3-month period; sample sizes are N=166, 165, 163
(baseline, mid-course, post-course).
2
Scores based on the Depression Anxiety Stress Scale (DASS); sample sizes are N=166, 164, 164 (baseline, mid-course, post-course).
3
Depression symptoms were not calculated for mid-course or post-course because the recall period was 12 months (see Table 15).
4
The life events score is the sum of events/intensity of event; sample sizes are N=166, 166, 164 (baseline, mid-course, post-course).
5
to α=.05 for five tests).
* No evidence of a linear trend of change.
40
At the follow-up interview parents reported the reported the number of stress, anxiety
use of fewer health services for their children and depression symptoms had all shown
than at baseline (p=.006). The number of stressful improvements compared to the baseline, with
life events had also decreased between the first small to medium effect sizes (d=.29 to d=.54,
interview and the follow-up (p<.001). median d=.39). Depression symptoms had
decreased the most compared to the baseline,
Although stress symptoms were the only health
with an effect size of d=.54 (p<.001). See Table 15.
measure that showed an improvement at the
post-course measure, 6 months later parents
Table 15. Health and stressful life events: Mean parent reports before IYP programme
and at follow-up

Measures of child health
Health and services1 1.7 (1.7) 1.3 (1.5) .006 .21 (-.01 to .43)
Measures of parental health

Stress2 10.8 (3.6) 9.4 (3.2) <.001 .39 (.17 to .61)
Anxiety 7.4 (1.9) 6.8 (1.9) .001 .29 (.07 to .51)
Depression symptoms3 4.7 (3.0) 3.1 (3.0) <.001 .54 (.32 to .76)
Stressful life events4

Score of life events 6.9 (7.1) 4.4 (4.5) <.001 .36 (.14 to .58)
1
Average number of visits to health agencies and/or child-related services in a 3-month period; sample sizes are N=166, 161 (baseline,
follow-up).
2
Stress and anxiety scores are based on the Depression Anxiety Stress Scale (DASS); sample sizes are N=166, 162 (baseline, follow-up).
3
Depression symptoms are based on the Composite International Depression Index (CIDI) and are a count of symptoms only; samples
sizes are N=166, 162 (baseline, follow-up).
4
The life events score is the sum of events/intensity of event; sample sizes are N=166, 162 (baseline, follow-up).
5
to α=.05 for five tests).
41
6.2 Outcomes for children in
clinical and sub-clinical groups
Group Leaders asked parents to complete the As shown in Table 16, for Conduct Disorder, ODD,
ECBI Intensity and Problem Scales and the Social and ADHD (all p<.001) and Self-control (p=.009),
Competency Scale as standard practice before the there was a significant interaction between the
start of the IYP programme, and again on completion. change in behaviour and the clinical/sub-clinical
These scales measure the level of Conduct Problem sub-group. The interaction was marginal for
and Social Competency in the children. Anxiety/Withdrawal (p=.046). This suggests the
children with clinical level ECBI Intensity scores at
baseline benefitted more from the IYP programme
The ECBI Scaled Intensity measure
on most of the behaviour measures. The interaction
About one-quarter (28%) of the children in the for Social Competence was not significant.
Incredible Years Pilot Study had a pre-course score
The multivariate analysis of IYP programme
of less than 60 on the ECBI Scaled Intensity measure outcomes on overall Child Behaviour confirmed
(N=42 of 151 matched records), suggesting a that the effect sizes for children in the sub-clinical
sizeable proportion of the children had sub-clinical and clinical Conduct Problem groups were
levels of Conduct Problem at baseline. The size of different (p<.001), where the median effect sizes
the sub-clinical sample made it possible to test were d=.51 for sub-clinical children (ranging from
whether or not IYP programme efficacy differed for d=.28 to d=.90) compared to d=.83 for the clinical
children in the clinical and sub-clinical groups. children (ranging from d=.59 to d=1.4).
change, by sub-clinical and clinical ECBI Intensity
Sub-clinical Conduct Clinical1 Test for

Problem Conduct Problem interactions5
d (95% CI) d (95% CI) p
Child behaviours2,6
Conduct Disorder (CD) .43 (-.01 to .86) .84 (.56 to 1.1) <.001*
ODD3 .90 (.45 to 1.3) 1.4 (1.1 to 1.7) <.001*
ADHD4 .28 (-.15 to .71) .80 (.52 to 1.1) <.001*
Self-control .86 (.41 to 1.3) 1.4 (1.1 to 1.6) .009*
Anxiety/Withdrawal .38 (-.06 to .81) .59 (.32 to .87) .046*
Social Competence .59 (.15 to 1.0) .82 (.54 to 1.1) .120
1
Clinical Conduct Problem is defined by the pre-course ECBI Intensity Score being 60 or greater.
2
N (baseline, post-course) for sub-clinical Conduct Problem=42, 42; for clinical Conduct Problem=109, 107.
3
4
5
The results of tests for interactions between the sub-groups and the observations at baseline and course completion (outcomes) for
each measure are represented by the p-values, where p<.05 is evidence the change over time for a measure was not similar between
the sub-groups of sub-clinical and clinical Conduct Problem.
6
A test for an interaction between the sub-groups and the observations at baseline and course completion for all six Child Behaviour
measures showed a significant interaction between the overall outcome and the clinical/sub-clinical Conduct Problem sub-groups:
F(6, 142)=4.10, p<.001.
*The change in scores over time between clinical and sub-clinical Conduct Problem children was not similar.
42
The evidence presented in Table 17 confirmed the parents of the children in the sub-clinical group.
differences for some of the measures detected Improvement in Social Competence remained
post-programme between the clinical and similar for both groups of children at the
sub-clinical groups of children at the 6-month follow-up interviews.
follow-up. The sub-clinical group typically had
Effect sizes for the sub-clinical children were small
smaller effect sizes for the child behaviour
to large, ranging from d=.40 to d=1.3 (median
measures than did the clinical group. There was
d=.64), while the clinical children had medium
a significant interaction between the change in
to large effect sizes, ranging from d=.61 to d=1.4
behaviour and the clinical/sub-clinical sub-groups
(median d=.94). These differences in the median
on the Conduct Disorder (p<.001) and ODD
effect sizes were confirmed by the multivariate
(p=.003) measures, as was found at the end of
analysis of the overall Child Behaviour outcome
the IYP course. However, the ADHD, Self-control
interaction between the two observation points
and Anxiety/Withdrawal measures were no
and the clinical/sub-clinical sub-groups (p<.001).
longer significantly different, perhaps reflecting
the continued improvement reported by
Table 17. Child behaviours at follow-up: Effect sizes of mean parent reports of change, by
sub-clinical and clinical ECBI Intensity

Sub-clinical Clinical1 Test for
Conduct Problem Conduct Problem interactions5
d (95% CI) d (95% CI) p
Child behaviours2,6
Conduct Disorder (CD) .40 (-.04 to .83) .96 (.68 to 1.2) <.001*
ODD 3
1.0 (.54 to 1.5) 1.4 (1.1 to 1.7) .003*
ADHD 4
.52 (.08 to .96) .83 (.55 to 1.1) .053
Self-control 1.3 (.78 to 1.7) 1.4 (1.1 to 1.7) .459
Anxiety/Withdrawal .58 (.14 to 1.0) .61 (.34 to .89) .432

1
Clinical Conduct Problem is defined by the pre-course ECBI Intensity Score being 60 or greater.
2
N (baseline, follow-up) for sub-clinical Conduct Problem=42, 41; for clinical Conduct Problem=109, 107.
3
4
5
The results of tests for interactions between the sub-groups and the observations at baseline and follow-up for each measure are
represented by the p-values, where p<.05 is evidence the change over time for a measure was not similar between the sub-groups of
sub-clinical and clinical Conduct Problem.
6
A test for an interaction between the sub-groups and the observations at baseline and follow-up for all six Child Behaviour measures
showed a significant interaction between the overall outcome and the clinical/sub-clinical Conduct Problem sub-groups:
F(6, 141)=4.50, p<.001.
*The change in the scores over time between clinical and sub-clinical Conduct Problem children was not similar.

43
The Social Competence Scale measure This implies a greater improvement on these two
measures for the clinical SCS children. However,
The Social Competence Scale (SCS) measures
the multivariate analysis on a combined measure
pro-social behaviours, communication skills and
of Child Behaviour indicates the effect sizes were
self-control. Table 18 examines the behaviour
broadly similar for the two groups of children
outcomes for children scoring above and below
(p=.100) at the end of the IYP course. The median
the clinical cut-off score of 17 on the Social
effect sizes were d=.62 for sub-clinical children
Competence Scale at baseline.
(d=.41 to d=.96) compared to d=.87 for clinical
The reported changes in the Conduct Disorder children (d=.63 to d=1.4).
and ODD measures were significantly different
(p=.003 and p=.007 respectively) for the clinical
and sub-clinical SCS children, with larger effect
sizes evident for children in the clinical group.
change, by sub-clinical and clinical Social Competence Scale (SCS)

Sub-clinical1 SCS Clinical SCS Test for interactions5
d (95% CI) d (95% CI) p
Child behaviours2,6
Conduct Disorder (CD) .53 (.18 to .87) .86 (.54 to 1.2) .003*
ODD 3
.86 (.50 to 1.2) 1.3 (.96 to 1.6) .007*
ADHD4 .49 (.14 to .83) .78 (.46 to 1.1) .058
Self-control .96 (.60 to 1.3) 1.4 (1.1 to 1.8) .079
Anxiety/Withdrawal .41 (.07 to .75) .63 (.31 to .95) .100

1
Sub-clinical Social Competence is defined by the pre-course Social Competence Scale scoring 17 or more.
2
For sub-clinical Social Competence Scale scores N=68, 68 (baseline, post-course); for clinical Social Competence Scale scores N=82, 80
(baseline, post-course).
3
4
5
the sub-groups of sub-clinical and clinical SCS.
6
measures showed no significant interaction between the overall outcome and the clinical/sub-clinical SCS sub-groups:
F(6, 141)=1.82, p=.100.
*The change in scores over time between clinical and sub-clinical SCS children was not similar.

44
Six months after the completion of the IYP course The multivariate analysis for the combined Child
parent reports of Conduct Disorder and ODD Behaviour measure confirmed these findings.
remained significantly different for children with The effect sizes for the children in the sub-clinical
clinical and sub-clinical baseline SCS scores (Table SCS group were medium to large (d=.51 to
19). For Conduct Disorder the effect size was d=.51 d=1.0, median d=.61), whereas for the clinical
for those in the sub-clinical group compared to the SCS children the effect sizes were large (d=.63 to
effect size of d=1.0 for those in the clinical group d=1.6, median d=1.1) (p=.004). The implication
(p<.001). For ODD the sub-clinical SCS effect size is that, in the medium term, improvements in
was d=.84 compared to the clinical SCS effect child behaviour may be greater for children with
size of d=1.4 (p=.008). In addition, effect sizes for a clinical level SCS pre-course score than for those
the clinical and sub-clinical SCS children were with a sub-clinical level pre-course score.
significantly different at follow-up for both Self-
control (p=.039) and Social Competence (p=.009),
a difference not evident immediately after finishing
the IYP programme.
by sub-clinical and clinical Social Competence Scale (SCS)
Sub-clinical1 SCS Clinical SCS Test for interactions5

d (95% CI) d (95% CI) p
Child behaviours2,6
Conduct Disorder (CD) .51 (.16 to .85) 1.0 (.72 to 1.4) <.001*
ODD 3
.84 (.48 to 1.2) 1.4 (1.0 to 1.7) .008*
ADHD 4
.55 (.21 to .90) .88 (.56 to 1.2) .057
Self-control 1.0 (.66 to 1.4) 1.6 (1.2 to 2.0) .039*
Social Competence .66 (.31 to 1.0) 1.1 (.75 to 1.4) .009*

1
Sub-clinical Social Competence is defined by the pre-course Social Competence Scale scoring 17 or more.
2
For sub-clinical Social Competence Scale scores N=68, 67 (baseline, follow-up); for clinical Social Competence Scale scores N=82, 80
(baseline, follow-up).
3
4
5
sub-clinical and clinical SCS.
6
showed a significant interaction between the overall outcome and the clinical/sub-clinical SCS sub-groups: F(6, 140)=3.32, p=.004.
*The change in the scores over time between clinical and sub-clinical Conduct Problem children was not similar.
45
6.3 Outcomes for phase 1
and phase 2
The Incredible Years Pilot Study was carried out over These results may suggest the programmes
two time periods in 2011 to ensure an adequate delivered in Phase 2 were slightly less effective
sample size for the research: Phase 1 was February to than those delivered in Phase 1. The multivariate
July (N=88); Phase 2 was August to December (N=78). analysis of the combined Child Behaviour
Table 20 examines the interactions between the measure indicated the Phase 1 programmes
phase samples and the change in outcomes across achieved marginally higher effect sizes than
the Child Behaviour measures to test for possible the Phase 2 ones as assessed at the post-course
differences in outcomes by phase at the end of the interview (p=.033). The in-course ECBI Intensity
IYP programme. The test for differences in outcomes score also showed a difference between Phase
at programme completion yielded marginal p-values 1 and Phase 2 outcomes, whereby the Phase
for ODD (p=.041) and Social Competence (p=.048). 1 effect size was d=1.2 compared to the Phase
Phase 1 had a median effect size of d=.77 compared 2 effect size of d=.76 (p=.041). The reasons for
to Phase 2 with a median effect size of d=.55. these differences are not known.
change, by study phase
Phase 1 Phase 2 Test for interactions6

d (95% CI) d (95% CI) p
Child behaviours1,7
Conduct Disorder (CD) .71 (.41 to 1.0) .51 (.19 to .83) .130
ODD 2
1.1 (.78 to 1.4) .81 (.48 to 1.1) .041*
ADHD 3
.57 (.26 to .87) .53 (.21 to .85) .664
Self-control 1.0 (.72 to 1.4) .88 (.54 to 1.2) .631
Social Competence .83 (.52 to 1.1) .51 (.19 to .83) .048*

ECBI Intensity 4
1.2 (.83 to 1.6) .76 (.35 to 1.2) .041*
Social Competence Scale 5
.81 (.43 to 1.2) .94 (.53 to 1.4) .853

1
Sample sizes for Phase 1 are N=88, 86; for Phase 2 N=78, 77 (baseline, post-course).
2
3
4
Sample sizes for Phase 1 on the ECBI Intensity scores are N=80, 50; for Phase 2 N=70, 51 (baseline, post-course).
5
Sample sizes for Phase 1 on the Social Competence Scale are N=80, 57; for Phase 2 N=70, 50 (baseline, post-course).
6
the sub-groups of the study phase.
7
measures showed a significant interaction between the overall outcome and the study phase sub-groups: F(6, 156)=2.36, p=.033. This
statistic does not include the in-course ECBI and SCS assessments.
*The change in score over time between Phase 1 and Phase 2 samples was not similar.
46
However, the observed differences between the d=.61) were not significantly different (p=.492).
study phases at post-course were not sustained to The lack of evidence of a sustained difference in
the follow-up interviews (Table 21). Results from outcomes between Phase 1 and Phase 2 children
the multivariate analysis showed Phase 1 (median at follow-up justifies combining the results for
effect size d=.79) and Phase 2 (median effect size these two groups.
by study phase
Phase 1 Phase 2 Test for interactions4

d (95% CI) d (95% CI) p
Child behaviours1,5
Conduct Disorder (CD) .76 (.45 to 1.1) .59 (.26 to .91) .238
ODD 2
1.1 (.79 to 1.4) .84 (.50 to 1.2) .118
ADHD 3
.66 (.35 to .96) .57 (.24 to .89) .948
Self-control 1.1 (.78 to 1.4) .97 (.54 to 1.2) .883
Social Competence .82 (.51 to 1.1) .63 (.30 to .95) .477

1
Sample sizes for Phase 1 are N=88, 87; for Phase 2 are N=78, 75 (baseline, follow-up).
2
3
4
represented by the p-values, where p<.05 is evidence the change over time for a measure was not similar between the sub-groups
of the study phase.
5
showed no significant interaction between the overall outcome and the study phase sub-groups: F(6, 155)=0.91, p=.492.

47
6.4 Outcomes for the three sites
The study was conducted at three sites: Bay of median d=.87) compared to both the Bay of
Plenty, Canterbury and Mid-Central. Table 22 Plenty (d=.46 to d=1.0, median d=.63) and
presents evidence of an interaction between the Canterbury (d=.35 to d=.85, median d=.60). This
pilot site and the change in Child Behaviour at finding was confirmed by the multivariate analysis
programme completion for the measures of ODD on the combined Child Behaviour measure
(p=.047), ADHD (p=.005), Anxiety/Withdrawal (p=.002). Mid-Central also had the largest
(p=.021), and the ECBI Intensity scores (p<.001). difference in effect sizes on the ECBI Intensity
Mid-Central had consistently larger effect sizes (d=1.6) compared with the Bay of Plenty (d=.69)
across the behaviour measures (d=.71 to d=1.4, and Canterbury (d=.95).
of change, by study site
Test for
Bay of Plenty Canterbury Mid-Central interactions6
d (95% CI) d (95% CI) d (95% CI) p
Child behaviours1,7
Conduct Disorder (CD) .59 (.21 to .97) .56 (.17 to .94) .83 (.43 to 1.2) .234
ODD2 .84 (.45 to 1.2) .85 (.46 to 1.2) 1.4 (.93 to 1.8) .047*
ADHD 3
.48 (.10 to .86) .36 (-.02 to .74) .90 (.50 to 1.3) .005*
Self-control 1.0 (.63 to 1.4) .74 (.35 to 1.1) 1.2 (.76 to 1.6) .168
Anxiety/Withdrawal .46 (.08 to .83) .35 (-.03 to .73) .71 (.31 to 1.1) .021*
Social Competence .66 (.28 to 1.0) .63 (.24 to 1.0) .73 (.33 to 1.1) .615

ECBI Intensity4 .69 (.24 to 1.1) .95 (.46 to 1.4) 1.6 (1.0 to 2.2) <.001*
Social Competence Scale5 .56 (.12 to 1.0) .82 (.33 to 1.3) 1.3 (.77 to 1.9) .090
1
Sample sizes for Bay of Plenty are N=56, 56; for Canterbury N=57, 55; for Mid-Central N=53, 52 (baseline, post-course).
2
3
4
Sample sizes on ECBI Intensity for Bay of Plenty are N=52, 41; for Canterbury=53, 36; for Mid-Central=46, 31 (baseline, post- course).
5
Sample sizes for Social Competence Scale for Bay of Plenty are N=52, 41; for Canterbury=52, 35; for Mid-Central=46, 31
(baseline, post-course).
6
the sub-groups of the study site.
7
measures showed a significant interaction between the overall outcome and the study site sub-groups: F(12, 310)=2.64, p=.002. This
statistic does not include the in-course ECBI and SCS assessments.
*The change in scores over time between the study sites was not similar.
48
Reported Child Behaviour outcomes at course for all three sites were good, but Mid-Central
completion were sustained at the follow-up (d=.91) was relatively better than Canterbury
interviews at the three sites (Table 23), with (d=.71) and the Bay of Plenty (d=.66).
medium to large effect sizes in the Bay of
The reason for this site difference is not
Plenty and Canterbury and consistently higher
known, but it does not appear to be an issue of
effect sizes again observed in Mid-Central.
programme fidelity (see Chapter 9, Section 9.1).
The multivariate analysis revealed a significant
difference among the sites (p<.001). Effect sizes
Table 23. Child behaviours at follow-up: Mean parent reports, by study site
Test for
Bay of Plenty Canterbury Mid-Central interactions4
d (95% CI) d (95% CI) d (95% CI) p
Child behaviours1,5
Conduct Disorder (CD) .63 (.25 to 1.0) .67 (.28 to 1.0) .87 (.46 to 1.3) .211
ODD 2
.78 (.40 to 1.2) .88 (.49 to 1.3) 1.5 (1.1 to 1.9) .014*
ADHD 3
.55 (.18 to .93) .43 (.05 to .81) .95 (.54 to 1.4) .032*
Self-control 1.1 (.74 to 1.5) .81 (.42 to 1.2) 1.2 (.80 to 1.6) .170
Anxiety/Withdrawal .51 (.13 to .88) .40 (.02 to .77) .79 (.38 to 1.2) .012*
Social Competence .78 (.40 to 1.2) .65 (.26 to 1.0) .75 (.35 to 1.2) .447
1
Sample sizes for Bay of Plenty are N=56, 56; for Canterbury N=57, 55; for Mid-Central N=53, 51 (baseline, follow-up).
2
3
4
the study site.
5
showed a significant interaction between the overall outcome and the study site sub-groups: F(12, 308)=2.80, p=.001.
*The change in scores over time between the study sites was not similar.
49
Chapter 7:
Programme outcomes for
Maori and non-Maori
An important issue raised by the preceding results

concerns the extent to which the study findings
held for Māori and non-Māori families. This
section addresses this question by comparing the
outcomes for Māori children in the study with the
outcomes for non-Māori children on the pre-test/
post-test and follow-up comparisons of child
behaviours, parenting practices and relationships
reported from Table 4 to Table 9 in Chapter 5.
The independent kaupapa Māori study is also
discussed in this chapter. Satisfaction ratings
of the IYP programme for Māori and non-Māori
parents are presented in Chapter 9.
7.1 The Incredible Years

Pilot Study
Pre-test/post-test comparisons:
Child behaviour
Table 24 contains the effect sizes for Māori and
non-Māori children on the Child Behaviours
measured in the Incredible Years Pilot Study as
well as an assessment of the extent to which
outcomes varied by ethnicity. If reported
outcomes differed for Māori and non-Māori, tests
of the interaction of change by ethnicity should
be significant.
50
It is evident from the data that the effect sizes reflected the fact that, although both Māori and
of the before-and-after scores for Māori and non-Māori children showed improvement in Social
non-Māori children were similar. The child Competence, the improvement for non-Māori was
behaviour effect sizes for Māori children ranged larger (d=.79) than the improvement for Māori
from d=.43 to d=.82 (median d=.56) compared to (d=.47).
the effect sizes for non-Māori, which ranged from
Although estimated effect sizes across the
d=.46 to d=1.1 (median d=.77). In all but one case
measures in Table 24 appear typically better for
the change by ethnicity interaction was
non-Māori children, the multivariate analysis
non-significant, implying an absence of difference
demonstrated the overall programme outcome
in the outcomes for Māori and non-Māori children.
on Child Behaviours for Māori (median effect size
The exception was a before-and-after difference
d=.56) and non-Māori (median effect size d=.73)
in the level of improvement in parent-reported
was similar (p=.072).
Social Competence (p=.012). This interaction
change, by Māori and non-Māori children

Māori Non-Māori Test for interactions6
d (95% CI) d (95% CI) p
Child behaviours1,7
Conduct Disorder (CD) .53 (.16 to .90) .67 (.39 to .95) .186
ODD2 .82 (.44 to 1.2) 1.1 (.76 to 1.3) .087
ADHD3 .43 (.06 to .79) .62 (.34 to .90) .131
Self-control .81 (.43 to 1.2) 1.0 (.76 to 1.3) .119
Social Competence .47 (.10 to .83) .79 (.51 to 1.1) .012*

ECBI Intensity4 .93 (.41 to 1.4) 1.1 (.73 to 1.4) .980
Social Competence Scale5 .96 (.43 to 1.5) .88 (.55 to 1.2) .786
1
Sample sizes for Māori are N=60, 59; for non-Māori are N=106, 104 (baseline, post-course).
2
3
4
5
6
The results of tests for interactions between the sub-groups and the observations at baseline and course completion (outcomes)
for each measure are represented by the p-values, where p<.05 is evidence the change over time for a measure was not similar
between the sub-groups of Māori and non-Māori children.
7
measures showed no significant interaction between the overall outcome and the Māori/non-Māori sub-groups: F(6, 156)=1.98,
p=.072. This statistic does not include the in-course ECBI and SCS assessments.
*The change in scores over time between Māori and non-Māori was not similar.
51
Data from the follow-up interviews given in (median d=.80) remained consistently better than
Table 25 show that, as with the post-course for Māori (median d=.63). This was confirmed by
findings above, only the Social Competence the multivariate analysis for the combined Child
measure showed a significantly (p=.008) smaller Behaviour measure (p=.025). This significant
effect size (d=.50) for Māori than for non-Māori difference was not evident at post-programme,
(d=.86) children. However, the effect sizes across but became apparent by the 6-month follow-up.
all the Child Behaviour measures for non-Māori
by Māori and non-Māori children

Māori Non-Māori Test for interactions5
d (95% CI) d (95% CI) p
Child behaviours1,4
Conduct Disorder (CD) .58 (.20 to .95) .74 (.46 to 1.0) .102
ODD 2
.81 (.43 to 1.2) 1.1 (.79 to 1.4) .068
ADHD 3
.57 (.20 to .94) .64 (.36 to .91) .671
Self-control .89 (.51 to 1.3) 1.1 (.83 to 1.4) .168
Anxiety/Withdrawal .68 (.31 to 1.1) .49 (.22 to .77) .231
Social Competence .50 (.13 to .87) .86 (.58 to 1.1) .008*

1
Sample sizes for Māori are N=60, 58; for non-Māori are N=106, 104 (baseline, follow-up).
2
3
4
Māori and non-Māori children.
5
showed a significant interaction between the overall outcome and the Māori/non-Māori sub-groups: F(6, 155)=2.49, p=.025.
*The change in scores over time between Māori and non-Māori were not similar.
These findings show that, with the possible However, the effect sizes for Māori children were
exception of Social Competence, the parents consistently smaller than those for non-Māori
of both Māori and non-Māori children reported children, suggesting the IYP programme was
improvements in their children’s behaviour that less effective for Māori. Multivariate analyses on
was sustained at the 6-month follow-up. It is a combined Child Behaviour measure revealed
possible that, given the number of comparisons that, although there was no significant difference
made, the result for Social Competence reflects a between Māori and non-Māori children
chance variation rather than a genuine post-programme, a significant difference was
Māori/non-Māori difference in outcomes. evident at follow-up.
52
Parenting practice
The effect sizes for parenting practices before and both Māori (d=.44) and non-Māori (d=.83) parents
after the IYP programme for Māori and non-Māori reported an improvement. However the change for
shown in Table 26 were similar. The effect sizes for non-Māori parents was larger.
Māori parents ranged from d=.28 to d=.72 compared
to the effect sizes for non-Māori parents, which The Arnold-O’Leary Total Scale of Dealing with
ranged from d=.24 to d=.90. Hostile discipline on Misbehaviour showed similar outcome scores for the
the Arnold-O’Leary Parenting Scale was the only parents of Māori and non-Māori children, with large
significant change by ethnicity interaction (p=.006). effect sizes of d=.72 for Māori children and d=.90
The improvement appeared to be primarily on the for non-Māori children. Moreover, the multivariate
item measuring bad language only, not on the other analysis on a combined Parenting measure of
two items (slapping and insulting) underlying this outcomes between Māori (median effect size d=.47)
factor. Both groups reported low levels of Hostile and non-Māori (median effect size d=.60) indicated
discipline before the beginning of the IYP course and no significant difference (p=.385).
Table 26. Parenting practices after IYP programme: Effect sizes of mean parent reports
of change, by Māori and non-Māori children
Parenting measures1,3 Māori Non-Māori Test for interactions4

d (95% CI) d (95% CI) p
Parenting practices
Poor supervision .28 (-.08 to .64) .24 (-.03 to .51) .895
Positive parenting .45 (.09 to .82) .47 (.19 to .74) .735
Corporal punishment .56 (.19 to .93) .60 (.33 to .88) .932
Parental involvement .54 (.17 to .90) .52 (.25 to .80) .954
Inconsistent discipline .41 (.05 to .78) .63 (.35 to .90) .378
Dealing with misbehaviour

Lax discipline .47 (.10 to .83) .57 (.30 to .85) .749
Over-reactive discipline .63 (.26 to 1.0) .77 (.49 to 1.1) .400
Hostile discipline .44 (.07 to .80) .83 (.55 to 1.1) .006*
Total Scale2 .72 (.34 to 1.1) .90 (.61 to 1.2) .406
1
Sample sizes for Māori are N=60, 59; for non-Māori N=106, 105 (baseline, post-course).
2
The Total Scale includes all the questions from the Arnold-O’Leary Parenting Scale including additional questions that are not part of the
Lax, Over-reactive, or Hostile discipline factors.
3
A test for an interaction between the sub-groups and the observations at baseline and course completion for all nine Parenting
Practice measures showed no significant interaction between the overall outcome and the Māori/non-Māori sub-groups:
F(9, 154)=1.07, p=.385.
4
the sub-groups of Māori and non-Māori children.
53
Effect sizes for changes in parenting practices measures whereby the change for non-Māori
reported by Māori and non-Māori parents were parents was greater than that for Māori parents.
again broadly similar across the measures at
On a combined Parenting measure, multivariate
follow-up (Table 27). While the Dealing with
analysis suggested no differences in outcomes
Misbehaviour Total Scale showed similar
between the parents of Māori and non-Māori
changes in practices for both groups of parents,
children. The effect sizes for Māori children
an examination of the subscales suggests an
ranged from d=.22 to d=.62 with a median d=.41,
interaction between ethnicity and the change in
and for non-Māori from d=.22 to d=.89 with a
the outcome scores for both the Hostile discipline
median d=.56.
(p<.017) and the Over-reactive discipline (p<.001)
Table 27. Parenting practices at follow-up: Effect sizes of mean parent reports of change,
by Māori and non-Māori children
Parenting measures1,3 Māori Non-Māori Test for interactions4

d (95% CI) d (95% CI) p
Parenting practices
Poor supervision .32 (-.05 to .68) .22 (-.06 to .49) .483
Positive parenting .22 (-.15 to .58) .38 (.11 to .66) .175
Corporal punishment .38 (.01 to .75) .54 (.26 to .82) .480
Parental involvement .44 (.07 to .81) .48 (.21 to .76) .573
Inconsistent discipline .41 (.04 to .78) .65 (.37 to .93) .162

Dealing with misbehaviour
Lax discipline .45 (.08 to .82) .56 (.29 to .84) .639
Over-reactive discipline .49 (.12 to .85) .81 (.52 to 1.1) .017*
Hostile discipline .32 (-.05 to .68) .77 (.49 to 1.1) <.001*
Total Scale 2
.62 (.25 to .99) .89 (.60 to 1.2) .099
1
Sample sizes for Māori are N=60, 58; for non-Māori N=106, 104 (baseline, follow-up).
2
the Lax, Over-reactive, or Hostile discipline factors.
3
showed no significant interaction between the overall outcome and the Māori/non-Māori sub-groups: F(9, 152)=1.51, p=.150.
4
54
Conflict and relationships
It appears both Māori and non-Māori parents Reported improvements in inter-partner and
reported broadly similar improvements in parent/child relationships (Verbal aggression and
their parenting practice at the end of the Physical assault) were similar for Māori and
IYP programme and 6 months beyond. It is non-Māori parents at post-course (Table 28). The
possible, as with Social Competence, the effect sizes for Māori parents ranged from d=.41 to
difference found for Hostile discipline results d=.49 (median d=.46) compared to the effect sizes
from a chance variation due to the number for non-Māori parents, which ranged from d=.48
of comparisons made rather than a genuine to d=.67 (median d=.53). There were no significant
Māori/non-Māori difference. interactions for change by ethnicity, which suggests
both Māori and non-Māori parents reported similar
improvements in their relationships. The multivariate
analysis produced no significant differences.
Table 28. Conflict and relationships after IYP programme: Effect sizes of mean parent
reports of change, by Māori and non-Māori children
Relationship types5 Māori Non-Māori Test for interactions6

d (95% CI) d (95% CI) p
Verbal aggression .49 (.13 to .86) .67 (.39 to .95) .338
Physical assault .45 (.08 to .81) .53 (.26 to .81) .946

Verbal aggression .46 (-.10 to 1.0) .52 (.15 to .89) .992
Physical assault .41 (-.14 to .96) .48 (.11 to .85) .550
Violence to partner .30 (-.25 to .84) .21 (-.16 to .57) .168
Violence from partner .40 (-.15 to .95) .22 (-.15 to .58) .143
Child-rearing disagreement .45 (-.11 to 1.0) .53 (.16 to .90) .877
Relationship quality .30 (-.25 to .84) .18 (-.19 to .54) .742
1
Scores are based on the Parent/Child Conflict Tactics Scale; for Māori primary caregivers sample sizes were N=60, 59; for non-Māori
N=106, 105 (baseline, post-course).
2
For the other parent and inter-parental factors (where applicable) samples sizes were for Māori N=27, 26 (baseline, post-course); for
non-Māori N=64, 58.
3
Inter-parental violence is from the Revised Conflict Tactics Scale (CTS2).
4
Child-rearing disagreement is from the Parent Problem Checklist, and Relationship quality is from the Partner Attachment Scale.
5
A test for an interaction between the sub-groups and the observations at baseline and course completion for all eight Relationship and
Conflict measures showed no significant interaction between the overall outcome and the Māori/non-Māori sub-groups: F(8, 72)=.59, p=.783.
6
the sub-groups of Māori and non-Māori children.
55
As was the case at post-course, Table 29 shows Relationship quality for the parents of both
consistent outcomes at the 6-month follow-up the Māori and non-Māori study children. The
across the measures of Conflict, Inter-parental multivariate analysis produced no significant
violence, Child-rearing disagreement and differences.
Table 29. Conflict and relationships at follow-up: Mean parent reports of change, by Māori
and non-Māori children
Relationship types5 Māori Non-Māori Test for interactions6

d (95% CI) d (95% CI) p
Verbal aggression .45 (.08 to .82) .68 (.40 to .96) .131

Verbal aggression .35 (-.18 to .87) .54 (.18 to .90) .753
Violence to partner .19 (-.34 to .71) .40 (.05 to .76) .939
Violence from partner .37 (-.16 to .90) .34 (-.02 to .69) .885
Child-rearing disagreement .41 (-.12 to .93) .58 (.22 to .93) .969
Relationship quality .19 (-.33 to .72) .14 (-.21 to .49) .926
1
Scores are based on the Parent/Child Conflict Tactics Scale; for Māori primary caregivers sample sizes were N=60, 58; for non-Māori
N=106, 104 (baseline, follow-up).
2
For the other parent and inter-parental factors (where applicable) samples sizes were for Māori N=27, 28 (baseline, follow-up); for
non-Māori N=64, 62.
3
Inter-parental violence is from the Revised Conflict Tactics Scale (CTS2).
4
Child-rearing disagreement is from the Parent Problem Checklist, and Relationship quality is from the Partner Attachment Scale.
5
A test for an interaction between the sub-groups and the observations at baseline and the follow-up for all eight Relationship and
Conflict measures showed no significant interaction between the overall outcome and the Māori/non-Māori sub-groups:
F(8, 68)=.56, p=.805.
6
56
Summary of the multivariate analyses As shown in the summary of the doubly multivariate
for Māori and non-Māori children tests on the previous tables presented in Table 30,
at IYP programme completion the differences
Although on the individual outcome measures between Māori and non-Māori children were not
of child behaviour, parenting practice and significant. However at the follow-up, significant
relationships the differences between Māori and differences were evident on the measures of Child
non-Māori children were not significant, the effect Behaviours (p=.025), suggesting the outcomes
sizes were typically better for non-Māori children on these measures for non-Māori children were
compared to Māori children. As reported in this stronger than those for Māori children, albeit Māori
chapter, the trend indicating possible differences in children still showed improvement. Differences in
outcomes for Māori and non-Māori was examined outcomes for Māori and non-Māori were not found
by a doubly multivariate repeated measures by Fergusson et al (2009), which may reflect the
Analysis of Variance, which tested for interactions more intensive measurement used in the present
between the combined outcome measures, the study or the lack of follow-up in the earlier study.
repeated interviews, and the ethnicity sub-groups, These findings indicate the need for culturally
thereby verifying the multiple hypothesis tests responsive refinements to IYP programmes to
in each table. Baseline differences in reported ensure the maximum effectiveness for all parents
behaviour for Māori and non-Māori children were and children and the maintenance of behaviour
taken into account in the model fitted. improvements.
Table 30. Summary of the multivariate analyses for Māori and non-Māori children at IYP
programme completion and follow-up
Effect size range for Effect size range for

Māori non-Māori Test for interactions4
d (median) d (median) p
Programme outcomes
Child behaviours1 .43 to .82 (.56) .46 to 1.1 (.73) .072
Parenting practices 2
.28 to .72 (.47) .24 to .90 (.60) .385
Relationships 3
.30 to .49 (.43) .18 to .67 (.50) .783
Follow-up outcomes
Child behaviours 1
.50 to .89 (.63) .49 to 1.1 (.80) .025*
Parenting practices 2
.22 to .62 (.41) .22 to .89 (.56) .150
Parent/Child relationships 3
.19 to .46 (.36) .14 to .68 (.47) .805
1
Child behaviour outcomes are from measures of: Conduct Disorder, ODD, ADHD, Self-control, Anxiety/Withdrawal, and Social Competence.
2
Parenting practices include measures of: Poor supervision, Positive parenting, Corporal punishment, Parental involvement, and
Inconsistent discipline, Lax discipline, Over-reactive discipline, Hostile discipline, and the Total Scale of Dealing with Misbehaviour.
3
Relationships include measures of: Verbal aggression, Physical assault, Violence to/from partner, Child-rearing disagreement, and
Relationship quality.
4
The results of tests for interactions between the sub-groups and the observations over time for the groups of measures are
represented by the p-values, where p<.05 is evidence the change over time for a group of measures was not similar between the
sub-groups of Māori and non-Māori children.
*The change in scores over time across grouped measures between Māori and non-Māori was not similar.
57
7.2 The kaupapa
Māori study
The independent kaupapa Māori study (Berryman In their discussion the authors draw out three
et al 2012) explored the experiences of four Māori themes:
mothers who participated in one of the evaluated 1. Understanding culturally responsive adult
IYP programmes in 2011 in Canterbury or the learning: “A better understanding of culturally
Bay of Plenty, and the experiences of four Māori responsive andragogy may be needed if
Special Education staff members who were Group the Incredible Years programme is going to
Leaders of evaluated IYP programmes. The full work for more whānau Māori…. Berryman,
report is available from the Ministry of Education. SooHoo and Nevin (in print) identified the
The interviews are presented in the report as importance of the space within which the
two collaborative stories, one the combined responsive dialogic rituals must proceed in
experiences of the four Māori mothers and the order for a relationship of trust and respect
other the combined experiences of the four Māori to be nurtured amongst both parties. …
Group Leaders. From these stories the authors (They) suggest that listening to the other is
conclude that: more likely to occur when spaces to develop
respectful relationships are given priority
“Māori whānau wanted and needed to
before engaging in any joint project….
participate but they wanted to do so on their
Whakawhanaungatanga provides a cultural
own terms. A culturally responsive programme
space for this to occur; where things are
should enable this to happen, however, it was
not done to people but where people are
clear that for at least two of these mothers this
accorded the autonomy to contribute on their
is not what had happened.” (page 56)
own terms.” (page 56)
“Māori facilitators understood that the most
2. Programme fidelity (doing what we know
important element for them was to establish
works): “… the Māori facilitators explained that
and consolidate the kaupapa, that is, what
they modified and varied their delivery style of
the delivery of Incredible Years required of
the Incredible Years programme so as to better
management and of colleagues trained to
engage with whānau Māori. Māori facilitators
facilitate Incredible Years and with whom
learned to make the programme as effective
they worked, to be most effective for whānau
as possible for Māori families by drawing on
Māori. When this was firmly established,
the resources (knowledge and experiences) of
connecting with Māori parents, no matter
their participants to find solutions that were
their circumstances, was more likely to lead
a better fit with Māori values and practices….
to the strengthening of collaborative learning
(They) showed that when facilitators are
that in turn led to whānau uptake of Incredible
able to make the ‘connects’, and when time
Years and as a consequence the further
and resources are available to facilitate the
enhancement of the mana of the parent and
programme responsively, then the Incredible
the wellbeing of their whānau.” (page 56)
Years programme can be more effective for
whānau and welcomed by them.” (page 57)
58
7.3 Incredible Years for
whānau Māori
3. A differentiated response: “… a differentiated The Incredible Years Pilot Study provides evidence
response … should be available and doable of benefits from IYP training for whānau Māori.
within the Incredible Years programme. However, the benefits appear to be consistently
Beginning from a point of getting to ‘really’ lower than those reported for non-Māori families,
know whānau by listening to them is the particularly for ODD and Social Competence.
starting point. … Incredible Years is a The authors of the independent kaupapa
comprehensively researched and studied Māori study, while acknowledging the IYP
programme internationally and there is also programme’s potential effectiveness for whānau
an increasing body of literature pertaining to Māori, emphasise the importance of being more
its delivery in New Zealand and its potential responsive to their needs (Berryman et al 2012).
effectiveness for whānau Māori…. In New There are grounds for making further investment
Zealand, Berryman, Woller and Glynn (2009) in developing culturally responsive refinements to
previously concluded that Incredible Years was the IYP programme to maximise the programme’s
effective for Māori when whānau were able to effectiveness for whānau Māori.
develop relationships of trust with facilitators
The Werry Centre has hosted a series of annual
to the extent that their own experiences were
hui (eg Werry Centre 2008, 2010) for Māori Group
validated and they were able to contribute
Leaders to share innovations in IYP programme
and to learn. Māori facilitators understood
delivery and to develop resources to ensure
that they needed to both listen to whānau
cultural responsiveness when delivering IYP
(responsive) and respond accordingly, and
courses to whānau Māori. At these hui Māori
to bring a Māori perspective (appropriate) to
Group Leaders have consistently advocated for
the programme to help facilitate success for
the need to implement, as standard practice, the
whānau.” (page 58)
many contributions they have naturally included
Berryman et al conclude that, although the in their delivery as culturally responsive practices
findings of this kaupapa Māori study reiterate to gain the best results for whānau Māori.
those of other studies, work must be done to
Recently the Werry Centre (2012) published
provide a more responsive, cohesive (and at times
Ngā Tau Mīharo ō Aotearoa, a set of IYP
differentiated) and aligned approach to meet the
programme resources for Māori Group Leaders.
needs of Māori families.
These resources recognise the importance of
Te Ao Māori in delivering IYP courses to whānau
Māori and were developed in collaboration
with experienced Māori Group Leaders and
Kaumātua. They include a DVD featuring Māori
Group Leaders and whānau Māori, a workbook
for Māori accreditees and practical resources for
use in group delivery. The concept of weaving
together the strands from two worlds derives
from the metaphor of the weaving of harakeke
(flax). In respecting the different meanings and
understandings within both Te Ao Māori and
Te Ao Pākehā, the Werry Centre acknowledges
the partnership between these two worlds and
how the skill sets may be aligned to maximise
outcomes for whānau.
59
Chapter 8:
Single Case Studies
Single Case Studies were included in the Incredible

Years Pilot Study to provide an alternative
methodology to test the hypotheses of behaviour
change and to investigate variability in individual
child behaviour. Weekly Parent Reports were
conducted at all three sites in Phase 1.
8.1 Parent Reports

A sub-sample of participants was randomly
selected at each of the three sites for the Parent
Report component in Phase 1 of the study. Ten
parents each from Mid-Central and Canterbury
consented to participate and five parents from
the Bay of Plenty, making a total of 25 parents. The
Parent Report sample consisted of nine Māori and
16 non-Māori children with a roughly even split of
children aged under 5 years and 5 years or older.
Of the 25 children, 20 had a clinical ECBI score (60
or more on the scaled Intensity score). Only two
of the 25 parents did not complete the IYP course.
Twenty parents completed all 18 weekly reports, a
further three completed 16 and two completed 15.
Twenty-three parents completed the four weekly
reports at the follow-up stage, while two parents
missed one interview each.
60
The weekly Parent Report12 in the Incredible Years Initially, the positive and negative behaviour
Pilot Study was a 10-minute telephone interview scores were plotted together on graphs for each
consisting of a checklist of 23 questions assessing individual child. By observing the plots on each
positive behaviours and 26 questions assessing graph, it was found that:
negative behaviours to which parents answered The plots for many children contained
‘yes’ or ‘no’ regarding their child’s behaviour unexplained variation, as demonstrated in the
during the previous day. The interviews were example graphs for Child 101 (Figure 4) and
conducted for every week of the 18-week IYP Child 311 (Figure 5), where behaviour changed
course and for 4 weeks at the 6-month considerably from week to week.
follow-up. It was based on the Parent Daily Report
The plots of positive behaviour started with
(PDR) developed at the Oregon Learning Centre
high scores. These data were therefore less
(Chamberlain and Reid 1987). This measure was
informative because the scope for behaviour
chosen because it is well suited for repeated daily
change was restricted.
assessments and because it is especially effective
in picking up low-frequency behaviour.
Addressing the unexplained variance
The data obtained from the Parent Reports
Given the amount of unexplained variance
provided two additional sources of information
in the scores, trend lines through the weekly
about the IYP programme: an alternative measure
scores were calculated to determine whether
to the main interviews to assess programme
the children demonstrated trends of change in
efficacy and the ability to examine the individual
their behaviour. These trend lines were produced
progress for each child throughout the
using a mixed effects linear regression model.
programme and at follow-up.
This method of statistical analysis provided a line
for each child that represented her/his change in
Initial data investigation behaviour from the start of the course to course
The Parent Reports recorded negative and completion.
positive behaviours throughout the IYP course
and at the follow-up for each of the 25 children,
thereby providing a week-by-week behaviour
assessment at a single case level.
12 Descriptions of source instruments are in Appendix 2.
61
Figure 4. Example of no behaviour change (Child 101)
Figure 5. Example of behaviour change towards improvement (Child 311)
Note for both figures above: a) the vertical dashed lines represent the days on which IYP courses were
run; b) the horizontal axis break represents the period before the follow-up where no data were collected.
62
Addressing the positive behaviour scores
It was evident from Figure 6 that positive behaviour or maximum assessed score (-0.12 correlation
trended towards improvement but was limited by with the intercept and slope). Because the
the maximum score assessed or ceiling. This ceiling positive behaviour scores were considered less
effect on the scores was verified by the covariance informative than the negative behaviour scores,
parameters produced by the model, which showed the two scores were combined by adding the
a 0.81 correlation between the starting point of inverse, or absence, of positive behaviours to the
positive behaviour (the intercept) and the rate of negative behaviours.
improvement (the slope).
The results from the combined behaviour trends
As shown in Figure 7 there were definite trends seen in Figure 8 demonstrate that trend lines can
to the scores of negative behaviours. However, be used to describe the behaviour scores observed
unlike the positive behaviour scores there was no and that the combined behaviour score was an
effect of the scores being limited by the minimum effective measure of the overall behaviour change.
Figure 6. Predicted positive behaviour scores for each child and for the IYP programme overall
Note: the thick line represents the linear regression of the IYP programme overall.
63
Figure 7. Predicted negative behaviour scores for each child and for the IYP programme overall
Note: the thick line represents the linear regression of the IYP programme overall.
Findings from the observations during

the programme
The overall trend of behaviour change Trends of behaviour change for each child
The mixed effects model provided trend lines for The individual trend lines presented in Figure 8
each child (random effects) and an overall IYP show improved behaviour reported for most of
programme effect trend line (fixed effect). Using the children. The behaviour of at least one child
the combined behaviour scores, the overall trend got worse, while for some children behaviour
throughout the course, represented by the thick remained the same. Figure 8 also highlights the
line in Figure 8, was towards improved behaviour. wide range of initial behaviours exhibited by the
From week 0 to week 21 of elapsed time, the children whose parents began the IYP programme
estimated IYP programme effect size was d=1.513. and the varying rates of behaviour change their
This is in the upper range of effect sizes obtained parents reported throughout the course.
from the Family Interview data.
13 An effect size calculation can be estimated based on the slope of the overall regression line, where the baseline
behaviour score is taken from the intercept at week 0 of elapsed time (α=14.4, sd=4.58), and using the regression
formula (α=14.4 and β= -.32 points per week) a post-course behaviour score at week 21 of 14.4 – 7.7 (6.7) is taken.
Therefore, based on Cohen’s d the effect size of d=1.5 is calculated from 14.4 less 6.7 divided by 4.58.
64
Figure 8. Predicted combined behaviour scores for each child and for the IYP programme overall
Note: the thick line represents the trend of behaviour change for the IYP programme overall.
Findings from the follow-up data

The Parent Reports at follow-up measured Figure 11 demonstrates the combined positive
each child’s behaviour 6 months after course and negative behaviours to the follow-up. The
completion. Both positive and negative behaviour overall IYP programme trend (thick line) shows
scores were tracked at the follow-up by four the general pattern of change throughout the
weekly interviews over 1 month. An average score programme followed by a levelling out after
was calculated for each child individually and course completion. Again, while the overall trend
aggregated for all 25 children to provide trend confirms the effect of the IYP programme and the
lines from course completion to the follow-up. sustained improvement to the 6-month follow-up,
Figure 11 demonstrates the different paths each
As demonstrated by Figure 9 and Figure 10,
child experienced as well as a few examples of
positive and negative behaviours tended to level
children who did not show behaviour change
out to a horizontal line representing that both
and/or who did not maintain behaviour change.
positive and negative behaviours had stayed
much the same as at course completion. This
indicated the effect of the IYP programme had
been maintained. This pattern was reflected in
most of the 25 individual lines. However, some
children demonstrated further gains in behaviour
after course completion while others appeared to
lose ground.
65
Figure 9. Predicted positive behaviour scores for each child through the programme
to the follow-up
Figure 10. Predicted negative behaviour scores for each child through the programme to
the follow-up
Note: a) the vertical line represents the week of IYP course completion; b) the horizontal axis break
denotes a shortened time scale where no data were collected (follow-up data collection started in week
49); c) the thick line represents the trend of behaviour change for the IYP programme overall.
66
Figure 11. Predicted combined behaviour scores for each child through the programme to
the follow-up
Conclusions from the Parent Reports

The results from the weekly Parent Reports highlighted the variability in the children’s
obtained throughout the programme and at behaviour and demonstrated that treatment
the 6-month follow-up confirmed the efficacy effects differed across children. Parents reported
of the IYP programme evident from the Family improved behaviour for most children that
Interviews. ranged from large to moderate, while a small
Furthermore, the individual trend lines for each number showed no change.
of the 25 children in the Single Case Studies
67
Chapter 9:
Programme fidelity,
Group Leader feedback
and parent satisfaction
programme delivery. The Group Leaders provided
checklists for pre-programme preparation,
programme delivery and post-programme
follow-up and reflection. Only one checklist was
missing across the three sites and 12 programmes
in the Incredible Years Pilot Study. The completed
checklists provided documentation of fidelity to
the Incredible Years processes and delivery.
Although Group Leaders cannot change the actual
parenting skills taught in the IYP programme
without reducing treatment integrity and hence
treatment effectiveness, cultural enhancements
are encouraged. Indeed, Webster-Stratton (2007)
states that the IYP programme is effective across
groups of culturally diverse parents due to its
principles that promote and guide a culturally
responsive structure for delivering the programme
to diverse populations. She stresses the important
role Group Leaders have in affirming the diversity
of the families that take part in IYP training. Cultural
diversity is viewed as integral to a successful
learning process for parents.
Of particular interest, then, is the documentation
9.1 Programme fidelity: of any culturally responsive actions that occurred
Unified protocols in the evaluated IYP programmes. The following
practices were adopted to a greater or lesser extent
To obtain similar results to those published by the in all but one of the 12 courses: manaakitangata,
developers of evidence-based programmes, it is whakawhanaungatanga, greetings, powhiri, waiata,
important to ensure the programme is delivered te reo, whakatauaki, Māori values (eg wairua),
with fidelity. ‘Fidelity’ means the programme is whare tapa whā concepts, wharenui metaphor,
delivered in its entirety, using all the components asking the group for cultural preferences around
and processes recommended by the developer. the opening and closing of sessions and session
Special Education developed a Unified activities, consulting Kaitakawaenga/Kaumatua and
Protocol for the IYP courses in the research that having a Māori Group Leader. The exception was
required Group Leaders to complete and return a course in Canterbury which reported no cultural
checklists that documented their adherence to enhancements at the specific request of the one
the stipulated Incredible Years processes and Māori parent attending the programme.
68
9.2 Group Leader feedback 9.3 Parent satisfaction with
Group Leaders and their supervisors held the Incredible Years
debriefing sessions following the completion of Parenting programme
the IYP courses. Their feedback included:
Conducting two 18-week programmes in Parent satisfaction from the
1 year was challenging for the Bay of Plenty Family Interview
as the same Group Leaders facilitated Phase 1
and Phase 2 courses. In Canterbury, the tight In the baseline interview parents were asked to rate
timeframe exacerbated problems caused by their satisfaction with the way the IYP programme was
organised using a 3-point scale: not at all, somewhat
the earthquake.
and very. These same questions were repeated
Existing casework impinged on the .5 release following course completion. In addition, parents were
time to deliver IYP training, resulting in Group asked to rate their agreement with 11 statements
Leaders effectively doing the same amount of about the IYP course overall after completion of the
work in less time. 18-week course. These items were based on the
Programme resources were considered Incredible Years Parent Satisfaction Questionnaire, but
adequate and continue to be developed to the scoring was altered from a 7-point to a 3-point
suit the New Zealand context, particularly for scale: not at all, somewhat and a great deal.
Māori parents.
Satisfaction data are presented for the 146 parents
Petrol vouchers and taxi chits were provided (N=51 Māori and N=95 non-Māori) who attended
where required, as was assistance with some or all of the IYP sessions they were originally
childcare. Food, warm drinks and treats enrolled in. Of the 166 parents in the study, two
contributed to the welcoming environment. parents did not respond to the question about
Venues were sometimes an issue when they attendance and a further 18 parents stated they did
had to be physically rearranged for each not attend their designated course.
session. In Canterbury, earthquake damage Satisfaction with the organisation of the IYP
meant inadequate facilities were the only ones programme was generally high for both Māori and
available. non-Māori parents before the course started and at
Drop-outs were primarily due to life course completion, with no significant difference
circumstances (work, sick children, moving) between the two groups of parents14 (Table 31).
rather than to the IYP programme. Parents were very satisfied with aspects such as the
As is standard practice for IYP programmes, Group Leader’s initial home visit, the information
Group Leaders did a lot of work between provided, and the professionalism and friendliness of
the staff (88% to 100%). Satisfaction with the meeting
sessions to support or to find support for
times (77% to 88%) and suitability of the venue (75%
parents with personal and family issues.
to 78%) were slightly lower. Assistance with attendance
Where appropriate, sessions began and ended was the exception to this high level of satisfaction.
with a karakia or whakatauaki. Approximately half (45% for Māori and 55% for
All three sites mentioned the need for Group non-Māori) of the parents were very satisfied with this
Leaders to have experience in working with aspect of the IYP programme’s organisation before the
challenging children and families and to start of the course, although the proportion increased
have sufficient IYP training to run successful (61% and 70% respectively) by course completion.
programmes.
14 Differences between responses from the parents of Māori and non-Māori children were tested using chi-square
tests of significance.
69
Table 31. Satisfaction with course organisation: opinions before and after the course
by ethnicity
Very satisfied with: Before course started After course completion
Māori (N=51) Non-Māori (N=95) Māori (N=51) Non-Māori (N=95)
Group Leader’s first visit 94% 88% NA NA

Information provided 94% 90% 86% 91%
Professionalism of staff 100% 95% 88% 93%
Friendliness of staff 100% 98% 98% 96%
Suitability of venue 78% 77% 75% 76%
Course meeting times 85% 88% 77% 84%
Assistance with attendance 45% 55% 61% 70%
At course completion parents were also asked the extent to which the IYP training helped with
about their satisfaction with the overall IYP other personal and family problems. Only half of
programme. The data presented in Table 32 show the respondents (49%; 48%) felt the learnings had
the generally high level of agreement expressed generalised beyond the target child. Notably, the
by both Māori and non-Māori parents with most majority of both Māori parents (84%) and
statements about the Incredible Years approach non-Māori parents (94%) agreed a great deal
and the progress they made. There was no that the IYP programme was appropriate to their
significant difference between Māori and cultural identity. None of the Māori participants
non-Māori parents in their level of satisfaction. felt the IYP course was ‘not at all’ appropriate to
Of particular note, nearly all parents said they their cultural identity. Similar generally high levels
would recommend the IYP course to others of satisfaction with the IYP programme have been
(92% for Māori and 94% for non-Māori). The reported in other studies (eg Gardner et al 2006;
exception to the high level of satisfaction was Larsson et al 2009).
70
Table 32. Satisfaction with the course overall at course completion by ethnicity
Agree ‘a great deal’ that: Non-Māori

Māori (N=51) (N=95)
they would recommend the programme 92% 94%

IY was appropriate to their cultural identity 84% 94%
IY was the right approach 71% 79%
they can their achieve goals in the IY programme 71% 77%
they’re optimistic that IY will give good results 69% 76%
they’re confident they can manage problems by themselves 67% 70%
they can manage problems because of lessons learnt 65% 70%
the problems with their child are improving 55% 67%
they’re satisfied with their child’s progress 59% 61%
they are using lessons learnt to improve their child’s behaviour 53% 60%
IY helped with other personal and family problems 49% 48%
Comments from the primary caregivers reflected the majority view of satisfaction with the IYP
programme. For example:
“Hugely increased my confidence as a parent and this is reflected in a better relationship with my child.”
“… it has been life changing for our family.”
“IY changed the way I speak to my child. I respond rather than yell and scream.”
“Really enjoyable – got a lot out of it.”
“IY has given me more of a backbone in my parenting.”
Parent satisfaction from the in-course

measure
Parent satisfaction was also assessed at the end their children’s behaviour had improved and
of the IYP course by the Group Leaders. Parents they were confident in their ability to manage
completed a Parent Satisfaction Survey that behaviour problems. Course content was deemed
covered overall programme satisfaction, course extremely/useful, as were parenting techniques.
content and specific parenting techniques. On These data provided further evidence of the high
a 7-point scale, the vast majority of responses level of satisfaction with the IYP programme
for all categories were six and seven (positive). expressed by the majority of parents.
These ratings reflect parents’ assessments that
71
Chapter 10:
Policy implications
The aim of this chapter is to present the views

of the Incredible Years EAG on the policy
implications and applications of the findings of
the Incredible Years Pilot Study described in the
preceding chapters. Issues addressed are:
The efficacy of the Incredible Years Parenting
programme in New Zealand
Extensions of the present research
The selection of client families
The effects of context
Māori/non-Māori differences
Participants who do not respond to Incredible
Years Parenting training
Site differences
The need for an effectiveness study of the
current Incredible Years Parenting programme
in New Zealand
The organisational implications of the
Incredible Years Pilot Study.
72
10.1 The efficacy of the
programme in New Zealand
The primary purpose of this evaluation was Both designs showed that the IYP programme
to obtain evidence of the efficacy of the IYP was an effective intervention, with evidence
programme in New Zealand contexts using a of efficacy being observed up to 6 months
before-and-after design applied to three sites post-treatment. It can be argued, therefore, that
running established IYP programmes. This research the time and expensive of a RCT is not warranted.
produces good evidence of the efficacy of the IYP
While there is good evidence the IYP programme
programme at the chosen sites with estimates
has good efficacy in New Zealand in the short to
of effect sizes comparable with those found in
medium term, the Incredible Years Pilot Study
previous randomised trials and before-and-after
findings also raise a number of issues that require
designs (see Section 2.2). On the basis of these
consideration in the roll out of IYP courses as part
findings there is adequate evidence to support
of the Ministry of Education’s PB4L programme.
the conclusion that the IYP programme, if
These issues are discussed below.
well delivered, is likely to be effective in a
New Zealand context.
10.2 Extensions of the
This research was conducted as a pilot study
preliminary to a possible further randomised Incredible Years Pilot Study
controlled trial of IYP training. The authors of The Incredible Years Pilot Study shows evidence
this report are of the view that a RCT of the of good efficacy in the short to medium term.
IYP programme in New Zealand is probably However, a longer-term assessment of the
unnecessary given the strong and consistent programme is needed to examine the extent
evidence from this Incredible Years Pilot Study. to which the benefits are maintained and the
Two lines of evidence support this conclusion. extent to which these benefits may dissipate with
First, there is extensive international evidence the passage of time. To address the issue of the
demonstrating the efficacy of IYP programmes maintenance of intervention gains the Advisory
in many settings including the United States, Group recommends the present study is extended
Europe and Asia (for reviews see Kazdin 2007; to longer-term follow-ups of the children and
Ministry of Social Development 2009b). Given this parents in the present trial. This research design
international evidence there are strong a priori would require the re-assessment of the study
grounds for believing the IYP programme will be group at 1 year and 2 years post-treatment using
effective in New Zealand. Second, this conjecture the Family Interview administered at the 6-month
was supported by consistent evidence of the follow-up.
efficacy of IYP courses in New Zealand using
pre/post and single subject research designs.
73
10.3 The selection of
programme client families
In the present study all families enrolled in the There is no clear solution to this dilemma. A
IYP courses were included in the evaluation case can be made for both treating all client
irrespective of their baseline scores on the initial families or for restricting the programme to only
ECBI assessment. This procedure was used to those families with children in the clinical range.
examine the extent to which the benefits of The EAG suggests that consideration should
IYP training varied with the level of behavioural be given to providing families whose children
problems evident at the first assessment. The have ECBI scores in the sub-clinical range with
findings of the study (see Section 6.2) showed a less intensive parent management training
clear evidence of two general trends: i) the IYP programme than the IYP programme. One such
programme was effective for client families with programme is Triple P Level 4 which provides
children in both the clinical and sub-clinical an 8–10 week programme (Ministry of Social
ranges of the ECBI scales; and ii) IYP training was Development 2009b).
more effective for client families with children in
the clinical range.
10.4 The effects of context
These findings have complex implications for
the identification of client families eligible for One of the important findings of the present
IYP courses. On the one hand it can be argued study was that the benefits of IYP training
that, since IYP training is effective for families in were greater for parent reports of behaviour in
both the sub-clinical and clinical ranges of the the home context than for teacher reports of
ECBI, the programme should be offered to all behaviour in the school context. These findings
families irrespective of their baseline scores. The are consistent with a large body of evidence
potential cost of this decision may be delivering suggesting the rates of problem behaviours
IYP training to a number of families who do not in children vary with the context within which
need the programme and will not benefit from the behaviour is assessed (Webster-Stratton
it. On the other hand it can be argued that, given and Reid 2010). The important implication of
the limited resources available for delivering this observation is that programmes targeted
the IYP programme, the greatest benefit can be at parents are likely to have their greatest
obtained by delivering it to only those families impact in changing behaviours at home
with children in the clinical range. The potential whereas programmes targeted at teachers
cost of this decision will be failing to deliver the are likely to have their greatest impact on
programme to families in the sub-clinical range behaviours at school. This conclusion suggests
who would benefit from it. the importance of having programmes in
different settings and very clearly supports the
approach used in the PB4L strategy to invest in
both parent behaviour management training
and teacher behaviour management training
(Ministry of Education 2011).
74
10.5 Māori/non-Māori 10.6 Participants who do not
differences respond to the Incredible
An important policy debate has concerned the Years Parenting
extent to which the IYP programme is culturally programme
appropriate, and effective, for Māori families (eg
Altena and Herewini 2009). The findings of the While the evaluation has shown IYP training
Incredible Years Pilot Study suggest that, as a has good efficacy as a parent behaviour
general rule : i) IYP training is effective for Māori management programme, it must be recognised
and ii) the IYP programme is seen as culturally the programme is not effective for all families.
appropriate by over 80 percent of Māori parents. An important finding from the Single Case
Studies analyses reported in Section 8.1 was
At the same time, small but statistically detectable that in the region of one in four families enrolled
differences were found between Māori and in IYP courses failed to report significant
non-Māori client families, with the effect sizes change in the client child’s behaviour. This
for Māori being slightly smaller than those for suggests that, while the programme benefits
non-Māori. These findings suggest that, while the majority of families receiving IYP, it does not
IYP is an effective programme for Māori, there benefit all families. This is a pattern common
is a case for further investment in programme for interventions of this type. Two extensions to
development to improve the programme’s the current IYP programme are indicated. The
delivery and to maximise the gains for Māori. The first is the need to identify those families and
Nga Tau Miharo programme resource developed children who have not benefited from the IYP
by the Werry Centre is an important innovation programme. The second is to develop further
in this area. It provides a manual and DVD for programmes and resources to meet the needs
the delivery of IYP training to Māori families. of families who have attended IYP courses but
Nga Tau Miharo was not in place during the who have not received the expected benefits.
current evaluation and it is possible that, had it
been available, the small ethic differences found
in this study would have been eliminated. 10.7 Site differences
More generally, views about the effectiveness An unexpected finding from this evaluation
of the IYP courses for Māori have been divided. was the presence of significant between-site
Previous research (Fergusson et al 2009) found the differences in the efficacy of the IYP programme,
programme was equally effective for Māori and with one site producing greater benefits than the
non-Māori whereas Māori commentators have other two sites. The reasons for these differences
suggested it is not acceptable for Māori unless it are unknown. Nevertheless, the findings
is substantially changed (eg Altena and Herewini have important policy implications since they
2009). The findings of this evaluation suggest a highlight the need for well-developed processes
middle position is the most tenable. Specifically, to audit the delivery of the IYP programme
while there is good evidence to suggest the IYP and to compare the performances of different
programme is effective for Māori families there providers with benchmarks based on the
is also evidence indicating the benefits of IYP present evaluation. An important next step in
training are slightly but detectably smaller for the development of IYP training in New Zealand
Māori than the benefits for non-Māori. These is the need to conduct a national effectiveness
findings clearly justify the delivery of the IYP study which examines a range of issues relating to
programme to both Māori and non-Māori but also variability in the outcomes of the IYP programme.
suggest the need for continuing investments in
improving the delivery and efficacy of IYP training
to Māori families. It is likely the implementation of
Nga Tau Miharo will provide an important step in
this process.
75
10.8 The need for an 10.9 Organisational
effectiveness study implications
The various issues raised above regarding An important feature of this evaluation was to
ethnic differences (Section 10.5), clients who develop and explore a new model for evaluating
do not improve (Section 10.6) and between-site government-funded programmes for the
differences (Section 10.7) highlight the need prevention, treatment and management of
for a further evaluation of the IYP programme conduct problems in childhood and adolescence.
using data from the Incredible Years National This model was based on a consortium approach
Register held by the Ministry of Education. This that brought together the skills of a number
register contains data on client ethnicity, pre and of groups: the service providers delivering the
post scores on the ECBI and measures of client programme; the government researchers who
satisfaction. The availability of this database evaluated the programme; academic advisors
makes it possible to examine the extent to which with expertise in research design and analysis;
the findings from the present evaluation can and Māori researchers.
be applied to the IYP programme as it is being
This new model was developed in response
delivered by multiple providers in multiple
to concerns about previously used evaluation
centres. Some of the important questions
models in which new policies and programmes
regarding the IYP programme include:
were evaluated using a tendering process.
Overall, how effective is the IYP programme in This resulted in external groups and agencies
reducing childhood problem behaviours? conducting evaluations on behalf of the
To what extent is the programme effective for Government. These processes were seen as being
Māori and non-Māori? less efficient because of the limited participation
What proportion of children enrolled in of government service providers and research/
the programme fail to show significant policy makers in the evaluation process. The
improvements in behaviour? model proposed for the assessment of the IYP
programme involved a collaborative process
How much variation is there in the efficacy of
that included: representatives of the service
the IYP programme among different sites?
developer (Ministry of Education); the funder of
How well is the IYP programme being delivered the fieldwork (Ministry of Health); the research
at different sites? evaluation team (Ministry of Social Development);
It is the view of the Advisory Group that using the and academic advisors with expertise in the area
Incredible Years National Register data to address of conduct problems research and the collection
these and related questions is an essential next step of developmental data.
in the development and evaluation of IYP courses
in New Zealand. While the Incredible Years Pilot
Study has provided in-depth information about the
performance of the IYP programme at three sites,
this evaluation fails to provide an overview of how
well the programme is being delivered at a national
level. Learning about this is important both for the
quality control of programme delivery and for the
further development of the IYP programme in a
New Zealand context.
76
10.10 Summary of
Evaluation Advisory
The present study provides a case history of Group recommendations
the effectiveness of this model in producing a The Incredible Years Pilot Study should be
well-designed and effective evaluation of the viewed as providing adequate evidence of the
IYP programme. The EAG believes the success of efficacy of the IYP programme in New Zealand.
the evaluation speaks for itself and the model
Investments should be made into the
developed in the process of evaluating the IYP
longer-term follow-up of the families
programme can be generalised to many other
participating in this evaluation, with further
contexts. The critical features of this model are:
assessments being made at 1 year and 2 years
It requires developing a consortium in which post-treatment.
different members bring different skills to
Consideration should be given to providing
the project.
parent management training programmes,
The control of the evaluation remains within such as Triple P Level 4, that are less intensive
government and contributes to building than the IYP programme for client families
government research capacity. with children whose behaviour scores
The costs of research are reduced since the on the ECBI scales place them in the
conduct of the research does not require the sub-clinical range.
payment of the overhead and profit costs Consideration should be given to developing
associated with outside tendering. parallel research to evaluate the Incredible
It is the view of the EAG that this model of Years Teacher programme to determine the
evaluation should continue to be used in the benefits of this programme for behaviours in
evaluation of other programmes targeted at the classroom and for behaviours at home.
the prevention, treatment and management of Further investment is needed to increase
childhood conduct problems. Some important the efficacy of the IYP programme for Māori
areas for further development include: families.
Longer-term follow-up of the families studied There is a need to develop further
in this evaluation (see 10.3) programmes and interventions to meet the
Evaluation of the effectiveness of the IYP needs of families whose children do not show
programme using the Incredible Years sufficient behaviour change at the end of the
National Register data (see 10.8) IYP programme.
Evaluation of the Incredible Years Teacher Investment should be made into a national
programme (see 10.4). effectiveness study using the Incredible Years
National Register data held by the Ministry of
The EAG firmly believes the proposed programme
Education.
of research will be better achieved by maintaining
the current evaluation team and EAG who have The present consortium structure for
overseen the Incredible Years Pilot Study, thus evaluating programmes targeted at childhood
transferring the skills and learning acquired over conduct problems should be maintained.
the course of this evaluation to further projects. A future research programme is outlined in
Section 11.3.
77
Chapter 11:
Summary and
conclusions
11.1 Summary of findings

related to research
questions
The findings from the Incredible Years Pilot Study
presented in this report provide the following
evidence relating to the research questions stated
in Chapter 3.
1. Programme efficacy: To what extent was there
evidence to suggest that the IYP programme was
effective in a New Zealand context?
Before-and-after comparisons of families
enrolled in the IYP programme produced
effect size estimates for change in child
behaviour, parenting practice and
conflict and relationships consistent with
previous Incredible Years evaluations using
randomised trials. These RCTs reported effect
sizes for child behaviour change ranging
from d=.48 to d=.89 and effect sizes for
parenting practice ranging from d=.38 to
d=1.44.
Parents receiving IYP training reported
significant improvements in the following
areas: child behaviour (effect sizes d=.51 to
d=.96), parenting practices (effect sizes d=.26
to d=.83) and conflict and relationships
(effect sizes d=.21 to d=.60).
Effect sizes for improvements in parenting
practices and family relationships were
typically lower (d=.26 to d=.71) than effect
sizes for parent-reported improvements
in child behaviour. This indicated small
changes in parenting practice may produce
substantial improvements in child behaviour.
78
The Parent Report Single Case Studies reflected On the combined outcome measures at
the findings of improved child behaviour programme completion the differences
at course completion reported for the main between Māori and non-Māori children
study. Furthermore, these data highlighted the were not significant. However, at
variability in child behaviour and showed that follow-up a small but statistically significant
treatment effects were not equal for all children difference between responses by parents
in the study. of Māori and non-Māori children was
detected for the overall child behaviour
Small to medium reductions in caregiver stress
outcome measure (p=.025). This suggests
and anxiety were reported at post-course there may be a particular challenge
and follow-up, as were fewer child visits to in maintaining the benefits of the IYP
health services and fewer stressful life events. programme for Māori families.
Parents also reported a decrease in depressive
4. Programme entry measures: To what extent do
symptoms at follow-up.
the outcomes of the IYP programme vary with
Before-and-after comparisons of scores on the child’s behavioural adjustment at baseline?
the Social Development Scale completed by Children with pre-course scores in the clinical
teachers produced small effect sizes (d=.17 range of ECBI Intensity improved to a greater
at post-course and d=.29 at follow-up). This extent at post-course on all child behaviour
suggested some of the improvements in child measures except Social Competence than
behaviour carried over to the school setting. children with scores below the clinical
2. Sustainability of gains: To what extent were any cut-off. Significant differences in reported
benefits of IYP sustained over time? Conduct Disorder and ODD remained at
follow-up, whereas differences in ADHD,
The reported improvements evident at the
Self-control and Anxiety/Withdrawal were
completion of the IYP courses were generally
no longer evident. This perhaps reflected
sustained at the 6-month follow-up. Significant
the continued improvement for children in
improvements were maintained in the
the sub-clinical group.
following areas: child behaviour (effect sizes
The greater improvement reported for
d=.56 to d=1.0), parenting practices (effect sizes
children in the clinical range on the
d=.25 to d=.79) and conflict and relationships
pre-course SCS on measures of Conduct
(effect sizes d=.15 to d=.59).
Disorder and ODD at post-course were
The Parent Report Single Case Studies sustained at follow-up. Self-control and
reflected the findings of improved child Social Competence had also improved
behaviour sustained to follow-up reported to a greater extent for children in the
for the main study. SCS clinical group than for children in
3. Benefits for Māori: To what extent were the the sub-clinical group 6 months
programme outcomes for Māori similar to or following course completion.
different from the outcomes for non-Māori? 5. Between site differences: To what extent do the
The evidence suggests the IYP programme outcomes of IYP vary with the site at which the
programme was delivered?
was both effective for and accepted by Māori
families. However, at both post-course and The IYP courses were effective at each of
follow-up parents of Maori children reported the three sites. However, the significant
slightly smaller improvements in child behaviour interaction between the global child
behaviour measure and site (p<.002)
and slightly smaller reductions in negative
indicated the courses delivered in
parenting practices on the individual outcome
Mid-Central were particularly effective.
measures than did parents of non-Maori
The reason for this difference is not known.
children. These differences were not significant.
79
11.2 Conclusions
6. Client satisfaction: To what extent do the parents These results lead to the following conclusions:
who received the IYP training see the programme The IYP programme can be successfully
as: a) helpful; b) appropriately organised; c) implemented in New Zealand and retain its
effective in dealing with their child’s behavioural general level of effectiveness in reducing conduct
problems; d) culturally appropriate? problems and enhancing parenting skills.
Parent ratings in both the Family Interview Although these findings are generally
and the in-course questionnaire revealed consistent with the RCT studies summarised
high to moderate satisfaction with the IYP in Chapter 2, the before-and-after design
programme for both Māori and non-Māori of the Incredible Years Pilot Study may lead
parents on most measures. to an over-estimation of effect sizes. The
7. Other findings comparison presented in Table 33 of the effect
The study reached the target group of sizes for child behaviour and parenting practices
children and parents. obtained post-treatment in this research with
The programme was delivered with fidelity the effect sizes reported in a Norwegian RCT
to the delivery, processes and techniques before-and-after study (Larsson et al 2009)
stipulated by Incredible Years. is reassurance the present study provides an
adequate estimation of the efficacy of the IYP
programme in New Zealand. On the basis of
these data it seems unlikely a RCT study of
IYP courses in New Zealand would produce
results materially different from the estimates
provided by the before-and-after comparisons
reported here.
The findings provide a series of performance
benchmarks against which the future roll out of
the IYP programme in New Zealand should be
assessed.
Table 33. Effect sizes for child behaviour and parenting practices reported in Larsson et al
(2009) and the Incredible Years Pilot Study
Larsson et al (2009) Incredible Years Pilot Study

Measure Effect size Instruments used Effect size Instruments used
Child Behaviour d=.65 ECBI Intensity d=1.00 ECBI Intensity
Parenting Practice
Positive parenting d=1.44 Alabama Parenting Questionnaire d=.46
Parenting Practices
Harsh discipline d=.61 d=.59
Arnold-O’Leary Parenting Scale Interview
Inconsistent discipline d=1.05 d=.54
80
11.3 Future research
The findings from the Incredible Years Pilot Study provided by the Incredible Years Pilot Study
leave important unanswered questions that need to assess between-site heterogeneity and to
to be addressed. The following future research determine the extent of variation.
programme is recommended: The suggestions that the benefits of IYP
A longer-term follow-up of the children and training may be slightly larger for non-Māori
parents in the present trial to examine the than for Māori families and that there appears
extent to which programme benefits are to be a particular challenge for Māori in
sustained and the extent to which these benefits maintaining behaviour change point to the
may dissipate over time. The research design need for further research to examine Māori/
would extend the present study to re-assess the non-Māori differences in greater detail. Findings
participants at 1 year and 2 years post-treatment from that research will inform the work on
using the Family Interview administered at the Māori enhancements for IYP courses. Regular
6-month follow-up. monitoring of changes in ECBI and SCS scores
A national effectiveness study based on the for Māori and non-Māori children at every site is
Incredible Years National Register data held necessary to ensure the effectiveness of the IYP
by the Ministry of Education to investigate programme for Māori is maximised.
whether or not the effect sizes for the changes An evaluation of Ngā Tau Mīharo ō Aotearoa,
in the ECBI and SCS measures obtained in the the Incredible Years resources developed by
Incredible Years Pilot Study can be replicated the Werry Centre for Māori Group Leaders, is
on national aggregated data and for individual required to ascertain whether or not differences
courses at all sites. The study should evaluate: in outcomes for Māori and non-Māori families
i) the overall effectiveness of the IYP courses; are evident in courses using these resources. A
ii) the IYP programme’s effectiveness for Māori comparison of before-and-after ECBI and SCS
and non-Māori; iii) possible variation among scores at sites with Māori families that use
providers in the effectiveness of IYP training; Ngā Tau Mīharo ō Aotearoa with sites that do
iv) the proportion of client families who fail not use these resources will reveal whether or
to show improvement; v) the adequacy of the not the differences reported here remain for
delivery of the IYP programme. IYP courses that have implemented culturally
The performance benchmarks reported here responsive refinements for Māori.
provide measures against which the future roll The lower effect sizes found for the teacher
out of the IYP programme should be judged. measure suggests parent management training
Consistent and rigorous quality control monitoring programmes like Incredible Years, although
of provider outcomes will ensure the programme is beneficial in reducing behaviour problems
effective and the funding is well spent. in schools, are not sufficient. It is likely parent
Further investigation is required to understand management training should be used in
the circumstances that produced site variation conjunction with school-based programmes
in outcomes to inform best practice principles for maximum return in the school environment.
and to ensure quality control. The site differences The evaluation of the Incredible Years Teacher
found in the Incredible Years Pilot Study programme is required.
demonstrate that differences may occur in Although the evidence presented in this report
programme outcomes for a variety of reasons strongly supports the effectiveness of the IYP
even at well-established and well-functioning programme in the New Zealand context, it is
sites. This finding points to the need for possible existing parenting programmes produce
consistent quality control of provider delivery similar effects. Further research is required to
and monitoring outcomes. It indicates the need compare the cost benefit of providing the IYP
for fidelity checks and an audit of each IYP programmes against the cost benefit of other
programme in every site using the benchmarks parenting programmes.
81
Appendix 1.
The Incredible Years
Pilot Study Evaluation
Advisory Group
Dr John Church Adjunct Senior Fellow, Psychology Department, University of
Canterbury
Professor David Fergusson Executive Director, Christchurch Health and Development Study,
University of Otago
Professor Richie Poulton Director, Dunedin Multidisciplinary Health and Development
Research Unit, University of Otago
Dr Sarah Hayward Formerly Senior Project Manager, Ministry of Health; currently

Principal Advisor, Ministry of Justice
Brian Coffey Group Manager, Strategy (Special Education), Ministry of Education
John McGovern Service Manager, Ministry of Education (Special Education)
Ross Mackay Principal Advisor, Ministry of Social Development
Dorian Gray Senior Analyst, Ministry of Social Development
Fiona Sturrock Research Lead, Ministry of Social Development (Chairperson)
82
Appendix 2.
Source instruments for
the Incredible Years
Pilot Study
See The Incredible Years Pilot Study Technical the number of problem behaviours and the
Appendix (Gray 2013) for more details on the frequency with which these behaviours occur. The
source instruments used in the study. scale demonstrates good stability, with reliability
coefficients from 0.86 (test-retest) to 0.98
The Family Interview Questionnaire (internal consistency) (Robinson et al 1980). Good
convergent validity is demonstrated by significant
The Family Interview Questionnaire is based on
correlations with the Child Behaviour Checklist
a composite of instruments determined by the
and the Parenting Stress Index (reported in Jones
outcome variables of primary interest, and the
et al 2007). The ECBI is a well-respected and
mediating and contributing variables associated
well-used measure for assessing the frequency of
with them. There are four main blocks of outcome
conduct problem behaviour that is reliable and
variables underlying the interview:
valid, and identifies change due to intervention
Child behaviour – decreased problem over time (Rhodes 2009). It has been used
behaviour extensively within the field of parent training
Parent behaviour – increased positive intervention including in several studies of the
parenting IYP programme (Webster-Stratton 1998; Scott et
Family relationships – improved parent/child al 2001; Hutchings et al 2007). The ECBI has two
and parent/partner relationships scales, the Intensity scale and the Problem scale,
but the latter was not used in the Family Interview
Other relevant outcomes – parental
Questionnaire.
depression, anxiety and stress; stressful
life events. The Strengths and Difficulties Questionnaire
(SDQ) (Goodman 1997) is a 25-item inventory
Child behaviour designed as a behavioural screening measure to
assess the occurrence of particular behaviours
Child behaviour was measured with four
associated with conduct problems, hyperactivity,
instruments adapted for use in this context.
emotional symptoms and peer problems in
Approval was granted to adapt the Eyberg
children. The scale has demonstrated good
Child Behaviour Inventory with the payment of
stability as judged by internal consistency
copyright fees. Scale items were all rated on a
(mean Cronbach’s alpha=0.73), cross-informant
consistent 3-point scale: not at all, somewhat, a
correlation (mean=0.34) and test-retest stability
great deal.
after 4–6 months (mean=0.62) (Goodman 2001).
The Eyberg Child Behaviour Inventory (ECBI) There are versions for parents and teachers.
(Eyberg and Ross 1978; Eyberg 1980) is a 36-item Both versions contain five subscales: emotional
inventory measuring child problem behaviours problems, conduct problems, hyperactivity, peer
perceived by the caregiver, and is normed for problems and pro-social behaviour. The additional
children aged 2–16 years. The ECBI measures Impact Supplement scale, which measures how
83
the caregiver or teacher perceives the child’s allowing rules to go unenforced or providing
behaviour and the impact of the problem on the positive reinforcement for misbehaviour.
child’s daily life, was not used in the Incredible ‘Over-reactivity’, refers to displays of anger,
Years Pilot study. The SDQ is a screening measure meanness or irritability. ‘Hostility’ refers to the use
and not as sensitive to clinical change as the ECBI. of verbal or physical force. The scale has adequate
internal consistency and has been found to have
The Social Competence Scale – Parent Version
good test-retest reliability. It looks specifically
(P-COMP), developed by the Conduct Problem
at parenting practices rather than at child
Prevention Research Group (Fast Track), consists
misbehaviour. This measure provides an accurate
of 12 items that assess the child’s positive
assessment of parental discipline strategies,
social behaviours as perceived by the parent. It
yielding correlations with independent blind
includes a measure of frustration tolerance and
observation of parenting behaviours. Scale items
communication skills. This instrument is also used
were all rated on a consistent 3-point scale: never,
by the Incredible Years Group Leaders to assess
sometimes, often.
the participating parents’ children (CPPRG 1995;
Corrigan 2002).
Relationships
Some of the child behaviour items were based The relationship between the caregiver and their
on those used in the Early Start Evaluation child and partner were measured using three
(Fergusson et al 2005b). These items have been instruments. Approval was granted to adapt the
tested in a New Zealand context and were found Conflict Tactics Scales CTSPC and CTS2 with the
to provide robust measures of child behaviour. payment of copyright fees.
Parenting practices The Conflict Tactics Scales (Straus et al 1993;

Straus et al 1998) have been used for decades
The Alabama Parenting Questionnaire (APQ) was
to evaluate violence within families and
designed to tap the parenting dimensions that
intimate relationships. Two updated versions,
are risk factors associated with child conduct
the Conflict Tactics Scale Parent/Child (CTSPC)
disorder (Shelton et al 1996). It consists of 42
and the Revised Conflict Tactics Scale 2 (CTS2),
items, scored with a 5-point rating scale, that
were used in the Incredible Years Pilot Study.
load onto five subscales: parental supervision,
The CTSPC items related to the parents’ use of
positive parenting, corporal punishment, parental
verbal aggression and physical assault in their
involvement and inconsistent discipline. Dadds
relationships with their children. Scale items were
et al (2003) evaluated the APQ with a community
all rated on a consistent 3-point scale: never, less
sample of Australian children aged 4–9 years. The
than once a week, once a week or more. The CTS2
results showed good internal consistency, validity
focused on violence between the parents. Scale
and test-retest reliability for the measure. Scale
items were all rated on a consistent 3-point scale:
items were all rated on a consistent 3-point scale:
never, sometimes, often.
never, sometimes, often.
The Arnold-O’Leary Parenting Scale (Arnold
et al 1993) is a 30-item inventory of parenting
competencies that measures dysfunctional
and/or ineffective parenting practices of parents
with younger children. The scale yields an overall
score and three revised subscale scores of
dysfunctional strategies used by parents tackling
problem behaviour. ‘Laxness’, refers to insufficient
monitoring of the child and her/his behaviour,
84
The Parent Problem Checklist (PPC) was Study based on Holmes and Rahe (1967). These
developed as a measure of inter-parental conflict, items have been tested in a New Zealand context
especially as it relates to the parents’ ability to co- and have been found to provide a robust measure
operate and to act as a team in performing the of a family’s exposure to stressful and adverse
executive parenting functions within the family. life events.
It contains 16 items measuring the presence or
absence of parental disagreement over rules and Cultural participation
discipline for child misbehaviour, the occurrence Cultural identity and participation for parents
of open conflict over child-rearing issues and of Māori descent was measured using a Mason
whether or not parents undermine each other’s Durie questionnaire adapted for the Christchurch
relationships with the children. The PPC is a Health and Development Study (Broughton
unidimensional measure with moderately high et al 2000).
internal consistency and high test-retest reliability
(Dadds and Powell 1991). Scale items were all Parent satisfaction
rated on a consistent 3-point scale: not at all,
Parent satisfaction was assessed with items from
somewhat, a great deal.
the Incredible Years Parent Program Satisfaction
The Partner Attachment Scale measures the Questionnaire Basic Parent Programme (1–4), as
quality of the relationship between parents. The well as specific questions about expectations of
items used in the study are based on a selected and satisfaction with the IYP programme.
series of items from Braiker and Kelley (1979) as
used in the Christchurch Health and Development Demographics
Study (CHDS) 21-Year Interview. Scale items were The background information obtained in the
all rated on a consistent 3-point scale: doesn’t Family Interview Questionnaire covered such
apply, somewhat applies, definitely applies. things as family size, housing, education, and
employment. The New Zealand Deprivation Index
Child health 2006 was determined through ‘meshblocking’ the
A question from the Early Start Field Trial 5-Year addresses of the Family Interview participants and
Follow-up questionnaire (Fergusson et al 2005b) extracting the relevant New Zealand Deprivation
assessed the use of child health services. Index to each meshblock.
Parental depression The Teacher Questionnaire

The Composite International Diagnostic Interview Child behaviour in the classroom was assessed
(CIDI) version of the Diagnostic and Statistical using the Canterbury Social Development Scale
Manual of Mental Disorders (DSM) assessed (Church et al 2006). This is a 30-item rating
depression symptoms (World Health Organisation scale which consists of brief descriptions of 15
1993) and the anxiety and stress scales of the antisocial behaviours and 15 positive social
Depression Anxiety Stress Scale (DASS) measured behaviours which are likely to occur in the
anxiety and stress (Lovibond and Lovibond 1995). classroom (or early childhood centre). Each item
is rated on a 5-point scale from ‘never’ through to
Life events ‘very frequently’. The CSDS exists in four versions:
The occurrence of stressful life events was one for the preschool level, one for Years 1 to
measured using items from the Early Start Field 4, one for Years 5 to 8, and one for Years 9 and
Trial 5-Year Follow-up questionnaire (Fergusson 10. The versions differ only with respect to age
et al 2005b), which was an expansion of the items appropriate wording for individual items. Teacher
used in the Christchurch Health and Development responses to the antisocial items are reverse
85
scored so the total score provides a standardised initiations, and the third column was used to
measure of the level of social development. For code positive and negative reactions made by
the present study the rating scale was changed the parent in response to child statements and
from a 5-point to a 3-point scale which meant the child behaviour (Church 2012b).
clinical cut-offs for the scale could not be used. Parent initiations (Column 1) were coded
into three classes: compliance requests,
The Single Case Studies questions and other initiations,
The weekly Parent Report was based on the and other talk.
Parent Daily Report (PDR) developed at the Child behaviour following a compliance
Oregon Learning Centre (Chamberlain and request (Column 2) was classified as
Reid 1987). The PDR is administered to parents compliance/non-compliance and child
by the Interviewer via the telephone. The responses to other parent talk were
caller asks parents to report whether or not classified as appropriate/inappropriate
any of a list of child behaviour problems had reply. Other child talk was classified as
occurred at home in the previous 24-hour other talk.
period. If a behaviour has occurred, parents
Parent reactions (Column 3) to child
are asked how stressed the behaviour made
responses to compliance requests, to
them feel. This measure was chosen because
child replies, and to child behaviour were
it is well suited for repeated daily assessments
classified either as positive reactions or as
and because it is especially effective in picking
negative reactions.
up low-frequency behaviour.
All recording and all coding was done by a trained
The weekly Parent Report in the Incredible Years
early childhood lecturer working under the
Pilot Study was a 10-minute interview using a
supervision of Dr John Church at the University of
list of 23 questions on positive behaviours and
Canterbury.
26 questions on negative behaviours to which
parents answered ‘yes’ or ‘no’ regarding their
child’s behaviour the previous day. Parents were
not asked about stress.
For the Direct Observation trial, digital
recordings were made of parent-child
interaction during three consecutive 5-minute
play activities: child initiated play, parent
directed play and packing up. Recordings were
made in a clinic using a standard collection
of eight toys. Each 15-minute recording was
coded using a 3-column coding form. Each
line on the recording form recorded a single
parent-child or child-parent interaction.
Interactions were coded using commonly
employed coding categories similar to those
used by Eyberg (2010) and Forehand and
McMahon (1981). The first column was used
to record parent initiations (and responses
to child initiations), the second column was
used to record child responses and child
86
Appendix 3.
Statistical methods
Programme efficacy (outcome) and Programme effect sizes

sustainability In addition to the repeated measures test, the
The programme efficacy or outcome is analysis includes ‘effect size estimates’ to examine
determined by testing for a linear trend of change the size of the change in behaviours. Effect size
in the mean scores from the baseline through calculations are Cohen’s d (expressed as ‘d’).
the mid-course to the post-course interview Cohen’s d is the standardised difference between
(observation points or time). The sustainability of means of proportions (Cohen 1977). Cohen
programme outcomes is determined by testing suggests an effect size of d=.20 is small, an effect
for a linear trend between the baseline and the size of d=.50 is medium, and an effect size of
follow-up interviews. A general linear model d=.80 is large. These interpretations are arbitrary
is used to test the hypothesis that there was a but provide an indication of how large the
linear trend in the scores by applying an Analysis behaviour change is. A positive effect size (d>0)
of Variance (ANOVA) for repeated measures. indicates improved behaviours while a negative
‘Repeated measures’ refers to the comparing effect size (d<0) indicates worsened behaviours.
of the three interviews over time for the same The effect size for each measure is calculated
children. If the p-value for time is small (p<.05) as the difference between the baseline and
then the null hypothesis that there was no linear comparison mean scores divided by the standard
trend of change through the course can be deviation of the scores for that measure at
rejected, which suggests the programme was baseline, where the comparison is either post-
effective in changing behaviours. course (interview 3) or follow-up (interview 4).
The algebraic form of the general linear model In both cases, the baseline standard deviation is
can be expressed as: used, which may produce conservative estimates
of effect size.
Yᵢ = ı
Where:
Y = outcome (Conduct Disorder, etc)
i = observations 1 to N
j = observation points (Time) 1, 2, and 3
(or 1 and 4 for the follow-up)
87
Effect size between baseline and post-course results: Where:
Mean score at interview 1 (baseline)

Mean score at interview 3
(post-course)
Effect size between baseline and follow-up results:
Mean score at interview 4 (follow-up)
Standard deviation of scores at
interview 1 (baseline)

Confidence intervals on the effect sizes Programme outcomes between

Because the effect sizes are estimates based sub-groups
on samples, 95% confidence intervals (CI) are Programme outcomes are tested between
calculated to provide a range for the estimated sub-groups such as Māori/non-Māori children
effect sizes. The confidence intervals are and clinical/sub-clinical Conduct Problem, for
calculated using a SAS® Macro developed by instance. This is because any improvements
Hess and Kromrey (2003), which calculates the from the programme could differ across the
confidence intervals based on three inputs: i) sub-groups being compared. The test is based on
the effect size; ii) the sample size of the effect the interaction between the sub-group and the
size calculation at the comparison interview change in the outcome scores as determined by
(treatment); and iii) the sample size of the a Multivariate Analysis of Variance. Evidence of a
effect size calculation at baseline (control). The significant interaction between sub-group and
comparison sample size is the sample size at the the change in outcome scores is demonstrated
post-course or follow-up interview, which is used where p<.05. The absence of a significant
in the macro instead of the baseline or control interaction was taken to imply the relative change
sample size as it is usually the smallest sample was broadly similar across the sub-groups being
size. Hence the confidence limits calculated are compared.
conservative.
In the example of the Māori/non-Māori ethnicity
The calculated confidence intervals will be fairly sub-group, the algebraic form of the general
wide for small sample sizes. In some cases of linear model can be expressed below, where the
small sample sizes and small effect sizes, the hypothesis that the programme outcomes were
lower confidence limit may be less than zero. The broadly similar between Māori and non-Māori is
p-values for the tests of a linear trend should be taken from the p-value for the ethnicity by time
used as evidence of change and not the effect interaction.
size confidence intervals.
88
Where:
Y = outcome (Conduct Problem, etc) j = observation points 1 and 3 (or 1 and 4)
i = observations 1 to N
Corrections to multiple hypothesis testing

The tables reported contain multiple hypotheses outcome tables (Table 34). It was thought a
testing, which requires some consideration of the Bonferroni adjustment was sufficient for the overall
possibility of low p-values occurring by chance. outcome tables because of their strong findings.
Two methods are employed to reduce the chance Table 34 shows the Bonferroni adjustments made
of false-positive conclusions from hypothesis to the α=.05 level for each of the programme
testing. First, a Bonferroni adjustment to the α=.05 outcome and sustained outcome tables.
is applied to the overall programme and sustained
Table 34. Bonferroni adjustments applied
Tables Bonferroni adjustment to α=.05 Number of tests
Child Behaviours (programme outcomes)* p-values <.006 are significant 8

Child Behaviours (sustained outcomes) p-values <.008 are significant 6
Parenting Practices p-values <.005 are significant 9
Conflict Tactics/Relationships p-values <.006 are significant 8
Other outcomes p-values <.010 are significant 5
*Includes the ECBI and SCS scores as collected by the IYP course Group Leaders.
Second, the sub-group interaction tables are The multivariate analysis provides a p-value that
summarised with a doubly Multivariate Analysis of represents a broad outcome for each table such as
Variance, which tests the hypothesis that there is Child Behaviour or Parenting Practices as a group.
no interaction between the broad outcome being In the example of the Māori/non-Māori ethnicity
tested (combining all the measures in a table), sub-group, the algebraic form of the general
observation point (time), and the sub-group of linear model can be expressed below, where the
interest. The purpose of this is to ensure, where only hypothesis that for all k=1 to K where
a few of the measures show interactions between each k is an individual measure such as Conduct
sub-groups, the sub-groups were or were not similar. Disorder, Oppositional Defiant Disorder, etc.
Where:
Y = outcome (Child Behaviour, etc) j = observation points 1 and 3 (or 1 and 4)
i = observations 1 to N k = outcome k (k= 1 to K, Conduct Disorder,
Oppositional Defiant Disorder, etc)
89
Summary of statistics and tests
The following table summarises the statistics calculated, and the tests for linear trends that provide the
evidence to reject the null hypothesis of no linear trend in scores throughout the Incredible Years Pilot
Study, meaning no improvement.
Table 35. Summary of analysis methods used
Outcome results for each measure Statistical tests
Scores from each measure Test for a linear trend of improvement

Mean (average) scores and standard deviations (std) at A repeated measures Analysis of Variance is used to test for
baseline, mid-course, and post-course. a linear trend through the observation points. A p-value,
Mean scores and std at baseline and follow-up. with a Bonferroni adjustment, provides evidence of a
linear trend in the mean scores.
Effect sizes (Cohen’s d)
The effect sizes ‘d’ with 95% CI:
Effect size of the programme outcomes, calculated for
baseline and post-course.
Effect size of the outcomes 1 year after baseline,
calculated for baseline and the follow-up.
Effect sizes of sub-groups for each measure Test for interactions between sub-groups and time
Effect size of the programme outcomes by sub-groups (changes between observations over time)
with 95% CI, calculated for baseline to post-course. A repeated measures Analysis of Variance is used to test for
Effect size of the outcomes 1 year after baseline by interactions between sub-group and time or the change
sub-groups with 95% CI, calculated for baseline to in outcomes scores:
follow-up. through baseline to post-course observations
baseline with follow-up observations.
A p-value of p<.05 suggests the sub-groups experienced
different effect sizes.
Overall outcomes and sub-groups Test for interactions between sub-group, grouped
Range and median of effect sizes for each overall outcome measures, and time
outcome group (table) by sub-group calculated for A doubly multivariate (with repeated measures) Analysis
baseline to post-course and baseline to follow-up. of Variance was used to test for interactions between
sub-groups, grouped outcomes (in each table), and the
changes through observation points in time. This test
was used to confirm there were no overall trends in the
outcomes for the sub-groups.
90
References
Altena, I and Herewini, T (2009) Incredible Years Marae Based Group. Auckland: Werry Centre
Aos, S, Miller, M and Drake, E (2006) Evidence-based public policy options to reduce future prison
construction, criminal justice costs and crime rates. Olympia: Washington State Institute for Public Policy
Arnold, P S, O’Leary, S G, Wolff, L S and Acker, M M (1993) The Parenting Scale: A measure of dysfunctional
parenting in discipline situations. Psychological Assessment 5 (2), 137–144
Belsky, J, Vandell, D L, Burchinal, M, Clarke-Stewart, K S, McCartney, K and Owe, M T (2007) Are there
Long-Term Effects of Early Child Care? Child Development, March/April, Vol 78, No. 2, 681–701
Berryman, M, Woller, P and Glynn, T (2009) The Incredible Years: Learning from the experiences of Māori
whānau and Māori staff in Special Education. Wellington: Ministry of Education
Berryman, M, Woller, P and Glynn, T (2012) Māori Whānau and Māori Facilitators: A Look at their Experiences
in Incredible Years in 2011. Wellington: Ministry of Education
Braiker, H and Kelley, H H (1979) Conflict in the development of close relationships. In R L Burgess and
T L Huston (Eds) Social exchange in developing relationships. New York: Academic
Brestan, E V and Eyberg, S M (1998) Effective psychosocial treatments of conduct-disordered children and
adolescents: 29 years, 82 studies, and 5,272 kids. Journal of Clinical Child Psychology, 1998 27(2):180–189
Broughton, J, Fergusson, D M, Rimene, C, Horwood, L J and Sporle, A (2000) Nga Tini Aho O Te Ao Hou.
The many strands of contemporary Māori society: Māori ethnicity and identity in the Christchurch Health and
Development Study. The Ngai Tahu Māori Health Research Unit (NTMHRU)
Cargo, T (2008) Māori experiences of delivering the Incredible Years Parenting Programme (Reflections).
Auckland: Werry Centre
Chamberlain, P and Reid, J B (1987) Parent observation and report of child symptoms. Behavioral
Assessment, 9, 97–109
Church, R J (2012a) Incredible Years Pilot Study: The Repeated Measures Direct Observation Study
(unpublished)
Church, R J (2012b) Parent-Child Interaction Coding Scheme (P-CICS): Coding manual. University of
Canterbury: School of Educational Studies and Human Development
Church, R J, Tyler-Merrick, G and Hayward, H (2006) Identifying antisocial development in New Zealand
children. Phase 1: Development of the screening procedure. Report to the Ministry of Education. University
of Canterbury: School of Education
Cohen, J (1977) Statistical Power Analysis for the Social Sciences. New York: Academic Press
Conduct Problems Prevention Research Group (CPPRG) (1995) Psychometric Properties of the Social
Competence Scale – Teacher and Parent Ratings. (Fast Track Project Technical Report). University Park, PA:
Pennsylvania State University
Corrigan, A (2002) Social Competence Scale – Parent Version, Grade 1/Year 2. (Fast Track Project Technical
Report). Available from the Fast Track Project Web site, http://www.fasttrackproject.org
Dadds, M, Maujean, A and Fraser, J A (2003) Parenting and Conduct Problems in Children: Australian Data
and Psychometric Properties of the Alabama Parenting Questionnaire. Australian Psychologist, Vol 38,
No. 3, 238–241
91
Dadds, M and Powell, M (1991) The relationship of interparental conflict and global marital adjustment
to aggression, anxiety and immaturity in aggressive and non-clinic children. Journal of Abnormal Child
Psychology, 19, 553–561
Durie, M (2004) Race and ethnicity in public policy. Does it work? The Social Policy, Research and Evaluation
Conference, 25 November. Wellington
Durie, M (2006) Māori Education 2026. Post Primary Teachers Association Conference. Wellington
Eyberg, S M (1980) Eyberg Child Behaviour Inventory. Journal of Clinical Psychology 9, 27
Eyberg, S M (2010) Abridged Manual for the Dyadic Parent-Child Interaction Coding System, 3rd edition.
Child Study Laboratory, University of Florida
Eyberg, S M and Ross, A W (1978) Assessment of child behavior problems: The validation of a new
inventory. Journal of Clinical Child & Adolescent Psychology, 7(2), 113-116
Fergusson, D M, Horwood, L J and Ridder, E (2005a) Show me the child at seven: The consequences of
conduct problems in childhood for psychosocial functioning in adulthood. J Child Psychol Psychiatry,
2005, 46(8):837–849
Fergusson, D M, Horwood, L J, Ridder, E M and Grant, H (2005b) Early Start: Evaluation report. Retrieved
7 May 2008 from http://www.earlystart.co.nz/pdf/evalreport.pdf
Fergusson, D M, Stanley, L and Horwood, L J (2009) Preliminary Data on the Efficacy of the Incredible
Years Basic Parent Programme in New Zealand. Australian and New Zealand Journal of Psychiatry 43, 76–79
Fergusson, D M, Boden, J and Hayne, H in P Gluckman (2011) Improving the Transition. Wellington
Feinstein, L and Sabates, R (2006) Predicting adult life outcomes from earlier signals: Identifying those at
risk. Centre for Research on the Wider Benefits of Learning, Institute of Education, University of London
Forehand, R and McMahon, R J (1981) Helping the noncompliant child: A clinician’s guide to parent training.
New York: Guilford Press
Gardner, F, Burton, J and Klimes, I (2006) Randomised Controlled Trials of a Parenting Intervention in the
Voluntary Sector for Reducing Problems in children: Outcomes and Mechanisms of Change. Journal of
Child Psychology and Psychiatry, 47:11, 1123–1132
Goodman, R (1997) The Strengths and Difficulties Questionnaire: A research note. Journal of Child
Psychology, Psychiatry, and Allied Disciplines 38 (5), 581–586
Goodman, R (2001) Psychometric properties of the Strengths Questionnaire (SDQ). Journal of the
American Academy of Child and Adolescent Psychiatry, 40, 1337–1345
Gray, A (2011) Investing in the early years from http://www.nzfamilies.org.nz/web/investing-early-years/
index.html
Gray, D (2013) Incredible Years Pilot Study: Technical Appendix. Ministry of Social Development,
Wellington (unpublished)
Gluckman, P (2011) Improving the Transition. Wellington
Heckman, J J (2006) Skill formation and the Economics of Investing in Disadvantaged Children. Science
30 June 2006, Vol 312, No. 5782, 1900–1902
Hendricks, A K and Balakrishnan, R (2005) Review of parenting programmes. Wellington: The Families
Commission
Hess, M R, Kromrey, J D (2003) EFFECT_CI: A SAS® Macro for Constructing Confidence Intervals Around
Standardized Mean Differences. University of South Florida. Retrieved from http://analytics.ncsu.edu/
92 sesug/2003/SD04-Hess.pdf
Holmes, T H and Rahe, R H (1967) The social readjustment rating scale. Journal of Psychosomatic Research,
Vol 11(2), 1967, 213–218
Horwood, L J, Gray, D S and Fergusson, D M (2011) The Psychometric Properties of the Child Behaviour
Rating Scales used in the Incredible Years Pilot Study (unpublished)
Hutchings, J, Bywater, T, Daley, D, Gardner, F and Whitaker, C, Jones, K, et al (2007) Parenting intervention
in Sure Start services for children at risk of developing conduct disorder: Pragmatic randomised
controlled trial. British Medical Journal, 334(7595), 1–7
Jones, K, Daley, J, Hutchings, T, Bywater, T and Eames, C (2007) Efficacy of the Incredible Years Basic parent
training programme as an early intervention for children with conduct problems and ADHD. Child: Care,
Health and Development, 33, (6) 749–756
Kazdin, A E (2007) Psychosocial treatments for conduct disorder in children and adolescents. New York,
NY: Oxford University Press. (2007). A guide to treatments that work (3rd edition). 71–104, xxv, 758
Karoly, L A, Kilburn, M R and Cannon, J S (2005) Early childhood interventions: Proven results, future
promise. Santa Monica: RAND Corporation. Retrieved 7 May 2008 from http://www.rand.org/pubs/
monographs/2005/RAND_MG341.pdf
Kelleher, C and McGilloway, S (2006) The Incredible Years Basic Parenting Programme: The Clondalkin
Partnership Study in Ireland. Retrieved December 2012 from http://www.incredibleyears.com/library
Larsson, B, Fossum, S, Clifford, G, Drugli, M B, Handegård, B H and Mørch, W-T (2009) Treatment of
oppositional defiant and conduct problems in young Norwegian children: results of a randomized
controlled trial. European Child and Adolescent Psychiatry, 18(1):42–52
Lindsay, G, Strand, S, Cullen, M A, Cullen, S, Band, S, Davis, H, Conlon, G, Barlow, J and Evans, R (2011)
Parenting Early Intervention Programme evaluation: DFE-RR121A. London: DfE
Lovibond, S H and Lovibond, P F (1995) Manual for the Depression Anxiety Stress Scales (2nd edition).
Psychology Foundation, Sydney
MacFarlane, A (2011) Te Ao Māori perspective on understanding conduct problems, a chapter in

Advisory Group on Conduct Problems (2011) Conduct Problems: Effective Programmes for 8–12 Year Olds.
Wellington: Ministry of Social Development
Ministry of Education (2011) Positive Behaviour for Learning Action Plan Update. Wellington, New Zealand
Ministry of Social Development (2007) Interagency plan for Conduct Disorder/Severe Antisocial behaviour
2007–2012. Wellington: Ministry of Social Development
Ministry of Social Development (2009a) Conduct problems: Best practice report. Wellington: Ministry of
Social Development
Ministry of Social Development (2009b) Conduct problems: Effective programmes for 3–7 year olds.
Wellington: Ministry of Social Development
Odgers C L, Caspi, A, Broadbent, J M, Dickson, N, Hancox, R J, Harrington, H (2007) Prediction of

differential adult health burden by conduct problem subtypes in males. Archives of General Psychiatry,
2007, 64:476–84
O’Neill, D (2009) A Cost-Benefit Analysis of Early Childhood Intervention: Evidence from a Randomised
Evaluation of a Parenting Programme. IZA Discussion Paper No. 4518, October
Posthumus, J A, Raaijmakers, M a J, Maassen, G H, van Engeland, H and Matthys, W (2011) Sustained

effects of Incredible Years as a Preventive Intervention in Preschool Children with Conduct Problems.
J Abnorm Child Psychol
93
RAND (2007) Promising Practices Network: Programmes that Work: Incredible Years. Retrieved 25 June 2012
from http://www.promising practices.net/program.asp?programid=134
Rhodes, H (2009). Briefing sheet: Examples of Effective Measuring Tools, Family and Planning Institute
Robinson, E A, Eyberg, S M and Ross, A W (1980) The standardisation of an inventory of child conduct
problem behaviours. Journal of Clinical Child Psychology, 57, 628–635
Romeo, R, Knapp, M and Scott, S (2006) Economic cost of severe antisocial behaviour in children – who
pays it? British Journal of Psychiatry (2006) 188, 547–553
Scott, S, Knapp, M, Henderson, J and Maughan, B (2001) Multicentre controlled trial of parenting groups
for childhood antisocial behaviour in clinical practice. British Medical Journal, 323, 194–203
Scott, S (2007) Conduct disorders in children. British Medical Journal, March, 334:646
Shelton, K K, Frick, P J and Wooton, J (1996) Assessment of parenting practices in families of elementary
school-age children. Journal of Clinical Child Psychology, 25, 317–329
Smith, L T (1999) Decolonizing methodologies: Research and indigenous peoples. Dunedin: University of
Otago Press
Straus, M, Hamby, S, Boney-McCoy, S and Sugarman, D (1996) The revised Conflict Tactics Scales (CTS2):
Development and preliminary psychometric data. Journal of Family Issues, 1998, 17(3), 283–316
Straus, M A, Hamby, S L, Finkelhor, D, Moore, D W and Runyan, D (1998) Identification of child

maltreatment with Parent-Child Conflict Tactics Scales: Development and psychometric data for a
national sample of American parents. Child Abuse and Neglect, 22(4), 249–270
Webster-Stratton, C (1994) Advancing videotape parent training: A comparison study. Journal of

Consulting and Clinical Psychology, 62(3), 583–593
Webster-Stratton, C (2007) Affirming diversity: multi-cultural collaboration to deliver the Incredible Years
Parent Programs. Retrieved 8 June 2012 from http://www.incredibleyears.com/library
Webster-Stratton, C and Hammond, M (1997) Treating children with early-onset conduct problems: a
comparison of child and parent training interventions. Journal of Consulting and Clinical Psychology, 65(1),
93–109. Supplementary analysis retrieved December 2012 from http://www.incredibleyears.com/library
Webster-Stratton, C and Reid, M J (2010) The Incredible Years Parents, Teachers and Children Training
Series: A Multifaceted Treatment Approach for Young Children with Conduct Disorders, book chapter in
J Weisz and A Kazdin (Eds) Evidence-based Psychotherapies for Children and Adolescents, 2nd edition. New
York: Guilford Publications
Webster-Stratton, C, Reid, M J and Hammond, M (2004) Treating Children with Early-Onset conduct
Problems: Intervention Outcomes for Parent, Child, and Teacher Training. Journal of Clinical Child and
Adolescent Psychology, Vol 33, No. 1, 105–124
Webster-Stratton, C (2012) Presentation given at The Incredible Years Parent Management Training
Conference. Wellington, New Zealand
Werry Centre (2008) Māori Experiences of Delivering the Incredible Years Parenting Programmes (Reflections).
Auckland: Werry Centre
Werry Centre (2010) Incredible Years Ngā Tau Miharo Hui. Auckland: Werry Centre
Werry Centre (2012) Ngā Tau Mīharo ō Aotearoa. Auckland: Werry Centre
World Health Organisation (1993) Composite International Diagnostic Interview (CIDI). World Health
Organisation, Geneva, Switzerland
94

Incredible Years Parent

Uploaded by

Copyright:

Available Formats

Incredible Years Parent

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Incredible Years Parent

Uploaded by

Copyright:

Available Formats

What is the purpose of the report?

What is the purpose of the report?

What parenting programmes are discussed in the report?

What parenting programmes are discussed in the report?

Incredible

Fiona Sturrock and Dorian Gray

1.1 Background to the

1 See Appendix 1 for a list of the members of the EAG.

2.1 An overview of the

Figure 1. Incredible Years Parenting

(sourced from www.incredibleyears.com)

Gardner et al (2006) 76 2 to 9 England ECBI d=.55

ECBI = Eyberg Child Behaviour Inventory, Intensity Score

2 Effect sizes are explained in Chapter 3, Section 3.3 and in Appendix 3.

Gardner et al (2006) 76 2 to 9 England RCT

Hutchings et al (2007) 153 3 to 5 Wales RCT

Larsson et al (2009) 127 4 to 8 Norway RCT

AOPS = Arnold-O’Leary Parenting Scale Total

On the recommendation of the AGCP, the

3.1 Research design

3 Analyses of outcomes by phase are reported in Chapter 6, Section 6.3.

Phase 1: Jan to June 2011 Phase 2: JULY to DEC 2011

Main Study Single Case Main Study Single Case

75 + parents 25 parents 75 + parents 8 parents

N Parenting practices, dealing with misbehaviour, child management (partner)

O Partners and partner relationships

P Primary carer’s health/depression

Q Stressful life events

S Expectations of/satisfaction with Incredible Years

Teacher Questionnaires Single Case Studies

4 Analyses of outcomes by phase are reported in Chapter 6, Section 6.3.

Details of the statistical models used

4.1 The referral process

5 Data re-entry checks produced an infinitesimal

Pre-test/post-test comparisons:  Social Competence: This dimension was based

6 See Appendix 2 for descriptions of source instruments.

Baseline Mid-course Post-course Trend Post-course effect

In-course behavioural assessments

Baseline Follow-up Trend Follow-up effect size

5.2 Parenting practices and

Pre-test/post-test comparisons: disorder. It loads onto five subscales: parental

8 See Appendix 2 for descriptions of source instruments.

Parenting measures1 Baseline Mid-course Post-course Trend Post-course effect

Dealing with misbehaviour3

Parenting measures1 Baseline Follow-up Trend Follow-up effect size

Dealing with misbehaviour3

9 See Appendix 2 for descriptions of source instruments.

Relationship types Baseline Mid-course Post-course Trend Post-course effect

Other parent2 and child

Relationship types Baseline Follow-up Trend Follow-up effect size

Other parent2 and child

Baseline Post-course Trend Post-course effect size

Sample sizes are N=150 baseline, 139 post-course.

Baseline Follow-up Trend Follow-up effect size

Sample sizes are N=150 baseline, 124 follow-up.

Non-school Difference between Test1 for no difference

Non-school Behaviour change test

The previous chapter presented the findings

6.1 Outcomes for health and

Baseline Mid-course Post-course Trend Post-course effect size

Pre-test/post-test comparisons: Social Competence: This dimension was based