See

discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/232543280

Effectiveness of Comprehensive Professional

Development for Teachers of At-Risk
Preschoolers

Article in Journal of Educational Psychology · May 2009

DOI: 10.1037/a0013842

CITATIONS READS

118 660

4 authors, including:

Susan H Landry Jason L Anthony

University of Texas Health Science Center at … University of Texas Health Science Center at …
166 PUBLICATIONS 5,912 CITATIONS 55 PUBLICATIONS 3,122 CITATIONS

SEE PROFILE SEE PROFILE

Paul R Swank
University of Texas Health Science Center at …
263 PUBLICATIONS 7,079 CITATIONS

SEE PROFILE

All content following this page was uploaded by Jason L Anthony on 24 July 2014.

The user has requested enhancement of the downloaded file. All in-text references underlined in blue
are linked to publications on ResearchGate, letting you access and read them immediately.
Journal of Educational Psychology © 2009 American Psychological Association
2009, Vol. 101, No. 2, 448 – 465 0022-0663/09/$12.00 DOI: 10.1037/a0013842

Effectiveness of Comprehensive Professional Development for

Teachers of At-Risk Preschoolers
Susan H. Landry, Jason L. Anthony, Paul R. Swank, and Pauline Monseque-Bailey
Children’s Learning Institute at University of Texas Health Science Center at Houston

This study compared effectiveness of “business as usual” to that of 4 professional development (PD)
programs that targeted teachers of at-risk preschool children. A 2 ⫻ 2 design was used to cross mentoring
and progress monitoring conditions among the 4 PD programs. Specifically, some teachers received both
in-classroom mentoring and detailed, instructionally linked feedback concerning children’s progress in
language and literacy. Some teachers received no mentoring but did receive the detailed, instructionally
linked feedback concerning children’s progress. Some teachers received in-classroom mentoring but only
limited feedback on children’s progress, which was not linked to curricular activities. Finally, some
teachers received no mentoring and only limited feedback concerning children’s progress. All 4 PD
conditions included the same year-long, facilitated online course that emphasized language and literacy
instruction, practice of learned material in one’s classroom, and participation in online message boards
with fellow teachers. Across 4 states, 158 schools (N ⫽ 262 classrooms) were randomly assigned to 1
of the 4 PD conditions or business as usual. The condition that included online coursework combined
with mentoring and detailed, instructionally linked feedback yielded the greatest improvements in
teaching behavior and children’s school readiness.

Keywords: professional development, preschool, mentoring, curriculum-based measurement, technology

Understanding how to provide children with an early foundation (e.g., DiPietro, 2000; Landry, Smith, Swank, Assel, & Vellet,
in school readiness skills is critical given that many states estimate 2001; Neville et al., 1998). Children from lower SES backgrounds
that half of their students arrive at kindergarten already far behind are at the highest risk for not receiving the type of experiences that
of where they need to be to have a good chance at succeeding in are most likely to promote school readiness. For example, because
school (“Highlighting NAEP 2003,” 2003; Zill & West, 2001). of life stresses, psychological distress, and poor parental role
The most recent results of the National Assessment for Educa- models, economically disadvantaged parents often use parenting
tional Progress indicated that only 31% of fourth-grade children in approaches that inadequately support child development (Aber,
the United States performed above the proficient level in reading Jones, & Raver, 2007; McLoyd, 1998). More specifically, children
and only 32% performed above the proficient level in math in economically disadvantaged families experience fewer in-
(“Highlighting NAEP 2003,” 2003). In both domains, children stances of rich language input (Hart & Risley, 1995), fewer op-
from lower socioeconomic status (SES) backgrounds (i.e., children portunities to interact with children’s literature (Neuman, 1996),
eligible for free- or reduced-price lunch) performed significantly
and parenting styles that are less responsive to children’s interests
worse than children from higher SES backgrounds. Experts now
(Landry et al., 2001). Children in economically disadvantaged
generally agree that these disparities in children’s skill levels begin
families also experience poorer health care and nutrition than their
before elementary school, and they persist throughout children’s
economically advantaged peers. Thus, compounding risk factors
formal schooling (National Center for Education Statistics, 2001a,
2001b; Snow, Burns, & Griffin, 1998). place children from economically disadvantaged backgrounds at
Recent evidence from basic science research and longitudinal increased risk for poor academic achievement, school dropout,
and intervention studies has reaffirmed that a child’s experience social maladjustment, criminal behavior, and physical and mental
during the early years directly affects brain development in ways illness.
that affect later learning, behavior, and physical and mental health Quality early childhood education is a primary means by which
children from low-income backgrounds can start kindergarten with
the skills necessary to succeed (Bowman et al., 2001; Snow et al.,
Sandy H. Landry, Jason L. Anthony, Paul R. Swank, and Pauline 1998). Evidence has shown that when children are supported by
Monseque-Bailey, Children’s Learning Institute at University of Texas teachers with specialized techniques who are sensitive to emerging
Health Science Center at Houston. developmental skills, they generally achieve at higher levels (e.g.
This study was supported by Interagency Education Research Institute Howes, 1997). In fact, children from impoverished backgrounds
Grant HD25128, funded by the Institute of Educational Sciences, U.S.
demonstrate average levels of development at the time of entry
Department of Education.
Correspondence concerning this article should be addressed to Susan H.
into kindergarten if they receive such support (Landry et al., 2001).
Landry, Children’s Learning Institute, Department of Pediatrics, University Because families at poverty levels of income are more likely to
of Texas Health Science Center at Houston, 7000 Fannin, Suite 2300, rely on center-based early childhood programs that accept federal
Houston, TX 77030. E-mail: Susan.Landry@uth.tmc.edu subsidy (Phillips, Voran, Kisker, Howes, & Whitebook, 1994), it is

448
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 449

important that these programs provide learning environments that supporting that discovery. This explicit belief system often is at
optimally promote school readiness. odds with professional development that encourages teachers to
Although there is growing consensus that high-quality preschool provide explicit information about vocabulary, number concepts,
experiences can lay a strong foundation of school readiness even and letters in a more intentional approach (Bereiter, 1972). Thus,
among economically disadvantaged children, there is a serious one challenge for professional development programs for early
mismatch between the preparation of most early childhood educa- childhood educators is how to encourage a properly sequenced and
tors and the preparation needed to optimize classroom practices. balanced integration of teachers’ efforts at supporting children’s
Educating young children is a complex and challenging endeavor play and self-discovery while also providing planful and purpose-
because teachers are asked to promote high levels of child achieve- ful instructional activities.
ment, have a deep understanding of curriculum content, pay atten- Recent theoretical frameworks for teacher professional devel-
tion to young children’s capacity to learn, understand the optimal opment describe the importance of acknowledging teachers’ phi-
conditions under which they learn, and have an ability to engage in losophies in an effort to help them understand how new informa-
reflection on their own teaching practices (Bowman et al., 2001). tion can fit within existing attitudes and beliefs (Bransford, Brown,
Promoting optimal learning for young children is thought to re- & Cocking, 2000; Kennedy, 1997; Richardson, Anderson, Tid-
quire some unique instructional strategies. Theories of early learn- well, & Lloyd, 1991; Speck, 1996). Intellectual engagement in the
ing suggest that early childhood educators need to be responsive subject matter through opportunities to understand the theory and
and sensitive to individual differences in children’s abilities, home rationale for new instructional practices is of critical importance.
environments, and cultural characteristics (Bowman et al., 2001). Such frameworks additionally indicate that adults will learn most
Although early childhood educators may not always have the effectively when (a) the learning is situated in authentic contexts
formal educational background to prepare them for the classroom, (e.g., demonstrating techniques with teachers in classroom set-
effective professional development has been shown to improve the tings, which is perceived as realistic and relevant to day-to-day
quality of early childhood programs (Howes, Phillips, & White- activities), (b) there are opportunities to do collaborative problem
book, 1992; Kontos, Howes, & Galinsky, 1997). Therefore, pro- solving and practice specific skills, and (c) learning experiences
fessional development may serve as a buffer to inadequate teacher are extended over time rather than in one workshop (Bransford et
preparation, which is commonplace in early childhood education. al., 2000; Elmore, 2002; Learning First Alliance, 2000; Putnam &
In the past decade, knowledge has burgeoned about the key foun- Borko, 2000; Speck, 1986; Sullivan, 1999).
dational skills children need to enter kindergarten ready to learn Professional development may be further enhanced if teachers
academic skills. This includes an understanding and use of vocab- receive ongoing, in-classroom coaching, or mentoring (Spodek,
ulary, complex oral language, and early writing (National Institute 1996). Mentoring is thought to provide teachers with opportunities
for Literacy, 2007). Preschool children’s phonological processing to try new approaches with guided support and a knowledge
skills, such as phonological awareness, phonological short-term resource without concerns regarding the mentor’s having a super-
memory, and the efficient use of phonological representations of visory role (Eisenhower National Clearinghouse for Mathematics
words, are also important precursors to acquiring literacy & Science Education, 1998; International Reading Association &
(Anthony et al., 2006; Anthony, Williams, McDonald, & Francis, National Association for the Education of Young Children, 1998).
2007). A child’s knowledge of letters, including naming letters and The availability of a network of professional support, which could
knowing that they are associated with sounds, is also a critical include mentoring, has been shown to improve teacher– child
foundation skill. In a recent meta-analysis, these foundational interactions (Corsini & Caruso, 1989), to decrease feelings of
skills were unique predictors of reading achievement in early isolation (Hayes, Palmer, & Zaslow, 1990), and to reduce high
elementary grades (National Institute for Literacy, 2007). It has rates of teacher turnover (Landry, Swank, Smith, Assel, & Gun-
been documented that teachers trained in instructional strategies newig, 2006). With “in-house” trainers who tailor the program to
that expose children to experiences with emergent literacy skills teachers’ needs, higher levels of adult– child interaction and more
are more likely to have students who show cognitive gains that positive child developmental outcomes are observed (Epstein,
carry into kindergarten (Whitehurst & Lonigan, 1998; Zevenber- 1993). An advantage of mentoring is its ability to individualize
gen et al., 1997). professional development to the needs of the learner, which may
be particularly important for early childhood teachers who vary in
Theoretical Frameworks for Proposed education and training. Although the importance of mentoring is
Professional Development becoming more recognized among early childhood educators, ex-
perimental studies of the mentor–teacher process and outcomes
A first step toward the goal of school readiness for all children remain limited. Of note is a lack of studies with adequate meth-
is to examine whether teachers can be provided with professional odology, as most are quasi-experimental or correlational with no
development that results in teachers providing early learning ex- control group (Kontos, Machida, Griffin, & Read, 1992).
periences that foster school readiness. To accomplish this goal, it Assessment of students by their teachers for the purpose of
is often necessary for teachers to change their beliefs about appro- informing instruction is becoming a more common practice in
priate content and pedagogy and to change their beliefs about their early childhood education, especially for early language and liter-
roles and responsibilities in supporting children’s learning in the acy (McConnel, 2000; Phaneuf & Silberglitt, 2003). Teacher-
classroom. One belief that can interfere with teachers making use administered assessments of children’s academic progress, also
of professional development is the longstanding belief that chil- known as curriculum-based measures (CBMs), are usually very
dren need to construct their own knowledge through self-directed brief tests, may use timed tests, and are administered repeatedly
discovery and that the teacher’s responsibility and role is one of across the school year. CBMs help teachers focus their instruction
450 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

on important learning outcomes. Children’s scores on CBMs can elements of high-quality professional development outlined above.
be used not only to gauge children’s response to instruction but Teachers in the four professional development conditions were
also to group children according to their learning needs (Wesson, also provided with the same supplemental curricula and associated
1992), which has been shown to maximize instructional impact materials. Teachers in the four professional development condi-
(e.g., Bowman et al., 2001; Leeper & Witherspoon, 1984). Re- tions also administered the same CBMs. The four professional
search on teachers’ use of CBMs to inform instruction is limited development conditions differed in whether they included regu-
(Fuchs, 2004; Madelain & Wheldall, 2004), and there has been a larly scheduled in-classroom mentoring and whether they included
call to examine the extent to which technology that directly links detailed feedback regarding the progress monitoring data that
CBM results to grouping of children and to selection of instruc- included recommendations for grouping children and for instruc-
tional activities can improve children’s learning outcomes (Fuchs, tional activities included in the supplemental curriculum. The
2004). study’s design, which crossed mentoring and feedback conditions
among the four professional development conditions and also
Bringing Professional Development Programs to Scale included a business-as-usual control group, allowed us to test for
the main effects and interaction effects of mentoring and detailed
Literature on scaling educational programs is not well devel- feedback and to compare each of the professional development
oped (National Academy of Education, 1999), but successful mod- groups with the control group. Before implementing the main
els are available as guides (Culp & Honey, 2000; Slavin, Madden, study, all core components and condition-specific components of
Dolan, & Wasik, 1996). The current intervention includes a highly the professional development programs were field tested locally as
specified framework of what is needed in classrooms without part of a feasibility pilot (described below). We hypothesized that
scripting the program and flexibility that allows administrators’ the main evaluation study would demonstrate effectiveness of the
and teachers’ input into implementation. The need to ensure a core professional development program and each of the supple-
balance between faithful implementation of the key program com- mental components (i.e., in-classroom mentoring and instruction-
ponents and local input was a goal (McLaughlin, 1990). This was ally linked feedback concerning children’s academic progress).
accomplished, in part, through the development of partnerships
with all stakeholders because gaining the support of local stake-
holders is a critical process for bringing projects to scale success- Method
fully (Culp & Honey, 2000).
Consistent with successful scaling research, the current profes- Feasibility Pilot Project
sional development programs included a highly systematic ap-
proach in which training goals were carefully sequenced and A pilot project was conducted locally with 25 teachers to ex-
spread out over time to support transfer of goals into the classroom amine the feasibility of small-group, facilitated, online profes-
(Center for Children & Technology, 2000). Also, the programs sional development with teachers in early childhood education
were built from solid theoretical and research bases, but training settings. All small-group training sessions were attended by the
was grounded in practical application. For example, professional project manager to identify areas in need of change and the amount
development first focused on how to organize the classroom for of specificity that needed to be included in the facilitator guide.
effective teaching and behavior management. Subsequently, pro- The goals of the pilot study were to determine feasibility of (a)
fessional development focused on how teachers could promote access to computer labs for small-group training sessions, (b)
important school readiness skills. school programs’ providing time for teachers to attend the classes,
(c) teachers’ willingness to attend classes, (d) teachers’ comfort
with Internet-based training, (e) teachers’ willingness to post their
Project Overview
efforts to try activities in their classrooms and to engage in online
The main objective of this study was to demonstrate that teach- discussion outside of class, and (f) implementation of trained
ers across the three primary types of early childhood education activities in classroom settings.
programs serving low-income children (i.e., subsidized childcare, The pilot study highlighted a number of aspects of the program
Head Start, and public school prekindergarten) could be facilitated that needed to be changed or refined. For one, it was not possible
to use effective instructional practices that, in turn, promote chil- for the early childhood educators to be relieved of their teaching
dren’s development of language and emergent literacy. Partici- responsibilities during work days. So, the small-group professional
pants were teachers from all three types of early childhood edu- development sessions had to be held after hours. Therefore, it was
cation programs that serve economically disadvantaged families. deemed necessary to compensate teachers for their attendance with
This multisite study tested the effectiveness of four overlapping a stipend or college credit. It was also discovered that more
professional development conditions that were based on scientif- detailed training materials were needed to ensure consistency of
ically based research and modern theoretical frameworks of pro- course content across small-group facilitators. Other necessary
fessional development. To determine the effectiveness of the pro- refinements included greater specificity of frequency of specific
fessional development programs, schools were randomized into professional development activities, timelines for progress mon-
one of five conditions, control (i.e., business as usual) or one of the itoring, and teacher availability for mentoring to ensure the
professional development programs. The four professional devel- project’s fidelity. Finally, the pilot study brought to light that
opment conditions included a common set of core components. facilitators should have teaching experience in the early child-
Specifically, all professional development conditions involved the hood field and that they should not have competing, nonstudy
same small-group online training that included all the essential responsibilities.
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 451

Research Design of Main Study Participants

After conclusion of the feasibility pilot, we conducted a ran- This multisite study took place in Ohio, Maryland, Florida, and
domized experimental study to evaluate the effectiveness of four Texas. Classrooms in Ohio, Maryland, and Florida participated in
overlapping professional development programs relative to a the project during the 2004 –2005 school year. Classrooms in
business-as-usual control group. The programs overlapped in that Texas participated during the 2005–2006 school year. Across the
teachers in all four programs received the same small-group, four sites and 2 years, participants included 262 early childhood
facilitated online training (see Online professional development educators who were housed in 158 schools. These teachers were
section, below). They were also provided with the same Center for distributed across the sites as follows: Ohio, n ⫽ 65; Maryland,
Improving the Readiness of Children for Learning and Education n ⫽ 59; Florida, n ⫽ 65; and Texas, n ⫽ 73. Participating schools
(CIRCLE) Preschool Early Language and Literacy Training man- primarily served children and families from low-SES backgrounds.
ual and supplemental CIRCLE curricular materials (Center for Demographic characteristics of participating early childhood edu-
Improving the Readiness of Children for Learning and Education, cators and classrooms are summarized in Table 1.
To be eligible for participation, early childhood educators were
2006). Teachers in the four professional development conditions
required to follow a published curriculum, but they were not
also administered the same CBMs. The research design addition-
required to follow any particular published curriculum. Curricula
ally allowed for evaluation of the value added by regularly sched-
represented in participating classrooms included Creative Curric-
uled in-classroom mentoring and by immediate, instructionally
ulum with training in Ohio, DLM Express with training in Mary-
linked feedback from progress monitoring. Finally, the 2 ⫻ 2
land, Creative Curriculum online and some home-grown curricu-
research design permitted examination of the interaction between
lum activities in Florida, and Scholastic’s Building Language and
mentoring and instructionally linked feedback. Specifically, par- Literacy and DLM Express in Texas. In general, the curricula used
ticipating schools were randomly assigned to one of five experi- in control and intervention classrooms incorporated a scope and
mental conditions: (a) mentored with personal digital assistant sequence for language and literacy learning activities that was
(PDA)-based progress monitoring (detailed feedback), (b) non- implemented in a purposeful but playful way. The professional
mentored with PDA-based progress monitoring, (c) mentored with development model was created so that it could be used with a
paper-and-pencil progress monitoring (limited feedback), (d) non- broad range of curricula, as participating programs in this inter-
mentored with paper-and-pencil progress monitoring, or (e) con- vention were allowed to choose their classroom curriculum.
trol. Observation of teachers’ behavior during classroom instruc- Parents from participating classrooms were sent information
tion and testing of children’s literacy and language outcomes were letters and consent forms. From among those who returned signed
conducted in the beginning of the school year and again at the end consent forms, up to eight children were randomly selected from
of the school year. Therefore, changes in teaching behaviors and each classroom to participate in the assessments that informed the
children’s school readiness could be attributed to experimental evaluation. Across the four sites, selected children ranged in age
condition and associated components of the professional develop- from 3 to 5 at the time of pretesting (M ⫽ 4.3, SD ⫽ 0.5). The
ment programs. sample of 1,786 children was 50% boys. Approximately 42% of

Table 1
Classroom and Teacher Characteristics by Study Site

Characteristic Florida (n ⫽ 65) Maryland (n ⫽ 59) Ohio (n ⫽ 65) Texas (n ⫽ 73) p⬍

Classroom type (%) .0001

Public school 0 74 0 38
Head Start 27 26 100 37
Child care 73 0 0 25
Language of instruction (%) .0001
English 40 96 100 85
Spanish 60 4 0 15
Length of day (%) .0001
Full day 88 96 35 77
Half day 12 4 65 23
Teacher education (%) .0001
High school/CDA 97 0 26 23
2-year college 3 0 40 30
4 or more years college 0 100 34 47
Teacher ethnicity (%) .0001
African American 19 53 37 6
Caucasian 6 42 60 22
Hispanic 75 5 3 72
Pre-kindergarten teaching experience .13
Mean years (SD) 7.31 (6.6) 6.00 (6.2) 8.55 (6.0) 8.15 (6.7)

Note. CDA ⫽ Child development associate.

452 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

the sample were Hispanic, 34% were African American, 17% were al., 2006). Given the challenge of scaling up the professional
Caucasian, 2% were Asian, and 5% were other. English was development program across multiple states, the multiple-day
spoken in the homes of 73% of participating children. Spanish was workshops were adapted to be appropriate for an online applica-
spoken in the homes of 32% of participating children, and a tion, called eCIRCLE. The nine courses covered the following
language other than English or Spanish was spoken in the homes topics: classroom management, best practices and responsive
of 5% of the sample. (Percentages are greater than 100% because teaching, setting the stage for children’s talk, reading aloud, pho-
some families spoke multiple languages in their home.) nological awareness, letter knowledge, mathematics, written ex-
pression, and language development. Consistent with theoretical
Procedures Related to Implementation of Professional frameworks, the online course involved (a) small-group interactive
Development Programs learning facilitated by a trainer, (b) extensive videotaped modeling
of content-related activities and expert commentaries that allowed
Project management and oversight. Pauline Monseque- teachers to see examples in realistic contexts that were relevant to
Bailey, as project manager, supervised eight facilitators. There their classroom experiences, (c) interactive engagement with on-
were two facilitators in each state. Facilitators were responsible for line coursework and online assessments of knowledge, (d) oppor-
both facilitating the online group training and mentoring teachers tunity for independent review of all course content, (e) opportunity
who were assigned to mentoring conditions. Each facilitator had to practice specific skills within the small group (e.g., role playing
individual, weekly telephone conference calls with the project and development of lesson plans), (f) practice of specific instruc-
manager to review the project status at their site, ask questions, and tional activities in one’s own classroom, (g) teacher postings of
address any concerns. In addition, a weekly conference call that classroom experiences, and (h) trainer review of postings and
included the facilitators across the sites and the project manager feedback. The eCIRCLE online professional development program
was conducted to encourage collaboration, build a facilitator net- was developed to provide teachers with the appropriate balance
work, and discuss the online platform. The project manager made between implementing developmentally appropriate activities that
several scheduled visits to each site to visit classrooms and observe are teacher directed and designed to foster development of specific
online professional development sessions, called eCIRCLE (de- skills and implementing activities that are child directed and de-
scribed in the Online professional development section). Addi-
signed to allow children to enhance mastery and breadth of skills
tional visits were conducted to address issues when they occurred.
through active exploration. By acknowledging and working within
Susan H. Landry, as principal investigator, also visited each site at
teachers’ existing philosophies of instructional practices, we ex-
least once. The research team at the University of Texas Health
pected that the program would be more effective in facilitating a
Science Center, along with researchers from the online course
high fidelity of implementation. Teachers in all four professional
development team, held half-day meetings three times each year to
development conditions attended small-group eCIRCLE sessions
review the status of the project.
(n ⫽ about16 teachers) that were facilitated by trained and expe-
Facilitators. Facilitators were required to have a background
rienced early childhood educators (see Facilitators, above). Teach-
in early childhood education and to have an appreciation for the
ers in the four professional development conditions attended 2-hr
research process. At the beginning of the school year, facilitators
eCIRCLE classes twice per month for nearly the entire school
also attended 4 days of training at the University of Texas Health
Science Center. This training included CIRCLE’s 2-day workshop, year.
which informed the online professional development course (see Mentoring. Mentoring is a complex and multidimensional
Online professional development section, below). The 2-day work- process of guiding teaching practices and influencing and support-
shop covered topics of classroom management; language-rich ing teachers. The goal of this professional development was to
classroom environments; literacy-rich classroom environments; provide opportunities in an interconnected way that helped teach-
child development in areas of oral language, phonological aware- ers work through roadblocks, discuss instructional strategies, and
ness, letter knowledge, reading, and writing; and instructional discuss how instruction changes and develops over time.
strategies and activities that promote preschool children’s lan- During initial classroom visits, the facilitators assessed the in-
guage and literacy. Facilitators were introduced to the online struction being used in the classroom and evaluated the classroom
professional development and the facilitator guide. The training environment using the Teacher Behavior Rating Scale (TBRS;
also covered effective mentoring strategies, how to administer the Landry, Crawford, Gunnewig, & Swank, 2001). Subsequent men-
CBMs, and how to use results from the CBMs to guide instruction. toring visits consisted of helping teachers with classroom arrange-
The facilitators had two roles: One, to guide small groups of ment, instructional lessons, and instructional planning. Facilitators
teachers through the online professional development course, and also provided teachers with reflective follow-up designed to help
two, to mentor those teachers who were additionally assigned to the facilitator and teacher discuss positive instructional pieces
receive mentoring. Each facilitator was responsible for around 30 and those that needed attention. Written feedback was provided
teachers who attended the small-group online sessions twice per using the CIRCLE Glows and Grows Mentoring Tool (Tuyman,
month. Approximately half of these teachers also received 4 hr of Aston, & Gunnewig, 2001). Facilitators mentored teachers twice a
in-classroom mentoring each month. Separate training sessions month for 2 hr each visit.
were conducted for mentored teachers versus nonmentored teach- Across the year, facilitators videotaped teachers during three
ers whenever possible. activities: book reading, center time, and small-group instruction.
Online professional development. The genesis of the online The facilitators rated each videotape and provided feedback re-
professional development program used in this study was the garding how the teacher could improve his or her instructional
face-to-face training workshops developed by CIRCLE (Landry et skills. Data gathered from the videotaping session and a copy of
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 453

the video were sent to University of Texas Health Science Center. Measures Used for Program Evaluation
The project manager, Pauline Monseque-Bailey, reviewed these
Teacher outcomes. Seventy-five teachers evenly distributed
materials and provided written feedback to each of the facilitators.
across the four sites and five experimental conditions were ran-
This was an effective strategy for supporting fidelity of implemen-
domly selected for observation. Each of the 75 teachers was
tation in a large, scale-up study. Through repeated use of the
observed for about 120 min early in the school year, before any
observation tool, facilitators were able to see how teachers devel-
professional development. Each was observed again for about 120
oped an awareness of including quality literacy instruction over the
min at the end of the school year, after completion of the profes-
course of a school year.
sional development program. Observations typically occurred in
Progress monitoring. The CIRCLE–Phonological Awareness, one session with the exception of a few instances when the daily
Language, and Literacy Screener (C-PALLS) helps early child- schedule was disrupted (e.g., field trips or class party). Observers
hood educators monitor how well their pupils are learning some of were unaware of classrooms’ study condition.
the major school readiness skills necessary for literacy acquisition. Observers used the CIRCLE TBRS (Landry et al., 2000) to rate
C-PALLS was administered by classroom teachers in all four of the quality and frequency of occurrence of specific teaching be-
the professional development programs during the fall, winter, and haviors. The TBRS contains multiple subscales with a total of 50
spring. Teachers in all four professional development programs items measuring multiple indicators of teaching behaviors. For the
were encouraged to use C-PALLS results to help determine class- following subscales, quantity and quality summary scores are
room grouping strategies, guide lesson planning, develop appro- obtained: Oral Language, Book Reading, Print and Letter Knowl-
priate center activities, and inform parent–teacher conferences. edge, Written Expression, Phonological Awareness, and Total
C-PALLS includes CBMs for phonological awareness, vocab- Composite, combining all individual subscales. For the Lesson
ulary, and letter knowledge. The Phonological Awareness screener Plan and Portfolio subscales, only a quantity score is obtained, and
evaluates a range of phonological awareness skills using short (six- for the Centers subscale, only a quality score is obtained. Quality
to nine-item) untimed tasks. Skills evaluated include auditory scores are based on a 3-point rating scale (i.e., low, medium, and
discrimination, rhyme recognition, rhyme production, alliteration high). Quantity scores are based on a 3-point rating scale, on which
detection, sentence segmentation, syllable clapping, and onset- a score of 1 indicates 0 –1 activities observed, 2 indicates 2–3
rime blending. For the separate Vocabulary and Letter Naming activities observed, and 3 indicates 4 or more activities observed.
CBMs, children are asked to name as many pictures or letters as An exception is the Portfolio quantity score, which involves a
possible within 1 min. Scores from C-PALLS demonstrate good 5-point rating system. Specifically, portfolios of five children from
reliability and validity, in relation to their brevity (Swank et al., a classroom are randomly selected for review. The highest score
2006). For example, 3-month test–retest correlations range from reflects that work samples across three skill areas were included in
.79 to .86, internal consistency of the phonological awareness at least four of the five portfolios reviewed. Generalizability co-
screener is .93, and correlations of scores from C-PALLS CBMs efficients, which index interrater reliability for the various sub-
with scores from standardized measures of like constructs range scales, are high, ranging from .80 to .98 (Mitchell, 1979). Internal
from .40 to .80. Moreover, scores from C-PALLS CBMs have consistency also is high (.96). Although significant correlations
been shown to be sensitive to individual differences in rates of between subscales are found, these are not so high that the infor-
growth and classroom differences in rates of growth. mation is redundant (Assel, Landry, & Swank, 2007).
For two professional development conditions, the PDA version Child outcomes. Examiners spent time talking with each child
in a playful manner to help him or her become comfortable with
of C-PALLS was used by teachers, whereas in the other two
the assessment process. Children were usually assessed across
professional development conditions the paper-and-pencil version
multiple visits to avoid fatigue. Children took frequent breaks for
of C-PALLS was used. The paper-and-pencil version requires that
toileting, snacks, and/or to allow for movement. Children were
the teacher (a) tabulate scores for each child; (b) develop and
assessed in the language of their instruction if instruction was
maintain a tracking system to monitor individual children’s change
predominately delivered in English or Spanish. For classrooms
in scores over time; (c) use a high, medium, or low categorization
with bilingual instruction, teachers were interviewed by an exam-
of children’s skills as outlined in the manual to cluster children for iner who used a standard set of questions to ask about each child
small-group instruction; and (d) choose appropriate activities for selected for testing. From the structured interview data, a deter-
small-group instruction from the CIRCLE training manual. Within mination was made to test individual children in English or Span-
a week of each progress monitoring wave, teachers who used the ish. Child outcomes included scores on tests of vocabulary, com-
paper-and-pencil version were provided a spreadsheet of chil- posite language, phonological awareness, letter knowledge, and
dren’s scores for that particular wave. In contrast, the PDA version print awareness.
of C-PALLS completes all four of the steps above as part of the The Expressive One-Word Picture Vocabulary Test (EOWPVT;
software programming. Content experts had predetermined which Brownell, 2000) was used to measure children’s expressive vo-
activities from the CIRCLE manual were appropriate for children cabulary. There are English and Spanish versions that are appro-
of different skill levels within each school readiness domain. Thus, priate for examinees ages 2 to 18. Both versions present examinees
the amount of effort and skill required by teachers in the paper- with stimulus pages containing a colored picture that depicts an
and-pencil conditions was considerably more than that required by action, object, category, or concept. Examinees are asked to label
teachers in the PDA conditions. Of course, the PDA version also each picture. Internal consistency values for 2- to 5-year-olds range
systematically guided teachers through the assessment procedures, from .93 to .98, depending on index of internal consistency. Test–
assuring standardized administration. retest reliabilities over a 20-day interval are .88 and .89 for 2- to
454 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

4-year-olds and 4- to 7-year-olds, respectively. A number of studies for the lack of independence among observations, given that chil-
have shown concurrent correlations of EOWPVT scores with scores dren were nested in classrooms. In analyses of both teacher and
from other language measures that range from .64 to .87 and concur- child outcomes, a priori contrasts had been specified. A priori tests
rent correlations of EOWPVT scores with scores from other vocab- included (a) control group versus professional development groups
ulary measures that range from .67 to .90 (Brownell, 2000). collectively, (b) control group versus each of the four professional
The Auditory Comprehension subtests of the English and Span- development groups individually, (c) two mentored groups com-
ish versions of the Preschool Language Scale—fourth edition bined versus two nonmentored groups combined, (d) two PDA
(PLS– 4; Zimmerman, Steiner, & Pond, 2002) were used to assess groups combined versus two paper-and-pencil groups combined,
complex receptive language. These measures are highly sensitive and finally (e) a test of the interaction between the mentoring and
to changes in young children’s language development in response PDA conditions. All the contrasts except (b) are independent.
to teacher enhancement projects (Landry et al., 2006). Test–retest
reliabilities over a 1-week time span are .87 and .95. Internal Teacher Behavior Outcomes
consistency ranges from .91 to .93. Validity of PLS– 4 scores has
also been established by correlating them with scores on other We used general linear modeling, specifically ANCOVA, to
measures of language and accurately identifying children with predict teachers’ TBRS scores at the end of the year from exper-
differences in language development (Zimmerman et al., 2002). imental condition and teachers’ pretest scores. Because pretest
English and Spanish versions of the Developing Skills Checklist differences were covaried, only teachers who provided both pretest
(DSC and La Lista, respectively) evaluate a range of skills that data and posttest data were included in analyses. Thus, sample
develop during preschool and kindergarten (“Developing Skills sizes ranged from 62 to 69 depending on the number of teachers
Checklist,” 1990). For this study, the Auditory Analysis subtests with both pretest and posttest data on each separate TBRS scale.
were used to assess phonological awareness. Items evaluate the No differences in pretest scores were found between teachers who
following phonological awareness skills: rhyme recognition, seg- had both pretest and posttest data and teachers who had only
mentation of sentences into words, segmentation of compound pretest data. Interactions between group and pretest scores, which
words into words, and segmentation of words into syllables. Items reflect moderation of the intervention effects, were tested and if found
also evaluate speech perception for phonemes by asking children significant were retained in the models. The effect sizes (ESs) re-
whether two stimulus words are the same or different. Auditory ported were based on the difference between posttest means that were
Analysis does not assess higher levels of phonological awareness, adjusted for covariates and interactions, if present, and then divided
such as blending or deletion of onset and rime or phonemes. The by the pooled pretest standard deviations of the groups.
Kuder-Richardson reliability coefficient for this subtest is .84, and Although most teacher behavior outcomes were normally dis-
the standard error of measurement is 1.87 ( “Developing Skills tributed, six TBRS scores remained significantly skewed at post-
Checklist,” 1990). test (i.e., quantity and quality of writing instruction, quantity and
The Preschool Comprehensive Test of Phonological Processing quality of phonological awareness instruction, and quantity and
(PCTOPPP; Lonigan, Wagner, Torgesen, & Rashotte, 2003) was quality of child portfolios). Because these instructional behaviors
designed as a downward extension of the Comprehensive Test of were so infrequently observed, they were recoded as observed or
Phonological Processing (Wagner, Torgesen, & Rashotte, 1999). not observed. To appropriately model these data, we used gener-
PCTOPPP provides assessment of phonological processing and alized linear modeling with a logit link function to predict out-
print knowledge, the latter of which was included in this study. comes having a binomial distribution (i.e., no–yes). Scores for
The Print Awareness subtest assesses children’s knowledge of these three areas represent the proportion of teachers rated as yes.
print concepts, letter discrimination, word discrimination, letter- Table 2 summarizes the content of each subscale, the anticipated
name knowledge, and letter-sound knowledge. Internal consis- quantity and quality scores, and the expected means by study
tency for the Print Awareness subtest is moderate to high for 3- to condition.
5-year-old children (i.e., ␣s ⫽.89 to .95, respectively), as are Overall teaching quantity. Collectively, teachers in the four
test–retest reliabilities (rs ⫽ .50 to .90) and validity coefficients intervention groups demonstrated significantly more positive
(rs ⬎ .43; Lonigan et al., 2003). teaching behaviors across academic domains than did teachers in
the control group, t(69) ⫽ 2.45, p ⬍ .05, ES ⫽ 0.86 (Figure 1, top).
Collectively, the two professional development groups who used
Results
the PDA-based progress monitoring tool were observed teaching
Approach to Data Analyses more than the two professional development groups who used the
paper-and-pencil– based progress monitoring tool, t(69) ⫽ 2.27,
The overarching approach we took to examine the effectiveness p ⬍ .05, ES ⫽ 0.64. In addition, the professional development
of the professional development programs was to predict end-of- group who received both mentoring and the PDA demonstrated
year outcomes from experimental group membership while statis- more teaching than the professional development group who re-
tically controlling for variability in beginning-of-the-year scores ceived mentoring but no PDA, t(69) ⫽ 1.97, p ⬍ .05, ES ⫽ 0.79.
on the outcome measures. This tends to be the most powerful Individual group comparisons also showed that teachers in the two
approach to analyzing repeated measures data when pretest and professional development groups who received detailed feedback
posttest scores are correlated. This approach translates to analysis from the PDA-based progress monitoring tool were observed
of covariance (ANCOVA) of teachers’ behavior outcomes and to engaging in more teaching than teachers in the control group,
multilevel ANCOVA of children’s language and literacy out- ts(69) ⫽ 3.29 and 2.28, ps ⬍ .001 and .03, ESs ⫽ 1.40 and 0.96,
comes. In the latter case, we used multilevel ANCOVA to account for mentored PDA and nonmentored PDA groups, respectively.
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 455

Table 2
Teacher Behavior Rating Scale Subscale Expected Means by Study Condition

Subscale and pretest score Control Mentor ⫹ PDA Mentor ⫹ P/P PDA only P/P only
a
Total teaching: Summary scores obtained by averaging
across subscale scores. 1.38 1.68 1.51 1.59 1.48
Quantity
Quality
Low 1.62 2.40 1.97 2.07 2.22
Mean 2.07 2.53 2.35 2.40 2.28
High 2.52 2.66 2.72 2.72 2.33
Phonological awareness:b Provides activities that follow
the sequence of the developmental continuum
with evidence that activities are integrated into
other learning situations.
Quantity 0.00 0.36 0.42 0.25 0.29
Quality 0.08 0.75 0.53 0.18 0.47
Writing instruction:b Teacher provides modeling and a
variety of opportunities and materials to support
this early skill.
Quantity 0.02 0.11 0.04 0.08 0.04
Quality 0.04 0.26 0.11 0.14 0.17
Book reading:a Teachers expected to demonstrate two or
more discussions about book features, vocabulary
words, and details related to the book.
Quantity 1.67 2.04 1.85 2.05 1.89
Quality 2.13 2.57 2.47 2.56 2.32
Print and letter knowledge:a Engages children in topic-
related activities that promote letter–word
knowledge and help them learn to associate
names of letters with shapes and/or begin to
make sound–letter matches. Teachers expected
to show two or more examples of activities to
promote this skill.
Quantity 1.38 1.87 1.60 1.65 1.57
Quality 1.73 2.44 1.91 2.19 1.93
Play and learning centers:a Number of centers with
clear boundaries that cover critical learning
activities and objectives linked to the theme
with rotation/refreshment of materials. Teacher
explains routine to children regarding entering
centers, moving between centers, etc.
Quality
Low 1.50 2.48 1.76 1.98 2.36
Mean 2.07 2.57 2.12 2.37 2.16
High 2.64 2.65 2.49 2.74 1.96
Lesson plan:a Evidence that theme-related materials and
activities are linked to learning objectives.
Teacher is observed implementing lesson-plan
activities.
Quantity
Low 1.24 1.74 1.54 1.75 1.88
Mean 1.78 2.24 2.01 2.13 1.69
High 2.32 2.74 2.49 2.51 1.51
Portfolios:b Adequate use of portfolios as samples of
work was seen in at least three portfolios.
Quantity 0.16 0.56 0.34 0.46 0.66
Oral language:a Encourages children’s use of language by
modeling language, asking questions, and/or
making comments; engaging children in
conversations. Teachers expected to show at least
two to three examples of this language support.
Quantity 1.76 2.03 2.02 2.01 1.93
Quality 2.42 2.77 2.62 2.69 2.5

Note. PDA ⫽ progress monitoring with the personal digital assistant (PDA) system; P/P ⫽ progress monitoring with paper and pencil; P/P only ⫽ no
mentor condition. Pretest score: Expected quantity and quality scores are ⬎2 unless otherwise noted. Results that are related to pretest score are indicated
where low ⫽ 1 standard deviation below the mean, M ⫽ average score, high ⫽ 1 standard deviation above the mean.
a
Based on 3-point rating scale. b Based on percentage of teachers coded as demonstrating instructional practice; expected scores for quality and
quantity ⬎ .60.
456 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

Figure 1. Quantity of total positive teaching behaviors (top), quality of phonological awareness instruction
(middle), and quality of written expression instruction (bottom) by study condition collapsed across sites.

Overall teaching quality. The effects of the professional de- Even with the Pretest ⫻ Group interaction, teachers in the
velopment programs on overall teaching quality at the end of the four intervention groups collectively still demonstrated higher
year were moderated by teaching quality at the beginning of the quality teaching across academic domains than the control
year, t(65) ⫽ 2.54, p ⬍ .05. Interpreted another way, the naturally group across all levels of pretest, t(65) ⫽ 2.14, p ⬍ .05, ES ⫽
strong relation between quality of teaching at the beginning of the 0.76. Individual group comparisons showed that teachers in the
year and quality of teaching at the end of the year was interrupted professional development group who received mentoring plus
by the interventions. Specifically, there was a trend toward a PDA progress monitoring demonstrated teaching at the end of
stronger pretest effect in the control group than in the four inter- the year that was of significantly higher quality than that of
vention groups combined, t(65) ⫽ 1.65, p ⬍ .10. This trend was teachers in the control group, t(65) ⫽ 2.51, p ⬍ .01, ES ⫽ 1.11.
driven by a pretest effect in the control group that was significantly In fact, a significant Mentoring ⫻ Feedback interaction, t(65) ⫽
stronger than those in the mentored PDA group and the nonmen- 2.70, p ⬍ .01, revealed that being provided with both a mentor
tored paper-and-pencil group, ts(65) ⫽ 1.95 and 2.42, ps ⬍ .05 and and detailed feedback on children’s progress was associated
.01, respectively. The Group ⫻ Pretest interaction revealed that the with the highest quality teaching and that being provided with
four professional development programs were most beneficial for neither a mentor nor detailed feedback was associated with the
teachers who evidenced poor quality teaching or only average poorest quality teaching among intervention groups. Nonethe-
quality teaching at the beginning of the year. less, even teachers in the nonmentored, limited feedback inter-
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 457

vention group still demonstrated significantly higher teaching teachers, ts(69) ⫽ 3.61 and 1.97, ps ⬍ .001 and .05, ESs ⫽ 1.46
quality than teachers in the control group, t(65) ⫽ 2.67, p ⬍ .01, and 0.79, for mentored PDA and nonmentored PDA groups, re-
ES ⫽ 0.49. spectively. The two groups who received detailed feedback also
Phonological awareness. When compared with control teach- demonstrated higher quality print and letter knowledge instruction
ers, those across the intervention conditions showed higher quan- than did control teachers, ts(69) ⫽ 3.76 and 2.48, ps ⬍ .001 and
tity, ␹2(1, N ⫽ 75) ⫽ 8.80, p ⬍ .01, ES ⫽ 0.76, and higher quality, .01, ESs ⫽ 1.38 and 0.90, for mentored PDA and nonmentored
␹2(1, N ⫽ 75) ⫽ 7.19, p ⬍ .01, ES ⫽ 0.91, of phonological PDA groups, respectively. When comparing teachers from the two
awareness instruction. All comparisons of control teachers with PDA conditions with teachers from the two paper-and-pencil con-
teachers in individual professional development conditions ditions, those who had the PDA that provided immediate detailed
showed a significantly higher quantity of phonological awareness feedback showed higher quantity, t(69) ⫽ 1.96, p ⬍ .05, ES ⫽
instruction for intervention teachers, mentored PDA group, ␹2(1, 0.52, and higher quality, t(69) ⫽ 3.22, p ⬍ .01, ES ⫽ 0.78, print
N ⫽ 28) ⫽ 7.70, p ⬍ .01, ES ⫽ 0.81; mentored paper-and-pencil and letter knowledge instruction.
group, ␹2(1, N ⫽ 27) ⫽ 9.05, p ⬍ .01, ES ⫽ 0.91; nonmentored
Play and learning centers. The effects of the professional
PDA group, ␹2(1, N ⫽ 29) ⫽ 5.17, p ⬍ .05, ES ⫽ 0.64; and
development programs on quality of play and learning centers was
nonmentored paper-and-pencil group, ␹2(1, N ⫽ 27) ⫽ 6.01, p ⬍
moderated by the quality of learning centers at the beginning of the
.05, ES ⫽ 0.70. With the exception of the teachers who were not
year, t(65) ⫽ 3.09, p ⬍ .05. Specifically, quality of centers at
mentored but had the PDA progress monitoring technology, those
posttest was much more closely related to quality of centers at
in all other intervention conditions showed higher quality imple-
mentation of phonological awareness activities when compared pretest in the control group than in the four intervention groups,
with control teachers, mentored PDA group, ␹2(1, N ⫽ 28) ⫽ t(65) ⫽ 1.88, p ⬍ .06. The nature of the Pretest ⫻ Group inter-
13.57, p ⬍ .001, ES ⫽ 1.49; mentored paper-and-pencil group, action was such that the professional development programs were
␹2(1, N ⫽ 27) ⫽ 6.69, p ⬍ .01, ES ⫽ 1.03; and nonmentored most beneficial for teachers who had low-quality centers or
paper-and-pencil group, ␹2(1, N ⫽ 27) ⫽ 5.25, p ⬍ .05, ES ⫽ 0.90 average-quality centers at the beginning of the year.
(see Figure 1, center). When comparing teachers in the two inter- Even with the Pretest ⫻ Group interaction, teachers in the four
vention conditions with mentoring with teachers in the two inter- intervention conditions collectively still had higher quality centers
vention conditions without mentoring, results showed that men- at posttest than teachers in the control group across all levels of
tored teachers had significantly higher quality phonological pretest, t(65) ⫽ 2.27, p ⬍ .05, ES ⫽ 0.43. Individual group
awareness instruction than those who did not receive mentoring, comparisons found that the group of teachers who received men-
␹2(1, N ⫽ 63) ⫽ 6.99, p ⬍ .01, ES ⫽ 0.70. toring and the PDA progress monitoring tool had better play and
Writing instruction. Relative to control teachers, teachers in learning centers at the end of the year than did the control group,
all four of the facilitated online intervention conditions demon- t(65) ⫽ 2.31, p ⬍ .05, ES ⫽ 0.91. Individual group comparisons
strated higher quality instruction in written expression at the end of also found that the group who received no mentoring and the
the year, ␹2(1, N ⫽ 75) ⫽ 4.10, p ⬍ .05, ES ⫽ 0.43 (Figure 1, paper-and-pencil progress monitoring tool had higher quality cen-
bottom). Teachers in the four professional development conditions ters at posttest than the control group across all levels of pretest,
also demonstrated a trend toward more frequent writing instruction t(65) ⫽ 3.12, p ⬍ .01, ES ⫽ 0.17.
at the end of the year than did control teachers, ␹2(1, N ⫽ 75) ⫽ Lesson plans. Collectively, teachers in the four intervention
3.27, p ⬍ .07, ES ⫽ 0.24. Those receiving mentoring plus the PDA groups were found to have significantly more lesson plans by
technology that provided immediate detailed feedback regarding the end of the year than teachers in the control group, t(65) ⫽
children’s progress were observed engaging in more writing ac- 1.99, p ⬍ .05, ES ⫽ 0.41. Interestingly, the two professional
tivities, ␹2(1, N ⫽ 28) ⫽ 5.32, p ⬍ .05, ES ⫽ 0.42, and higher development groups who received regular mentoring were ob-
quality writing activities with their children, ␹2(1, N ⫽ 28) ⫽ 5.71, served to have more written documentation of planned instruc-
p ⬍ .01, ES ⫽ 0.66, relative to control teachers.
tional activities than teachers in the two professional develop-
Book reading. Results of comparisons on amount of shared
ment groups who did not receive mentoring, t(65) ⫽ 2.15, p ⬍
reading performed by teachers in the four intervention conditions
.05, ES ⫽ 0.56.
with the amount performed by teachers in the control group was in
Portfolios. Relative to control teachers, teachers across the
the expected direction and approached significance, t(62) ⫽ 1.89,
four intervention conditions that included online professional de-
p ⬍ .06, ES ⫽ 0.77 (Figure 2, top). This trend was driven by the
two intervention groups who received PDA progress monitoring in velopment composed higher quantity portfolios of children’s work
that they demonstrated significantly more read-aloud activities samples and progress, ␹2(1, N ⫽ 75) ⫽ 5.39, p ⬍ .05, ES ⫽ 0.67.
than did teachers in the control group, ts(62) ⫽ 2.04 and 2.18, ps ⬍ Teachers who were mentored and received detailed feedback con-
.05, ESs ⫽ 1.00 and 1.02, for mentored PDA and nonmentored cerning children’s academic progress had higher quality portfolios
PDA groups, respectively. than did control teachers, ␹2(1, N ⫽ 28) ⫽ 4.96, p ⬍ .05, ES ⫽
Print and letter knowledge instruction. Relative to teachers in 0.77. Also, teachers who were not mentored and who received
the control condition, teachers across all four intervention condi- limited feedback concerning children’s academic progress had
tions showed higher quantity scores, t(69) ⫽ 2.59, p ⬍ .01, ES ⫽ higher quality portfolios than did control teachers, ␹2(1, N ⫽
0.86, and higher quality scores, t(69) ⫽ 2.48, p ⬍ .01, ES ⫽ 0.75 27) ⫽ 7.02, p ⬍ .01, ES ⫽ 0.98.
(Figure 2, bottom). Furthermore, the two groups of teachers who Oral language instruction. Although in the expected direction,
received detailed feedback from PDA progress monitoring dem- no significant differences were found between the study conditions
onstrated more print and letter knowledge instruction than control on oral language instruction quantity or quality scores.
458 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

Figure 2. Quality of book reading (top) and print and letter knowledge activities (bottom) by study condition
collapsed across sites.

Children’s Language and Literacy Outcomes Because pretest differences were covaried, we included only
children who provided both pretest data and posttest data for a
We used multilevel ANCOVA to predict the scores that children
given outcome measure in analyses. Thus, sample sizes ranged
obtained on the language and literacy measures at the end of year.
from 1,607 to 1,678. There were no significant pretest differences
Level 1 included fixed effects of children’s pretest score on the
between children with complete data and children with only pretest
outcome measure, amount of time between pretest and posttest,
data on tests of receptive language, phonological awareness, or
children’s chronological age, language of test administration, and
any significant interactions among these covariates and experi- print knowledge. However, children missing posttest data scored 1
mental condition. Level 1 also included a random intercept to to 2 raw score points lower on average on the expressive vocab-
account for children’s classroom membership. We found the ran- ulary measure (i.e., EOWPVT).
dom effect of classroom membership to be a significant predictor Vocabulary. There were no significant interactions of study
of all child outcomes and therefore retained it in all models. Level condition with any of the covariates. A priori group contrasts
2 included a fixed effect of site and a Site ⫻ Experimental revealed that the two PDA progress monitoring groups had signif-
condition interaction if significant. Level 2 also included the fixed icantly higher EOWPVT posttest scores than the two paper-and-
effects of experimental condition via inclusion of a priori contrasts pencil groups, t(237) ⫽ 2.18, p ⬍ .05, ES ⫽ 0.15 (Figure 3). The
of groups’ effects on classrooms’ average performances. interaction between the mentoring and progress monitoring con-
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 459

Figure 3. Standard scores on the Expressive One Word Picture Vocabulary by study condition (top) and letter
knowledge by study condition and site (bottom).

ditions approached significance, t(239) ⫽ 1.81, p ⬍ .07, and in the control condition, t(227) ⫽ 2.17, p ⬍ .05, ES ⫽ 0.15, and
favored those children whose teachers were mentored and received t(213) ⫽ 3.54, p ⬍ .001, ES ⫽ 0.47, for the mentored PDA and
the PDA progress monitoring technology. In fact, this group of nonmentored PDA groups, respectively. Collectively, Texas chil-
children had significantly higher posttest scores than the control dren in these two PDA groups also had higher letter and print
group, t(254) ⫽ 2.08, p ⬍ .05, ES ⫽ 0.19 (see Figure 3). knowledge scores at the end of the year than did children in the
Letter knowledge and print awareness. Analysis of children’s two groups who received only limited feedback from paper-and-
end-of-year print awareness scores revealed a significant Interven- pencil progress monitoring, t(229) ⫽ 4.38, p ⬍ .0001, ES ⫽ 0.44.
tion Condition ⫻ Site interaction, F(12, 248) ⫽ 2.54, p ⬍ .01 In Florida, there was a significant Mentoring ⫻ Progress Moni-
(Figure 3). There was also a significant Group ⫻ Pretest interac- toring interaction, t(252) ⫽ 3.18, p ⬍ .001, ES ⫽ 0.75, that
tion, F(4, 1635) ⫽ 4.58, p ⬍ .001, in which pretest scores were favored children in the mentored PDA group versus the nonmen-
less related to posttest scores for children in the intervention tored PDA group. In fact, Florida children whose teachers received
conditions compared with the control group. Thus, the intervention both mentoring and detailed feedback from the PDA had higher
had more to do with child outcomes than did the child’s initial print awareness scores at the end of the year than did Florida
letter knowledge ability. The results illustrated are across all levels children whose teachers were in the control condition, t(282) ⫽
of pretest. Examination of comparisons within each site showed 2.56, p ⬍ .01, ES ⫽ 0.20. In Ohio, children in the two conditions
that in Texas the two groups that received detailed feedback from in which teachers received mentoring achieved higher end-of-year
PDA progress monitoring had better print awareness than children print awareness scores than did children of nonmentored teachers,
460 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

t(266) ⫽ 2.45, p ⬍ .01, ES ⫽ 0.75. Across sites and pretest levels, Discussion
there was a significant Mentoring ⫻ Progress Monitoring interac-
tion, t(238) ⫽ 2.41, p ⬍ .01, ES ⫽ 0.26, that favored children The findings from this four-site experimental study show that
in the mentored PDA group. Also, collectively children in the two research-based best practices can be scaled in typical child care
PDA groups achieved higher end-of-the-year letter knowledge and early childhood education settings via high-quality, ongoing
than did children in the two paper-and-pencil groups, t(247) ⫽ professional development. The findings also demonstrate that pro-
fessional development programs are most effective when they are
2.86, p ⬍ .01, ES ⫽ 0.16.
comprehensive and well integrated. This study provided clear
Phonological awareness. There were no significant interac-
evidence for such a professional development program for teachers
tions of study condition with any covariates. Collectively, children
of at-risk prekindergarten children across multiple types of early
in all four professional development conditions had significantly
childhood settings.
higher posttest scores on the DSC/La Lista than did children in the
control group, t(241) ⫽ 2.12, p ⬍ .05, ES ⫽ 0.14. When compar-
ing individual intervention groups with the control group, children Gains in Teachers’ Instructional Practices
of teachers in the nonmentored PDA progress monitoring group
The most powerful of the four professional development ap-
had significantly higher posttest scores than did children in the
proaches for understanding greater increases in teachers’ practices
control group, t(230) ⫽ 1.96, p ⬍ .05, ES ⫽ 0.16.
was the most comprehensive and well-integrated approach that
Composite language. We found a significant Study Condi-
involved having teachers participate in a year-long online profes-
tion ⫻ Site ⫻ Pretest interaction, F(12, 1566) ⫽ 3.97, p ⬍ .0001, sional development course with classroom mentoring and with
for PLS– 4 scores, indicating that the effects of the professional progress monitoring that provided immediate, instructionally
development programs on children’s oral language were moder- linked feedback. This combination of professional development
ated by their pretest oral language levels and site differences. components provided teachers with opportunities to practice what
Across the study conditions, the moderating effects of children’s they learned in their classrooms and to communicate regularly
pretest language abilities were stronger in Texas than all other with other teachers participating in small-group learning and via
sites, t(1570) ⫽ 2.11, p ⬍ .05. In Florida, the moderating effects online message boards. It also provided weekly mentoring over the
of pretest language abilities were weaker than all other sites, course of a year and detailed feedback on individual children’s
t(1530) ⫽ 4.27, p ⬍ .0001. The weaker effect of pretest language academic progress along with recommended child groupings and
scores in Florida was because these pretest scores were less related instructional activities. Teachers who received this comprehensive
to children’s end-of-year composite language scores if their teach- professional development package became better teachers. That is,
ers were in one of the four professional development programs, they improved the overall quality of their teaching and specifically
t(1314) ⫽ 3.29, p ⬍ .001. There was a Control versus Mentored the quality and frequency of their instruction of early writing,
PDA by Pretest interaction, t(1435) ⫽ 2.38, p ⬍ .05, ES ⫽ 0.51. phonological awareness, letter knowledge, and shared reading.
These results suggest that the intervention moderated the effect of They also showed more effective center-based instruction and kept
children’s initial language abilities for understanding their gains in more complete portfolios on children. In general, teachers who
language across the school year. In other words, amount of growth received the most comprehensive package were demonstrating
in language was less dictated by children’s initial language ability shared reading and early literacy instruction at or above expected
if the children were in the mentored PDA group. frequencies and/or levels of quality. Differences between teachers
In Maryland, there was a significant Mentoring ⫻ Feedback ⫻ in this comprehensive condition versus those without the profes-
Pretest interaction, t(1609) ⫽ 3.02, p ⬍ .01, ES ⫽ 0.84 (Figure 4). sional development program were of high practical significance
This interaction demonstrated that children’s oral language skills because effect sizes were consistently moderate to large.
were benefited most if they had limited oral language at the The effectiveness of the most comprehensive approach on the
full range of instructional practices was seen across all four sites.
beginning of the year and had teachers who were both mentored
This is a particularly encouraging finding because there was con-
and received detailed feedback from the PDA (see Figure 4). There
siderable variability across the sites in factors such as the type of
was also a Control versus Mentored PDA ⫻ Pretest interaction,
early childhood setting, teachers’ cultural backgrounds, and level
t(1616) ⫽ 3.84, p ⬍ .0001, ES ⫽ 0.61. This demonstrated that
of teacher education, all of which might be expected to interact
there was less of a pretest effect in the mentored PDA group than
with program effectiveness. For example, in Maryland the class-
in the control group. rooms were all in the public schools and all teachers had at least a
In Ohio, children in the mentored groups had higher end-of-year 4-year college degree. Historically, there had been a focus by
PLS– 4 scores than did children in the nonmentored groups, administrators on improving the quality of the prekindergarten
t(290) ⫽ 2.89, p ⬍ .01, ES ⫽ 0.24. However, the control group program to better prepare children from low-income families to
had higher PLS– 4 scores than the four intervention groups com- enter kindergarten with age-level language and early literacy skills.
bined, t(250) ⫽ ⫺3.33, p ⬍ .001, ES ⫽ 0.26. This finding was In contrast, the classrooms in Florida were in childcare settings,
largely because the control group had higher PLS– 4 scores than and most of the teaching staff did not have a 2- or 4-year college
the two nonmentored groups, t(232) ⫽ ⫺3.89, p ⬍ .0001, ES ⫽ degree. Also, the programs in Florida did not have as strong a
0.37, and t(238) ⫽ ⫺4.02, p ⬍ .0001, ES ⫽ 0.41, for the non- mandate to support children to be school ready in cognitive skills.
mentored paper-and-pencil group and nonmentored PDA groups, Ohio classrooms were all part of the National Head Start program,
respectively. The controls were comparable to the two mentored and like Florida teachers, the majority did not have 4-year degrees.
intervention groups. However, unlike Florida, Ohio classrooms had a specified set of
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 461

Figure 4. Relation between children’s pretest and posttest scores on Preschool Language Scales (PLS) at the
Maryland site.

school readiness standards and practices as part of the program’s course with the PDA monitoring may have been explicit enough in
culture. Finally, the classrooms in Texas included a relative bal- how to carry out these activities that mentors did not provide
ance of the three types of settings found in the other three sites. In measurable additional support. Also, the Print Knowledge subscale
spite of these differences, the professional development program of the TBRS was unique in showing a significant advantage of the
was equally effective in changing teaching practices across sites PDA, with or without mentoring, over the paper-and-pencil ap-
because there was no Professional Development Program ⫻ Site proach. In both of these conditions, teacher quality of print knowl-
interaction. Thus, although teachers’ initial scores on the TBRS edge instruction exceeded the expected criteria, and the frequency
may have varied across the sites, the comprehensive program of instruction was, on average, just below the expected score. The
resulted in similar gains in instructional practices. information teachers received from the PDA on gains in children’s
Although the online program with the PDA progress monitoring letter and print knowledge and how to promote further learning
approach consistently showed strong effects, the efficacy of men- may function as a technology mentor for this particular type of
toring was not always apparent. Specifically, having a mentor did instruction.
not increase program effectiveness when examining gains in teach- Mentoring was found to be particularly important for increasing
ers’ book reading or print knowledge instructional practices. One teachers’ quality of phonological awareness and writing instruc-
possible explanation may be that implementing effective book tion. Teachers across all sites were not used to incorporating these
reading and early literacy activities that promote print awareness instructional activities into their daily practices. Thus, modeling
skills may already be more familiar to the teachers, and the online and coaching concerning strategies for these skill areas may be
462 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

more important than mentoring other areas of school readiness that varied across sites. It does not seem related to teacher education
teachers were more familiar with and incorporated at some level of levels because the two sites where effects were seen—that is,
competence. Although mentored teachers did evince better pho- Maryland and Florida— had the biggest education discrepancy.
nological awareness and writing instruction, the frequency of That is, Maryland had the most educated teachers and Florida had
activities in these areas was still less than desirable. the least educated teachers. These two sites also differed on chil-
dren’s home language. In Florida, most of the children were
Child Outcomes learning English as a second language because their home lan-
guage was Spanish. In contrast, most of the children in Maryland
Not only was the most comprehensive professional development came from English-speaking homes. In light of the fact that Mary-
effective in improving the quality of teaching and classroom land and Florida probably showed the biggest differences on all
environments, but it was also effective in promoting children’s teacher, child, and classroom characteristics (e.g., school district
learning. Teachers who received the most comprehensive combi- vs. childcare and degree of school-ready focus), the current results
nation of integrated professional development components gradu- demonstrate that the professional development program can be
ated children with better language comprehension, more advanced effective across diverse early childhood settings.
phonological awareness, larger breadth of expressive vocabulary,
and more print and letter knowledge than children in control Technology as an Instrument of Change
classrooms. These effects were both reliable and practically im-
portant because statistically significant and educationally mean- The use of technology is thought to be an important key to the
ingful improvements in children’s school readiness were demon- success of this professional development program. The Web-based
strated. Furthermore, that the program yielded small to moderate instructional course was developed with extensive use of video-
effect sizes through only indirect intervention is impressive. That taped examples of how teachers could implement the activities in
is, children’s learning outcomes were appreciably improved through the classrooms. Each lesson had videotapes of experts in the field
professional development of hundreds of teachers rather than through explaining the importance of the particular area of focus. Com-
costly and labor-intensive direct intervention with thousands of mentary guided learners to note key aspects of the classroom
children. videos. Because the lessons also included text that explained
Unlike the findings for teachers’ behaviors, there were findings course objectives and included links to learning activities, the
for children’s abilities that varied by site. For example, Texas facilitators did not need the same level of expertise to facilitate
children whose teachers had the online course with the PDA teachers’ learning as they would without the online course. One
showed greater gains in letter and print knowledge than control question that was not addressed in this experimental study was
children whether mentoring was part of the program or not. How- whether the online course could be effective without a person
ever, in Florida only children whose teachers had the most com- facilitating small groups of educators. As the success of the pro-
prehensive approach showed a significant advantage in letter and gram was thought to be a result, in part, of the network of learners
print knowledge over control children. Because teachers in Flor- established through the facilitation of a trainer or coach, a key
ida, on average, had less education and the childcare centers had component for CIRCLE professional development is a supportive
not been as focused on children’s school readiness before this relationship (Landry et al., 2006). The benefit of this relationship
program, the addition of mentoring associated with the most com- is further demonstrated in the many teaching practices that were
prehensive approach seems to have been more important for these most effectively facilitated with the addition of an in-classroom
Florida teachers to effectively support children’s development of mentor.
this important school readiness domain. Some of the most robust findings from this study concerned the
The effectiveness of the program in promoting children’s gen- benefits of the PDA-based progress monitoring tool. Use of
eral receptive language also varied across the four sites. The mCLASS C-PALLS consistently resulted in improvements in
positive effects of the professional development programs on this teachers’ instruction and children’s learning. Although the actual
multifaceted aspect of children’s school readiness were most no- assessments were identical, the PDA version of the CBMs pro-
ticeable among children with low levels of language comprehen- vided teachers with immediate feedback about children’s learning
sion at the beginning of the school year. In Florida, the four from one assessment to the next, and it provided comparisons
professional development approaches were more strongly related across multiple skill areas for each child. Perhaps more important,
to children’s language outcomes than their pretest, or initial, lan- the PDA version recommended how to group children from the
guage abilities. In Maryland, the intervention also had an effect on same classroom into small groups and specific instructional activ-
children’s receptive language skills. However, at this site the effect ities to use for particular small groups of children. Extending the
was seen most strongly for the children who started the year with work of Fuchs, Fuchs, Hamlett, and Phillips (1994), who studied
the lowest language skills and particularly when their teachers had elementary school teachers’ use of CBMs in mathematics instruc-
the most comprehensive professional development program, the tion, this study demonstrated the usefulness of technology to help
PDA progress monitoring plus in-classroom mentoring. Also over- early childhood educators use CBMs to plan their language and
all, in Maryland the children whose teachers had the PDA instead literacy instruction.
of the paper-and-pencil approach had better language comprehen-
sion at the end of the year. In contrast, the effects of the profes- Challenges to Bringing the Program to Scale
sional development program on general receptive language were
not as strong in Texas, and no effects were seen in Ohio. It is not Similar to others’ efforts to bring an educational program to
clear why professional development effects on receptive language scale, we learned many things that could inform future attempts in
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 463

this area. Because of the tight timelines to carry out federally least competent teachers to change their instructional practices to
funded research, adequate time is often not available to develop an extent that advanced children’s learning. Identification of rec-
relationships with administrators and teachers before starting a ommended dosage levels for teachers of different competence
study. However, commitment to the project from multiple levels of levels is an important area for future study because it will help
staff (e.g., superintendents, directors, coordinators, and teachers) is ensure that resources earmarked for professional development are
critically important. A common problem is the person at the appropriately allocated.
highest level in the program believing in the importance of the A major goal of this study was to determine whether a profes-
intervention but not gaining support from others in the organiza- sional development program could be brought to scale across a
tion. Thus, it is important for the research group to talk and work diverse range of early childhood education settings. In three of the
with staff at all levels to explain the intervention, including dis- four sites, early childhood services were delivered in a specified
cussion of the demands on a teacher’s time and the level of way such that in any one site, only childcare, Head Start, or public
commitment requirement to achieve effects. prekindergarten were available for participation. In the fourth site,
Not surprisingly, the extensive use of technology needed to it was possible to have classrooms from all three types of pro-
deliver this professional development program brought about grams. This meant that factors, such as teacher education, that are
unique challenges. Local and centralized supports for technology highly associated with the type of program delivery, were con-
were essential. Challenges in this study included minor problems founded by study site. It would take a much larger sample of
with the technology platform, locating computer labs for group classrooms within sites, stratified by teacher education, to most
sessions, and teachers’ inexperience with computers and PDAs. adequately address the question of whether the effectiveness of
These obstacles, though, were surmountable. this professional development program varied by teacher educa-
That the implementation of this project occurred across four tion. Another area that may be important for understanding pro-
states made assuring fidelity of implementation one of our greatest gram effectiveness that was not specifically measured was teach-
challenges. Nonetheless, the programs’ efficacy was supported ers’ philosophies, including their attitudes and beliefs. This may be
across four states with only limited oversight from the research an important area to examine in future research.
group who developed it. Fidelity was supported through regular A second limitation was the inability to examine the extent to
conference calls among the project manager and facilitators, as which curricula that have a specific focus on language and literacy
well as regular monitoring of teachers’ postings of their experi- would provide an additional advantage for understanding gains in
ences and questions. Although funding only allowed for infrequent prekindergarten children’s learning. Recent studies of the efficacy
visits to the sites to work directly with staff, these visits appeared of specific language and literacy curricula suggest that this may be
to be important, and more visits may have supported stronger an important factor (Assel, Landry, Swank, & Gunnewig, 2006).
results. Finally, future research needs to address other factors (e.g., lan-
Finally, that the professional development program did not guage of instruction for English language learning children and
mandate a particular curriculum was both an asset and a liability. teacher– child ratios) that may play an important role in under-
The benefit of such an approach was that the program can be standing variability in the effectiveness of professional develop-
widely used and then individualized through mentoring. However, ment programs for preparing teachers to effectively support young
experience with classrooms that used curricula that did not have a children’s language and literacy development.
strong focus on emergent literacy seemed to put those teachers at
a disadvantage because they did not have readily available a scope References
and sequence of specific instructional activities that paralleled the
objectives described in the online courses. Aber, J. L., Jones, S. M., & Raver, C. C. (2007). Child development and
social policy: Knowledge for action. In J. L. Aber (Eds.), Poverty and
child development: New perspectives on a defining issue (pp. 149 –166).
Limitations and Future Directions Washington, DC: American Psychological Association.
Anthony, J. L., Williams, J. M., McDonald, R., Corbitt-Shindler, D.,
It is important to note that this study demonstrated impacts on
Carlson, C. D., & Francis, D. J. (2006). Phonological processing and
teachers’ behavior, classroom environments, and children’s learn-
emergent literacy in Spanish speaking preschool children. Annals of
ing within the same year that teachers received the professional Dyslexia, 56, 239 –270.
development. Effect sizes tended to be moderate to large on Anthony, J. L., Williams, J. M., McDonald, R., & Francis, D. J. (2007).
teacher and classroom outcomes, but they were sometimes small Phonological processing and emergent literacy in younger and older
on child outcomes (e.g., vocabulary and phonological awareness). preschool children. Annals of Dyslexia, 57, 113–137.
As improving children’s school readiness occurs further down the Assel, M. A., Landry, S. H., & Swank, P. R. (2007). Are early childhood
chain of linked events than does changing teachers’ behavior, it classrooms preparing children to be school ready? The CIRCLE Teacher
will be important to assess effects of the programs after teachers Behavior Rating Scale. In L. Justice & C. Vukelich (Eds.), Achieving
participate for a second year. This will help determine whether an excellence in preschool literacy instruction (pp. 120 –135). New York:
additional program “dose” provides an opportunity for teachers to Guilford Press.
Assel, M. A., Landry, S. H., Swank, P. R., & Gunnewig, S. (2006). An
hone their skills, which may result in even better child outcomes.
evaluation of curriculum, setting, and mentoring on the performance of
In line with trying to find the best dosage of professional devel- children enrolled in pre-kindergarten. Reading and Writing, 20, 463–
opment, it may be that teachers with the least amount of experience 494.
or formal training in early childhood education require the most Bereiter, C. (1972). An academic preschool for disadvantaged children:
professional development. This study anecdotally found that the Conclusions from evaluation studies. In J. S. Stanley (Ed.), Preschool
more comprehensive, or intensive, approach was necessary for the programs for the disadvantaged: Five experimental approaches to early
464 LANDRY, ANTHONY, SWANK, AND MONSEQUE-BAILEY

childhood education (pp. 1–21). Baltimore: Johns Hopkins University Family day care: Current research for informed public policy (pp.
Press. 188 –208). New York: Teachers College Press.
Bowman, B. T., Donovan, M. S., & Burns, M. S. (Eds.). (2001). Eager to Landry, S. H., Crawford, A., Gunnewig, S., & Swank, P. R. (2000). The
learn: Educating our preschoolers. Washington, DC: National Academy CIRCLE-Teacher Behavior Rating Scale. Unpublished research
Press. instrument.
Bransford, J., Brown, A., & Cocking, R. R. (Eds.). (2000). How people Landry, S. H., Smith, K. E., Swank, P. R., Assel, M. A., & Vellet, S.
learn: Brain, mind, experience, and school. Washington, DC: National (2001). Does early responsive parenting have a special importance for
Academy Press. children’s development or is consistency across early childhood neces-
Brownell, R. (2000). Expressive One-Word Picture Vocabulary Test. No- sary? Developmental Psychology, 37, 387– 403.
vato, CA: Academic Therapy. Landry, S. H., Swank, P. R., Smith, K. E., Assel, M. A., & Gunnewig, S.
Center for Children & Technology. (2000). Transformation of Union City (2006). Enhancing early literacy skills for pre-school children: Bringing
1989 to present. New York: Educational Development Center. a professional development model to scale. Journal of Learning Dis-
Center for Improving the Readiness of Children for Learning & Education. abilities, 39, 306 –324.
(2006). Preschool early language and literacy training. Houston: Uni- Learning First Alliance. (2000). Every child reading: A professional de-
versity of Texas, Health Science Center at Houston. velopment guide. Baltimore: Association for Supervision & Curriculum
Corsini, D. A., & Caruso, G. (1989). High quality family day care: Development.
Financial considerations (ERIC Document #ED320707). (Available Leeper, S. H., & Witherspoon, D. B. D. (1984). Good schemes for young
from ERIC Document Reproduction Service.) children (5th ed.). New York: Macmillan.
Culp, K. M., & Honey, M. (2000, October). Scale and localization: The Lonigan, C. J., Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (2003).
challenge of implementing what works. Paper presented at the Wing- Preschool Comprehensive Test of Phonological and Print Processing.
spread Conferences on Technology’s Role in Urban School Reform. Austin, TX: PRO-ED.
Developing Skills Checklist. (1990). Monterey, CA: CTB/McGraw-Hill. Madelain, A., & Wheldall, K. (2004). Curriculum-based measurement of
DiPietro, J. A. (2000). Baby and the brain: Advances in child development. reading: Recent advances. International Journal of Disability, Develop-
Annual Review of Public Health, 21, 455– 471. ment, and Education, 51, 57– 82.
McConnel, S. (2000). Individual growth and development indicators. Min-
Eisenhower National Clearinghouse for Mathematics & Science Education.
neapolis, MN: Author.
(1998). Ideas that work science professional development. Columbus:
McLaughlin, M. W. (1990, December). The Rand Change Agent Study:
Ohio State University.
Macro perspectives and micro realities. Educational Researcher, pp.
Elmore, R. R. (2002). Bridging the gap between standards and achieve-
11–16.
ment. The imperative for professional development in education. Wash-
McLoyd, V. C. (1998). Socioeconomic disadvantage and child develop-
ington, DC: Albert Shanker Institute.
ment. American Psychologist, 53, 185–204.
Epstein, A. S. (1993). Training for quality: Improving early childhood
Mitchell, F. (1979). Interobserver agreement, reliability, and generalizabil-
programs through systematic inservice training (Monographs of the
ity of data collected in observational studies. Psychological Bulletin, 86,
High/Scope Educational Foundation, No. 9). Ypsilanti, MI: High/Scope
366 –370.
Press.
National Academy of Education. (1999). Recommendations regarding
Fuchs, L. S. (2004). The past, present, and future of curriculum-based
research priorities: An advisory report to the National Education Re-
measurement research. School Psychology Review, 33, 188 –192.
search Policy and Priorities Board. Washington, DC: Author.
Fuchs, L. S., Fuchs, D., Hamlett, C. L., & Phillips, N. B. (1994). Classwide
National Center for Education Statistics. (2001a). The nation’s report card:
curriculum-based measurement: Helping general educators meet the
Fourth-grade reading 2000. Retrieved April 6, 2001, from http://
challenge of student diversity. Exceptional Children, 60, 518 –537. nces.ed.gov/pubsearch/pubsinfo.asp?pubid⫽2001499
Hart, B., & Risley, T. R. (1995). Meaningful differences in the everyday National Center for Education Statistics. (2001b). The nation’s report
experiences of young American children. Baltimore: Paul H. Brookes. card: Mathematics 2000. Retrieved from http://nces.ed.gov/pubsearch/
Hayes, J. L., Palmer, J. L., & Zaslow, M. J. (Eds.). (1990). Who cares for pubsinfo.asp?pubid⫽2001517
America’s children? Washington, DC: National Academy Press. National Institute for Literacy. (2007). [Early literacy predictors of later
Highlighting NAEP 2003. (2003). National Assessment for Educational reading outcomes]. Unpublished raw data from the National Early Lit-
Progress Oregon, 1(1), 1. Retrieved July 17, 2007, from http:// eracy Panel. Louisville, KY: National Center for Family Literacy.
www.ode.state.or.us/initiatives/naep/naepnews_vol01num01.pdf Neuman, S. B. (1996). Children engaging in storybook reading: The
Howes, C. (1997). Children’s experiences in center-based child care as a influence of access to print resources, opportunity, and parental interac-
function of teacher background and adult-child ratio. Merrill-Palmer tion. Early Childhood Research Quarterly, 11, 495–513.
Quarterly, 43, 404 – 425. Neville, H. J., Bavelier, D., Corina, D., Rauschecker, J., Karni, A., Lal-
Howes, C., Phillips, D., & Whitebook, M. (1992). Thresholds of quality: wani, A., et al. (1998). Cerebral organization for language in deaf and
Implications for the social development of children in center-based child hearing subjects: Biological constraints and effects of experience. Pro-
care. Child Development, 63, 449 – 460. ceedings of the National Academy of Sciences of the United States of
International Reading Association & National Association for the Educa- America, 95, 922–929.
tion of Young Children. (1998). Learning to read and write: Develop- Phaneuf, R., L., & Silberglitt, B. (2003). Tracking preschoolers’ language
mentally appropriate practices for young children. Young Children, and preliteracy development using a general outcome measurement
53(4), 30 – 46. system: One education district’s experience. Topics in Early Childhood
Kennedy, M. (1997). The connection between research and practice. Ed- Special Education, 23, 114 –123.
ucational Researcher, 26, 4 –12. Phillips, D. A., Voran, M., Kisker, E., Howes, C., & Whitebook, M.
Kontos, S., Howes, C., & Galinsky, E. (1997). Does training make a (1994). Child care for children in poverty: Opportunity or inequity.
difference to quality in family child care? Early Childhood Research Child Development, 65, 472– 492.
Quarterly, 43, 351–372. Putnam, R. T., & Borko, H. (2000). What do new views of knowledge and
Kontos, S., Machida, S., Griffin, S., & Read, M. (1992). Training and thinking have to say about research on teacher learning? Educational
professionalism in family day care. In D. L. Peters & A. R. Pence (Eds.), Researcher, 29, 4 –15.
 COMPREHENSIVE PROFESSIONAL DEVELOPMENT 465

Richardson, V., Anderson, P., Tidwell, D., & Lloyd, C. (1991). The Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (1999). Comprehensive
relationship between teachers’ beliefs and practices in reading compre- Test of Phonological Processing. Austin, TX: PRO-ED.
hension instruction. American Educational Research Journal, 28, 559 – Wesson, C. L. (1992). Using curriculum-based measurement to create
586. instructional groups. Preventing School Failure, 36, 17–20.
Slavin, R. E., Madden, N. A., Dolan, L. J., & Wasik, B. A. (1996). Every Whitehurst, G. J., & Lonigan, C. J. (1998). Child development and emer-
child, every school: Success for all. Newbury Park, CA: Corwin. gent literacy. Child Development, 68, 848 – 872.
Snow, C. E., Burns, M. S., & Griffin, P. (Eds.). (1998). Preventing reading Zevenbergen, A. A., Whitehurst, G. J., Payne, A. C., Crone, D. A., Hiscott,
difficulties in young children. Washington, DC: National Academy M. D., Nania, O. C., et al. (1997). Outcomes of an emergent literacy
Press. intervention in Head Start homes and classrooms. National Head Start
Speck, M. (1996, Spring). Best practice in professional development for Association Research Quarterly, 1, 137–147.
sustained educational change. ERS Spectrum, pp. 33– 41. Zill, N., & West, J. (2001). Entering kindergarten: A portrait of American
Spodek, B. (1996). The professional development of early childhood children when they begin school (NCES 2001– 035). Washington, DC:
teachers. Early Child Development and Care, 115, 115–124. U.S. Department of Education, Office of Educational Research &
Sullivan, B. (1999). Professional development: The linchpin of teacher Improvement.
quality. Infobrief, No. 18. Zimmerman, I. L., Steiner, V. G., & Pond, R. E. (2002). Preschool
Swank, P. R., Assel, M. A., Anthony, J. L., King, T., Gunnewig, S., & Language Scale—Fourth edition. San Antonio, TX: Psychological
Landry, S. H. (2006). Psychometric analyses of English m:Class data. Corporation.
Presentation at the annual meeting of the Society of the Scientific Study
of Reading. Received November 15, 2007
Tuyman, B., Aston, L., & Gunnewig, S. (2001). CIRCLE Glows and Grows Revision received June 18, 2008
Mentoring. Unpublished measure. Accepted June 23, 2008 䡲