PHYSICAL REVIEW SPECIAL TOPICS - PHYSICS EDUCATION RESEARCH 9, 020116 (2013)

Extending positive CLASS results across multiple instructors and multiple classes
of Modeling Instruction
Eric Brewe,1,2 Adrienne Traxler,2 Jorge de la Garza,2,3 and Laird H. Kramer2
1
Department of Teaching and Learning, Florida International University, Miami, Florida 33199, USA
2
Department of Physics, Florida International University, Miami, Florida 33199, USA
3
Departamento de Fı́sica, Tecnológico de Monterrey, Campus Monterrey, Monterrey, Mexico 64849
(Received 14 March 2013; published 30 October 2013)
We report on a multiyear study of student attitudes measured with the Colorado Learning Attitudes
about Science Survey in calculus-based introductory physics taught with the Modeling Instruction
curriculum. We find that five of six instructors and eight of nine sections using Modeling Instruction
showed significantly improved attitudes from pre- to postcourse. Cohen’s d effect sizes range from
0.08 to 0.95 for individual instructors. The average effect was d ¼ 0:45, with a 95% confidence interval of
(0.26–0.64). These results build on previously published results showing positive shifts in attitudes from
Modeling Instruction classes. We interpret these data in light of other published positive attitudinal shifts
and explore mechanistic explanations for similarities and differences with other published positive shifts.

DOI: 10.1103/PhysRevSTPER.9.020116 PACS numbers: 01.40.Fk, 01.40.gb

I. INTRODUCTION literature investigating mechanistic explanations for

attitudinal shifts and characteristics of implementations
A tremendous amount of productive research has of transformed curricula that promote enhanced student
addressed improving the conceptual understanding and attitudes.
problem-solving skills of students in physics courses. Brewe et al. [11] and de la Garza and Alarcon [12], both
More recently, a growing number of studies have consid- using variants of the Modeling Instruction curriculum,
ered the roles that attitudes, beliefs, and expectations play published the first favorable attitudinal shifts in
in learning. Though less immediately familiar to many university-level calculus-based introductory courses with
physics instructors than evaluation of free-body diagrams single classes. Because these prior results came from single
or correct use of conservation of energy, attitudinal factors courses, the possibility exists that these are the result of
have been recognized as a key developing area of research instructor effects. To address this concern, we have under-
in STEM education [1–3]. taken a multiyear follow-up study of attitudinal shifts in the
In rough analogy to the fruitful measurement of student Modeling Instruction sections of calculus-based physics I
conceptual gains by diagnostics such as the Force Concept at Florida International University. The goal of this paper is
Inventory [4] or the Force and Motion Conceptual threefold: to reiterate the Brewe et al. and de la Garza and
Evaluation [5], several instruments have been developed Alarcon positive attitudinal shifts across multiple years and
and used to evaluate students’ ‘‘belief gains’’ (or losses) instructors, to describe features of the Modeling Instruction
[6–8]. We have chosen to use the Colorado Learning curriculum that may promote this pattern of favorable
Attitudes about Science Survey (CLASS) [7]. No clear shifts, and to add to the ongoing discussion of conditions
consensus has yet emerged as to what types of instruction
that may be necessary or sufficient to cause attitudinal
may reliably improve students’ attitudes toward science
gains in introductory physics. We consider several candi-
from pre- to postcourse. In the absence of any massive
dates for mechanistic explanations for positive attitudinal
attitudinal data aggregation showing distinct grouping
shifts: use of the Modeling Instruction curriculum and
by instructional style [9], researchers are still seeking
pedagogy, instructor participation in weekly planning
common factors in the relatively few classrooms with
meetings, the epistemological framing of the curriculum,
positive gains to report. Further, in light of Pollock and
the size of the class, and even whether the CLASS is
Finkelstein’s [10] work on sustaining educational reforms,
measuring elements of self-efficacy.
this article demonstrates an example of a reform that seems
Section II reviews previous results in the literature.
to work consistently across multiple implementations
and multiple instructors. We contribute to the ongoing Section III describes the relevant features of Modeling
Instruction, the curriculum common to the data in this
paper. Section IV describes our institutional context and
methods of data collection. Section V presents results from
Published by the American Physical Society under the terms of
the Creative Commons Attribution 3.0 License. Further distri- nine new classes of physics I. Section VI discusses possible
bution of this work must maintain attribution to the author(s) and interpretations of those results, and Sec. VII presents our
the published article’s title, journal citation, and DOI. final conclusions.

1554-9178=13=9(2)=020116(10) 020116-1 Published by the American Physical Society

BREWE et al. PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

Elby [19], where they use a resources perspective. Lising

II. PREVIOUS ATTITUDE RESULTS
and Elby found that students’ ability to make effective use
Physics instructors care about student attitudes and of physics knowledge they demonstrably possess is medi-
beliefs for a variety of reasons, both personal and pragmatic. ated by their deployed epistemological resources about
At the large scale, the public’s perception of science has where and how that knowledge is applicable.
far-reaching consequences for funding of research in a Both the ‘‘beliefs’’ and ‘‘resources’’ perspectives on
democracy. Whether or not students eventually major in student epistemologies are found in physics education
science, they are exposed to attitudes toward science and research. They share a broad end goal of promoting a
learning science from their K–12 and college instructors. In view of physics as a coherent system of knowledge, pro-
the college classroom, students’ attitudes toward science duced by and accessible through systematic reasoning. A
and math have been shown as key predictors of their success growing body of evidence demonstrates that understanding
in those classes [13,14]. Narrowing the scope further to students’ attitudes and epistemologies and how they change
physics, students’ pretest performance on a survey of atti- as a result of instruction is a necessary consideration for any
tudes and beliefs has been linked to their decision to take complete picture of science education. When we wish that
physics courses, as well as their conceptual gains once they students would finish our courses better able to ‘‘think like
do [8,15]. Longitudinal work using the CLASS shows a scientists,’’ their attitudes and beliefs about the subject form
strong connection between students’ initial attitudes and a real and measurable part of that goal—one which can be
their persistence in the physics major [16]. supported through appropriate curriculum.
One subset of the broad category of ‘‘attitudes and
beliefs,’’ that of epistemological beliefs, warrants further A. Attitudinal shifts in literature
detail here. The term ‘‘beliefs’’ itself carries some theo- A number of survey instruments have been developed
retical implications. Researchers investigating student to assess student attitudes and beliefs in physics courses,
epistemologies differ on whether those epistemologies and to measure the effect that instruction has on these
take the form of broad developmental stages, domain- beliefs. The earliest surveys are the Maryland Physics
specific but coherent beliefs, or context-sensitive activa- Expectations Survey (MPEX) [6] and the Views about
tions of resources (see Elby [3] for expansion of these Science Survey (VASS) [20], with later work producing
distinctions). However, across these theoretical commit- the Epistemological Beliefs Assessment for Physical
ments, there is evidence that students’ understanding of the Science (EBAPS) [21] and the Colorado Learning
nature of knowledge and learning can influence their aca- Attitudes about Science Survey (CLASS) [7]. A common
demic development in a variety of ways. Schommer [17] feature of these instruments is that surveyed students show
classifies student beliefs into categories about the certainty, negative attitudinal shifts from the beginning to end of an
source, and nature of knowledge. In a study of students’ introductory course. These negative shifts are common even
comprehension of a mathematical passage, Schommer in transformed classes with demonstrated improvements in
et al. [18] found that a belief in ‘‘simple knowledge’’ conceptual learning [6,7]. Furthermore, students are able
(the idea that knowledge is a set of discrete, unambiguous to accurately identify favorable or expertlike responses—
facts) predicted poor performance on a mastery test, as both before and after instruction. In short, they know how
well as overconfidence in that performance. More gener- a physicist would answer the survey questions, but they
ally, we might expect students’ epistemological beliefs to do not identify with those responses in their personal
affect their choice of study strategies and ability to gauge experience [22].
their own learning. For example, if a student views physics A small but growing number of courses serve as
as a series of disconnected definitions and math problems, counterpoints to the negative shifts, demonstrating
he or she might be completely satisfied with a learning positive shifts in attitudes and beliefs. These results
strategy of memorizing equations without thinking about occur for different student populations and on different
their interrelationships. Some evidence to support such a instruments: in high school physics on the EBAPS
causal link exists in case study work by Lising and and MPEX [23], for preservice elementary teachers on

TABLE I. Summary of previously published positive CLASS shifts. Standard errors of the mean are shown in pre, post, and shift
data, and the 95% confidence interval (C.I.) on the effect is shown in the effect size.

Overall percent favorable response

Group Pre Post Shift Effect size (95% C.I. on d)
Brewe et al. 2009 (MI-B) 68:6
2:8 77:5
2:0 9:0
2:7 0.71 (0.08, 1.30)
de la Garza and Alarcon 2010 (MI-G) 68:4
2:4 71:4
2:1 3:1
2:2 0.21 ( 0:21, 0.63)
Otero and Gray 2010 (PET) 53:8
1:2 62:6
1:2 8:8
1:1 0.59 (0.38, 0.80)
Lindsey et al. 2012 (PbI) 52:0
0:7 60:6
0:7 8:6
0:7 0.52 (0.40, 0.63)

020116-2
EXTENDING POSITIVE CLASS RESULTS ACROSS . . . PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

80 models [27] that structure the course content in MI are

shared among members of the learning environment, and
the validity, deployment, and interpretation of the models
are established through classroom activities and dis-
60 course.1 In this approach, models serve as conceptual
Overall Favorable C−LASS %

resources that can be used to develop understanding of a

variety of phenomena.
Modeling Instruction’s origins in the Modeling Theory
40
of Science provide an epistemological foundation that is
evident in the curriculum as student activities are focused
on the process of building, validating, and deploying mod-
els. This process of modeling replicates the central activity
of practicing scientists and therefore promotes students
20
engaging with the practices and norms of physics [28].
The explicit epistemological foundation of physics as
model building and use requires that students are active,
engaged participants in the learning environment. We con-
0 tend that the participation of students in model building
MI−B MI−G PET PbI and use within the Modeling Instruction learning environ-
Implementations ment contributes to the conceptual and attitudinal gains
documented through research [29].
FIG. 1 (color online). Published positive shifts on the CLASS;
An ongoing, NSF-supported, Modeling Instruction cur-
see also Table I. Blue and red represent the pre- and postcourse
percentages, respectively, of overall favorable responses. The
riculum development and research project has been under
error bars show standard error for pre and post. way at Florida International University (FIU) since 2007.
This effort has built on the success of the High School
Modeling Instruction project to develop a set of unique,
the CLASS [24,25], in algebra-based introductory college model-centered curriculum materials appropriate for imple-
physics on the MPEX II [24]. Finally, in introductory mentation of Modeling Instruction in university physics.
calculus-based physics, Modeling Instruction courses Materials include conceptual reasoning, problem solving,
have demonstrated positive shifts in student beliefs as and representational competence building student activities,
measured by the CLASS [11,12]. It is these results, com- as well as model building labs. Instructors’ materials include
piled in Table I and Fig. 1, that we expand upon in this guides for implementing the curriculum materials and pro-
article. We acknowledge that other instruments could have vide example questions to guide discussion, logistical notes,
been used, but we have chosen to use the CLASS due to its and brief descriptions of the purpose of all student activities.
greater prevalence in the current literature [7,11,12,24,25]. Finally, the curriculum materials include annotated video
examples of the materials being used in a Modeling
III. MODELING INSTRUCTION CURRICULUM Instruction class. The materials developed by this project
AND PEDAGOGY were used by all instructors in this study.
Modeling Instruction courses at FIU have primarily
We contend that the structure of Modeling Instruction operated as a collaborative learning environment, with 30
(MI) has contributed to the achievement of positive shifts students in a studio-format class with integrated lab and
on the CLASS. This structure includes the epistemological ‘‘lecture’’ time. Students in Modeling Instruction classes
foundations which have shaped the development of curric- are active participants, with minimal time spent listening to
ular materials and discourse management practices which lectures. Students learn science as scientists, by engaging
ultimately comprise the implementation of MI. Modeling in inquiry labs, activities targeting model-based reasoning,
Instruction is a curriculum and pedagogy developed for and problem solving; these are the primary mechanisms for
university physics with an explicit epistemological foun- building, validating, and using models [26]. In a Modeling
dation in the Modeling Theory of Science [26]. According Instruction class, the students typically work on an activity
to the Modeling Theory of Science, science is essentially in groups of three students and then present their results on
an ongoing process of model development, validation, a small portable whiteboard to the whole class. In these
deployment, and revision. The basic building blocks in ‘‘board meetings,’’ students discuss their whiteboards
science are the models and science is a modeling process.
Modeling Instruction contends that science instruction 1
Our analysis of models considers scientific conceptual mod-
should therefore teach students the basic rules of modeling els, which for us are in the shared domain of classroom dis-
and should organize the course content around a small set course, rather than mental models, which for us refer to cognitive
of scientific conceptual models. The scientific conceptual structures inside minds, and about which we are less certain.

020116-3
BREWE et al. PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

and the instructor facilitates and guides the discussion with candidates for attribution of this growth, one of which is
the goal of helping the class to reach consensus. This the implementation of Modeling Instruction in a selection
modeling cycle builds a view of physics as a coherent of sections of introductory calculus-based physics. The
system of knowledge where formulas are constructed Modeling Instruction sections of introductory physics are
from conceptual understanding and laboratory evidence, popular, with demand outpacing availability by nearly a
and where models are subject to change as new data factor of 4. In order to handle the placement of students in
become available. the MI sections the PER group at FIU has implemented a
Support for instructors in implementing the Modeling lottery system which both eases administration of registra-
Instruction pedagogy includes an instructor’s guide with tion and provides some randomization of the class makeup.
linked videos and guiding questions for facilitating student The student participants in this study are somewhat
discourse. The instructor materials were used as the basis representative of the FIU student population. Of the 221
for a weekly Modeling Instruction planning meeting for participants, 217 reported ethnicity, 76% report Hispanic,
instructors 1–4 and 6 represented in this new data. 9% Black, 7% White, 6% Asian or Pacific Islander. The
Instructor 5 utilized the materials but was not able to attend sample also includes 120 (47%) male participants and 115
the planning meeting. The Modeling Instruction weekly (53%) female participants. This gender distribution is
planning meetings lasted one hour and were designed to similar to the makeup of the university; however, there is
ensure that the instructors in this data set were using the a greater representation of females than in typical physics
Modeling Instruction materials and were facilitating stu- classes.
dent discourse using Modeling Discourse Management
practices [30,31]. During the meetings, the instructors B. Pre- and posttesting in all Modeling Instruction
reviewed the instructional plan for the week and reviewed sections of introductory university physics
and modified student activities and labs. Reviewing the
instructional plan typically involved a discussion of the Beginning in Fall 2008 and continuing through Fall
weekly goals and purposes for specific activities. Because 2012, the PER group at FIU has administered the CLASS
of the attention to goals and purposes, the planning in all Modeling Instruction sections. The data we report
meetings were also a time when the epistemological foun- here include nine sections of the calculus-based mechanics
dations of the materials were discussed and debated. The sections (the first semester) of Modeling Instruction.
planning meetings were typically run by the first author, Classes ranged in size from 20 to 30, with an average
and were attended by all instructors teaching during any size of 24.6. We have constrained our analysis to the
given semester (with the exception of 5). mechanics section in order to allow comparisons to other
Several features of the MI learning environment are studies with positive attitudinal results. These nine sections
viable candidates for the influence on attitudes: the active include six instructors, two of which have been the instruc-
nature of the pedagogy, the explicit epistemological focus tor for multiple sections. In all cases, the CLASS was
on modeling, the small class size, and the effects on administered on the first day of the semester and again
students’ self-efficacy [32]. In addition, the role of the on the last day of the semester. The survey was adminis-
instructor’s guide and the weekly planning meetings are tered as a paper-and-pencil assessment. Student responses
worth considering as candidates for the influence on stu- were analyzed using the template provided by the
dent attitudes. In Sec. VI we revisit these possibilities in University of Colorado PER group [33]. Students who
light of accumulated positive CLASS results. did not participate in both the pre- and posttest were
removed from the data set, as were students who did not
IV. METHODS respond appropriately to question 31, which indicates if
students are reading the questions. Finally, the students in
A. FIU context description the initial study [11] were not included in this data set to
Florida International University is a large urban avoid double counting; thus, the data presented here are
research-intensive Hispanic-serving institution. As of Fall denoted MI-New. After removing these students, a total of
2012, enrollment was 50 394 students, 91% of whom are 221 students remained.
commuters, ensuring that FIU reflects the ethnic diversity Student responses from six different instructors are
of Miami and South Florida. The student body at FIU is included in this study. These instructors have a range
62% Hispanic and 14% Black, making FIU an important of experience with University Modeling Instruction. Two
source of STEM graduates from underrepresented groups. instructors (1 and 2) have used University Modeling
The physics department at Florida International Instruction curriculum and pedagogy more than five times,
University has been experiencing continuous growth in two instructors (3 and 4) were implementing for the first time
the number of declared and intended physics majors begin- as lead instructor but had each spent a year as an apprentice
ning in the early 2000s. The growth represents a nearly with experienced Modeling Instruction users, and two
400% increase normalized to the size of the university. instructors (5 and 6) were implementing for the first time
Within the physics department several changes are without apprentice experience. Instructors 3–6 were all

020116-4
EXTENDING POSITIVE CLASS RESULTS ACROSS . . . PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

teaching calculus-based introductory physics for the first

60
time, though all had teaching experience in physics labs.
Although the results published in Brewe et al. are not
included in this current analysis, more recent data from the 40

same instructor are part of this set (as instructor 1). Further,
data from classes taught by Brewe are included in this 20

Overall shift
data set.
In this analysis, we are primarily concerned with the 0
shifts in the overall CLASS profile, and are not including
an in-depth analysis of the categories. Also, we are only −20
looking at shifts in favorable responses as has been done
in recent analyses [11,25,34]. Finally, we calculate
−40
effect sizes and confidence intervals on these effects as
a way to provide data that are comparable across other
−60
studies. The effect size we use is Cohen’s d, calculated
according to Eq. (1). The 95% confidence intervals on the
effect can be calculated according to Smithson [35]: 1 2 3 4 5 6
Instructor

post
pre
d¼ : (1) FIG. 3. Box plot of the shift in overall percent favorable
pooled
responses for each of the instructors in the new data set. The
Here,
represents the mean overall percent favorable box width is scaled by the square root of n for each instructor.
responses for pre- and postinstruction, and  is the pooled In each box, the thick center line indicates median shift, and
the lower and upper bounds represent the first and third
standard deviation for both sets.
quartiles, respectively. The whiskers show the extent of the
remaining data out to 1:5 the box size; points outside this
V. RESULTS AND ANALYSIS range are marked as circles, and represent students with very
large shifts.
Five of the six instructors who implemented the
University Modeling Instruction curriculum and pedagogy
on the effects for each of the instructors. These are
achieved significant positive shifts in the overall favorable
plotted in Fig. 2 and range from d ¼ 0:08 for instructor
responses from preinstruction to postinstruction, as seen in
5 to d ¼ 0:95 for instructor 4. Based on these effects,
Figs. 2 and 3.
students of all instructors—except instructor 5—showed
In order to identify the magnitudes of the shifts, we
positive shifts on the overall CLASS score. Instructor 5
calculated Cohen’s d and the 95% confidence intervals
showed no shift as indicated by the effect of d ¼ 0:08
and the confidence intervals including d ¼ 0. When data
1.5

from all instructors is aggregated and effect size and

confidence intervals on the effect are calculated, we find
an effect size of d ¼ 0:45, with a 95% confidence interval
of (0.26–0.64).
1.0

Although we do not include an analysis of the shifts

by category, Appendix A gives the results for all categories
Cohen’s d

as well as the aggregated data by category to allow

for meta-analysis of these results. These results indicate
0.5

significant positive shifts in all categories for aggregated

data.
These data do not indicate that all instructors imple-
menting Modeling Instruction would necessarily achieve
0.0

positive shifts on the CLASS, as instructor 5 showed no

positive shift. We do see consistent positive shifts overall
across the majority of instructors, including instructors
with varying levels of experience with the University
1 2 3 4 5 6 Modeling Instruction curriculum and pedagogy. Further,
Instructor
we see positive shifts across eight of nine sections in this
data set. The consistency of these results suggests that the
FIG. 2. Plot of Cohen’s d by instructor. Error bars reflect the University Modeling Instruction curriculum and pedagogy
95% confidence interval on the effect. support the development of more favorable attitudes

020116-5
BREWE et al. PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

toward learning physics, independent of instructor. The we anticipate larger sections as classroom space becomes
null result from instructor 5 warrants further attention. available. Future research will probe whether the positive
attitudinal shifts continue in classes 2 to 3 times the size of
those reported here.
VI. DISCUSSION
The results presented indicate a consistent pattern of A. Considering mechanisms to explain positive shifts
positive attitudinal shifts across a variety of instructors.
Positive shifts are rare among research reported on A classroom is a complex learning environment influ-
calculus-based introductory physics classes. This compi- enced by a multitude of internal and external factors.
lation of positive shifts, especially when contrasted with Claims that a particular measured result is due to any one
negative shifts commonly reported, leads us to attribute of these factors, such as curriculum, must be made with
the shifts to the Modeling Instruction curriculum and great caution. However, by comparing our results with
pedagogy. The commonalities across these courses pro- other published studies of students’ attitudinal shifts, we
vide insight into the factors that could mechanistically can eliminate some factors as likely sources, and
explain positive attitudinal shifts. However, the existence strengthen the possibility of attribution to others.
of these results alone is not adequate to draw causal
conclusions regarding to what specifically the shifts 1. Unique class and instructor
should be attributed. Further, we should attend carefully The Brewe et al. [11] results come from one fall-spring
to instructor 5 who shared many commonalities with the sequence of a course, with one instructor. The MI-New
other instructors, but showed nonsignificant positive atti- data presented here, spanning five years and five additional
tudinal shifts. instructors (Fig. 3), drastically reduce the possibility that
The most obvious commonality among the instructors positive CLASS shifts arose from a ‘‘good semester’’ or
is the use of the Modeling Instruction curriculum mate- any unique expertise of the professor. Further, these data
rials and curriculum guide. Modeling Instruction has suggest that the curriculum and the pedagogy that are
bounded this investigation and most clearly distinguishes conveyed by the Modeling Instruction instructor’s guide
the participants in this study from others who have not are replicable, even by novice instructors. The role that the
demonstrated positive shifts. Attributing positive shifts to weekly planning meetings play in the implementation of
a broad feature such as implementation of a curriculum the curriculum or pedagogy seems to be important. This
and pedagogy is unsatisfying, as it does not clearly preliminary finding fits well with research on dissemina-
identify specific features of the implementation that tion of transformed curricula by Henderson et al. [36]
lead to the positive shifts. Yet the curriculum and peda- and on the characteristics of high quality professional
gogy are the features that both unite the instructors in this development [37].
study and distinguish them from others who have not
demonstrated positive attitudinal shifts. In the second 2. Class size
half of this section, we provide some suggestions on
The size element of the instructional environment is less
what aspects of the Modeling Instruction learning envi-
easily dismissed. All of the MI sections in this paper have a
ronment are particularly relevant to improving student
maximum enrollment of 30. This small class size is a
attitudes.
common feature of several reported positive CLASS
Instructor 5 also implemented the Modeling Instruction
results—in Otero, de la Garza et al., and Lindsey et al.
curriculum and pedagogy, but as with all implementations,
[12,25,34], there is one section of 100 students but all
variations naturally occur. One major variation in imple-
others are 50 students or smaller, with 30 a more typical
mentation is that instructor 5 was unable to participate
size. Only Redish and Hammer’s positive MPEX-II results
in the weekly Modeling planning meetings. This could
come systematically from large courses (100–200 stu-
plausibly have had impacts on implementation. Instructor
dents) [38]. While Redish and Hammer provide counter-
5 missed out on opportunities to learn from the shared
evidence, it should not be dismissed that the majority of
experiences of other instructors and on discussion of the
classes showing positive CLASS shifts are small enroll-
conceptual and epistemological resources that would be
ment classes. Some Modeling Instruction curriculum
valuable in implementation. Either way, the difference is
features such as consensus-reaching board meetings are
suggestive, and we will in future work pay closer attention
currently embedded in the small-class structure; as noted
to the value of the planning meetings.
above, it remains to be seen if they will scale successfully
A second commonality is that Modeling Instruction at
to a larger course.
FIU is implemented in classes of 30 students. It is difficult
to assume that the small class size, which promotes close
faculty and student interaction, does not contribute to 3. Epistemological framing of class
improved attitudes. The Modeling classes at FIU are Modeling Instruction is built on an explicit epistemolog-
limited to 30 students due to space constraints; however, ical foundation and the curricular materials and pedagogy

020116-6
EXTENDING POSITIVE CLASS RESULTS ACROSS . . . PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

are designed to promote the use of productive conceptual expertlike response on this statement would reflect a combi-
and epistemological resources [26]. This epistemological nation of expectations about learning and self-efficacy. (This
focus is one plausible mechanism for the consistent positive interpretation seems consistent with students’ ability to pre-
shifts across implementations of Modeling Instruction. dict expert CLASS responses even when they do not share
In this scenario, students who engage in model building, them [22]—a student might anticipate that a physicist could
validation, and revision have authentic scientific experien- memorize the requisite information, even if the student could
ces. These experiences promote certain attitudes about not.) Modeling Instruction has been shown to have either no
learning physics: that it is not simply about memorizing change or positive shifts on self-efficacy beliefs, whereas
formulas, and that models in physics are coherent, con- standard lecture courses shift self-efficacy beliefs negatively
structed by students (and scientists), and subject to change. [32,39]. This pattern of Modeling Instruction demonstrating
These attitudes are more aligned with expert attitudes about positive shifts and lecture demonstrating negative shift on
learning in science, possibly leading to positive shifts on the self-efficacy instruments is reflective of typical CLASS
CLASS. results. This explanation suggests that studying the self-
This scenario is consistent with other positive shifts on efficacy beliefs embedded in the CLASS statements and
the CLASS and other attitude surveys when explicit atten- how self-efficacy is shifted in other classroom contexts
tion to epistemology was a guiding theme of the course (e.g., the Physics by Inquiry setting) is a viable candidate
[24,38]. However, the positive shifts obtained by Lindsey for deeper understanding of CLASS shifts.
et al. [25] used Physics by Inquiry, which they emphasize
does not include any explicit epistemological framing, but VII. CONCLUSION
instead includes an implicit focus on epistemology. More We close by returning to the larger picture, considering
generally, one aspersion on the epistemological framing the relevance of attitudinal results in physics education
argument is that many transformed physics classes have an research. In this subfield, as in education more generally,
epistemological component, either explicit or implicit. The the past decades have seen growing awareness that no list
prevalence of negative or null shifts in transformed of facts can encompass mastery of a discipline. But even
courses, which presumably share epistemological features, deeper conceptual understanding, while necessary and
is perhaps surprising and a counter argument to the epis- desirable for that goal, presents an incomplete picture.
temological framing of the class as an explanatory mecha- Skills such as scientific reasoning, experimental design,
nism. We suggest that the theoretical tools available to the and critical evaluation of results have achieved recognition
physics education research community for characterizing as teachable and measurable aspects of physics. Our results
these aspects of curriculum are still developing, and a contribute to a growing body of evidence that goes further
common language is not yet decided [3]. Documentation to address the set of motivations and beliefs that drive and
of curricular features that address epistemological issues, mediate students’ learning efforts.
and how and why they do so, is crucial to furthering Evidence links student attitudes to their interest and persis-
understanding of their impact on student attitudes toward tence in the discipline (cf. Refs. [15,16]). The consistent
learning physics. positive CLASS results at FIU are especially powerful in
combination with our concurrent consistent increase in the
4. Self-efficacy number of physics majors and the support mechanisms for a
One final, albeit speculative, potential explanatory student community of physics [29,40,41]. We have attempted
mechanism is in the CLASS instrument itself. The ‘‘learn- above to highlight features of Modeling Instruction which
ing attitudes’’ of the CLASS title may be an umbrella term may contribute to this pattern of positive shifts, and to rule out
for various, more specific things like expectations, self- some confounding factors. Moving forward, key issues are to
efficacy, and affect. Self-efficacy is a person’s confidence explore the question of class size and to continue to continue
in their ability to accomplish some particular goal articulating the epistemological features of classroom and
(e.g., confidence in their ability to learn physics). This instructional environments.
explanation would allow positive shifts to reflect both
apprehension prior to the class and then a relief from that ACKNOWLEDGMENTS
apprehension following instruction. Consider the CLASS This work has been supported by the National Science
statement, ‘‘A significant problem in learning physics is Foundation, (NSF No. 0802184, DUE No. 1140706) and
being able to memorize all the information I need to by the Howard Hughes Medical Institute Grant
know.’’ This statement includes an expectation about the No. 52006924. We also thank Vashti Sawtelle, Renee
nature of learning physics, that ‘‘. . .learning physics is Michelle Goertzen, Idaykis Rodriguez, Jared Durden, and
being able to memorize all the information I need to Sean Stewart for their efforts developing the curriculum
know.’’ It is plausible that this may cause the student to materials and instructor’s guides. Finally, we acknowledge
consider her confidence in her ability to memorize all the the work of Dwain Desbein as a curriculum development
information, which is a statement of self-efficacy. Thus, an partner.

020116-7
BREWE et al. PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

APPENDIX
Although CLASS statistics by category are not essential to the main point of this paper, they are commonly reported in
the literature for the CLASS instrument. We include them in Fig. 4 and Table II to facilitate meta-analysis and comparison
with other results.

75
Overall Favorable C−LASS %

50

25

0
All categories

Personal Interest

RW Connection

PS General

PS Confidence

PS Sophistication

Sense−Making/Eff.

Concept. Connect.

Applied Concept.
Overall

C−LASS Categories

FIG. 4 (color online). MI-New CLASS data (N ¼ 221) for all categories. Blue and red represent the pre- and postcourse
percentages, respectively, of overall favorable responses. The error bars show standard error for pre and post.

TABLE II. MI-New CLASS data (N ¼ 221) for all categories
standard error of the mean.
Category Pre Post Shift
Overall 63:8
1:0 70:8
1:1 6:9
1:0
All categories 64:8
1:2 72:8
1:3 7:9
1:2
Personal interest 68:9
1:8 74:5
1:8 5:6
1:7
Real-world connection 79:7
1:8 83:8
1:6 4:2
1:9
Problem solving–general 70:8
1:4 76:4
1:6 5:5
1:6
Problem solving–confidence 71:0
1:6 77:2
1:9 5:9
2:0
Problem solving–sophistication 46:5
1:9 61:3
2:0 14:8
2:0
Sense-making or effort 78:0
1:3 82:4
1:5 4:3
1:6
Conceptual connections 56:1
1:7 67:2
1:8 11:1
2:0
Applied conceptual understanding 42:6
1:6 57:8
1:8 15:1
1:9

020116-8
EXTENDING POSITIVE CLASS RESULTS ACROSS . . . PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

[1] National Research Council, Discipline-Based Education [18] M. Schommer, A. Crouse, and N. Rhodes,
Research: Understanding and Improving Learning in Epistemological beliefs and mathematical text compre-
Undergraduate Science and Engineering (National hension: Believing it is simple does not make it so, J.
Academies Press, Washington, DC, 2012), pp. 1–231. Educ. Psychol. 84, 435 (1992).
[2] J. L. Docktor and J. P. Mestre, A synthesis of discipline- [19] L. Lising and A. Elby, The impact of epistemology on
based education research in physics, National Research learning: A case study from introductory physics, Am. J.
Council White Paper, 2011. Phys. 73, 372 (2005).
[3] A. Elby, Getting started with research on epistemologies [20] I. A. Halloun, Views about science and physics achieve-
and expectations, in Getting Started in PER, Vol. 2, http:// ment: The VASS story, AIP Conf. Proc. 399, 605 (1997).
www.per-central.org/items/detail.cfm?ID=10578. [21] A. Elby, ‘‘Epistemological beliefs assessment for physical
[4] D. Hestenes, M. Wells, and G. Swackhamer, Force con- science’’ (unpublished).
cept inventory, Phys. Teach. 30, 141 (1992). [22] K. Gray, W. Adams, C. Wieman, and K. Perkins, Students
[5] R. K. Thornton and D. R. Sokoloff, Assessing student know what physicists believe, but they don’t agree: A
learning of Newton’s laws: The force and motion con- study using the CLASS survey, Phys. Rev. ST Phys. Educ.
ceptual evaluation and the evaluation of active learning Res. 4, 020106 (2008).
laboratory and lecture curricula, Am. J. Phys. 66, 338 [23] A. Elby, Helping physics students learn how to learn, Am.
(1998). J. Phys. 69, S54 (2001).
[6] E. F. Redish, J. M. Saul, and R. N. Steinberg, Student [24] V. Otero and K. E. Gray, Attitudinal gains across
expectations in introductory physics, Am. J. Phys. 66, multiple universities using the Physics and Everyday
212 (1998). Thinking curriculum, Phys. Rev. ST Phys. Educ. Res. 4,
[7] W. Adams, K. Perkins, N. Podolefsky, M. Dubson, N. 1 (2008).
Finkelstein, and C. Wieman, New instrument for measur- [25] B. Lindsey, L. Hsu, H. Sadaghiani, J. Taylor, and K.
ing student beliefs about physics and learning physics: Cummings, Positive attitudinal shifts with the Physics
The Colorado Learning Attitudes about Science Survey, by Inquiry curriculum across multiple implementations,
Phys. Rev. ST Phys. Educ. Res. 2, 010101 (2006). Phys. Rev. ST Phys. Educ. Res. 8, 8 (2012).
[8] I. Halloun and D. Hestenes, Interpreting VASS dimensions [26] E. Brewe, Modeling theory applied: Modeling instruction
and profiles for physics students, Sci. Educ. 7, 553 (1998). in introductory physics, Am. J. Phys. 76, 1155 (2008).
[9] R. Hake, Interactive-engagement versus traditional meth- [27] I. A. Halloun, Modeling Theory in Science Education
ods: A six-thousand-student survey of mechanics test data (Kluwer, Dordrecht, 2004).
for introductory physics courses, Am. J. Phys. 66, 64 [28] E. Brewe, Energy as a substancelike quantity that flows:
(1998). Theoretical considerations and pedagogical consequences,
[10] S. J. Pollock and N. D. Finkelstein, Sustaining educational Phys. Rev. ST Phys. Educ. Res. 7, 020106 (2011).
reforms in introductory physics, Phys. Rev. ST Phys. [29] E. Brewe, L. Kramer, and V. Sawtelle, Investigating stu-
Educ. Res. 4, 010110 (2008). dent communities with network analysis of interactions in
[11] E. Brewe, L. Kramer, and G. O’Brien, Modeling instruc- a physics learning center, Phys. Rev. ST Phys. Educ. Res.
tion: Positive attitudinal shifts in introductory physics 8, 010101 (2012).
measured with CLASS, Phys. Rev. ST Phys. Educ. Res. [30] D. Desbien, ‘‘Modeling discourse management compared to
5, 013102 (2009). other classroom management styles in university
[12] J. de la Garza and H. Alarcon, Assessing students’ atti- physics,’’ Ph.D. dissertation, Arizona State University, 2002.
tudes in a college physics course in Mexico, AIP Conf. [31] J. Durden, E. Brewe, L. Kramer, N. S. Rebello, P. V.
Proc. 1289, 129 (2010). Engelhardt, and C. Singh, ‘‘Implicit action’’: Understanding
[13] J. D. House, Student motivation and achievement in col- discourse management in modeling instruction, AIP Conf.
lege chemistry, Int. J. Instr. Media 21, 1 (1994). Proc. 1413, 187 (2012).
[14] J. D. House, Student motivation, previous instructional [32] V. Sawtelle, E. Brewe, and L. H. Kramer, Positive impacts
experience, and prior achievement as predictors of per- of modeling instruction on self-efficacy, AIP Conf. Proc.
formance in college mathematics, Int. J. Instr. Media 22, 1289, 289 (2010).
157 (1995). [33] Analysis template is available from http://www
[15] K. K. Perkins, M. M. Gratny, W. K. Adams, N. D. .colorado.edu/sei/class/.
Finkelstein, and C. E. Wieman, Towards characterizing [34] V. Otero, S. Pollock, and N. Finkelstein, A physics depart-
the relationship between students’ self-reported interest ment’s role in preparing physics teachers: The Colorado
in and their surveyed beliefs about physics, AIP Conf. learning assistant model, Am. J. Phys. 78, 1218 (2010).
Proc. 818, 137 (2006). [35] M. Smithson, Confidence Intervals (SAGE Publications,
[16] K. Perkins and M. Gratny, Who becomes a physics major? Inc., Thousand Oaks, CA, 2003).
A long-term longitudinal study examining the roles of pre- [36] C. Henderson, A. Beach, and N. Finkelstein, Facilitating
college beliefs about physics and learning physics, inter- change in undergraduate STEM instructional practices:
est, and academic achievement, AIP Conf. Proc. 1289, 253 An analytic review of the literature, J. Res. Sci. Teach.
(2010). 48, 952 (2011).
[17] M. Schommer, Effects of beliefs about the nature of [37] L. M. Desimone, Improving impact studies of teachers’
knowledge on comprehension, J. Educ. Psychol. 82, 498 professional development: Toward better conceptualiza-
(1990). tions and measures, Educ. Res. 38, 181 (2009).

020116-9
BREWE et al. PHYS. REV. ST PHYS. EDUC. RES. 9, 020116 (2013)

[38] E. F. Redish and D. Hammer, Reinventing college physics [40] R. M. Goertzen, E. Brewe, and L. Kramer, Expanded
for biologists: Explicating an epistemological curriculum, markers of success in introductory university physics,
Am. J. Phys. 77, 629 (2009). Int. J. Sci. Educ. 35, 262 (2013).
[39] V. Sawtelle, E. Brewe, and L. H. Kramer, Exploring [41] R. Goertzen, E. Brewe, L. Kramer, L. Wells, and D. Jones,
the relationship between self-efficacy and retention in Moving toward change: Institutionalizing reform through
introductory physics, J. Res. Sci. Teach. 49, 1096 implementation of the Learning Assistant model and Open
(2012). Source Tutorials, Phys. Rev. ST Phys. Educ. Res. 7, 9 (2011).

020116-10