Cbe.12 11 0201

CBE—Life Sciences Education
Vol. 12, 59–72, Spring 2013
Article
CREATE Cornerstone: Introduction to Scientific Thinking, a

New Course for STEM-Interested Freshmen, Demystifies
Scientific Thinking through Analysis of Scientific Literature
Alan J. Gottesman and Sally G. Hoskins
City College and the Graduate Center, City University of New York, New York, NY 10031
Submitted November 21, 2012; Revised December 15, 2012; Accepted December 15, 2012
Monitoring Editor: Jose Vazquez
The Consider, Read, Elucidate hypotheses, Analyze and interpret data, Think of the next Experiment
(CREATE) strategy for teaching and learning uses intensive analysis of primary literature to im-
prove students’ critical-thinking and content integration abilities, as well as their self-rated science
attitudes, understanding, and confidence. CREATE also supports maturation of undergraduates’
epistemological beliefs about science. This approach, originally tested with upper-level students,
has been adapted in Introduction to Scientific Thinking, a new course for freshmen. Results from this
course’s initial semesters indicate that freshmen in a one-semester introductory course that uses a
narrowly focused set of readings to promote development of analytical skills made significant gains
in critical-thinking and experimental design abilities. Students also reported significant gains in their
ability to think scientifically and understand primary literature. Their perceptions and understand-
ing of science improved, and multiple aspects of their epistemological beliefs about science gained
sophistication. The course has no laboratory component, is relatively inexpensive to run, and could
be adapted to any area of scientific study.
INTRODUCTION to attract and retain students who might be considering sci-

ence studies would be to give them an opportunity to de-
We think a significant number of students lose interest in velop their reading and analytical skills and gain a realistic
studying science early in their college careers, because many sense of scientific thinking as soon as they started college. A
science curricula do not promote open-ended discussion, crit- Consider, Read, Elucidate hypotheses, Analyze and interpret
ical analysis, and creative study design—activities that char- data, Think of the next Experiment1 (CREATE)-based course
acterize science as it is practiced. We thought that one way focused on scientific thinking, using a novel selection of read-
ings whose analysis did not require years of content mastery,
would, in principle, give freshmen a chance to engage deeply
DOI: 10.1187/cbe.12-11-0201
Address correspondence to: Sally G. Hoskins (shoskins@ccny.cuny in activities characteristic of actual science practice. We hy-
.edu). pothesized that such an experience could have a positive
This project addresses a new course developed by S.G.H. influence on students’ scientific abilities, their attitudes to-
based upon work supported by the National Science Foundation ward science, and their understanding of the research process
(NSF) CCLI/TUES (Course, Curriculum and Laboratory Improve- early in their academic careers. To test this idea, we devel-
ment/Transforming Undergraduate Education in Science) program oped a new elective, Biology 10050: Introduction to Scientific
under NSF grant 0942790. Any opinions, findings, and conclusions or
recommendations expressed in this material are those of the authors Thinking.
and do not necessarily reflect the views of the NSF.
c 2013 A. J. Gottesman and S. G. Hoskins. CBE—Life Sciences Edu-
cation
c 2013 The American Society for Cell Biology. This article is BACKGROUND
distributed by The American Society for Cell Biology under license
from the author(s). It is available to the public under an Attribution– Biology 10050 was developed as an adaptation of an upper-
Noncommercial–Share Alike 3.0 Unported Creative Commons Li- level course, Biology 35500: Analysis of Scientific Literature
cense (http://creativecommons.org/licenses/by-nc-sa/3.0).
“ASCB R
” and “The American Society for Cell Biology R
” are regis-
tered trademarks of The American Society for Cell Biology. 1 See Hoskins et al., 2007.
59
A. J. Gottesman and S. G. Hoskins
with CREATE. That course, offered at City College of New on broader scientific issues, including interlaboratory com-
York (CCNY) since 2004, aims to demystify and humanize petition, peer review, and the factors that might influence
science through intensive analysis of primary literature. In principal investigator (PI) decisions about what direction to
Biology 35500, “modules”—sets of journal articles published take next.
sequentially from single laboratories—are the focus for an Late in the semester, students, as a class, develop a list of
intensive elective. Students learn a new set of pedagogical 10–12 questions for paper authors. These are emailed as a sin-
approaches, including concept mapping, cartooning of gle survey to each author (PIs, postdocs, graduate students).
methodology, figure annotation, use of templates to parse Many authors reply with thoughtful comments about their
experimental logic, and design of follow-up studies (Hoskins own paths to science, their motivations, and their lives be-
and Stevens, 2009; Hoskins, 2010b). These methods are ap- yond the laboratory. Discussion of authors’ varied responses
plied first to an article from the popular press and then in the complement the in-class data analysis with insight into the
analysis of a series of primary literature papers that follow a lives and motivations of “the people behind the papers.”
particular scientific question (e.g., “How do axons find their Our upper-level course led to gains in students’ content
targets in the embryo?,” “How is axis polarity maintained integration and critical-thinking ability, as well as in their
during regeneration?”). By examining module articles in a self-assessed learning gains (Hoskins et al., 2007). We also
stepwise manner, we develop a “lab meeting” atmosphere found that undergraduates’ self-assessed science abilities, at-
in the class, with experimental findings discussed as if they titudes, and epistemological beliefs changed during the CRE-
had been generated gradually by the students themselves. ATE semester (Hoskins et al., 2011). Upper-level students’
Within individual articles, every figure or table was analyzed postcourse interviews (see Tables 1 and S1 in Hoskins et al.,
with recognition that each specific question being addressed 2007), as well as conversations with alumni of Biology 35500
or question asked created a data subset that contributed to (“You have to do a version of this for freshmen—it changed
the major finding of the paper. how I read everything” and “If I had known sooner that
In CREATE class sessions, multiple aspects of study design research wasn’t boring, I might have joined an undergrad
are scrutinized closely as we work backward from data in research program”) inspired us to consider adapting upper-
each figure and table to reconstruct details of the particular level CREATE for freshmen.
experiment that generated those data before we analyze the A related motivation for developing the CREATE Corner-
findings. In the process of examining specific experiments stone course was that the biology department at CCNY, like
and their outcomes, we repeatedly consider questions funda- its counterparts elsewhere, loses many would-be majors dur-
mental to much research, (e.g., “What is n?,” “How was the ing the early years of the biology curriculum. Some students
sample selected?,” “What controls were done and what did who start with the intention of declaring a biology major
each control for?,” “How do the methods work?,” “What is do not follow through. Others who do choose biology later
the basis of ‘specificity’ in staining, binding, or expression?,” change majors and leave science altogether, with multiple fac-
“How convincing are the data?”). In addressing such ques- tors likely playing a role. Students may be poorly prepared
tions, students gain insight into the design and interpreta- for college-level science, feel overwhelmed by the amount of
tion of research beyond the individual study under analysis. new information covered in the introductory-level courses
Because methods are examined in terms of fundamental bio- (Seymour and Hewitt, 1997), or be discouraged by textbooks’
logical and chemical properties (e.g., “What makes antibodies depiction of biology as a largely descriptive science (Dun-
‘specific’?,” “Do antibody probes bind the same way that ribo- can et al., 2011). Nationwide, some students get the impres-
probes do?,” “How can you tell whether a particular stem cell sion from the laboratory components of introductory biol-
undergoes division after injury to an organism?”), students ogy, chemistry, or physics classes that lab work is routine,
review fundamental content from previous course work in predictable, and boring.
a new context. By considering “evolution of methodology” We felt that a CREATE Cornerstone course focused on sci-
(e.g., differential screening of cDNA libraries vs. gene chip entific thinking could support and build students’ science
analysis vs. RNAseq approaches; gene knockout vs. RNA in- interest at an early phase of their academic careers. In part,
terference) students become aware of the pace of technique adapting upper-level CREATE for freshmen might benefit
development and how the range of tools available may influ- students by teaching them a variety of techniques (the CRE-
ence the nature of questions asked. In this way, Biology 35500, ATE toolkit; Hoskins and Stevens, 2009) that make complex
the original CREATE course, involves both close analysis of material more accessible and understandable. At the same
papers presented in their original sequence as an individual time, the course seeks to provide students with an inside look
“module” but also consideration of broader nature of sci- at the workings of real-world biology research labs and the di-
ence issues. For example, discussion centered on the fact that versity and creativity of the scientists who work in them. We
what used to be considered “junk” DNA is now recognized hypothesized that students in such a course would become
as having a key role in microRNA pathways illustrates the more adept at thinking critically about scientific material and
malleability of scientific knowledge. at designing and interpreting experiments—key strategic foci
After completing analysis of each paper, and before mov- of the CREATE approach. In addition, we hypothesized that
ing to the next paper in the series, students create their own students would gain in their abilities to critically analyze
follow-up experiments, thereby building experimental de- scientific writing, deepen their understanding of the nature
sign skills, as well as awareness that a given study could, of science, and develop more mature epistemological beliefs
in principle, move forward in a variety of ways. Students’ about scientific knowledge. We also suspected that some stu-
proposed follow-ups are vetted in a grant panel exercise de- dents who had not considered careers in research, or others
signed to mimic activities of bona fide panels (see Hoskins who had but quickly rejected the idea, would consider re-
et al., 2007). In turn, these sessions lead to discussion focused search more positively as their college education progressed.
60 CBE—Life Sciences Education

CREATE Freshmen—Thinking/Attitude Gains
Introduction to Scientific Thinking is a three-credit, one- course versions of the EDAT present different scenarios. Pre-
semester elective for first-year college students with a de- course, students read a paragraph presenting the claim that
clared interest in science, technology, engineering, and math the herb ginseng enhances endurance; postcourse, the se-
(STEM) disciplines at the CCNY, a minority-serving institu- lected text alleged that iron supplements boost memory. The
tion. The course meets twice-weekly for 75 min/session, and EDAT survey was scored separately by two investigators fol-
on our campus is taken before the introductory-level courses lowing the scoring rubric created and explained in Sirum
in any of the basic sciences. The goal is to develop the science- and Humburg (2011). After the individual scoring, any dis-
related reading and analytical skills of freshmen by using the crepancies were discussed and reconciled. Tests for statisti-
CREATE strategy to critically evaluate a number of recent and cal significance were performed using the Wilcoxon signed-
ongoing research studies. Ideally, the experience should also rank test (http://vassarstats.net/index.html; Arora and
encourage students to persist in STEM disciplines, participate Malhan, 2010). Effect sizes (Cohen, 1992; Coe, 2002) were also
in undergraduate research experiences (UREs) in later years, determined.
and consider research as a career choice.
Survey of Student Self-Rated Abilities, Attitudes, and Be-
At CCNY, first-year students cannot declare a biology ma-
liefs (SAAB). To investigate students’ reactions to the CRE-
jor. The course is thus aimed at presumptive biology ma-
ATE course, we asked them to complete the SAAB. In this
jors and in principle could be taken concomitantly with the
Likert-style survey, students reported their degree of agree-
standard introductory biology (or other science) course. On
ment on a 5-point scale (range: strongly disagree to strongly
campuses where students can or must declare a major in
agree) with a series of statements concerning their attitudes,
the first year, this course would be appropriate for students
self-rated abilities, and beliefs about analyzing scientific lit-
who evince interest in biology studies. The data reported
erature; the research process; the nature of scientific knowl-
here address changes in Biology 10050 students’ critical-
edge; and scientists and their motivations. The surveys were
thinking/experimental design abilities and in their attitudes
identical precourse and post course, and used statements
and beliefs about science. The question of student persistence
whose derivation and description is described in Hoskins
in STEM and participation in undergraduate research projects
et al. (2011). Students were given 20 min to complete the
will be tracked in upcoming semesters.
survey. For statistical analysis, all response scores were ag-
gregated into their appropriate categories (see Supplemental
METHODS AND ASSESSMENT TOOLS Material for derivation of categories) and changes precourse
to postcourse were analyzed for statistical significance using
Participants in this study were first-year students at CCNY the Wilcoxon signed-rank test. Because these data and those
who enrolled in the semester-long Biology 10050: Introduc- of the EDAT are nonparametric (a score of “4” is not twice as
tion to Scientific Thinking course during Fall 2011 and Spring good as a score of “2,” for example) and noncontinuous, the
2012. In each semester, at the first class session, students were signed-rank test was deemed an appropriate analytical tool
invited to participate anonymously in our study on a volun- (Arora and Malhan, 2010).
tary basis that had no bearing on class grade. Precourse data The SAAB data for the Biology 10050 class include pooled
were collected during the first few classes and postcourse data results from the Fall and Spring sections (18 and 13 partici-
in the final class session of the semester. All participating stu- pating students, respectively). Data collected using the same
dents were asked to devise a “secret code” number known survey, administered in the same manner, were also obtained
only to them and to use this code on all surveys. Identifying from one contemporaneous section of the upper-level CRE-
surveys in this way allowed us to compare individual and ATE course (Biology 35500, 21 students; two meetings per
group scores pre- and postcourse, while preserving student week for 100 min/session). Additionally, the SAAB survey
anonymity (Hoskins et al., 2007). was administered to volunteers in a course in Organismic Bi-
ology (general physiology, 23 students; one 100-min lecture
Critical Thinking Assessment Test (CAT). Students in the Fall and one 3.5-h lab session/wk), none of whom had taken a
cohort of Biology 10050 completed the CAT (Stein et al., 2012). CREATE class. This group was not a matched-control pop-
In the CAT, which is a reliable and valid test of critical think- ulation (students were not freshmen). Rather, data from this
ing, students spent 1 h reading a number of informational cohort of students provided insight into potential changes
passages and writing responses to a variety of prompts ask- in attitudes, abilities, and epistemological beliefs that might
ing them to evaluate the information and draw conclusions. happen naturally during the course of a semester in a non-
The same test was taken again at the end of the semester. The CREATE science class. The CREATE classes were taught by
CAT tests were graded and analyzed statistically (Student’s the same instructor (S.G.H.); the Organismic Biology class
t test) by a scoring team at Tennessee Tech University, where was taught by a colleague not otherwise involved in this
this survey was created. study. Both instructors were experienced at teaching their
respective courses.
Experimental Design Ability Test (EDAT). Students in both
cohorts of Biology 10050 also completed the EDAT, the re- Student Comments on Author Emails. To gain insight into
liability and validity of which have been established by the students’ reactions to author email responses, we assigned
EDAT developers (Sirum and Humburg, 2011). In the EDAT, students to read and annotate the responses as they were
students were presented with a claim and challenged to “pro- received. Students included the responses in their note-
vide details of an investigative design” and indicate the ev- books/portfolios, with marginal notes indicating which as-
idence that would help them decide whether to accept the pects of each response they found most surprising and/or in-
claim. Students were given 15 min to respond to a writ- teresting. In the Spring session of Biology10050, we included
ten prompt that described the assertion. Precourse and post- a question on a late-semester (in-class, open-book) exam,
Vol. 12, Spring 2013 61

asking students whether the emails changed their ideas about CREATE Cornerstone Objectives and Selected
science research or scientists. We compiled responses and an- Exercises
alyzed them for repeated themes. Students learned to use CREATE tools, including concept
mapping, paraphrasing, cartooning, annotating figures, ap-
Student Participation. The CREATE study was approved plying templates to parse experimental logic, designing
by CUNY Institutional Review Board (Exemption category follow-up experiments, and participating in grant panels
1 and 2). Of the students in Bio 10050, 69% were female and (Hoskins and Stevens, 2009). The CREATE techniques aim to
59% were members of minority groups currently underrep- sharpen students’ analytical skills and build critical-reading
resented in academic science. Students were invited, in week habits that can be used in new situations. These approaches
1 of class, to anonymously participate in an education study also build students’ metacognition—the ability to track their
with the goal of “improving undergraduate education in sci- own understanding (Tanner, 2012). To construct a concept
ence.” Participation was optional and the instructor noted map successfully, for example, students need to understand
that student participation or nonparticipation had no bear- individual ideas and discern the relationships between them.
ing on course grade or any other relationships with CCNY. To sketch a cartoon that shows what took place in the lab
There were no points or extra credit awarded for participa- to generate the data presented in a particular figure, stu-
tion. We think that students who participated were motivated dents must make sure they understand the relevant method-
by the chance to take part in a science education study and/or ology. We applied concept mapping and cartooning along
to be part of a scientific experiment. with other CREATE tools to a novel combination of readings.
Articles selected for Biology 10050 were chosen because of
their topicality, relatively simple methodology, and aspects
CURRICULAR DESIGN of each that provoked controversy, exemplified the role of
Adapting CREATE for Freshmen controls, and/or highlighted important distinctions between
data and their interpretation. Goals for the Cornerstone stu-
In the original (upper-level) CREATE course, the class stud-
dents included learning: to read with skepticism, to critically
ied, sequentially, a series of papers published by a single lab
analyze data and generate alternative interpretations, to rec-
that tracked the development of understanding in a partic-
ognize the malleability of scientific knowledge, and to de-
ular field of scientific inquiry (e.g., how embryonic retinal
velop and evaluate experiments with particular emphasis on
axons find their targets in the brain; how planaria maintain
controls and their roles. A final goal was for students to de-
positional information during regeneration). For the fresh-
velop a more realistic view of research and researchers than
men, we changed the types of articles studied, using popular
the one often promoted in popular culture.
press articles and a wider range of scientific literature, but ap-
plied the same overall CREATE teaching/learning strategies.
The freshmen initially read and analyzed numerous popu- Developing an Appropriately Skeptical Reading Style
lar press stories based on journal articles. We also read a The class sessions were typically run as discussions or debates
variety of newspaper and magazine pieces describing sci- about points that arose in the assigned readings. We rarely
entific investigations or researchers. These warm-up exer- presented all the information at once, instead examining each
cises, used more extensively for the freshmen than in upper- reading in stages. For example, one unit early in the semester
level CREATE, started students toward developing the skills used an op-ed in the New York Times claiming that iPhone
they would need for reading and analyzing primary litera- owners experienced “love” for their phones and outlining
ture later in the semester. All the readings (in all CREATE study outcomes that purported to support this conclusion
courses) are actual texts as originally published. In some (Lindstrom, 2011). We also read a published refutation of the
cases, we read only parts of papers, but we did not rewrite op-ed signed by 44 neuroscientists (Poldrack, 2011a), and the
or simplify any of the material. The freshmen ultimately original version of the refutation letter before it was edited
read a pair of papers published in sequence that addressed by the New York Times (Poldrack, 2011b). We started with
a subject—the ability of infants to recognize and judge the the op-ed and only later distributed the challenge from the
social actions of others—related to a number of the shorter neuroscience community, considering:
readings.
Toward the end of the semester, the freshmen, as a class, How, in principle, would one determine “the most appeal-
composed a list of 10–12 questions about the studies we had ing sounds in the world,” whether babies “automatically”
read, “research life,” and the researchers themselves. These swipe iPhones expecting a response, or whether “love” is
questions were emailed as a single survey to each paper’s experienced by phone owners (as claimed by Lindstrom,
authors, with a cover letter explaining our approach and 2011)?
inviting a response. This key strategic component of CRE- What evidence would you find convincing?
ATE courses seeks to shift students’ often-negative precon- What studies would you do if you were interested in such
ceptions about what research/researchers/research careers issues?
are like. Many of the scientist-authors responded with com- How did Lindstrom make such determinations?
prehensive answers related to their personal and profes- On what basis do the neuroscientists challenge the stated
sional lives, their contributions to the work that we studied, conclusions?
and their scientific experiences as their careers developed. Do the New York Times’ edits shift the message of the original
The generosity of authors in preparing thoughtful responses letter to the editor? If so, how?
is especially valuable and memorable, according to our Taking all of the readings and analyses together, what do you
students. conclude about iPhone “love”? Why?

As they learned to use and apply CREATE tools, students ently helpful behavior: inquisitiveness, a pheromone signal,
accustomed to reading and passively accepting the informa- an aversion to squeaky distress calls, and the like. The pub-
tion encountered in their textbooks, on the Internet, or in lished paper provoked substantial interest and some contro-
newspapers began to recognize that just because something versy, as reported in Nature (Gewin, 2011). We reviewed the
is published does not mean it is beyond criticism (Hoskins, published critique, and students found that some of “our” al-
2010a). ternative interpretations had also been raised by top scientists
in the field, again recognizing that their own thinking was
scientific. Students also noted that even peer-reviewed work
Data Analysis—Developing Alternative
published in Science, where the original article appeared, can
Interpretations evoke intelligent criticism, and that scientists do not always
“Writing about Testing Worries Boosts Exam Performance in agree.
the Classroom” (Ramirez and Beilock, 2011) is a Science paper
examining the degree to which stress may contribute to un-
Established Knowledge Can Change
dergraduates’ “choking” on exams. We initially distributed
only some of the paper’s narrative and a single figure il- A provocative set of readings discuss the discovery that pep-
lustrating the first study performed, holding back the title, tic ulcers have a bacterial origin (Associated Press, 2005; Cen-
abstract, and all other information. During class, students di- ters for Disease Control and Prevention, 2005). It took the
agrammed the experiment, which compared test scores of PI’s ingestion of Helicobacter pylori, the suspected pathogen,
two groups of students. Each group had been administered hardly a canonical step in “The Scientific Method,” to gener-
a baseline math test. Posttest, both groups were told a stress- ate the conclusive data. This nature of science story illustrates
inducing story about how outcomes on a later test covering how established scientific knowledge—that ulcers had psy-
the same material would be used. Before taking the second chological not bacteriological etiology—can be wrong. Read-
test, one group wrote for 10 min about their fears of poor test ing the description of Dr. Barry Marshall being met with scorn
performance, while the other group sat for 10 min. The data at meetings where he initially presented his unconventional
revealing the test scores of the two groups show the nonwrit- hypothesis, students saw that novel (and possibly revolu-
ing group performing worse on the second test than they did tionary) ideas may not be instantly welcomed. This recent
on the first, thus “choking,” while the writing group scored scientific development highlighted the personal factors and
gains. We considered: genuine passion that can underlie science, making the point
that as scientific study continues, some established ideas of
Can we conclude that writing about one’s test concerns leads today will inevitably be supplanted. The ulcer readings also
to less choking on exams? How solid is that conclusion? illustrated the value of a healthy skepticism even about “ob-
If we had generated these data ourselves, could we publish vious” facts, such as that the stomach’s acidity would kill all
now? Why? Why not? bacteria within.
Are any alternative interpretations of the data plausible?
Through discussion, students proposed a third “write

Introducing Experimental Design and Peer Review
about anything” group as an additional control. We next pro- At the conclusion of many of the discussion units, the fresh-
vided the paper’s figure 2 and associated narrative. The au- men proposed follow-up experiments. The challenge: If your
thors had added a third group that was instructed to write research team had just performed the work we reviewed,
about “an unrelated unemotional event.” Students saw that what would you do next? Each student independently de-
the investigators had added the same control group they had vised two distinct follow-ups as homework. Three or four
asked for, extending the study to resolve the “writing-only” times during the semester, students formed teams of four
issue. This bolstered students’ sense that they were “thinking to act as grant panels charged with assessing the stud-
like scientists.” ies designed by their peers. The first time this was done,
we challenged the panels to establish appropriate funding
criteria before looking at the proposed studies. Discussions
Using Sketching to Clarify Design—Developing of criteria led to consideration of evolution, evolutionarily
Alternative Interpretations conserved mechanisms, and the meaning of model systems,
One paper’s abstract alone served as the focus for a class. as many groups only wanted to fund work that is “relevant to
The abstract for “Empathy and Pro-Social Behavior in Rats” humans.” We also discussed realities of reputation and how
(Bartal et al., 2011) outlines five individual experiments. As it may affect funding success. Some groups sought to fund
homework, students cartooned each experiment, all of which “established investigators who have already published in the
tested conditions under which one rat would open a trans- field,” leading other students to question how anyone gets
parent plastic container that restrained a second rat. Students started in research. Such discussions build students’ under-
defined the specific hypothesis being addressed in each study, standing of the sociological context of science.
the controls needed in each case (none are included in the After criteria had been discussed, each student submitted
abstract), the conclusions stated, and possible alternative in- one of his or her experiments, sans name or other identifier,
terpretations. into the grant pool. The instructor then presented each pro-
After comparing cartoons and resolving discrepancies, the posed follow-up study to the class without evaluative com-
class considered whether the behaviors observed were nec- ments. When the panels subsequently conferred to rank the
essarily signs of “empathy.” Might there be other explana- proposed experiments, students thought critically about the
tions? Working in small groups, students proposed multiple work of their peers, debating and defending their judgments
alternatives that could in principle account for rats’ appar- in the sort of open-ended give-and-take that characterizes
Vol. 12, Spring 2013 63

science as it is practiced. There is no single correct answer tell the students what additional experiments were done.
to the question: “Which of the ≈25 proposed studies is the Through class discussion, students developed their own
best?” Students were thus freed from the pressure to be right, questions and alternative interpretations (e.g., “maybe the
or to divine, somehow, what the instructor’s opinion might babies aren’t judging behavior; they just like yellow better
have been. than blue”). As in the discussions of “Babies recognize faces”
and “Writing about testing. . .,” only after the students raised
particular issues did we provide sections of the paper with
Using Multiple Popular Press Articles to Build
the relevant additional information and control experiments.
Toward a Mini-Module of Primary Literature After analyzing the full paper, students designed follow-up
We developed students’ critical-reading skills through re- experiments, vetted them in a grant panel, and then read and
peated practice with short articles. In the process, we pointed analyzed the authors’ actual next paper.
out multiple aspects of scientific thinking, and introduced the “Three-Month-Olds Show a Negativity Bias in Their So-
subject matter knowledge that would be needed in the later cial Evaluations” (Hamlin et al., 2010) was concerned with
reading of primary research reports exploring infant cogni- younger babies’ reactions to similar social interactions. This
tion. Early in the semester, we read and analyzed “Babies second paper used many of the same methods as the first,
Recognize Faces Better Than Adults, Study Says” (Mayell, facilitating students’ ability to read the material. Interest-
2005) and a popular press account of “Plasticity of Face Pro- ingly, the later work produced a different result, finding that
cessing in Infancy” (Pascalis et al., 2005), a study that tested younger babies were averse to hinderers but (unlike their
the memories of 6- to 9-mo-old infants. Students discovered “elders”) did not show any particular preference for helpers.
gaps in the popular press version (no information on “n” or As the authors discussed possible evolutionary implications
gender distribution of infant subjects, and unclear methodol- of their work, we were able to return to a critical theme that
ogy, for example). We added additional information from the had arisen earlier in the semester, in the “model systems”
Proceedings of the National Academy of Science paper as discus- discussion.
sion required it (for details of teaching with this paper, see
Hoskins, 2010b). Exercises of this sort challenge students to
read actively and seek key missing information (e.g., “How Assessment in Biology 10050
many female vs. male babies were studied?” or “Exactly The study presented here is based on tools (CAT, EDAT,
how was the visual training done?”) that is essential to their SAAB) administered anonymously pre- and postcourse. To
evaluations. evaluate students’ understanding of course material as a ba-
Two additional popular press stories (Talbot, 2006; Ang- sis for determining grades, we assess students in all CREATE
ier, 2012) and a study on babies’ perception of normal versus classes using a combination of in-class activities; writing as-
scrambled facial features (Maurer and Barrera, 1981) were signments; open-book, open-notes exams; and class partici-
critically analyzed in other class sessions. Discussions cov- pation. There is no assigned textbook, but students can con-
ered broader questions including: How can you tell whether sult during exams the notebooks/portfolios they compiled
a baby who is too young to talk notices something novel, throughout the semester (see Hoskins et al., 2007, for details).
and why might it matter? Because one of the studies was We find that open-book testing changes the classroom at-
funded by the National Institutes of Health, we considered mosphere and relieves students from the pressure to study
how a real-life grant panel might evaluate the work’s health primarily by memorizing, making it easier for them to focus
relevance. Students raised the possibility of using methods on critically evaluating scientific writing and explaining their
from the infant studies for early detection of neurological insights. With the exception of analysis of one exam ques-
abnormalities, such as autism, and discussed the degree to tion (see Student Reactions to Emails, below), the classroom
which environmental enrichment activities could be consid- assessments were not used as data for this study.
ered “health related.” These readings and discussions set the
stage for the analysis of two full-length papers.
“Social Evaluation by Preverbal Infants” (Hamlin et al., RESULTS
2007), examines 6- and 10-mo-old babies’ abilities to discrimi-
nate between and react to helpful, neutral, and hindering “be- CAT Outcomes
haviors” by observed “characters.” The babies witnessed sce- Students in the Fall CREATE Cornerstone course took the
narios in which experimenter-manipulated blocks of wood CAT (Table 1; Stein et al., 2012), and tests were scored by a
bearing large googly eyes interacted on a hill. One sort of trained team at Tennessee Tech University, where this test was
block (e.g., red circle) would move partway up the hill, but created. Biology 10050 students’ overall CAT scores improved
slide down before reaching the summit. Another block (e.g., significantly postcourse versus precourse, with a large effect
yellow square) might, in a subsequent “episode,” seemingly size (0.97). While there is overlap between categories, CAT
help it move up. A third block (blue triangle) might hinder questions address four main areas. Overall, the largest gains
upward movement. A series of control experiments explored made by CREATE Cornerstone students were on CAT ques-
the need for eyes and other animating traits on the blocks. tions that tested “evaluating and interpreting information.”
Other controls investigated whether babies preferred partic- Students also made gains on questions involving problem
ular colors/shapes, or upward motion to downward, rather solving, creative thinking, and/or effective communication
than seemingly helpful interactions (which moved the target (the other three subcategories addressed by the CAT). While
block up) to hindering ones (which moved it down). these findings must be interpreted with caution due to the
We started by providing the introduction, first figure, and small sample size, they suggest that students in the pilot
associated text for initial analysis. As before, we did not CREATE Cornerstone course made substantial gains in their

Table 1. CAT test resultsa
Critical Thinking Ability Test (CAT) Precourse Postcourse n Significance Effect size
Mean (SD) 9.6 (2.5) 13.0 (4.4) 15 p < 0.05 0.97

a The
CAT (duration 1 h) was administered pre- and postcourse to the Fall 2011 Biology 10050 class and scored at Tennessee Tech University.
We present the overall score for the test, precourse vs. postcourse. Fifteen students took both tests. Significance: Student’s t test.
ability to read, understand, and critically analyze informa- ures during the semester than did the first-year students. It is
tion, and that such gains are transferable to the content do- also interesting to note that the mid-level physiology course
main addressed by the CAT test, which was not related to the included a weekly laboratory, in which data were generated
material covered in the course. and analyzed, and one experimental design activity.
For epistemological beliefs categories, effect sizes in three
of the four categories that shifted significantly in the freshman
EDAT Outcomes CREATE group (certainty of knowledge, innate ability, cre-
Students in both Fall and Spring CREATE Cornerstone classes ativity of science) were moderate. The effect size of “sense of
completed a pre- and postcourse EDAT that was scored us- scientists as people” was large. Upper-level CREATE students
ing a 10-point rubric (Sirum and Humburg, 2011). Results also shifted significantly in this category, but with a smaller
are summarized in Table 2. Scores suggest that the first-year effect size, possibly reflecting the fact that many upper-level
students gained significantly in experimental design ability students were working in labs and had a better sense pre-
over the semester, citing more components of an “ideal” ex- course of what research scientists were like. Upper-level CRE-
perimental design postcourse than precourse. ATE students also showed significant changes in understand-
ing of the uncertainty of scientific knowledge (large effect
SAAB Outcomes size), and of “sense of scientists’ motivations” (moderate ef-
fect size).
Results from the SAAB surveys for each class are displayed in
Both the CREATE courses, but not the mid-level physiology
two groupings in Table 3. The upper group reflects the items
course, sent email surveys to authors of papers and discussed
related to students’ self-rated skills and understanding; the
author responses late in the semester. Different material was
lower group shows results for items that reflect students’
read and analyzed in each CREATE course; thus, different
epistemological beliefs about science (see Hoskins et al., 2011,
authors were queried and different responses were received
for a discussion of the derivations of all categories).
by the two groups. We think it likely that this component of
SAAB results show significant gains made by CREATE Cor-
the CREATE courses played a large role in changing students’
nerstone students in all six skills and attitudes categories and
opinions about what scientists are like and (for upper-level
in the majority (four out of seven) of epistemological cate-
CREATE students) why they do what they do.
gories. Students in the upper-level CREATE course (for which
a year of introductory biology, a semester of genetics, and a
semester of cell biology are prerequisites) shifted significantly Student Reactions to Emails
on all skills and attitudes categories, and three of the seven On the second exam in the Spring semester, we included
epistemological categories. Students in the mid-level physi- a question asking students about their reactions to the au-
ology course (for which a year of introductory biology and thor emails, focusing on their preconceptions about “scien-
a semester of genetics are prerequisites), in contrast, did not tists/research careers” and whether the author responses
shift significantly in any category. changed these views. We coded all the responses (n = 15),
Effect sizes help to determine whether statistically signif- extracting key themes from each, and summarize below the
icant changes are likely to be meaningful. For skills and at- themes mentioned by four or more students.
titudes shifts, effect sizes for freshmen were large (Cohen, The most prevalent response to the emails was students’
1992) in five of the six categories and moderate for “inter- statements that, precourse, they had assumed today’s re-
preting data.” Effect sizes for upper-level CREATE students searchers were “straight-A” students in college (14/15 re-
in these categories were all large. In this regard, it may be sponses; 93% of students). The same students (14/15) noted
relevant that upper-level students read literature that was that they no longer believed this to be true, citing several au-
substantially more complex and looked closely at more fig- thors who described academic struggles that preceded their
eventual success. Thirteen out of 15 students (86%) said that
the responses had changed their preconceptions about re-
searchers, and 9/15 (60%) noted that respondents stressed
Table 2. EDAT results: mean and SDa the importance of passion (as opposed to good grades) as
EDAT test Precourse Postcourse n Significance Effect size a key to research success. Seven out of 15 students (47%)
expressed enthusiasm on learning that the responding scien-
Mean (SD) 4.3 (2.1) 5.9 (1.4) 28 p < 0.01 0.91 tists described a great deal of work-related travel, includ-
ing international travel. Forty percent of students (6/15)
a Pool of two classes of Biology 10050: n = 28 total. Statistical signif- described having held one or more of the preconceptions
icance tested with Wilcoxon signed-rank test. Scores can range from
0 to 10, per the EDAT rubric (see Sirum and Humburg, 2011).
that 1) scientists were loners or nerds, 2) who lacked social
lives, 3) because science consumed all their time. A similar
Vol. 12, Spring 2013 65

Table 3. SAAB survey outcomes in three student cohorts: freshman CREATE students (n = 28), upper-level CREATE students (n = 19), and
mid-level non-CREATE students (n = 23)a
Freshman-level CREATE class
Category Precourse mean (SD) Postcourse mean (SD) Significanceb Effectc # Ssd
Decoding literature 17.3 (3.2) 21.9 (3.0) <0.001 1.48 6

Interpreting data 14.1 (2.6) 15.4 (2.3) 0.008 0.53 4
Active reading 13.0 (2.4) 16.1 (2.3) <0.001 1.32 4
Visualization 12.5 (2.8) 15.6 (2.1) <0.001 1.27 4
Think like a scientist 11.9 (2.2) 15.5 (1.6) <0.001 1.90 4
Research in context 10.9 (1.5) 13.8 (1.4) <0.001 2.00 3
Certainty of knowledge 22.1 (2.9) 24.3 (3.8) 0.002 0.66 6
Ability is innate 6.9 (1.4) 8.0 (1.6) 0.005 0.73 2
Science is creative 3.9 (0.8) 4.3 (0.7) 0.005 0.53 1
Scientists as people 2.8 (0.9) 3.6 (1.0) 0.004 0.84 1
Scientists’ motives 3.8 (1.1) 4.1 (0.8) ns 0.35 1
Known outcomes 3.9 (1.0) 4.2 (0.9) ns 0.32 1
Collaboration 4.2 (0.6) 4.3 (0.8) ns 0.14 1
Upper-level CREATE class
Category Precourse mean Postcourse mean Significanceb Effectc # Ssd
Decoding literature 15.5 (2.8) 20.3 (2.7) <0.001 1.75 6

Interpreting data 13.7 (2.2) 16.4 (1.7) <0.001 1.39 4
Active reading 13.9 (2.1) 16.9 (1.9) <0.001 1.50 4
Visualization 13.3 (2.1) 16.6 (1.7) <0.001 1.74 4
Think like a scientist 13.3 (2.5) 16.5 (2.1) <0.001 1.39 4
Research in context 13.5 (1.1) 14.3 (0.9) 0.037 0.80 3
Certainty of knowledge 23.0 (2.7) 26.1 (2.9) 0.021 0.82 6
Ability is innate 7.3 (1.9) 8.4 (1.5) ns 0.65 2
Science is creative 4.1 (0.7) 4.6 (0.9) ns 0.63 1
Scientists as people 2.5 (1.0) 3.9 (0.8) 0.007 0.44 1
Scientists’ motives 3.8 (1.0) 4.3 (0.6) 0.014 0.63 1
collaboration 4.4 (0.6) 4.5 (0.6) ns 0.17 1
Mid-level non-CREATE class
Category Precourse mean Postcourse mean Significanceb Effectc # Ssd
Decoding literature 19.6 (4.3) 20.4 (3.3) ns 0.21 6

Interpreting data 15.3 (2.3) 15.7 (2.6) ns 0.16 4
Active reading 14.7 (2.3) 14.7 (2.4) ns 0.01 4
Visualization 14.0 (2.3) 14.7 (1.8) ns 0.34 4
Think like a scientist 13.8 (2.7) 13.8 (2.5) ns 0.00 4
Research in context 13.5 (1.1) 13.5 (1.4) ns 0.00 3
Certainty of knowledge 23.7 (3.3) 23.6 (3.2) ns −0.03 6
Ability is innate 7.3 (1.5) 7.6 (1.5) ns 0.20 2
Science is creative 3.8 (1.2) 4.2 (0.7) ns 0.42 1
Scientists as people 3.1 (0.9) 3.3 (1.1) ns 0.03 1
Scientists’ motives 3.9 (0.9) 4.0 (1.1) ns 0.10 1
Collaboration 4.4 (0.5) 4.2 (0.6) ns −0.36 1
a Responses were tabulated using a 1–5 scale. (1 = “I strongly disagree”; 2 = “I disagree”; 3 = “I am neutral”; 4 = “I agree”; 5 = “I strongly
agree”). Some propositions were worded so that an answer reflecting a more mature understanding would get a lower score (“I accept the
information about science presented in newspaper articles without challenging it,” for example). These were reverse-scored for analysis.
The Wilcoxon signed-rank test for statistical significance was performed on precourse/postcourse raw data totals for all categories. Category
1–6: self-rated skills and attitude factors; categories 7–13: epistemological factors.
The survey was developed in a previous study of upper-level CREATE students (Hoskins et al., 2011). Different categories are probed by
different numbers of statements (#Ss).
b p values for statistical significance (Wilcoxon signed-rank test). ns = not significant.
c Mean difference/average SD.
d # Ss = number of statements in category.

percentage noted that precourse they had assumed all scien- ture, and that research science is neither routine, predictable,
tists had lofty goals of “helping people,” but they had come or boring, nor something found only in textbooks.
to realize that many had more personal goals of satisfying
Grant Panels Promote Open-Ended Thinking and Insight
their own curiosity. Five out of 15 students (33%) stated that
into the Nature of Science. CREATE Cornerstone students
precourse they had assumed most scientists did not enjoy
made significant gains on the EDAT, which presents a sce-
their jobs, that research was not fun, and that lab life was
nario distinct from that of any of the Cornerstone readings.
boring, but they no longer held these views. Five out of 15
Students’ gains on this test suggest that their general experi-
(33%) said they were surprised to learn scientists had flexi-
mental design skills have improved during the semester.
ble work schedules, and a similar percentage stated that they
Experimental design skills are honed in class through
had learned from the emails that motivation was very impor-
grant panel activities that focus on follow-up experiments
tant. Finally, 4/15 (27%) noted their surprise that the authors
to the studies we analyzed that are designed by the students
answered at all.
as homework. These are repeated several times during the
semester. Although panels focus specifically on experimen-
tal systems under study in class, they likely help students
DISCUSSION develop a more generalized skill in experimental design and
creative thinking. In each panel all students’ experiments are
Genesis of the CREATE Strategy reviewed, and the panels (groups of four students) discuss
The CREATE strategy originated as a response to the observa- the merits of each. Early in the semester, some experiments
tion that many upper-level undergraduate biology majors— must be culled based on absence of a hypothesis, absence of
despite the years spent studying a wide range of scientific a cartoon, or general lack of clarity (approximately five of 20
topics—were not well-prepared to read and understand pri- in early panels). In end-of-semester exercises, virtually every
mary literature; did not readily “think like scientists,” with an experiment meets the basic criteria and can be considered
appropriately critical eye; did not see science research as an at- seriously. Statements of hypotheses become clearer, controls
tractive career choice; and had little or no practical experience stronger, designs and procedures better illustrated, and po-
mustering their content knowledge to attack novel scientific tential outcomes well anticipated.
problems. Discussions with students in other courses over the Besides likely contributing to the development of students’
years, and with other faculty on our campus and elsewhere, experimental design skills, the grant panels provide insights
revealed that many students believed: research is dull, and into the nature of science. It becomes evident, as the activity
lab exercises formulaic and boring (Luckie et al., 2004); there is is repeated during the semester that, among the top experi-
a single and eternal right answer to every scientific question ments (typically four or five stand out), the study perceived
(Liu and Tsai, 2008); primary literature is written in a nearly by a particular panel to be “best” is to some degree a matter
unbreakable code; and scientists themselves are stereotypic of taste. Some students prefer a reductionist approach, others
nerds or “machinery kind of people” (Hoskins et al., 2007). an expansion of the study to encompass additional sensory
Our findings in the pilot CREATE Cornerstone course suggest modalities (e.g., an experiment investigating whether babies
that these viewpoints can be changed over a single semester learn to recognize faces faster if each face is associated with
through intensive analysis of scientific literature. a different musical tune). Some students focus mainly on ex-
periments aimed at developing treatments for humans (e.g.,
take genes involved in planarian regeneration and immedi-
Themes Highlighted in Readings ately seek their counterparts in mammals). Many of our stu-
dents are accustomed to “textbook” science where, typically,
The curriculum examples outlined above illustrate how fun-
only the (successful) end points of studies are described, and
damental features of scientific thinking can be studied in a
very little current-day work is featured. The grant panel ac-
realistic domain-specific context, which appears to be a key
tivity introduces the idea that working scientists likely select
element in developing critical-thinking skills (Willingham,
their follow-up experiment from a variety of valid possibili-
2007). Students repeatedly thought carefully about control
ties, and that personal styles and preferences could influence
groups—what they “control” for, how they are interpreted,
such decisions.
and why they are needed. Multiple studies underscored the
importance of careful attention to sample size and selec- Critical-Thinking and Experimental Design Skills—Tools
tion. In the experiments on infants, for example, students of Science. A significant number of students show interest
raised issues of possible gender-related behavioral differ- in science in high school or before (often significantly be-
ences, whether postnatal age is comparable between full-term fore [Gopnik, 2012]), but do not pursue STEM studies at
and premature infants, and the like. Students practiced devel- the tertiary level. Either they never consider studying sci-
oping alternative interpretations of data and noted that not ence in college, or they switch out of the field for a vari-
all conclusions are equally strong. Several studies highlighted ety of reasons in their first or second year (Seymour and
the potential for introducing unanticipated bias (see discus- Hewitt, 1997; Committee on Science and Technology, 2006).
sion of a possible “Clever Hans” effect in “Babies Recognize At the same time, for students who persist in STEM ma-
Faces Better Than Adults, Study Says” in Hoskins, 2010b). jors, some of the most creatively challenging and thought-
Students saw that original, interesting, and important inves- provoking courses—capstone experiences—are reserved for
tigations are currently ongoing (many readings were pub- seniors (Goyette and DeLuca, 2007; Usher et al., 2011; Wie-
lished in 2011–2012). Students also recognized that even very gant et al., 2011). We hoped to convey some of the ana-
early in their academic careers they are capable of reading, lytical and creative aspects of science at the outset of stu-
understanding, and intelligently criticizing scientific litera- dents’ college careers with a CREATE course designed for
Vol. 12, Spring 2013 67

freshmen. Providing this training early in students’ aca- The EDAT assessment called on students to think like sci-
demic experience might help students gain skills and develop entists: analyze a problem, determine evidence required to
attitudes that would support their persistence in STEM solve it, and design a properly controlled experiment that
(Harrison et al., 2011). could generate the relevant data. Students made statistically
We used the CAT and EDAT assessments to probe the de- significant gains in their experimental design ability, with
velopment of students’ abilities as they practiced the litera- their postcourse responses mentioning more of the points that
ture analysis process. The CAT test focuses on a content do- experts see as essential to good experimental design (Sirum
main distinct from that of the CREATE class but challenges and Humburg, 2011). In the Cornerstone classroom, students
students in some parallel ways. Students must determine repeatedly proposed and evaluated experimental designs as
what data mean, decide which data are relevant, draw conclu- they participated in multiple grant panels and worked with
sions based on their understanding, and explain themselves different student-colleagues. We suspect that these exercises
in writing. Many campuses are using the CAT test for pro- served as a form of practice during the CREATE semester,
grammatic assessment, comparing scores of freshman with helping students build competence in their ability to formu-
those of seniors, for example. We are aware of only one pub- late, express, and defend ideas about particular proposed
lished study using CAT in a pre/post, single-course situation. studies (Ambrose et al., 2010). At the same time, the challenge
First-year students in a semester-long inquiry-based micro- of producing an experiment that would be singled out by
biology module at Purdue University, performing hands-on a grant panel for “funding” may have stimulated some stu-
research in an introductory class, make significant CAT gains dents’ efforts to be particularly creative in their experimental
during the semester (Gasper et al., 2012). The finding that designs.
CREATE Cornerstone students at CCNY similarly made sig- The CAT and EDAT findings also support our sense that
nificant gains on this test in a single semester suggests that skills deemed important by many science faculty (e.g., prob-
transferable critical-thinking skills, such as those measured lem solving/critical thinking, data interpretation, written and
by the CAT, can also be built through classroom activities oral communication; Coil et al., 2010), including ourselves,
that do not involve hands-on inquiry labs. can be taught in a course that emphasizes the process of sci-
While the small sample size in this pilot study precludes ence, including close reading and critical analysis of primary
broad conclusions, it is interesting that our students made literature, creative experimental design, and a look behind
the largest gains on CAT questions whose solution required the scenes into the lives and dispositions of paper authors.
“evaluation and interpretation.” Introduction to Scientific While we teach or review relevant content in the context of
Thinking emphasizes looking closely at data, reconstructing particular reading assignments, we do not seek to achieve
the experiment or study that gave rise to the data, and rea- the broad coverage of a typical introductory course. Students
soning carefully about the logic of interpretations and the need not know the details of the electron transport chain in
significance of the findings. Students carry out this process order to analyze “Babies Recognize Faces Better Than Adults,
in a variety of content domains, engaging in friendly argu- Study Says,” although they do need to know the fundamen-
ments about whether rats are empathic or just noise-averse, tal logic of study design, use of controls, and the danger of
whether writing about fears really prevents choking on tests, being unwilling to think beyond your preferred hypothe-
and what it is that babies might prefer about a yellow square sis. To analyze the rat studies, students must understand the
with googly eyes (the color? the shape? the eyes? the “help- terms “empathy” and “prosocial behavior,” and know how
ful” behavior?). As noted by Stanger-Hall (2012), close to to think about variables, controls, and multiple aspects of
80% of U.S. high school seniors performed below the sci- animal behavior. In each case, they also need metacognitive
ence proficiency level on a recent national standardized test awareness—the ability to determine what they do and do not
(National Center for Education Statistics, 2009). Among un- understand, as well as “how we know what we know” (Tan-
dergraduates, barely more than half the students sampled at ner, 2012), another skill developed through practice during
24 institutions made gains in critical thinking during their the semester.
first 2 yr of college, as measured by the Collegiate Learning
Assessment (Arum and Roksa, 2011). These data suggest that Student Attitudes and Beliefs—Influences on Learning and
current course work in high school and during early college Career Options. On the SAAB survey, freshmen reported sig-
years (when standard introductory science courses are taken nificant gains in their self-rated ability to: “decode” primary
by STEM majors) is not promoting substantial development literature; interpret data; read actively (annotating, concept
of higher-order thinking and analytical reasoning skills. We mapping and/or cartooning the material they were read-
find CREATE Cornerstone students’ outcomes on the CAT as- ing); visualize scientific procedures; feel like they were think-
sessment encouraging in this regard. At the same time, some ing like scientists; and see experiments in a broader context
researchers suggest results of low-stakes tests like the Colle- (Table 3). Effect sizes (Cohen, 1992; Coe, 2002) were large for
giate Assessment of Academic Proficiency may be influenced five of the SAAB measures and moderate for “interpreting
by low performance motivation among test takers, because data.” With regard to students’ epistemological beliefs, pre-
participation in such exercises has no bearing on class grade vious researchers (Perry, 1970; Baxter Magolda, 1992) have
(Wise and DeMars, 2005). This issue could potentially influ- noted that students’ naı̈ve epistemological beliefs about sci-
ence our students’ performance on anonymous assessments. ence resist change, even after a 4-yr undergraduate program.
While we have no independent measure of students’ motiva- In some cases, such beliefs appear to regress after students
tion for participating in our study, we believe it likely that, as take introductory biology courses (Smith and Wenk, 2006;
participants in a novel course, they find the opportunity to Samsar et al., 2011). After one semester, the freshmen in the
be part of a scientific study to be intriguing and a motive to CREATE Cornerstone course reported significant increases
perform well. in four of the seven epistemological categories we surveyed:

the uncertain nature of scientific knowledge; the question of “there are many paths to science”]. To me this means
whether one needs to have a special innate ability to do sci- that there’s no one path or just one requirement like
ence; whether science is creative; and their sense of scientists reading a textbook, but many. I can conduct research
with as little a space as a backyard or in one of the
as “real people.” A concurrent upper-level CREATE class also biggest labs and each one could lead to success and
made gains in several epistemological categories, while stu- greatness. (Exam response, freshman in Biology 10050)
dents in a non-CREATE comparison course that included a
weekly laboratory session did not change significantly in any
Students’ self-reported reactions to author emails suggest
category (Table 3). These findings argue that the shifts we
that students starting college, at least at CCNY, harbor se-
see relate to the CREATE experience, rather than to intellec-
rious misconceptions about research/researchers that could
tual maturation that might occur naturally in college biology
likely interfere with their potential development as scien-
students over the course of a semester.
tists. Nearly all students in the class noted that before they
While student epistemology is rarely emphasized in col-
read the authors’ responses they had assumed that “only
lege teaching handbooks, students’ attitudes in this area can
straight A students can become scientists.” This supposi-
strongly influence their learning. For example, students who
tion changed when responding scientists recounted partic-
feel that intelligence is a fixed quantity in which they are
ular academic travails (e.g., rejection from some graduate
lacking may decrease their efforts to learn and study inef-
schools) that preceded their success. Other student comments
fectively as a result (Henderson and Dweck, 1990). The high
regarding their precourse suppositions that research is bor-
attrition rate of students from the biology major has been at-
ing; that researchers are both overworked and unhappy with
tributed in large part to students’ failure to connect intellec-
their jobs; and that such jobs allow no time for hobbies, fami-
tually with the subject, and the traditional mode of teaching
lies, or personal life, suggest that students’ precollege science
introductory courses itself can slow students’ development
experience has not presented research careers in an accurate
of higher-order thinking skills (e.g., analysis, synthesis, eval-
light. Notably these views defy logic, suggesting that some
uation; Bloom et al., 1956). While the majority of faculty mem-
adopted the stereotype without giving it much thought. Why
bers who teach introductory biology courses want students to
would people who are so smart (“like Einstein”) and who
learn higher-order skills, exams in such courses tend to focus
achieved “straight As” in college choose dull, boring careers?
at lower levels (Momsen et al., 2010). Multiple-choice testing
Why would someone engaged in a boring career that he or
(often considered a practical requirement for a large lecture
she did not enjoy, nevertheless work so intensely that he or
course) shapes students’ study habits in unproductive ways
she had time for nothing else? We have speculated elsewhere
and interferes with critical thinking (Stanger-Hall, 2012). Not-
that popular culture’s depictions of scientists may influence
ing that epistemological change is typically slow, Smith and
students negatively, starting in high school or before (Hoskins
Wenk point out that “. . .one cannot ignore the potential re-
and Stevens, 2009). Changing students’ negative views of re-
tarding effect of an entrenched instructional system of lecture,
searchers/research careers is a likely required first step, if
textbook readings, and recitation on the students’ epistemo-
such students are to be inspired to undertake undergraduate
logical development” (Smith and Wenk, 2006, p. 777). This
research experiences that can lead to research careers (Harri-
phenomenon may be reflected in the differences in responses
son et al., 2011). Given that the no-cost email survey of authors
of CREATE and non-CREATE students on the SAAB survey.
can have a strong positive impact on students’ views, we en-
The change in first-year students’ attitudes about scientists
courage other STEM faculty, particularly those working with
as people is large. We saw previously that upper-level stu-
high school students or first-year undergraduates, to consider
dents harbored negative opinions about scientists and the
this activity.
research life (Hoskins et al., 2007), but we did not know
whether these ideas developed during college or before. Find- Early Interventions. Traditionally, STEM-inclined students
ing that first-year students also assumed, precourse, that sci- spend their early college years in conventional core courses.
entists were antisocial and that research careers were dull sug- Electives, including capstone courses, are reserved for upper-
gests that students finish high school and enter college with level students. Recently, however, a number of colleges and
negative preconceptions about research/researchers. Multi- universities have begun developing nontraditional courses
ple years of college science education apparently do little to for entering students. A 5-d presemester “boot camp” for bi-
change these ideas. The shift we saw in students’ views of sci- ology students at Louisiana State University aims to teach
entists and research careers is likely attributable to one of the students about the different expectations at college versus
more unconventional techniques used in CREATE classes, the high school, focusing on study strategies and introductory
email survey of paper authors (see analysis of email reactions biology material. This brief presemester experience resulted
in Results, above). This student response to a Cornerstone in gains for boot-camp veterans, as compared with a matched
exam prompt regarding author replies is typical: control group, in classroom performance and in persistence in
the major (Wischusen and Wischusen, 2007). In a new course
I had this preconception [pre-course] that. . .you had focused on freshmen’s ability to reason scientifically, students
to be like Einstein to penetrate that field. I thought studied a variety of topics that a faculty group had deemed
you always had to have straight A’s and be highly valuable for introductory STEM courses. Students made
versatile, but after reading the e-mails from the authors significant gains in understanding of control of variables and
I know that’s definitely not the case. From what they proportional thinking, and also showed greater persistence
said I know that you don’t have to be perfect or like
Einstein. It’s the passion and motivation to learn and
in STEM (Koenig et al., 2012). Freshmen at Cabrini College
make discoveries. You have to have a drive that leads participated in Phage Genomics rather than a standard Intro-
you on. It was inspiring to hear that “science has many ductory Biology laboratory course. The novel course involved
paths” by S—[the quote in the author’s response was participation in a two-semester, hands-on research project
Vol. 12, Spring 2013 69

that significantly increased students’ interest in postgraduate ter prepared to master the material in any STEM major they
education, their understanding of scientific research, and choose, as gains in critical-thinking and reading/analytical
their persistence in the biology major (Harrison et al., 2011). skills should help them manage the information load in the
more content-heavy science courses to come.
Beyond Textbooks. Visual representations in journal articles
are both more frequent and more complex than those seen
in textbooks (Rybarczyk, 2011). When visual representations CONCLUSIONS
do appear in textbooks, they rarely illustrate the process of Introduction to Scientific Thinking, the CREATE Cornerstone
science (Duncan et al., 2011). Controversy, certainly a part of course, improved critical-thinking and experimental design
science, is virtually absent from textbooks (Seethaler, 2005). skills of freshmen at the same time that it positively shifted
Some faculty members feel that encounters with primary their attitudes about their reading/analytical abilities, their
literature, as well as capstone courses, and the majority of understanding of scientists as people, and multiple aspects
undergraduate research experiences, should be reserved for of their epistemological beliefs. There are few reported ap-
upper-level students, who have built a broad foundation of proaches to changing the critical thinking of first-year science
content knowledge in textbook-based courses. We agree that students, and it appears that epistemological beliefs among
understanding the nuts and bolts of a paper is a prerequisite college students at all undergraduate levels are quite stable.
for full understanding. Further, analysis and comprehension We find that a one-semester course positively affects both. The
skills are better taught in the context of a particular content course has no laboratory component, so it is relatively inex-
domain (Willingham, 2007). At the same time, particularly pensive to offer. Because the topic area of the articles that can
for biology, the explosion of fundamental content makes it be analyzed ranges broadly, readings can be selected for their
impossible for faculty to cover, let alone teach, “basic” mate- utility in a variety of introductory science courses. Finally,
rial even to the same depth it was covered in the introduc- the email survey responses from paper authors have a strong
tory courses of their own undergraduate years (Hoskins and effect on students’ sense of scientists as people, helping them
Stevens, 2009). In addition, despite having encountered them to overcome misconceptions of the sort that can dissuade stu-
in multiple courses, students may fail to retain key concepts dents from seeking research opportunities and, by extension,
(e.g., function of control experiments; see Shi et al., 2011). Our research careers. We are encouraged by the results of this pilot
compromise was to base a freshman course on particular ex- study and conclude that important gains—both practical and
amples of scientific literature, choosing topics in a limited attitudinal—with potential to help students make progress in
range of content areas and focusing in-depth on scientific STEM, can be achieved in a one-semester course that meets
thinking and data analysis. While we designed Introduction 2.5 h/wk and could, in principle, be added to many curricula.
to Scientific Thinking for future biology majors, the approach If, as exhorted by many science-education policy reformers,
could be easily adapted to other STEM domains. Interestingly, we are to do a better job at encouraging students to consider
a recent study argues that long-term cognitive advantages research careers seriously (National Research Council, 2003;
can arise from studying individual topics in depth. First-year American Association for the Advancement of Science, 2011),
undergraduates’ grades in science courses were highest for we need to move beyond standard first-year courses and re-
students who had studied a single topic in depth for a month veal scientific research as a creative and exciting career choice
or more at any time during high school. Grades showed no undertaken by interesting and diverse individuals, not unlike
correlation with economic status, region, class size, parents’ the first-year students themselves. While it would be grati-
academic level, or other factors (Schwartz et al., 2009). fying to see more students enter STEM research fields, the
Taken together, our findings support the hypothesis that a enhancement of skills, attitudes, and epistemological beliefs
CREATE Cornerstone course designed for first-year students concerning science engendered by CREATE Cornerstone is
can bring about gains in multiple areas, including critical- aligned with societal and civic goals, even for students who
thinking and experimental design ability, self-rated attitudes, go in other directions.
abilities and epistemological beliefs, and understanding of
scientists as people. Our freshman students did not have a
base of content knowledge in biology beyond what they re- ACKNOWLEDGMENTS
tained from high school or had absorbed from popular media. Many thanks to CCNY students for participating in the CREATE
By choosing articles from top journals (e.g., Science, Nature) courses and/or associated assessments. We thank Dr. Millie Roth
but focusing on topics that did not require deep under- and NKem Stanley-Mbamelu, Constance Harper, and Maria Harvey
standing of, for example, gene knockout techniques or elec- of City College Academy for Professional Preparation for assistance
trophoresis, we were able to give students a taste of the sorts with first-year students, and Drs. Shubha Govind, Anu Janakiraman,
Kristy Kenyon, Jonathan Levitt, and Leslie Stevens for assistance,
of design logic, interpretational challenges and controver- comments on the manuscript, and ongoing discussions of CREATE
sies, and creativity that are hallmarks of real-world scientific teaching/learning issues. Many thanks to Dr. Barry Stein, Elizabeth
investigation. At the same time that our students gained un- Lisic, and the CAT team at Tennessee Tech University for use and
derstanding of how authentic scientific studies are carried out grading of the CAT instrument. We also thank the anonymous re-
and interpreted, their email interviews of authors provided viewers, whose insightful comments strengthened the manuscript.
a personalized glimpse behind the scenes into the lives, atti- We are very grateful to the NSF for support.
tudes, and motivations of the researchers themselves. Ideally,
such insights will help to dispel misconceptions that can drive REFERENCES
students away from science. To the extent that students in the Ambrose SA, Bridges M, DiPietro M, Lovett M, Norman M, Mayer
one-semester Introduction to Scientific Thinking course make R (2010). How Learning Works: Seven Research-Based Principles for
significant gains in scientific thinking ability, they become bet- Smart Teaching, San Francisco, CA: Wiley.

American Association for the Advancement of Science (2011). Vision Henderson V, Dweck CS (1990). Achievement and motivation in
and Change in Undergraduate Biology Education, Washington, DC. adolescence: a new model and data. In: At the Threshold: The Devel-
http://visionandchange.org/finalreport (accessed 27 June 2012). oping Adolescent, ed. S Feldman and G Elliott, Harvard University
Press, Cambridge, MA, pp 308–329.
Angier N (2012). Insights from the minds of babes. New York Times
1 May, D1. Hoskins SG (2010a). “But if it’s in the newspaper, doesn’t that mean
Arora PN, Malhan PK (2010). Biostatistics, Mumbai, India: Global it’s true?”: developing critical reading & analysis skills by evaluating
Media. (eBook accessed 25 September 2012 through CCNY library). newspaper science with CREATE. Am Biol Teach 72, 415–420.
Arum M, Roksa J (2011). Academically Adrift—Limited Learning on Hoskins SG (2010b). Teaching science for understanding: focusing on
College Campuses, Chicago, IL: University of Chicago Press. who, what and why. In: Science and the Educated American: A Core
Component of Liberal Education, ed. J Meinwald and JG Hildebrand,
Associated Press (2005). Two Australians win Nobel Prize in Cambridge MA, American Academy of Sciences, 151–179.
medicine. www.msnbc.msn.com/id/9576387/ns/health-health_
care/t/two-australians-win-nobel-prize-medicine/#.UE_1RCLlaOx Hoskins SG, Lopatto D, Stevens LM (2011). The CREATE approach
(accessed 11 September 2012). to primary literature shifts undergraduates’ self-assessed ability to
read and analyze journal articles, attitudes about science, and epis-
Bartal IB-A, Decety J, Mason P (2011). Empathy and pro-social be- temological beliefs. CBE Life Sci Educ 10, 368–378.
havior in rats. Science 334, 1427–1430.
Hoskins SG, Stevens LM (2009). Learning our L.I.M.I.T.S.: less is more
Baxter Magolda MB (1992). Knowing and Reasoning in College: in teaching science. Adv Physiol Educ 33, 17–20.
Gender-Related Patterns in Students’ Intellectual Development, San
Francisco, CA: Jossey-Bass. Hoskins SG, Stevens LM, Nehm RH (2007). Selective use of the pri-
mary literature transforms the classroom into a virtual laboratory.
Bloom BS, Englehart MD, Furst EJ, Hill WH, Krathwohl D (1956).
Genetics 176, 1381–1389.
Taxonomy of Educational Objectives: Cognitive Domain, New York:
McKay. Koenig K, Schen S, Edwards M, Bai L (2012). Addressing STEM re-
Centers for Disease Control and Prevention (2005). Helicobacter pylori tention through a scientific thought and methods course. J Coll Sci
and Peptic Ulcer Disease. www.cdc.gov/ulcer/history.htm (accessed Teach 41, 23–29.
11 September 2012). Lindstrom M (2011). You love your iPhone. Literally. New York Times
Coe R (2002). It’s the effect size, stupid. what effect size is and why 1 October, A21.
it is important. Paper presented at the British Educational Research Liu S-Y, Tsai C-C (2008). Differences in the scientific epistemo-
Association Annual Conference, 12–14 September 2002, Exeter, UK. logical views of undergraduate students. Int J Sci Educ 30, 1055–
www.psych.yorku.ca/mcnelles/documents/ESguide.pdf (accessed 1073.
23 September 2012).
Luckie DB, Maleszewski JJ, Loznak SD, Krha M (2004). Infusion of
Cohen J (1992). Statistical power analysis. Curr Dir Psychol Sci 92, collaborative inquiry throughout a biology curriculum increases stu-
98–101. dent learning: a four-year study of “Teams and Streams.” Adv Physiol
Coil D, Wenderoth MP, Cunningham M, Dirks C (2010). Teaching the Educ 28, 199–209.
process of science: faculty perceptions and effective methodology. Maurer D, Barrera M (1981). Infants’ perception of natural and dis-
CBE Life Sci Educ 9, 524–535. torted arrangements of a schematic face. Child Development 52, 196–
Committee on Science and Technology (2006). Undergraduate sci- 202.
ence, math, and engineering education: what’s working? U.S. House Mayell H (2005). Babies recognize faces better than adults, study
of Representatives Committee on Science and Technology Subcom- says. National Geographic News. http://news.nationalgeographic
mittee on Research and Science Education. Second Session. 15 March. .com/news/2005/03/0321_050321_babies.html (accessed 12 Sept-
http://commdocs.house.gov/committees/science/hsy26481.000/ ember 2012).
hsy26481_0f.htm (accessed 1 October 2012).
Momsen JL, Long TM, Wyse SA, Ebert-May D (2010). Just the facts?
Duncan D, Lubman A, Hoskins SG (2011). Introductory biology text- Introductory undergraduate biology courses focus on low-level cog-
books under-represent scientific process. J Microbiol Biol Educ 12, nitive skills. CBE Life Sci Educ 9, 435–440.
143–151.
National Center for Education Statistics (2009). The Nation’s Report
Gasper BJ, Minchella DJ, Weaver GC, Csonka LN, Gardner SM (2012). Card 2009, Washington, DC: Institute of Education Sciences of the
Adapting to osmotic stress and the process of science. Science 335, U.S. Department of Education. www.nationsreportcard.gov/science
1590–1591. _2009/science_2009_report (accessed 3 November 2012).
Gewin V (2011). Rats free each other from cages. Nature 480, National Research Council (2003). BIO2010: Transforming Under-
doi:10.1038/nature.2011.9603. graduate Education for Future Research Biologists, Washington, DC:
Gopnik A (2012). Scientific thinking in young children: theoretical National Academies Press.
advances, empirical research, and policy implications. Science 337, Pascalis O, Scott LS, Kelly DJ, Shannon RW, Nicholson E, Coleman
1623–1627. M, Nelson PA (2005). Plasticity of face processing in infancy. Proc
Goyette SR, DeLuca J (2007). A semester-long student-directed re- Natl Acad Sci USA 102, 5297–5300.
search project involving enzyme immunoassay: appropriate for im- Perry W (1970). Forms of Intellectual Development in the College
munology, endocrinology, or neuroscience courses. CBE Life Sci Educ Years: A Scheme, San Francisco, CA: Jossey-Bass.
6, 332–342.
Poldrack R (2011a). The iPhone and the brain. New York Times
Hamlin JK, Wynn K, Bloom P (2007). Social evaluation by preverbal 5 October, A26.
infants. Nature 450, 557–559.
Poldrack R (2011b). NYT letter to the editor: The un-
Hamlin JK, Wynn K, Bloom P (2010). Three-month-olds show a neg- cut version. www.russpoldrack.org/search?updated-min=2011-01
ativity bias in their social evaluations. Dev Sci 13, 923–929. -01T00:00:00-08:00&updated-max = 2012-01-01T00:00:00-08:00&max
-results = 8 (accessed 11 September 2012).
Harrison M, Dunbar D, Ratmansky L, Boyd K, Lopatto D (2011).
Classroom-based science research at the introductory level: changes Ramirez G, Beilock SL (2011). Writing about testing worries boosts
in career choices and attitude. CBE Life Sci Educ 10, 279–286. exam performance in the classroom. Science 331, 211–213.
Vol. 12, Spring 2013 71

Rybarczyk (2011). Visual literacy in biology: a comparison of visual Stanger-Hall KF (2012). Multiple-choice exams: an obstacle for
representations in textbooks and journal articles. J Coll Sci Teach 41, higher-level thinking in introductory science classes. CBE Life Sci
106–114. Educ 11, 294–306.
Samsar K, Knight JK, Gülnur B, Smith MK (2011). The Colorado Stein B, Haynes A, Redding M (2012). Critical Thinking Assessment
Learning About Science Attitude Survey (CLASS) for use in biology. Test, Version 5, Cookeville, TN: Center for Assessment & Improve-
CBE Life Sci Educ 10, 268–278. ment of Learning, Tennessee Tech University.
Talbot M (2006). The baby lab. The New Yorker 82, 90–101.
Schwartz MS, Sadler PM, Sonnet G, Tai RH (2009). Depth versus
breadth: how content coverage in high school science courses re- Tanner D (2012). Promoting student metacognition. CBE Life Sci Educ
lates to later success in college science coursework. Sci Educ 93, 798– 11, 113–120.
826. Usher DC, Driscoll TA, Dhurjati P, Pelesko JA, Rossi LF, Shleininger
Seethaler S (2005). Helping students make links through science con- G, Pusecker K, White HB (2011). A transformative model for un-
troversy. Am Biol Teach 67, 265–274. dergraduate quantitative biology education. CBE Life Sci Educ 9,
181–188.
Seymour E, Hewitt N (1997). Talking about Leaving: Why Under-
graduates Leave the Sciences, Boulder, CO: Westview. Wiegant F, Scager K, Boonstra J (2011). An undergraduate course to
bridge the gap between textbooks and scientific research. CBE Life
Shi J, Power JM, Klymkowsky MW (2011). Revealing student think- Sci Educ 10, 83–94.
ing about experimental design and the roles of control experiments. Willingham DT (2007). Critical thinking: why is it so hard to teach?
Int J Sch Teach Learn 5, 1–16. Am Educ 31, 8–19.
Sirum K, Humburg J (2011). The Experimental Design Ability Test Wischusen SM, Wischusen EW (2007). Biology intensive orientation
(EDAT). Bioscene 37, 8–16. for students (BIOS), a biology “boot camp.” CBE Life Sci Educ 6,
Smith C, Wenk L (2006). The relation among three aspects of col- 172–178.
lege freshmen’s epistemology of science. J Res Sci Teach 43, 747– Wise SL, DeMars CE (2005). Low examinee effort in low-stakes as-
785. sessment: problems and potential solutions. Educ Assess 10, 1–17.

Cbe.12 11 0201

Uploaded by

Copyright:

Available Formats

Cbe.12 11 0201

Uploaded by

Document Information

Original Title

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

Cbe.12 11 0201

Uploaded by

Copyright:

Available Formats

CBE—Life Sciences Education

Vol. 12, 59–72, Spring 2013

CREATE Cornerstone: Introduction to Scientific Thinking, a

INTRODUCTION to attract and retain students who might be considering sci-

60 CBE—Life Sciences Education

Vol. 12, Spring 2013 61

62 CBE—Life Sciences Education

Through discussion, students proposed a third “write

Vol. 12, Spring 2013 63

64 CBE—Life Sciences Education

Table 1. CAT test resultsa

Mean (SD) 9.6 (2.5) 13.0 (4.4) 15 p < 0.05 0.97

Vol. 12, Spring 2013 65

Freshman-level CREATE class

Decoding literature 17.3 (3.2) 21.9 (3.0) <0.001 1.48 6

Upper-level CREATE class

Category Precourse mean Postcourse mean Significanceb Effectc # Ssd

Decoding literature 15.5 (2.8) 20.3 (2.7) <0.001 1.75 6

Mid-level non-CREATE class

Category Precourse mean Postcourse mean Significanceb Effectc # Ssd

Decoding literature 19.6 (4.3) 20.4 (3.3) ns 0.21 6

66 CBE—Life Sciences Education

Vol. 12, Spring 2013 67

68 CBE—Life Sciences Education

Vol. 12, Spring 2013 69

70 CBE—Life Sciences Education

Vol. 12, Spring 2013 71

72 CBE—Life Sciences Education

You might also like

Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.