ALL - Read pages 738-747

Read the charts on 748-50 (you may want to print

Pick any TWO Matrix (e.g A & C, D&E) Sections to Read
Skim Research Methods Journal of Engineering Education
Optional - 776-778 which talks about the implications
and use of the matrix. October 2012, Vol. 101, No. 4, pp. 738–797
© 2012 ASEE. http://www.jee.org

The Informed Design

Teaching and Learning Matrix
DAVID P. CRISMONDa AND ROBIN S. ADAMSb
a
City College,City University of New York, bPurdue University

BACKGROUND
Design experiences play a crucial role in undergraduate engineering education and are increasingly
important in K–12 settings. There are few efforts to purposefully connect research findings on how
people design with what teachers need to understand and do to help K–16 students improve their
design capability and learn through design activities.

PURPOSE
This paper connects and simplifies disparate findings from research on design cognition and presents
a robust framework for a scholarship of design teaching and learning that includes misconceptions,
learning trajectories, instructional goals, and teaching strategies that instructors need to know to
teach engineering design effectively.
The methods aren’t important for your reflection. Just know that a meta-literature review means
their research was actually collecting and analyzing all the existing research in the field to draw
METHOD overall conclusions.
A scholarship of integration study was conducted that involved a meta-literature review and led to
selecting and bounding students’ design performances with appropriate starting points and end
points, establishing key performance dimensions of design practices, and fashioning use-inspired
tools that represent design pedagogical content knowledge for teachers.

RESULTS
The outcome of this scholarship of integration effort is the Informed Design Learning and Teaching
Matrix that contains nine engineering design strategies and associated patterns that contrast beginning
versus informed design behaviors, with links to learning goals and instructional approaches that aim to
support students in developing their engineering design abilities.

CONCLUSIONS
This paper’s theoretical contribution is an emergent educational theory of informed design that iden-
tifies key performance dimensions relevant to K–16 engineering and STEM educational contexts.
Practical contributions include the Informed Design Teaching and Learning Matrix, which is fash-
ioned to help teachers do informed teaching with design tasks while developing their own design
pedagogical content knowledge.

KEYWORDS
engineering design, scholarship of integration, pedagogical content knowledge

INTRODUCTION

Design has been a prominent topic in shaping undergraduate engineering education

and the engineering profession. Reports on the goals of engineering education in the
United States and internationally (e.g., Mann, 1918; SPEE, 1930; Harris et al., 1994;
Goals Committee, 1968; Spinks et al., 2006) consistently identify design as central to
The Informed Design Teaching & Learning Matrix 739
Epistemology is the study of knowledge. Over the course
of OKEE we’re engaging in engineering as well as
reading about engineering to develop your understanding of how
engineering education. Bucciarelli (2003) considers design knowledge and knowing es- engineering knowledge
works.
sential elements of an epistemology
Text of engineering. Figueiredo (2008) describes four
key dimensions related to the nature of engineering: the basic sciences (engineer as sci-
entist), the human sciences (engineer as humanist), design (engineer as designer), and
the crafts (engineer as craftsworker). While proposed definitions of design are legion,
for the purpose of this paper, we see design as a goal-directed problem-solving activity
(Archer, 1965) that initiates change in human-made things ( Jones, 1992), and involves
optimizing parameters (Matchett, 1968) and balancing trade-offs (AAAS, 2001) to
meet targeted user needs (Gregory, 1966).
Design experiences are also playing a more substantive role in precollege students’
STEM (science, technology, engineering, and mathematics) education and career prepa-
ration. While some form of design capability has appeared in various national K–12 stan-
dards as a separate and distinct learning objective (AAAS, 1993; NRC, 1996; ITEA,
2000), until recently design challenges have been employed in K–12 settings more often
as performance projects (Kanter, 2010, p. 526) for learning concepts other than engineer-
ing. They have been used as contexts to teach mathematics ( Jacobson & Lehrer, 2000),
science (Sadler, Coyle, & Schwartz, 2000; Kolodner et al., 2003; Fortus, Dershimer, Kra-
jcik, Marx, & Mamlok-Naaman, 2004), computer programming (Kafai, Ching, & Mar-
shall, 1997), music (Bamberger, 2003), design and technology in the United Kingdom
(Kimbell, Stables, Wheeler, Wozniak, & Kelly, 1991; Barlex, 1995, 2000), and technology
education topics in the United States (LaPorte & Sanders, 1995; Hacker & Burghardt,
2004). Design-based learning experiences have been found to improve student learning
and achievement in mathematics and science – although these results are not always con-
sistent (Kanter, 2010, p. 3; Petrosino, 1998) – and to enhance students’ interest in STEM
subjects (Committee on Engineering Education in K–12, 2009).
These trends continue to evolve – in some cases merging and in other cases remaining
distinct and separate. A recent National Assessment of Educational Programs (NAEP)
initiative has formulated a plan to assess U.S. eighth-grade students in technological liter-
acy and engineering design in 2014. The Committee on K–12 Engineering Education
(2009) recommended that learning engineering design become a key feature of K–12 en-
gineering education after finding that significant learning in preschool through high
school classrooms was associated with the use of extended design activities that were pre-
sented in meaningful contexts. The Committee on Standards for K–12 Engineering Ed-
ucation, however, did not recommend developing separate content standards for engi-
neering education in K–12 settings (NAS, 2010). Instead, it urged the infusion and
mapping of engineering learning goals, topics, and skills into the standards of other
STEM disciplines. This approach has been adopted by the National Academy of Science This is what
was used to
in its Framework for K–12 Science Education: Practices, Crosscutting Concepts, and Core Ideas inform NGSS
(2012), where “engineering and technology are featured alongside the natural sciences.”
One consequence of K–12 schools adopting this approach will be that many students’ first
encounters with design activities will happen under the watchful eyes of teachers with few
to no design experiences under their belts and little training in using these activities.
Advancements in the scholarship of design teaching and learning must therefore ad-
dress two significant needs. First, the field lacks a coherent representation of design peda-
gogical content knowledge (Design PCK), despite the fact that it has produced a rich
body of research findings and hundreds of prescriptive and descriptive design process
models (Dubberly, 2004; Andrews & Goodson, 1980), as well as consensus models for
 740 Crismond & Adams

engineering and architectural/industrial design (Roozenburg & Cross, 1991). Design

PCK is content-specific, specialized teacher knowledge associated with instructional
techniques that are particularly suited to teaching effectively with design tasks (see Shul-
man, 1986). It entails knowing typical trajectories that different students can follow when
learning engineering design, the misconceptions and inefficient habits of mind to which
beginning designers are prone, and relevant learning goals that can help students acquire
competency in design in a timely fashion. Design PCK is distinguished from design con-
tent knowledge, which includes facts, concepts, and models related to design, and general
pedagogical knowledge that teachers need to know to do effective classroom and time man-
agement, lead whole-class or small-group discussions, and help establish a classroom cul-
ture for a community of learners.
While many have been motivated to integrate design into K–12 curricula or further its
This paragraph
speaks to some of use in undergraduate engineering education, these efforts have proven to be difficult, non-
the goals of the trivial endeavors. In precollege settings, teachers using design projects can miss critical
course in helping
you to be opportunities to support the meaningful learning of STEM concepts in students (Kanter,
critical consumers 2010). Lacking the knowledge needed to teach effectively with design challenges (Hynes,
of Engineering
curricula and 2010), they may inadvertently model for students inefficient design behaviors and habits
materials. of mind or reinforce students’ design misconceptions. Such teachers may be less reflective
about the actionable knowledge that they generate while teaching (Schön, 1995), or may
become uncritical users when implementing textbook-based design process models that
do not fully represent the space of design learning (see Adams, Turns, & Atman, 2003;
Mosborg et al., 2005) or the kinds of judgment necessary to choose one strategy over an-
other (Bransford, Franks, Vry, & Sherwood, 1989). For example, following a design pro-
cess model step-by-step may inadvertently communicate to learners that design is a set of
linear steps to be followed regardless of the situation, which can make students less adept
at switching design strategies when needed, transitioning among subproblems within the
larger design challenge (Strobel & Pan, 2011), or potentially less ready to deal with the
ambiguities of ill-defined problems (e.g., Adams et al., 2003; Cross, 2001; Daly, Adams &
Bodner, 2012; Lande & Leifer, 2010).
A second need for an integrative scholarship in engineering design is to help K–16
teachers access and interpret implications from design cognition research and render
it usable for everyday classroom teaching. This also is not a trivial endeavor: design
cognition and learning research spans extremely diverse disciplines, such as engineer-
ing, software design, architecture, writing, product design, and cross-disciplinary
areas like sustainable design, biomedical design, nanotechnology, and human-
centered design. Research suggests that undergraduate teachers, who may have con-
siderable disciplinary and experiential knowledge, rarely read design research or dis-
seminate their effective design teaching approaches to others (Pembridge & Paretti,
2010), although there are exceptions (Terpenny & Goff, 2005). When they do pub-
lish, they do not reference the broader body of research (Martin, Adams, & Turns,
2002). In other words, the problems of linking educational research with teaching
practice, which have been noted in other fields of STEM education (Turns et al.,
2006; Arbaugh et al., 2010), hold as well for engineering design education. Strength-
ening ties between research and practice may support the use of evidence-based deci-
sion making when choosing and sequencing instructional strategies and assessments,
and may also lay the groundwork for the development of empirically verifiable mod-
els of learning progressions.
The Informed Design Teaching & Learning Matrix 741

Learning progressions are conjectural models that take a stance regarding the nature
and sequencing of skills and ideas that learners should develop over time. They may be
empirically tested through validation studies, cross-sectional studies, and sequential stud-
ies (Duncan & Hmelo-Silver, 2009) to determine the degree to which the underlying
cognitive models hold true in different settings and for different learners. These progres-
sions may serve as templates for designing curricula and assessments (Songer, Kelcey, &
Gotwals, 2009) and help teachers select appropriate learning goals and instructional ac-
tivities (NRC, 2007), better understand what student behaviors or strategies to be watch-
ful for when teaching, enhance their reflective practice, and build in-depth Design PCK.
Learning progressions may also serve as templates for professional development products
(Songer et al., 2009) and guide the evaluation of programs that promise to foster more so-
phisticated thinking about a topic or skill by means of multiple learning experiences over
time (Songer et al., 2007; NRC, 2007).
The “scholarship of integration” (Boyer, 1990) can meet these needs by synthesizing a
broad and unwieldy research base to create a use-inspired framework and tools that can en-
able teachers to construct their own Design PCK. The scholarship of integration is “serious,
disciplined work that seeks to interpret, draw together and bring new insight to bear on
original research . . . fitting one’s own research – or the research of others – into larger intel-
lectual patterns” (Boyer, 1990, p. 19). This requires an integrative and potentially transdisci-
plinary synthesis that can bring together ideas across multiple perspectives. Kezar (2002)
makes a compelling argument that the goal of such work benefits researchers, educators,
and policymakers by pulling together various studies in a comprehensive way that gives
“meaning to isolated facts and putting them into perspective” and overcomes a tendency to
“split knowledge into ever more esoteric bits and pieces.” In the case of engineering design
education, the scholarship of integration may help support dialectic and interactive cycles
that link practical questions about design teaching with research questions about design
learning trajectories (Booth, Colomb, & Williams, 2008). The fruits of such a process could
help educators set more precise learning goals and select from a wider array of instructional
strategies and assessments when creating and implementing design-based activities.
In the following sections we first discuss our scholarship of integration approach and
then present the outcomes of this process: the formulation of the idea of informed design,
the depiction of Design PCK as the Informed Design Learning and Teaching Matrix,
and a review of the research base to support these ideas. We conclude the paper with a dis-
cussion of the Matrix as an emergent instructional theory from which research questions
can be generated and findings can be integrated, including results from testing the efficacy
of different instructional methods, and provide practical examples of how teachers may
use the Matrix to improve their classroom instruction and build their own Design PCK.

CONDUCTING A SCHOLARSHIP OF INTEGRATION STUDY

Skim lightly! This is on how the study was conducted!

This section describes the scholarship of integration approach that was used to articulate
the central notion in this paper of informed design and to create the Informed Design
Teaching and Learning Matrix (Table 1) associated with it. The Matrix presents nine
critical design practices or strategies and provides contrasting pattern statements that jux-
tapose how beginning versus informed designers do them. The table links these items to
lists of learning goals and instructional strategies that teachers may use to support learners
in becoming informed designers.
742 Crismond & Adams

The scholarship of integration has been slower than other forms of scholarship to gain
acceptance as an integral activity in academia. Some reasons for this include the isolation
of the disciplines, the view that interdisciplinary work is risky and located on the margins
of academic endeavors, the difficulty of the task, and a perceived disconnect between the
scientific community, the world of practitioners, and the larger public (Dauphinée &
Martin, 2000; Hofmeyer, Newton, & Scott, 2007; Kezar, 2002). Hofmeyer et al. (2007)
note, however, that this form of scholarship is becoming more central to academic work
because it is better equipped to build interdisciplinary partnerships, develop frameworks
that transcend disciplinary paradigms, and respond to complex, multifocal, contemporary
issues at the individual and societal level. While there are few examples of the scholarship
of integration, a review reveals certain shared attributes or goals. In the biomedical scienc-
es, one scholarship of integration effort focused on making connections across science dis-
ciplines, educating specialists, and placing the work of individual investigators into a larger
context (Dauphinée & Martin, 2000). In business management, this form of scholarship
has helped identify implications for practice that were not effectively communicated in
scholarly publications (Bartunek, 2007). In the health sciences, it has sought to address
how translational medicine, which adapts and converts findings from animal studies to
human studies, gets lost in translation, how an overabundance of information can be pro-
cessed into a useful compendium, and how feedback from clinics can help create and test
novel therapies (Mankoff, Brander, Ferrone, & Marincola, 2004). These examples illus-
trate themes of cycling between research and practice (Booth et al., 2008; Kezar, 2002)
and of moving beyond syntheses towards the creation of use-inspired frameworks. Turns
et al. (2006) identify the following goals for synthesizing research for practical use in the
context of engineering design education: support evidence-based decision making, answer
practical questions of high concern, promote reflective practice, and improve pedagogical
content knowledge in teachers.
A three-phase process for conducting a scholarship of integration study was used in
this paper. This process was based on examples of work in other disciplines
(Dauphinée & Martin, 2000; Kezar, 2009) as well as the concepts of use-inspired de-
sign (Turns et al., 2006), learning progressions (NRC, 2007; Duncan & Hmelo-Silver,
2009), and pedagogical content knowledge (Shulman, 1986). The first phase involves
bounding design performances within generic starting points and end points that are
appropriate learning targets for student designers. These in turn would guide what re-
search to review and how it should be interpreted and compiled. The second phase in-
volves conducting a meta-analysis that acts as a form of discovery in itself (Hofmeyer
et al., 2007), where the intended outcome is a set of key performance dimensions per-
tinent to the selected learning targets (Duncan & Hmelo-Silver, 2009). The third
phase involves representing these performance dimensions as a parsimonious set of
observable patterns that denote progressions in design learning, and incorporating
these patterns into a use-inspired tool for teachers and researchers. This tool aims to
enable teachers to develop their own Design PCK – including their ability to observe
and recognize the highly ineffective practices and habits of mind that beginning de-
signers employ, select learning goals, choose appropriate teaching strategies when
using design tasks, and assess students’ growth in design practices. An additional goal
is to facilitate empirical validation studies that will help in developing an instructional
theory of engineering design for K–16 students. The following paragraphs provide de-
tails of this scholarship of integration approach.
The Informed Design Teaching & Learning Matrix 743

Bounding Typical Student Designer Performances Keep SKimming

Bounding student design performance requires identifying appropriate starting points
and realistic end goals for K–16 design learners. Duncan and Hmelo-Silver (2009) sug-
gest that when creating a learning progression it is useful to identify (1) a lower anchor
that describes assumptions about the prior knowledge and skills of learners as they enter a
progression and (2) an upper anchor that depicts what learners are expected to know and
do by the end of a progression. While cognitive and learning scientists have long favored
contrasting the work and thinking of experts with novices, the range of definitions that
these researchers have employed for what constitutes a novice, however, has varied greatly.
Even graduate students have been classified as novices; for example, see Chi, Feltovich,
and Glaser’s (1981) often-cited novice-expert study of surface- versus deep-feature recog-
nition of problem sets in physics. The framework developed in this paper builds upon a
three-stage model of developing design expertise that includes naive, novice, and expert
levels of performance (Crismond 1997, 2001). Researchers have described telling differ-
ences between naive and novice levels of practice in various endeavors including perform-
ing science tasks (Zajchowski & Martin, 1993), solving physics problems (Chi & Bassok,
1988), learning nursing (Benner, Tanner, & Chesla, 1996), and doing wine tasting (Solo-
mon, 1997).
The starting point selected for this framework to depict a baseline in design capability
is the behavior and thinking of students with little or no experience and no formal training
in designing – the beginning designer. The end point in design capability is that of the “in-
formed designer,” whose level of competence lies somewhere between that of the novice
and expert designer, and in other learning contexts might be called an “advanced novice”
(Dreyfus, 2005), “expertlike novice” (Bereiter & Scardamalia, 1993), “expert student”
(Sternberg, 1997), or “competent performer” (Dreyfus & Dreyfus, 1986). Informed de-
signers will have had some formal training in design, and although not necessarily older
than beginning designers, will possess some design experience. The idea of a designer who
is informed grew out of a book of essays by David Hawkins called The Informed Vision
(2002), and alludes to notions of the engaged and knowledgeable citizen, to the wary and
savvy consumer, and has previously been applied to the context of learning and doing engi-
neering design (Crismond, Kolodner, Fasse, Gray, & Holbrook, 1999; Burghardt & Hack-
er, 2004; Crismond, 2005).
This midrange stage of the informed designer maps well onto a generic model of de-
sign expertise that corresponds to ways of perceiving, interpreting, structuring, and solving
complex problems (Dreyfus, 2005; Lawson & Dorst, 2009). As such, it is a more appro-
priate end point for school-based designers than design expertise. In many fields, exper-
tise, which involves deliberate practice and the use of well-connected and hierarchically
organized knowledge structures to solve problems (Dufresne, Gerace, Hardiman, &
Mestre, 1992) takes 10 or more years to achieve (Hayes, 1989). Students in K–16 learning
settings are not likely to accumulate the level of authentic practice necessary to develop
expert-like behaviors, although there are some exceptions (Atman, Chimka, Bursic, &
Nachtmann, 1999). While experts notice salient patterns quickly and have available much
situation-specific knowledge and many easily remembered cases available for recall
(Bransford, Brown, & Cocking, 1999), informed designers may hold fewer experiential
cases in mind and lack extensive practice in a domain. Compared to experts, informed de-
signers’ pattern-matching skills would be less reliable, and their retrieval and use of
learned ideas would be done less flexibly, since those ideas would have fewer connections
744 Crismond & Adams

to other thoughts. Their understandings would also be more situation-dependent and

wedded to their originally learned contexts. Piaget made note of a similar tendency in
cognitive systems that were still in formation (De Lisi & Goldbeck, 1999, p. 7).

Generating Performance Dimensions STOP! Skim the method but dig into the key performance dimension
Generating performance dimensions of informed designing involves a multidisciplinary
scholarship of integration process to bring together diverse knowledge sources and identi-
fy in a comprehensive way an inclusive set of intellectual patterns (Boyer, 1990). It is a
process of discovery (Hofmeyer et al., 2007) that aims to overcome knowledge isolation
and fragmentation (Kezar, 2002), while managing the trade-offs between simplicity and
parsimony when describing the complexities of learning (Lehrer & Schauble, 2009).
Identifying, reviewing, and uniting the broad literatures of design cognition research is
a nontrivial task. The literature spans many disciplines, making it difficult to navigate,
synthesize, and translate specific findings into recommendations for design teaching.
There are more than 170 peer-reviewed design journals, of which over 30 identify engi-
neering design as a primary subject of interest to their audiences (e.g., Design Studies, De-
sign Issues, Research in Engineering Design, Journal of Mechanical Design, Journal of Design
and Technology Education, Design Theory and Methodology, and Leonardo). Many other
journals treat engineering design as a topic of interest (e.g., Journal of Engineering Educa-
tion, International Journal of Engineering Education, Journal of the Learning Sciences, Cogni-
tion & Instruction, Cognitive Science, Journal of Research in Science Teaching, and Performance
Improvement Quarterly). Three additional challenges of this effort are that syntheses of
design cognition, learning, and teaching are not common, that terminologies from differ-
ent design domains can carry different meanings, and that studies report on a wide range
of ages and grade bands. While there are many useful design textbooks and anthologies,
they often emphasize design techniques (e.g., Dym & Little, 2004; Cross, 2000; Pahl &
Bietz, 1995) over how people learn design, and the anthologies often target design re-
searchers rather than design educators as their primary audience (e.g., Eastman,
Newstetter, & McCracken, 2001; Visser, 2006).
There are, however, exceptions to these trends. Kimbell, Stables, and Green (1996)
propose progressions in design capability for a series of specific “facets of performance” (p.
49) that include investigating, planning, modeling and making, design issues, evaluating,
extending knowledge and skill, and communicating. The authors also describe student
performances and teaching approaches for each design facet at four different grade bands.
Similarly, Lawson and Dorst (2009) published a book to help design practitioners learn
about and reflect upon their own development, design educators think about their teach-
ing and students learning, and design researchers connect to a broad body of research.
The following key performance dimensions, which are limited to foundational and
generative ideas and practices (Duncan & Hmelo-Silver, 2009), were identified as central
to doing informed design.
Learning while designing Informed designers are involved in continual learning
(Lawson & Dorst, 2009): learning by doing, learning from brainstorming and prototyp-
ing, learning by iteration and from feedback and failure, learning by noticing and trouble-
shooting, learning by drawing and dialoging with ideas, materials, and people, and learn-
ing from reflection. All of these emphasize the metacognitive and reflective practice
aspects of learning through design (Schön, 1987). One cognitive model of design empha-
sizes that “learning is central and inherent to designing” (Adams & Atman, 2000), in
The Informed Design Teaching & Learning Matrix 745

which designers continuously co-construct understandings of problems and possible solu-

tions. In contrast, the learning that beginning designers achieve is often not articulated
and is so fleeting and ephemeral that “conceptual closure is not attained” (Burghardt &
Hacker, 2004).
Making and explaining knowledge-driven decisions Informed designers use their
understanding of physical laws (Mann, 1981), of how things work, of methods of con-
struction (Zubrowski, 2002a, p. 57), and insights from experiments they
conduct to help make and explain their design decisions. This involves yet goes beyond
“common sense and everyday knowledge” (McCormick, 1993), a designer’s “craft-based”
application of known cases (Cross, 1999), and “intuition resolution” (Alexander, 1964, p.
5), and includes knowledge gained from revisions made as a design evolves (Perkins,
Crismond, Simmons, & Unger, 1995, pp. 72–73).
Working creatively to generate design insights and solutions Creativity and innova-
tion are cornerstones for all design work (Visser, 1996; Milne & Leifer, 1999; Shah &
Vargas-Hernandez, 2002; Barlex, 2011) both in the workplace and in schools. Design
must be informed by the creative insights (Smith, 1997, p. 143) that get generated when
framing a problem, generating potential solutions, and proposing novel ways to trouble-
shoot and iteratively improve prototypes. Helping students learn to deal with uncertainty
and to take productive risks while working with their ideas in creative ways can enhance
such work.
Perceiving and taking perspectives intelligently Informed designers achieve a per-
spective on the overarching goals and big picture in a product’s development that helps
them establish intentions and priorities in their design work (Christiaans & Dorst, 1992,
p. 135). They use empathy when imagining the experiences of products from the view-
points of a wide variety of users (Brown, 2009, p. 49). They can also employ “perceptive
observation” where they “learn what to focus on and what is relevant” (Kolodner & Wills,
1996) in order to detect positive and negative behaviors in a product’s performance.
Conducting sustained technological investigations Informed designers collect, or-
ganize, and analyze evidence and develop critical standards for performing technological
investigations and evaluating critical questions related to the device or system they are de-
veloping. Such work is analogous to scientists doing sustained inquiry when investigating
phenomena in nature (Erickson & Lehrer, 1998).
Using design strategies effectively Informed designers possess a range of design
practices and strategies, know when and how to use them (Adams, Turns, & Atman,
2003), and can alter their approaches to accommodate constraints of time and budget
(Wedman & Tessmer, 1991). They also work effectively in groups and can decide what
information sources to draw upon and what past experiences to apply most effectively
when addressing any number of problems embedded in a design challenge.
Integrating and reflecting on knowledge and skills Informed designers employ an
“integrated capability” where action, appraisal, and reflection are used in concert rather
than in isolation (Kimbell et al., 1991, p. 156) as they transition among the “intertwined . .
. compound problems” associated with design (Strobel & Pan, 2011). They combine skills
in design and fabrication with formal and everyday understandings of relevant disciplines
to create technological solutions (McCormick, 1994).
As can be seen from the reference section of this paper, the following journals and con-
ference proceedings were most represented in this synthesis: Design Studies (27 articles
cited), Proceedings of the Annual Meeting of the American Society of Engineering Educa-
tion Conference (12), Journal of Research in Science Teaching (9), International Journal of
746 Crismond & Adams

Engineering Education (9), International Conference on Design and Technology Educational

Research and Curriculum Development [IDATER] (9), Research in Engineering Design (8),
Journal of Engineering Education (7), and Journal of the Learning Sciences (4). The other
sources were represented to a far lesser extent. Studies in these publications reported on a
wide range of subjects, from young learners to practicing adult professionals, and perfor-
mances at different levels of design expertise, from rank tyro to expert designer.

Translating Performance Dimensions

The process of translating performance dimensions of informed designing into a use-
inspired, research-based instructional tool was accomplished through the following steps.
The first step involved selecting a parsimonious set of nine observable
design strategies that are fundamental to the performance dimensions of informed design
identified in the previous section. Four criteria were used in making these selections: (1)
criticality, strategies should be relevant and critical to design thinking and learning; (2)
observability, strategies should be observable or teachable by classroom instructors; (3)
frequency, strategies should be used multiple times during a design effort; and, (4) com-
municability, descriptions of strategies should make sense to non-designers.
The second step involved formulating “contrasting set” statements (Bransford
et al., 1989) that would be memorable to teachers and act as a guide for noticing how be-
ginning versus informed designers do the selected strategies. Each of the nine statements
begins by describing the telltale signs of beginning designers’ less effective approaches to
implementing that strategy, and ends with a contrasting depiction of how informed de-
signers would more effectively perform that same strategy (see Table 1). Taken together,
they work as a collection of two-step learning progression statements related to many (but
not all) elements that make up an effective practice of engineering design.
The final step was to place these statements into a single-table format that could be
easy for teachers to use and help develop their “professional vision” (Goodwin, 1994) as
informed users of design activities with students. The Matrix (Table 1) starts off by iden-
tifying a collection of nine design strategies (column 1), ones that could be found in any of
a number of models of the design process. It then identifies some of the highly ineffective
habits that beginning designers are prone to use (column 2), which are paired with de-
scriptions of the same strategies as done by informed designers (column 3). The Matrix
then attempts to articulate learning goals that are linked to the behaviors of informed de-
signers (column 4), and then identifies examples of teaching approaches aimed at helping
students reach those objectives (column 5).
Taken as a whole, the Matrix acts as an integrated framework that aims to facilitate
growth of Design PCK in teachers, improve design capability and practice in learners, and
help researchers advance the scholarships of design cognition, learning, and teaching for
engineering education. While the strategies and patterns listed in the Matrix (columns
1–3) may also be seen as orthogonal to what might constitute contextualized knowledge
about design, the ordering of strategies should not be viewed as an implicit suggestion
that they be performed lock-step in a given or preferred sequence.
Figure 1 summarizes how the nine contrasting patterns found in the Matrix map to
key performance dimensions associated with doing informed design. As shown here, the
same performance dimension may be associated with more than one Matrix pattern. Cer-
tain performance dimensions, such as “learning while designing” and “connecting and re-
flecting on knowledge and skills,” are associated with all Matrix patterns.
The Informed Design Teaching & Learning Matrix 747

UNPACKING THE MATRIX

In this section, we unpack and elaborate upon each of the nine contrasting patterns found
in the Informed Design Teaching and Learning Matrix by providing more detailed de-
scriptions of beginning and informed designers’ strategies, and situating them in the re-
search literature to illustrate their empirical base. The teaching strategies that are briefly
mentioned in the right-most column of the Matrix are also described in greater detail,
with citations provided when available. It should be noted that while some of these in-
structional approaches have undergone empirical testing, others have not, which high-
lights a critical need for future educational research. While we have tried to represent
ideas at a broad level, disciplinary differences in teaching design suggest that not all teach-
ing strategies will be appropriate for all settings and grade levels. On the other hand, some
teaching strategies may be useful in addressing more than one learning goal and Matrix
pattern.

Matrix Pattern A. Problem Solving vs. Problem Framing

THE STRATEGY OF UNDERSTANDING THE DESIGN CHALLENGE Beginning designers
feel that understanding the design challenge is straightforward, and a matter of comprehending
the basic task and its requirements. By perceiving the design task as a well-structured problem and
believing there is a single correct answer, they can act prematurely and attempt to solve it immedi-
ately. Informed designers seek initially to understand the challenge as best they can, but then
delay making design decisions in order to explore and comprehend the design challenge more fully.
They set out to learn through research, brainstorming, and doing technological investigations
what the critical issues are in order to frame the problem effectively. They will later return to assess
this framing after attempting to solve the challenge.
Observing the initial moments after beginning designers are given a design problem,
especially younger children, will probably reveal some team members quickly grabbing
some materials and attempting to solve the problem with little talk and forethought, or
immediately devising plans to solve the problem (Christiaans & Dorst, 1992, p. 132).
They are making “premature commitments” (Cross, 2000) to initial solutions. Beginning
designers must “frame a problematic design situation” (Schön, 1988), even though it may
be done impulsively, superficially, or without an awareness of the implicit assumptions
built into that framing. They may generate solutions prematurely because they assume
that all that they needed to know has been provided for them in the problem statement or
design brief, and do not believe that additional information is needed before they should
start generating solutions (Rowland, 1992). They can treat design challenges much as
they would well-defined, end-of-chapter textbook problems (Rowland, 1992, p. 74;
Atman & Bursic, 1996, p. 249) with clearly articulated initial states, identifiable collec-
tions of known variables, and set procedures for generating solutions that can be evaluated
unambiguously ( Jonassen, 1997, p. 68). Studies involving other problem-solving domains
report that novices tend to oversimplify the problems they face and begin their work by
suggesting solutions almost immediately (Elio & Scharf, 1990).
Beginning designers do not fully grasp just how complicated, fluid, and changeable
design problem-solving landscapes can be (Dorst, 2004), and can be unaware or unwary
of their potential for cascading complexity. For others with more experience, design tasks
are seen as “ill-structured” (Simon, 1984, pp. 152–153), and even “wicked” (Churchman,
1967; Rittel & Webber, 1984; Buchanan, 1995) because they are often ill-defined and
TABLE 1. The Informed Design Teaching and Learning Matrix.
748

This table links nine design strategies (Column 1) to contrasting pattern titles (across Columns 2 and 3) and statements of how beginning designers (Column 2)
versus informed designers (Column 3) do those strategies, and to relevant learning goals (Column 4) and instructional approaches (Column 5) that teachers can use.

BEGINNING vs. INFORMED DESIGNER PATTERNS

DESIGN
STRATEGIES WHAT BEGINNING WHAT INFORMED LEARNING GOALS TEACHING STRATEGIES
DESIGNERS DO DESIGNERS DO WHERE STUDENTS… WHERE STUDENTS…

Pattern A. Problem Solving vs. Problem Framing State criteria and constraints from
Define criteria and
design brief in one’s own words
constraints of challenge.
Understand Treat design task as a well- Delay making design decisions Describe how preferred design
Delay decisions until
the Challenge defined, straightforward in order to explore, comprehend solution should function and behave
critical elements of
problem that they prematurely and frame the problem better. Reframe understanding of problem
challenge are grasped.
attempt to solve. based on investigating solutions

Pattern B. Skipping vs. Doing Research Enhance background Do info searches/read case studies
sections.

knowledge, and build Write product history report

Build Do investigations and research
Skip doing research and understandings of users, Do studies/research on users
Knowledge to learn about the problem, how
instead pose or build solutions mechanisms and Reverse engineer existing products
the system works, relevant cases,
immediately. systems. Conduct product dissections
and prior solutions.
Pattern C. Idea Scarcity vs. Idea Fluency Generate range of design Do brainstorming and related
Work with few or just one Practice idea fluency in order to ideas to avoid fixation. techniques to achieve idea fluency
Generate Know guidelines/reasons for Relax real-world constraints or alter
Ideas idea, which they can get work with lots of ideas by doing
fixated or stuck on, and may divergent thinking, various divergent thinking original task to see it in new ways
not want to change or discard. brainstorming, etc. approaches. Do generative database searches

Pattern D. Surface vs. Deep Drawing & Modeling “Mess about” with given models
Explore and
Use words, gestures, artifacts to
investigate different design
Represent Propose superficial ideas that Use multiple representations to scaffold visualizing solutions
ideas via sketching,
This is a useful table to printout as you read along with your

Ideas do not support deep inquiry of explore and investigate design Do rapid prototyping using simple
modeling solutions, and
a system, and that would not ideas and support deeper materials or various drawing tools
making simple prototypes.
work if built. inquiry into how system works. Conduct structured review of ideas
Crismond & Adams

Continues
TABLE 1. Continued
Pattern E. Ignore vs. Balance Benefits & Tradeoffs Give explanations for design choices
Consider both the benefits Describe/portray pros and cons for all
Weigh Options Make design decisions without Use words and graphics to and tradeoffs of all ideas design options under consideration
& Make weighing all options, or attend display and weigh both benefits before making design Articulate design values and advice
Decisions only to pros of favored ideas, and and tradeoffs of all ideas before decisions. like KISS (Keep It Super Simple)
cons of lesser approaches. picking a design. and human-centered design
Pattern F. Confounded vs. Valid Tests & Experiments Create design advice for others and
Run valid “fair test” experi-
generalizations based on valid tests
Conduct Do few or no tests on proto- Conduct valid experiments to ments to learn how proto-
Do investigate-and-redesign and
Experiments types, or run confounded tests learn about materials, key design types behave and to opti-
product comparisons tasks
by changing multiple variables variables and the system work. mize their performance.
Do tests to optimize performance
in a single experiment.
Pattern G. Unfocused vs. Diagnostic Troubleshooting Follow troubleshooting steps:
Diagnose and troubleshoot observe, name, explain, and remedy
Use an unfocused, non- Focus attention on problematic
Troubleshoot ideas or prototypes based on Do troubleshooting stations/videos
analytical way to view areas and subsystems when
simulations or tests. Do modeling or cognitive training in
prototypes during testing and troubleshooting devices and
troubleshooting
troubleshooting of ideas. proposing ways to fix them.
The Informed Design Teaching & Learning Matrix

Pattern H. Haphazard or Linear vs. Manage project resources Student use design storyboards to
Managed & Iterative Designing and time well. record progression of their work
sections.

Do design in a managed way, Use iteration to improve Give instruction and scaffolding for
Design in haphazard ways ideas based on feedback. project management & design steps
Revise/Iterate where ideas are improved
where little learning gets done, Employ design strategies Encourage taking risks, learning
iteratively via feedback, and
or do design steps once in repeatedly in any order as while iterating, and reflecting on
strategies are used multiple
linear order. needed. how the design problem is framed
times as needed, in any order.

Pattern I. Tacit vs. Reflective Design Thinking Periodically reflect while Keep design diaries and portfolios
designing and keep tabs on Compare/contrast design cases of
Reflect on Do tacit designing with little Practice reflective thinking by
strategies used. approaches used by different groups
Process self-monitoring while working keeping tabs on design
Review to check how well Do computer-supported structured
or reflecting on the process strategies and thinking while
solutions met goals. reflections about design work
749

and product when done. working and after finished.

This is a useful table to printout as you read along with your
750 Crismond & Adams

MAKING KNOWLEDGE-DRIVEN DECISIONS

USING DESIGN STRATEGIES EFFECTIVELY

WORKING CREATIVELY TO GENERATE

CONNECTING AND REFLECTING ON

TECHNOLOGICAL INVESTIGATIONS
DESIGN INSIGHTS & SOLUTIONS

PERSPECTIVES INTELLIGENTLY
INFORMED DESIGN

LEARNING WHILE DESIGNING

DIMENSIONS OF

KNOWLEDGE AND SKILLS

CONDUCTING SUSTAINED
PERCEIVING & TAKING
MATRIX PATTERNS:
BEGINNING VS. INFORMED DESIGNERS
A. Problem Solving vs. Problem Framing • • • • • •
B. Skipping vs. Doing Research • • • • • •
C. Idea Scarcity vs. Idea Fluency • • • •
D. Surface vs. Deep Drawing & Modeling • • • • • •
E. Ignore vs. Balance Benefits & Trade-offs • • • • •
F. Confounded vs. Valid Tests & Experiments • • • • •
G. Unfocused vs. Diagnostic Troubleshooting • • • • • •
H. Haphazard or Linear vs. Managed & Iterative
Designing
• • • • • • •
I. Tacit vs. Reflective Design Thinking • • • •

FIGURE 1. Mapping the contrasting patterns of the Matrix to key dimensions of informed design.

involve elements and parameters that are only partially determined at the outset of work.
Design tasks can require understandings from an “extensive and unpredictable” range of
disciplines (McCormick, 1993, p. 309), and can swamp designers’ cognitive capacity with
“insoluble levels of complexity” (Alexander, 1964, p. 3). Particularly vexing for some be-
ginning designers is that there is no single right answer to the vast majority of design chal-
lenges. Multiple viable solutions are the norm for ill-defined design tasks in part because
“designers exercise the freedom to change goals and constraints” (Cross, 2001a, p. 82),
where each of many framings of a design problem can have its own optimal solution.
More experienced novice and expert designers approach these problems differently.
Some experts first aim to understand the challenge (Rowland, 1992; Akin & Lin, 1996)
by using “knowledge development strategies” to build rich representations of the unfamil-
iar challenge (Larkin, McDermott, Simon, & Simon, 1980). They attempt to identify key
issues associated with the problem, while scrupulously avoiding making any early design
decisions. Others do their initial explorations of a problem by posing a number of possible
solutions. Both approaches involve problem structuring (Goel & Pirolli, 1992), in con-
trast to problem solving, and involve iteratively formulating and framing the problem
(Adams et al., 2003), exercising a considerable level of freedom in order to explore all the
facets of the challenge (Daly, Adams, & Bodner, 2012), and avoiding making final design
decisions too early. Research of large-scale engineering projects, for example, has shown
The Informed Design Teaching & Learning Matrix 751

that postponing design decisions reduces the amount of redesign work needed to accom-
modate unanticipated changes (Gil, Tommelein, & Beckman, 2004). More generally,
those whom Bereiter and Scardamalia (1993) refer to as “effective learners” know that
identifying what is important in a problem can be hard to discern at the outset, and resist
making decisions too quickly about problems, especially in unfamiliar domains, until
more is known (Bruer, 1993).
Teaching strategies Educators have devised a number of ways to address the begin-
ning designer’s tendency to interpret design problems too simply, frame the problem
superficially, and make design decisions prematurely or treat design as a task of well-de-
fined rather than ill-defined problem solving. As shown in the paragraphs below, ap-
proaches include having students produce functional descriptions of products before
building prototypes, delaying decision making, and helping students do effective prob-
lem scoping.
Comprehending the problem statement Before students start even the most prelimi-
nary work on a design task, they should provide evidence of comprehending the design
challenge’s problem statement, often referred to as a design brief. Many design curricula
ask students to review the design brief and summarize in their own words its key points:
the context of the problem, its main goals, and the key criteria and constraints of the
challenge. Working within constraints while trying to achieve desired product behav-
iors is a “big idea” in design thinking (NAE, 2010) for all grade levels (ITEA, 2000).
Functional descriptions One way to restrain students from making premature design
decisions is to ask them to generate a functional description of what a viable solution
should do to be successful. Such descriptions can be reviewed periodically as students be-
come more familiar with the system they are developing and how to best frame the chal-
lenge they face. This approach supports the function-behavior-structure design process
model proposed by Gero and Kannengiesser (2004). Taking an integrated approach to
linking what a solution needs to achieve (function) with how it is used (behavior) and the
form it takes (structure) is an important attribute of design capability and measure of ef-
fective designing. For example, Turns (1998) noted in one study that senior undergradu-
ate engineers who produced high-quality designs discussed functional features of their
ideas rather than structural features early in their design process.
Problem framing and scoping Probably the greatest reason experienced designers delay
design decision making is their appreciation of the benefits of effective problem framing
and scoping. Simple comprehension of the task is not enough. They understand at the
outset of their work that a design problem’s most critical issues must still be discovered
and identified before they can be solved. This involves reviewing and elaborating on ele-
ments of the problem (Adams & Atman, 2000), identifying and stating user needs, and
articulating implicit assumptions during initial problem framing efforts (Bursic & Atman,
1997, p. 66). Teachers can stimulate “coupled iterations” – iterations that integrate prob-
lem and solution decisions (Adams et al., 2003) – by explicitly asking students early in the
design process about how qualities of their proposed solutions relate to their understand-
ing of the problem. They can then encourage students to review their initial framing as-
sumptions, and update their understanding of the problem as their proposed solutions
evolve through multiple iterations. Some curricula prompt students to review problem
specifications that they had previously summarized and to answer questions they
had raised during earlier efforts at addressing the design problem (Puntambekar &
Kolodner, 2005).
752 Crismond & Adams

Matrix Pattern B. Skipping vs. Doing Research

THE STRATEGY OF BUILDING KNOWLEDGE THROUGH RESEARCH Beginning de-
signers skip doing research in favor of generating solutions immediately. Informed design-
ers instead do research on users, write product histories, and collect information on manufac-
turing methods, materials, and product standards to build understandings of the problem
and potential solutions.
Start-from-scratch design challenges are among the hardest of design tasks for begin-
ning designers to address. Beginners, facing an open-ended design challenge with a blank
sheet of paper and few relevant cases or design heuristics in mind, are forced to make one
mystifying decision after another. For expert designers, such a task would involve making
many design choices that are routine, with only a handful of more difficult novel problems
interspersed among them (Akin & Lin, 1996, p. 42). The working memory of beginning
designers can become overwhelmed when they encounter a dauntingly high ratio of novel
to routine design decisions. Just as informed designers know when to delay design deci-
sions and why (Pattern A), they also know that doing research can make such a ratio more
manageable and help them make more informed design decisions.
When facing well-defined problems, students typically do not look for outside help
before attempting a solution (Woods & Crowe, 1984). They can thus skip doing research
on the problem, in favor of proceeding directly to generating solutions. In a comparative
study of second- and fourth-year industrial design undergraduates, students were asked to
redesign a litter system for a train. The less experienced designers would “solve a simple
problem; that is, they ignore a lot of the complications in the assignment” and were more
likely to produce “sparse problem prototypes organized by superficial features,” in part be-
cause they “don’t even realize they lack information” (Christiaans & Dorst, 1992, p. 132).
One of the major headaches and banes of design instructors is students doing poorly
structured and incomplete searches while designing. Beginners sometimes replace re-
search with “found-object designing” (W. Flowers, personal communication, 2006),
where nearby objects act as the main source of inspiration for design solutions. While at
times yielding creative and effective solutions serendipitously (Wills & Kolodner, 1994),
such a search method is considered by some to be neither methodical nor thorough.
Designers need both domain-specific knowledge and situation-relevant strategies to
design effectively. Acquiring information is integral to achieving such understandings
(Ennis & Gyeszly, 1991; Molina, Al-Ashaab, Ellis, Younger, & Bell, 1995; Wild &
McMahon, 2010). Supporting students in doing this kind of work is key to helping them
become informed designers. In one comparative study involving freshman and senior un-
dergraduate engineers, seniors who produced products of higher quality made more and
varied information requests while designing (Bursic & Atman, 1997, p. 62). A similar
trend was noted when professional engineering designers were compared with senior un-
dergraduate engineering students doing the same task (Atman et al. 2007).
Research can help designers change their focus or reframe a design problem, enrich
their representation of the problem in their minds, clarify relevant underlying principles,
as well as uncover clues to potential solutions. Research can involve working creatively
with “conceptual artifacts – such as theories, problem formations and interpretations”
(Bereiter & Scardamalia, 2006, p. 700) and can yield ideas that help refute proposed de-
sign ideas (Broadbent, 1984). It can also confirm designers’ “private frames of reference”
that “prestructure their views of the problem” and can help them “keep control of the
design process” (Powell, 1987, pp. 193–194). Bursick and Atman’s study (1997, p. 68)
The Informed Design Teaching & Learning Matrix 753

described how certain student designers may not use information they gather when mak-
ing their design choices, which suggests that what gets learned through research, like
other forms of learning, can become inert (Whitehead, 1929; Bransford et al., 1989).
Experienced designers do much of their research during early designing, when they
generate concepts and design ideas (Christiaans & Dorst, 1992, p. 135); however, the
need to do research can arise at almost any point in design work. These searches can help
in reviewing relevant product standards, finding instances of products for possible reuse
(Visser, 1995) or as potential exemplars (Rhodes, 1998, p. 134), and analyzing user prefer-
ences (Christianns & Dorst, 1992, p. 135). Research can help in determining methods of
manufacture (Barlex & Wright, 1998, p. 160) and construction (Kuffner & Ullman,
1990), provide details on materials’ physical specifications and costs (Bursic & Atman,
1997), and show how products work and why they are designed as they are (Kuffner &
Ullman, 1990). It can support investigations into constraints and performance parameters
of possible solutions (Ennis & Gyeszly, 1991) and help in articulating issues of safety, legal
liabilities, and maintenance (Bursic & Atman, 1997, p. 70). Information searches, espe-
cially when done using online tools, can involve large quantities of ill-structured informa-
tion that must be processed quickly (Baya & Leifer, 1994).
Studies have shown, however, that information searches are sometimes conducted as a
delaying tactic when groups have reached an impasse (Brereton et al., 1996, p. 334). Less
skillful designers in another study performed searches that were “not organized according
to some intention” (Christiaans & Dorst, 1992, p. 135), while subjects in the same study
who proposed higher quality solutions did more numerous information searches that took
less time to complete (p. 135). Atman et al. (1999) observed a similar pattern among cer-
tain engineering freshmen who spent considerable amounts of time gathering informa-
tion, but left little time to make actual design decisions.
Teaching strategies Curriculum developers and design educators have developed dif-
ferent ways to support students in exploring and doing research that informs their design
work. These include:
Focused information searches Beginning designers regularly conduct fruitless infor-
mation searches; in some cases, their queries fail to fit the formal categories used by
engineering databases (Kimbell, 1994). Teachers can model for students the use of ap-
propriate search terms and emphasize how the discriminating and strategic evaluation
of potential sources for yielding useful data can help. Periodically reviewing the design
brief can constrain and curtail runaway information search efforts (Rhodes, 1998, p.
137) and keep students’ research efforts more focused (Barlex & Wright, 1998, p. 165).
Having students submit a “results table” at regular intervals where search terms and re-
sults are gathered and itemized can provide feedback and formative assessment data
for teachers so that they can adjust instruction while assessing student progress.
Studying prior art Beginning designers can gain much by researching relevant prior art
or “handling collections” (Stables & Rogers, 2001; Kimbell & Stables, 2008) of devices or
systems whose functions approximate requirements noted in the design brief. Expert de-
signers who do not want to “reinvent the state of the art if it already exists” favor this ap-
proach (Cross & Cross, 1998, p. 144), and take the position that “much everyday design
work entails the use of precedents or previous exemplars . . . because the exemplars actually
contain knowledge of what the product should be” (Cross, 1999). Benenson and Neujahr’s
Stuff That Works! curriculum (2002) asks upper elementary students to conduct a scaven-
ger hunt in their shopping-bag design task. Students collect and bring to class a wide
754 Crismond & Adams

range of types of bags that exhibit different uses of materials, handle designs, and joining
and reinforcement methods. Students in teams group the bags according to criteria of
their own choosing, test the bags by loading them until they fail, and then conduct a post
mortem to determine the bags’ strengths and weaknesses before they design their own
shopping bag using a brown-paper lunch bag, string, tape, and other materials.
Writing a product history report Students can do meaningful research when they write a
product history of a device or system (e.g., espresso machine, power drill, or audio record-
er). Such a paper might describe how the selected product works, the context of its devel-
opment (country or company of origin), historical era, and milestones in the product’s
evolution. The inclusion of a product timeline could highlight the advent of new manu-
facturing methods or materials or detail a product’s reconceptualization based on chang-
ing user needs or market trends.
Researching users By doing research on a product’s customers, beginning designers can
build their own mental model of the user. The PIES instructional model (Barlex, 2004)
asks students to investigate and write about the physical, intellectual, emotional, and social
needs of users. Urban and Hauser (1993) describe a range of social science techniques in-
cluding user observations, surveys, interviews, and focus group meetings based on more
comprehensive methods used in industry. Instructional designers using the ADDIE
model (Smith & Ragan, 2005) perform a needs assessment when developing a profile of
potential users. Ullman’s quality function deployment method (1997) relies on surveys,
focus groups, and observations of customers to determine what various users want regard-
ing product features and quality. Role-playing activities can also yield insights about users
as well – including feigning injury to investigate the quality of care in an emergency room
(Brown, 2009, p. 50), wearing glasses smeared with petroleum jelly to mimic the experi-
ences of the visually impaired, or wearing gloves when using kitchen devices to experience
first-hand the challenges the arthritis-sufferers face.
Product dissections and reverse engineering Building a foundation of kinesthetic and
haptic experiences with materials and devices is seen as critical preparation for doing sci-
entific inquiry (Woolnough, 1991), science-based design (Zubrowski, 2009), and engi-
neering-oriented visualization (Scriber & Anderson, 2005). Sheppard and Jenison (1997,
p. 251) had engineering undergraduates build their own experiential base by conducting
product dissections, also known as product teardowns (Sandborn, Myers, Barron, &
McCarthy, 2009). With dissections, students are given time to investigate and use the de-
vices before taking apart and re-assembling them using simple tools. They then describe
how the devices function, make associations and analogies to similar devices, predict the
make-up of unseen subsystems, discuss manufacturing and assembly methods and costs,
and identify science and engineering concepts related to the artifacts and how they work.
Otto and Wood (1998) detail similar steps when describing a reverse engineering and re-
design methodology, and provide details on numerous strategies used in industry that un-
dergraduates also learn to do, including writing ‘black box’ product function descriptions,
conducting customer needs analysis, making various predictions, and performing experi-
ments on key components and the overall product. A recent study found significant pre-
post differences in students’ understanding of system interconnectivity and abilities to de-
scribe reasonable redesign solutions after doing a scaffolded design dissection activity
(Dalrymple, Sears, & Evangelou, 2010).
Case-based reasoning with catastrophic and other examples Another approach to doing
research involves using case-based reasoning (Kolodner, 1994), which can come into play
The Informed Design Teaching & Learning Matrix 755

when students read real-world case studies and, as Lundeberg argues, may make ideas
easier to recall because stories help learners organize and store information (NRC, 2011,
p. 30). Case stories about designing may involve individuals or entire firms and can high-
light various aspects of design thinking that resulted in successful enterprises. However,
insights into good design practice can be especially memorable for students when con-
structed from instances of failed solutions (Bursic & Atman, 1997, pp. 70–71) and from
catastrophic failure cases in engineering (Rendond-Herrero, 1993; Petroski, 1993;
Pietroforte, 1998). Using cases that represent disruptive innovations may also enable
counterintuitive thinking (Garcia, Sinfield, Yaday, & Adams, 2012) that leads to achiev-
ing creative breakthroughs to problems.

Matrix Pattern C. Idea Scarcity vs. Idea Fluency

THE STRATEGY OF GENERATING IDEAS Beginning designers can start their design
work with very few or even just one idea, which they may not want to discard, add to, or re-
vise. Informed designers want to design with an abundance of ideas and practice idea flu-
ency using techniques such as brainstorming and divergent thinking to explore the design
space and at least initially seek to avoid favoring any single solution.
To address a design challenge, designers need ideas, lots of ideas, and the wider the
range of ideas, the better. Sometimes, ideas are in short supply. Idea scarcity can arise from
designers’ “reluctance to spend the time and mental effort needed to conjure up a rich
storehouse of alternatives from which to choose” (Adams, 1986) or from a propensity to
shun novelty.
Like many other complex problem-solving enterprises, design is rarely a smooth flow-
ing series of procedures and events that move seamlessly from insightful ideas to inspired
prototypes to widely adopted products. At times, the pathway is blocked. Such “stuckness
in design” (Sachs, 1999) can arise from procrastination or poor scheduling (Hayes, 1989,
p. 331), or what has been called “functional fixedness” by some and “psychological inertia”
by others (Otto & Wood, 1998, p. 229). Idea fixation – which “deals with both the inabili-
ty of designers to see new ways of using objects they are exposed to and the inability to
present the use of attributes of an object whether appropriate or not” (Gero, 2011) – can
happen to individuals, teams, whole firms (especially large ones), and to entire industries.
It has also been noted in both experimental studies and classroom-based research with
problem solvers in many fields (Cross, 2000). While idea fixation may not be experienced
in the same way across different disciplines (Purcell & Gero, 1996), and may have diver-
gent and overlapping causes which fit that label, it is pervasive across different design do-
mains and persists despite warnings from teachers and consultants.
The situation can be more problematic for beginners, since they may not recognize
signals that their work has come to a halt: they may then fail to enact strategies for getting
unstuck in a timely way. Beginning designers are not aware of the impact that the invisible
yoke of unstated givens or spurious constraints (Akin & Akin, 1996) may have on the
questions they ask or the solution vectors they explore. Many designers, particularly nov-
ices, find it challenging to think divergently and get trapped by characteristics of known
solutions (Daly et al., 2011). Some beginning designers work “depth-first” (Cross, 2000)
and spend too much time developing a single idea, which may result in limited project
time getting used up and first plans being implemented out of sheer necessity.
Numerous explanations have been put forward to account for idea fixation, as it is an
issue found in many areas of problem solving. Studies from cognitive science suggest that
756 Crismond & Adams

limits of short-term and working memory can contribute to design fixation, despite a sub-
ject’s willingness to seek other alternatives (Smith, 1995; Kohn & Smith, 2009). From a
situated cognition perspective, designers’ thinking can become fixedly contextualized
within the boundaries and unquestioned assumptions that arise during problem framing
(Cross, 2001a), when designers can “strongly pre-structure their views of problems”
(Powell, 1987, p. 193). Curriculum or instructor pedagogy may also be a contributing fac-
tor. Early cues or hints from the teacher or illustrations in instructional materials could
lead students to favor some solutions over others. This could result in fixation by novice
and expert designers alike ( Jansson & Smith, 1991; Linsey & Viswanathan, 2010), al-
though this trend was found to be less prevalent in one study involving senior undergradu-
ate industrial designers (Purcell & Gero, 1996). Seen through a constructive-develop-
mental lens, designers may become embedded in and subject to (Kegan, 1982) their initial
design plans, a psychological dynamic that would render certain designers less able to re-
flect objectively upon their proposed ideas. Crismond (1997) noted how expert designers
were well practiced at generating one idea after another and were able to set aside each
idea in turn, in a deliberate act of letting that idea go, so that they could make a fresh start
at proposing yet another qualitatively different approach.
Teaching strategies Effective idea fluency should help designers both explore and ex-
pand the design space in which they are working. One psychometric tool for measuring
the products of idea generation focuses on four features that any collection of design ideas
should contain: novelty, variety, quality, and quantity of ideas (Shah & Vargas-Hernandez,
2002). Recent research has used these metrics to study the effectiveness of certain intuitive
ideation methods to produce collections of design ideas (Vargas-Hernandez, Shah, &
Smith, 2010). A concise review of the different types of approaches for generating many
design ideas can be found in Shah and Vargas-Hernandez (2002, p. 112). These ap-
proaches include provocative stimuli (exposure to new concepts), suspended judgment,
flexible representations (collaborative sketching), changing the frame of reference for see-
ing the problem, incubation, and exposure to example ideas (Shah, Smith, Vargas-
Hernandez, Gerkens, & Wulan, 2003).
Divergent thinking Techniques to help people achieve divergent thinking have been
proposed in various guises and have been described in books that catalog heuristics for
helping people achieve idea fluency through creativity. Among classics in this field are
Adams’s Conceptual Blockbusting (1986), de Bono’s Lateral Thinking (1970), Hayes’s The
Complete Problem Solver (1989), and McKim’s Experiences in Visual Thinking (1980).
Techniques presented in these books include delimiting and proposing alternative views
of the problem, doing idea sketching and visual recall, incubating ideas by stopping work
on a problem for a while, and generating personal and direct analogies.
To help designers explore the space of possible design solutions and facilitate the dis-
covery of novel concepts, Daly et al. (2011) developed an empirically based set of design
ideation strategies (Yilmaz & Seifert, 2011; Daly et al., 2010) called Design Heuristics.
Each heuristic appears on one of 77 strategy cards and suggests a different strategy (e.g.,
convert for second function, utilize opposite surface, nest, and use alternate energy source)
to use for generating ideas. Each card offers an action prompt, an abstract image to repre-
sent the strategy, and two examples that show the application of the heuristic to existing
consumer products. For example, the Utilize opposite surface card shows two examples: an
athletic shoe where the laces wrap around toward the bottom of the shoe to allow better
mobility and a dining chair where the back side is shown to reveal additional storage op-
tions. These divergent thinking strategies were tested in an undergraduate engineering
The Informed Design Teaching & Learning Matrix 757

course, where 48 students were provided with a short information session on the use of the
Design Heuristics, followed by a 25-minute idea generation session and a 20-minute de-
sign debrief session. An analysis of concepts students generated indicates that the Design
Heuristics facilitated exploration of the design space, which includes more original and
more diverse ideas.
Brainstorming One of the hallmark strategies of designers, brainstorming, involves
generating a wide-ranging collection of ideas while deliberately withholding criticisms
and deferring judgment on the quality of those ideas. Ideas may be generated using a wide
range of materials and modes of expression such as sketching, which is linked with the re-
interpretation of design ideas (Gero, 1999). Making analogies can encourage grouping
and connecting ideas in unexpected ways, which may enhance ideation (Ulrich & Ep-
pinger, 1995; Ball & Christensen, 2009; Stacey, Eckert, & Earl, 2009) and lead to the de-
velopment of new products (Perkins, 1997). Kimble and Sables (2007) and Johannson
(2006) note research that supports having designers first brainstorm individually and then
share their ideas with others.
Research has suggested that instructions emphasizing the withholding of criticism are
less effective than stressing the creation of large numbers of ideas in producing more ideas
and more good ideas (Paulus, Kohn, & Arditti, 2011). However, simply asking students to
generate lots of ideas with or without judging them ignores the nontrivial challenge of de-
veloping an ability to brainstorm. Without scaffolding, elementary and middle school
student designers often do not generate multiple solutions when facing design problems
(Welch, 1998; Welch, Barlex, & Lim, 2000). Attempts to mandate idea fluency have
backfired on teachers, especially when such edicts are issued without explaining to stu-
dents the reasons for such a dictate. McCormick, Murphy, and Davidson (1994) describe
cases where teachers required students to include three or four candidate ideas in their de-
sign portfolios, from which students purportedly would select one idea for implementa-
tion. The authors tell how students actually generated the requisite alterative solutions
after completing their design projects that were based on a single idea. They referred to
such student portfolio work as a “veneer of accomplishment.” When, without rationale,
brainstorming is proposed for students to do, they can treat it as a required classroom ritu-
al that they perform superficially, if at all.
Constraint relaxation and “dream designing” At certain junctures in a design project, ex-
pert designers may opt for a time to set aside all limitations – including those imposed by
nature, the design brief, or the client – in order to propose solutions from a fresh perspec-
tive. When asked about such “dream designing” during a post-design interview
(Crismond, 1997), one MIT design professor noted, “At some point in the design process
. . . you just forget about constraints and imagine the thing you are designing in an ideal
world, where limitations like cost and material are not an object.” Practitioners do “con-
straint relaxation” (Moorman & Ram, 1994) in an attempt to decouple the design task
from the circumscribed contexts in which their initial thinking has taken place, with the
aim of harvesting a new crop of ideas that they would not have thought of otherwise.
Later, they then adapt them so that they eventually address the constraints of the original
problem specifications.
Generative database searches Searching trade magazines, journals, and databases can
help designers generate new product ideas (Ulrich & Eppinger, 1995, p. 86; Ullman,
1997, p. 141), although such approaches can be unproductive for beginning designers
(Radcliffe & Lee, 1989, p. 206). Students can benefit from reviewing collections of rele-
vant design elements that can be found in classic works that illustrate the broad range of
758 Crismond & Adams

mechanical elements developed over the centuries ( Jones, 1930) or books that showcase
specialty collections of different paper mechanisms and folding techniques for making
pop-up books (Carter & Diaz, 1999). The Invention Machine software (Derringer, 1996)
uses a case-based reasoning AI engine that is linked to the U.S. Patent Office’s database
and aims to help designers make unexpected yet generative connections and approaches
during conceptual design and hence reduce the likelihood of idea fixation.
Starter vs. final project challenges Design challenges can be structured so that students
are less prone to idea fixation. One such pedagogical approach involves presenting stu-
dents with a starter challenge and then later modifying either the materials used for fabri-
cation (Zubrowski, 2009, p. 326) or changing the task specifications for the final design
task itself. In a cardboard chair design challenge (Goldman, 2002), for example, students
were asked first to build two or three qualitatively different miniature chair prototypes out
of index cards. They were then given two 4-by-8-foot cardboard sheets and asked to pro-
duce, without using glue, fasteners, or tape, a quarter-, half-, and finally full-scale card-
board chair that could support a 150-pound person sitting and leaning back in it. This in-
structional sequence can prevent students from fixating on their initial design ideas,
because by changing the scale of the prototype, the materials that can be used to make it,
or the specifications themselves, students are forced to make fundamental revisions to
their ideas in the face of design challenge requirements that keep evolving.

Matrix Pattern D. Surface vs. Deep Drawing and Modeling

THE STRATEGY OF REPRESENTING IDEAS FOR DEEP INQUIRY Beginning designers
propose and sketch ideas that superficially resemble viable solutions but that do not support
deep inquiry into how a solution might function, and would not work if built. Informed
designers use gestures, words, and artifacts to explore and communicate their design plans.
They make drawings, construct physical prototypes, and create virtual models that help them
develop deeper understandings of how their designs function.
Beginning designers can produce ideas and sketches that emphasize superficial aspects
of potential solutions, ones that lack crucial specifics necessary for their ideas to work
when built, and that do not contribute to conducting meaningful investigation into those
ideas. They ignore constraints and produce “ideation without substance” (Newstetter &
McCracken, 2001), spend little time doing preliminary design drawings (Welch, 1998),
and make scant use of such drawings later when making prototypes (Gustafson,
MacDonald, & Gentilini, 2007). Beginning designers may produce no drawings or faulty
ones because they lack the requisite graphic fluency (Fleer, 2000) in sketching, which “re-
quires considerable skill” (Welch et al., 2000, p. 141). They have patchy or inaccurate de-
vice knowledge ( Johnson, 1988; McCormick, 1994), ignore constraints, and create pro-
posals that could not be realized as actual products (Newstetter & McCracken, 2001, p.
67). When operating with little functional knowledge of the device, they can generate
plans that focus “almost entirely on aesthetic features” (Fortus et al., 2004, p. 1097). Miller
(1995) studied teams of three or four MIT undergraduate designers doing single-session,
materials-constrained design challenges where they created models of cranes and mechan-
ical triggers. Videotapes of these sessions showed students “building nonfunctional imita-
tions of known devices” (Miller, 1995, p. 15) and making “pre-Archimedean” errors where
“the preponderant overall form of mistake was to build a design visually resembling some
relevant generic device, but lacking functional connections or relationships among its
parts.” Students were unable to “form competent analytical models of whatever physical
The Informed Design Teaching & Learning Matrix 759

objects” they were designing, which required “applying basic physical knowledge to con-
crete design-and-build problems” (Miller, 1995, p. 14).
For graphically literate designers, sketching can support visual reasoning and design
thinking in a number of ways by (1) making internal thinking about aesthetics, ergonom-
ics, and mechanics explicit; (2) extending short-term memory so that more complex sys-
tems can be envisioned; (3) enabling problem scoping and solution archiving by enhanc-
ing collaboration and communication; and (4) supporting the designers’ own dialogs with
ideas and their evaluation of imagined solutions (Archer, 1979; Ullman, Wood, & Craig,
1990; Suwa & Tversky, 1997; Heiser, Tversky, & Silverman, 2004; Cardella, Atman, &
Adams, 2006; Goldschmidt, 1991). The inherent ambiguity of sketches can invite design-
ers to explore new directions when designing (Garner, 1989; Fish & Scrivener, 1990).
Sketches of imagined products sometimes act as drawing experiments that allow design-
ers “to test a hypothesis, explore phenomena, and affirm or negate the move” (Schön,
1984).
In design work, modeling can involve building a physical prototype – “an approxima-
tion of the product along one or more dimensions of interest” (Ulrich & Eppinger, 1995,
p. 219) – using easy-to-fabricate modeling materials, like cardboard and duct tape, or
easy-to-assemble structural elements, like LEGOs™. Mathematical models, including
those that are the basis of computer simulations, can represent the problem or potential
solutions and act as cognitive devices to enable thinking (see Visser, 2006). These ap-
proaches can help students visualize their product ideas more easily, especially those with
modest drawing skills (Lemons, Carberry, Swan, & Rogers, 2010), and are viewed by
many engineering educators and researchers as core competencies of effective design
practices (ITEA, 2000).
Teaching strategies Beginning designers’ inability to represent their design ideas accu-
rately or with sufficient detail can contribute to the superficial designs they produce. Ex-
tended instruction in sketching has been proposed (MacDonald & Gustafson, 2004) to
help establish graphical thinking and literacy as a “profound and diverse resource” for stu-
dents to use “at the very earliest conceptual stages and as a final act in the design process”
(Garner, 1989, p. 43). The following are some strategies that address deficiencies in sketch-
ing and drawing and introduce various forms of modeling to enhance design thinking.
Messing about with given models David Hawkins coined the term “messing about”
(2002) to describe a form of careful observation and hands-on investigation of materials
that precedes a child’s more formal scientific investigations. The technological equivalent
of messing about in design appears in curricula such as Learning by
Design™ (Kolodner, Crismond, Gray, Holbrook, & Puntembakar, 1998; Kolodner et al.,
2003) and Project-Based Inquiry Science (Georgia Tech Research Corp. [GTRC], 2010),
where students are given materials and plans for building and then exploring initial proto-
types. Sadler, Coyle, and Schwartz (2000) describe ways to support students’ modest de-
sign skills and background knowledge by providing them with plans for a barely working
yet functional prototype, which they then build, test, and improve upon through iterative
redesign. Having beginning designers fabricate flawed yet working models provides them
with an initial taste of success, yet leaves them plenty of room to improve their devices
over multiple iterations. The instructional sequence described in Zubrowski’s standard
model (2002a) has students first design an initial prototype without suggestions or help
from the instructor, and then build a teacher-supplied model of the same device. After
testing both models, students attempt to synthesize ideas to create an optimal solution.
Goldsmith College’s Richard Kimbell tells how students do the physical equivalent of
760 Crismond & Adams

brainstorming when they rapidly fabricate small-scale prototypes from easy-to-use mate-
rials, which he calls “conceptual making” (Georgia Tech Research Corp., 2004a). All of
these instructional techniques guide beginning designers to explore the design problem
space using a “solution-focused” approach (Lawson, 1979), support them in engaging in
dialogs with the materials they fabricate and use (Bamberger & Schön, 1983), and help
them conduct exploratory discussions and investigations that may make a problem more
tractable (Kavakli & Gero, 2003).
Building before sketching Many instructors ask beginning designers to follow a se-
quence for concept development that expert designers often use: first envision, then
sketch, and then build a model (Constable, 1994; Egan, 1999). Some younger designers
prefer to sidestep sketching in favor of immediately creating a physical model when de-
signing (Welch et al., 2000), in part because of a preconception they hold that says all
drawings must be presentable renderings of finished products (Constable, 1994). Since
students make better sketches after they have seen artifacts similar to their envisioned
prototypes or have such items in hand (Anning, 1997), the standard sketch-then-make
sequence might well be reversed. A study involving eight undergraduate engineering stu-
dents found that doing modeling first with an open-ended design task enhanced students’
visualization of problem solutions and improved their understanding of how the proto-
types worked without requiring additional time to complete their design projects (Lem-
ons et al., 2010).
Virtual drawing and computational modeling Computer-aided design (CAD) software,
including low- or no-cost programs such as Google Sketchup, can enable students to create
and consider details of design plans prior to building prototypes (Crismond, Howland, &
Jonassen, 2011). The use of such systems can aid students’ visualization of plans and has
long been an important goal for undergraduate engineering students (Committee on En-
gineering Design, 1961) and for those in middle and high school settings (Cline & Man-
dinach, 2000). Computer-based simulation and systems modeling programs such as
STELLA (Mandinach & Cline, 1996) can scaffold students’ engineering knowledge and
estimation skills, and enable them to make first-order approximations of a model’s perfor-
mance in the early phases of designing. These approaches, however, carry potential draw-
backs. The precision required in creating CAD drawings (e.g., explicit dimensions and
shapes) runs counter to the kinds of ambiguity in rough sketches that sometimes provoke
novel visual relationships (Goldschmidt, 1991) and promote divergent thinking (Fish &
Scrivener, 1990). These tools also carry with them their own learning curves and cognitive
demands, which can impede students’ causal reasoning about how the products worked,
and their ability to identify problems and communicate effectively about issues with their
designs (Kimbell & Stables, 2008, pp. 203–205).
Descriptions and structured reviews of design ideas Research has shown that both profes-
sional and student designers use verbal descriptions more frequently and in more critically
useful ways when communicating early design ideas than by freehand sketching or com-
puter-aided drawing ( Jonson, 2005). Verbal statements can be more effective than sketch-
ing in supporting designers’ explorations of evolving ideas and may be the most powerful
strategy teachers can recommend for helping students with meager sketching capabilities.
Periodic, structured conversations where students have small-group peer review discus-
sions can help advance students’ design ideas (Kimbell, 2004, p. 137). One such format
developed by Kimbell, Stables, and Green (1996, p. 14) had a significant impact on the
direction of students’ early design thinking and work, and involved each member of a de-
sign team giving a two-minute update to the rest of the group on her or his current project
The Informed Design Teaching & Learning Matrix 761

thinking, including what had been recently accomplished and why. The short presenta-
tion ended with what would be done next, after which the rest of the students gave feed-
back to the presenter.
Artifacts and gestures as stand-ins for drawings Harrison and Minneman (1996)
describe how experienced designers used nearby objects as props when discussing their
designs ideas. Hand gestures may act as stand-ins for objects or drawings and may help
direct the attention of others during design conversations (Heiser et al., 2004; Visser,
2009). Using and making reference to physical prototypes can help clarify and stabilize
meaning making among designers, while supporting the integration of relevant concepts
with these artifacts (Roth, 1995b). Such strategies, which Logan and Radcliffe (1998)
called “artefacting,” may help design team members from different disciplines communi-
cate better with one another.

Matrix Pattern E. Ignore vs. Balance Benefits and Trade-offs

THE STRATEGY OF WEIGHING OPTIONS AND MAKING DECISIONS Beginning de-
signers ignore or pay too little attention to design criteria and constraints, and focus only on
positive or negative aspects of their design ideas without thinking of associated benefits and
trade-offs. Informed designers balance systems of benefits and trade-offs when they consid-
er various plans, make design decisions, and justify them.
Design is a special kind of problem solving that is honeycombed – start to finish –
with decision points that require making dozens, even hundreds, of choices (Akin & Lin,
1996, p. 55; Strobel & Pan, 2011). Reasoning about the benefits and trade-offs of differ-
ent design alternatives is a cornerstone of design thinking and decision making (ITEA,
2000) and is needed for designers to resolve conflicts among a plurality of objectives when
designing (Rittel & Webber, 1984; Jonassen, Strobel, & Lee, 2006). Nelson and Stolter-
man (2003) describe the link between intention and judgment as a hallmark of “the de-
sign way.” Decisions abound as designers move from one problem to the next, including
picking the best material for fabrication, selecting from among a range of manufacturing
methods, and choosing among conflicting demands of different users’ needs. In the earlier
phases of designing, qualitative judgments about which solution pathways to pursue or
pass over can be made quite quickly – sometimes based on back-of-the-envelop calcula-
tions or estimations (Linder & Flowers, 2001), the invocation of practice-based guidelines
(Brand, 1994, p. 135), and patterns or “rules of thumb” (Alexander, 1979, p. 179).
Beginning designers can be oblivious to the unavoidable tensions and trade-offs asso-
ciated with design. Their decisions may be made based on criteria that are unstated, in-
choate, or in flux. When asked to describe their design decisions, they may tell only of the
benefits of their preferred design choices, while neglecting to mention associated trade-
offs. They likewise may highlight the negative aspects of less favored approaches while
passing over potential benefits. They may apply guidelines for making design decisions
without critically examining their applicability to the problem at hand (Goldstein &
Hogarth, 1997, p. 13). Informed designers, on the other hand, are practiced at weighing
and articulating both the positive features and drawbacks of ideas that they are about to
select or reject and look for potential downsides even with the most promising ideas.
Skill theory (Fischer, 1980, 2006) describes development using cognitive structures
called skills and rules for transforming those skills through increasing levels of complexi-
ty. Skill theory and its terminology can be used to describe how designers develop
762 Crismond & Adams

increasingly complex ways of reasoning about benefits and trade-offs when making de-
sign decisions. When beginning designers propose a new feature or product idea, they
initially would focus only on the positive [P] or negative [N] aspects of that idea but not
both. Each amounts to a separate, single representation set of that design idea, [P] or [N]
(Fischer, 1980, p. 490; Fischer & Bidell, 1998). A more complex way of thinking of the
design idea would involve coordinating or mapping the positive and negative features for
that design [P – N]. As more design ideas get considered and the strengths and weak-
nesses of each get identified and analyzed, a system composed of those ideas can form
[APN – BPN – CPN], making decision making among design options more thorough and
thus informed. Until students can coordinate the benefits and trade-offs of design ideas
they are considering, they will need support in doing this aspect of informed design
thinking.
Teaching strategies Research in judgment and decision making suggests two broad
approaches for helping people make informed choices, both of which may be applicable
when doing design. A reasoning-based approach would support decision makers in using
qualitative verbal arguments to articulate rationales for different alternatives they are con-
sidering. A second approach involves teachers presenting students with graphic represen-
tations of decisions based on formal, value-based models. Such systems quantify alterna-
tives by assigning values and weights to different factors, which are then aggregated to
make a final choice (Shafir, Simonson, & Tversky, 1997).
Explanation-based designing With this approach, teachers scaffold students’ thinking
about the benefits and trade-offs of different design options by regularly asking them to
provide explanations for their design decisions, including description of both the posi-
tives and negatives of different choices. Students can also use experiment-based design
advice or science and engineering principles to explain and support decisions they have
made. McKenna, Linsenmeier, and Glucksberg (2008) found that student designers in a
senior capstone course were more likely to support their design decisions with explana-
tions based on computational and analytical evidence than were students in a freshman
design course.
Decision diagrams In the Nuffield Design and Technology curricula, the Chooser Chart
is a highly visual and iconic rendering of a standard decision matrix (Barlex & Givens, 1995,
p. 50; see Figure 2). The chart supports students’ design thinking by representing and ag-
gregating the full range of design alternatives, with each option (row) scored using one to
four bullets for each of the criteria noted at the top of the chart’s columns. Students arrive at
a design decision regarding what option to use either by totaling the number of bullets for
the criteria they have selected as high priority or by simply eye-balling the chart for the op-
timal collection of bullets. Students can also be asked to create their own chooser charts,
based on research and tests they conduct.
The House of Quality diagram (Figure 3) is among a number of sophisticated decision
matrices that help designers translate user needs and preferences into measurable product
performance outcomes (Hauser & Clausing, 1988). Final design decisions are based on
totals of summed weighted or raw scores. Some experienced designers avoid slavishly fol-
lowing the calculated ranking these charts generate, holding that gut-feeling impressions
can be just as important for considering product feasibility (Ullman, 1997, p. 155) and
making design choices. Despite these differing views, the very act of having students cre-
ate decision charts or matrices may help them further develop their ideas by articulating
priorities and judging anticipated performances for a range of design alternatives.
The Informed Design Teaching & Learning Matrix 763

FIGURE 2. Nuffield’s Fastenings Chooser Chart helps students reason about the competing features of
different garment fasteners.

Design values and guidelines Design values permeate and impact designing; they influ-
ence both how designers initially perceive and frame the task and how they evaluate ideas
and complete their projects (Bucciarelli, 1984). These values can address issues related to
product quality, including designing for reliability, manufacture and assembly, and design-
ing for sustainability (McDonough & Braungart, 2002; McLennan, 2004; Ullman, 1997).
The human-centered innovation approach, for example, aims to have designs simultane-
ously meet the needs of desirability, viability, and feasibility (Brown, 2009). Simplicity is
another powerful design value relevant to many fields of design, since simpler systems
tend to be more robust and reliable than are more complex ones. Trade magazines and
books on design admonish both beginning and expert designers to avoid “creeping fea-
tures” (Ullman, 1997) and “feature creep” (Kemper, 2003), and buttress such advice with
764 Crismond & Adams

FIGURE 3. The House of Quality diagram graphically depicts different user preferences, product features,
and their relationships to one another (from Dym & Little, 2004).

descriptions of the dire consequences of creating designs that are overly complicated.
Teachers use various forms of the acronym KISS (keep it super simple) to promote ele-
gance, parsimony, and simplicity in their students’ design work.
Emotions and their role in design decision making Weighing benefits and trade-offs of
various design options is necessary to an informed designer’s thinking, but it is not suffi-
cient for making decisions. Findings from cognitive neuroscience studies point to the es-
sential role that emotions play in decision making. In Damasio’s study of a brain tumor
patient named Eliot (2005), damage during surgery to an area of this person’s prefrontal
cortex led to his complete loss of emotion, and with it any ability to function with friends,
family, and work. People who suffer from such a condition, when asked to make a simple
decision such as choosing between two dates for an appointment, are known for spending
extended periods of time “enumerating reasons for and against each of the two dates” and
conducting “a tiresome cost-benefit analysis, an endless outlining and fruitless comparison
of options and possible consequences,” yet are incapable of making a choice (Lehrer,
2010).
While a mystique regarding the role intuition plays in an expert’s decision making
should be avoided (Alexander, 1964, p. 6), incorporating both analytical insights and incli-
nations for some options over others based on one’s feelings should be considered as a
teaching strategy when supporting students in their design thinking. In his teaching of the
The Informed Design Teaching & Learning Matrix 765

undergraduate and graduate introduction to design courses at MIT, Pappalardo Professor

of Mechanical Engineering Woodie Flowers would ask his students – after they had spent
time analyzing their design options – a unique and subjective question: “Which idea smells
good to you?” This prompt aimed to get students in touch with the feeling side of their en-
gineering thinking and encouraged a productive dialog between rationality and emotions
when attempting to make a design decision.

Matrix Pattern F. Confounded vs. Valid Tests and Experiments

THE STRATEGY OF CONDUCTING TESTS AND EXPERIMENTS Beginning designers
run few or no tests on their design prototypes. When they do them, they conduct confounded
experiments by changing multiple variables in a single experiment, which yields little un-
derstanding about potential solutions. Informed designers run valid tests as part of techno-
logical investigations that help them to learn quickly about design variables, users, and ma-
terials, to understand how things work, and to optimize the performance of the prototypes
they decide to develop.
When the limits of research have been reached and Internet sources have been ex-
hausted, designers must generate their own data through conducting inquiry-based labo-
ratories (Litzinger, Lattuca, Hadgraft, & Newstetter, 2011) where students plan investi-
gations and interpret data they gather to check on their own predictions and possible
explanations (hypotheses) of prototype behavior. Professional designers, acutely aware of
project deadlines and conscious of time constraints, favor ways of doing tests that generate
insights quickly rather than doing more time-consuming approaches that possess greater
scientific rigor (Norman, 1996, p. 98). Still, they occasionally do controlled experiments,
although they do not run such tests to refute a formal hypothesis, since their results need
not be replicated (Cross, 2001b, p. 51).
In most professional settings, the scientific quest to understand why runs a distant sec-
ond to a designer’s need to know what and how. This is far less the case in schools where
design activities are being used to support students’ learning of STEM concepts (APASE,
1991; Frye, 1997; Kolodner et al., 1998; Sadler et al., 2000; Zubrowski, 2002a; Fortus
et al., 2004). Schauble, Klopfer, and Raghavan (1991) distinguish between engineering
and science models of experimentation. They suggest that the engineering model aims to
“optimize a desired outcome” and can address technical and social dimensions, while the
scientific model of experimentation aims to identify causal links between predictor and
outcome variables. They consider the engineering model to be more consistent with ev-
eryday problem solving and note that it appears in students’ behavior before students do
well-planned and properly controlled experiments, which is a key goal in the teaching of
science, of their prototypes.
Informed designers also use social science methods when conducting informal inter-
views, administering questionnaires, running focus groups, and observing users interact-
ing with a product. The educational opportunities to reinforce the authentic practice of
scientific practices while designing, especially inquiry, have been reported in a review of
the use of inquiry-based approaches in engineering education settings (Litzinger et al.,
2011, pp. 136–138), and have been incorporated into recently published K–12 curricula
and in undergraduate engineering education settings (Sheppard, 2009).
Teaching strategies A number of recently published design-based science curricula
(e.g., Project-Based Inquiry Science’s Diving into Science, GTRC, 2010; Science by
Design’s Construct-a-Catapult, TERC, 2000; Challenges in Physical Science’s
766 Crismond & Adams

Electromagnets, Coyle, 2001) ask students to plan and conduct controlled scientific experi-
ments where they identify a single product feature to vary, fix all other variables, and then
determine which product performance outcomes to measure and how (see
Harlen, 2001). Such plan-and-investigate tasks can be preceded by students having
whiteboarding discussions that build upon the KWL chart (What do you know? What do
you want to know? What did you learn?; Ogel, 1986) – where students list facts, ideas, and
learning issues related to investigating the impact key design variables have on a system’s
performance (Kolodner et al., 2003).
The following are types of investigations students can do to build the system and de-
vice knowledge they need to address design challenges in an informed way.
Experiment-based design advice Students can use results from tests that they conduct
on their prototypes to formulate experiment-based design advice that describes the im-
pact of altering single design variables on a product’s performance. Students report these
application-ready findings from their tests as design advice, also referred to as design rules-
of-thumb (Crismond, 2001, p. 796; Crismond, Camp, Ryan, & Kolodner, 2001; Kolodner
et al., 2003; Kolodner et al., 2004; Crismond, 2011), to help other classmates with their
own design planning and decision making. Such findings act as intermediate abstractions
(White & Frederiksen, 1993) that link science ideas and the concrete realities of particular
mechanisms and products and how they work.
Investigate-and-redesign task One form of technological investigation is the I&R task
sequence (Crismond 1997, 2001), which begins with a novel design task (Hawkins, 1990)
where students examine unfamiliar devices to try to identify them and describe their func-
tions. In the analyze devices step, students take multiple instances of the same product and
then use physical and engineering science concepts to explain how they work and make
predictions about their comparative performances on a given task. In the ideal feature step,
students list preferred functions of an ideal version of the device while disregarding issues
such as cost, feasibility, or customer appeal. They then plan an experiment that compares
different brands of the device being analyzed, do the conceptual redesign of the product, and
end by reflecting on the processes they used in doing this work.
Product comparisons Conducting a Consumer Reports-styled experiment that compares
different brands of the device, or product comparison, can build an understanding of key
product features and performance behaviors needed to do the informed redesign of a de-
vice (Crismond, 2001). Students doing product comparisons can report on how different
devices use different approaches to perform the same task and can elaborate on how differ-
ent user needs and preferences were addressed by different brands of the same device. Such
work may also enhance analogical transfer of solutions to the current design problem, while
helping students gain insight into science and engineering principles (Loewenstein,
Thompson, & Gentner, 1999). For teachers, device selection is critical in devising an effec-
tive product comparison activity: the design approaches and performances of candidate
items of the same product type need to be sufficiently different to make the comparison in-
teresting to students, while still illuminating key design problems and issues.

Matrix Pattern G. Unfocused vs. Diagnostic Troubleshooting

THE STRATEGY OF TROUBLESHOOTING Beginning designers have an unfocused,
non-analytical way of viewing the plans and performance tests of prototypes when trou-
bleshooting their designs. Informed designers focus their attention on
The Informed Design Teaching & Learning Matrix 767

problematic areas of their potential solutions and products while doing effective diagnostic
troubleshooting.
Some designers will not abandon their design ideas, even after running many tests and
design iterations that clearly demonstrate a plan’s ineffectiveness. Beginning designers can
look uncritically, in a coarse-grained, undifferentiated, and unfocused way, at their plans
and prototypes’ performances when troubleshooting these designs. They can generate and
test hypotheses about them at random ( Johnson, 1988), while not actively looking out for
worrisome patterns when testing prototype performance. This can result in them seeing
and describing as “satisfactory” what an experienced designer would call a “flawed perfor-
mance.” These designs are thus inoculated from change, despite numerous design itera-
tions, and can end up looking strikingly similar to concepts that were proposed from the
very beginning.
Design-based troubleshooting shares a number of traits with classical troubleshoot-
ing, the latter being a form of moderately ill-defined problem solving that involves an
attempt “to isolate fault states in a system and repair or replace the faulty components in
order to reinstate the system to normal functioning” ( Jonassen & Hung, 2006, p. 26). A
competent technician charged with fixing an existing product that is broken possesses
conceptual models and skills that include system and device knowledge, appropriate
domain knowledge, and awareness of a system’s physical layout and topography, as well
as knowledge of relevant procedures for troubleshooting the product and testing for po-
tential faults ( Jonassen & Hung, 2006). Topographic knowledge (Rasmussen, 1984)
involves an understanding of a device’s physical structure, which guides practitioners in
locating sources of particular problems. Functional knowledge speaks to how the system
works and how subsystems and their components interact. Domain knowledge ad-
dresses the science and engineering principles that explain the product’s basic functions
(e.g., Hooke’s law for the return spring of a water pistol’s trigger; Newton’s laws of mo-
tion for model car’s acceleration). While such ideas and practices alone are not sufficient
for effectively diagnosing faults ( Jonassen & Hung, 2006; Morris & Rouse, 1985), they
can enable practitioners to transfer troubleshooting skills to new settings (MacPherson,
1998), which can also be helpful when troubleshooting an evolving design concept or
prototype.
When the process of design-based troubleshooting occurs during conceptual design
and the artifact does not yet exist in a testable form, informed designers must run mental
simulations of how the envisioned device or system might work. They need to imagine a
“sequence of salient events in causal order” (Mioduser, Venezky, & Gong, 1996) in order
to make a feasibility judgment and to predict sources of poor product performance (Ull-
man, 1997; Adams et al., 2003). When testing physical prototypes, design-based trouble-
shooting involves actively looking for critical events and patterns of behaviors that diverge
from this mental model. These harbingers of lackluster performance or abject product
failure help to differentiate well-conceived products from those that need tweaking and
those plans that should be discarded. When faults or flaws are noticed, observers zoom
their attention in and then out when examining the system’s performance. Such narrow-
ing and focusing of attention helps practitioners isolate faults by reducing the complexity
of the system being considered. This lessens the load on working memory, which in turn
can improve troubleshooting performance (Axton, Doverspike, Park, & Barrett, 1997).
Informed designers also use case-based reasoning to recognize patterns of current system
performance based on similar cases they have encountered and can use causal reasoning to
follow anomalous events backwards to one or more root causes, which may reside in the
768 Crismond & Adams

design’s conceptualization, construction, or even the designer’s testing methodology. Ex-

perienced authors use similar processes in revising their compositions (Flowers, Hayes,
Carey, Schriver, & Stratman, 1986).
Teaching strategies The role that diagnostic troubleshooting plays in designing is an
understudied area of research. It resembles “diagnostic judgment for action” (Rasmussen,
1993), where practitioners search a system for behaviors, components, or subsystems upon
which to act, and parallels activities associated with scientific inquiry (Harlen, 2001), in-
cluding observing, noticing recurrent patterns in data, explaining phenomena, and doing
design-based experiments (Pattern F).
Disparities between intended function and actual product behavior during prototype
testing that go unnoticed cannot contribute to the iterative improvement of a design. The
potential for what students can learn through iterative design gets diminished when stu-
dents’ attention remains diffused and unfocused, and when students fail to notice and at-
tend to critical and problematic features in their designs. The following teaching strategies
aim to help students develop their design-based troubleshooting capabilities.
Diagnostic troubleshooting Diagnostic troubleshooting has been practiced and studied
as a structured, inquiry-based, four-step procedure (Crismond, 2008; 2011). The first step
involves a particular kind of observing that entails looking at a product’s overall perfor-
mance during early prototype testing in order to detect unexpected or out-of-range be-
haviors. The second step involves the actual diagnosis of the problem, where the designer
gives a name to the problems noticed in the product’s performance. Next, an explanation
of why those behaviors occur is offered, which can help students causally link faulty per-
formance with specific elements of the planned or actual design. The last step, less analyti-
cal than the others, involves proposing ways to remedy and fix the design or prototype.
Detected flaws can inspire ideas for simple fixes, additional features, or entirely new and
unimagined systems. A new iteration of the product then gets built and tested, which goes
through additional diagnostic troubleshooting cycles as needed. As with scientific discov-
ery, noticing unexpected properties or behaviors during testing can become a powerful
impetus for reconceptualizing product criteria and constraints (Wills & Kolodner, 1996).
Cognitive training in troubleshooting The cognitive training model has been used to
teach troubleshooting in a range of well- to ill-defined problem settings (Foshay, Silber, &
Steinicki, 2003). When used in an instructional context involving design challenges,
learning activities could primarily aim to help students build mental models of how the
device and its subsystems work. Lessons could review relevant science and engineering
principles and present case studies of typical failure modes with their causes and remedies.
Procedures for testing and repairing identical devices or analogous systems, as well as basic
troubleshooting heuristics, could be taught and practiced using real devices or computer
simulations of broken devices or systems ( Jonassen & Hung, 2006).
Troubleshooting stations Setting up separate activity stations, each with a device that
possesses a noticeable flaw that needs troubleshooting, can engage students in developing
(while helping assess) troubleshooting capabilities. Zubrowski produced a collection of
videos (EDC, 1999) that shows fourth grade students who were engaged in a model
windmill design activity visiting each of four troubleshooting stations. At each table, stu-
dents found a different windmill prototype with a significant design or fabrication flaw
that made it perform very poorly or not at all. Students were asked to identify and remedy
the problem at each troubleshooting station and then as a class discussed their insights
and ideas.
The Informed Design Teaching & Learning Matrix 769

Teacher modeling of troubleshooting Teachers can show students how to employ diag-
nostic troubleshooting strategies by modeling for students how they analyze poorly func-
tioning devices themselves. An instance of such coaching can be found in the Design in the
Classroom Web site (GTRC, 2004b) when a student appeared not to be attending to nota-
ble performance flaws when testing a model parachute. The technology education teacher
in the video, after noticing that the student was not watching the parachute’s descent care-
fully, modeled how to make more pointed observations:
“Watch it as it falls, from the beginning to the end.”
(Student releases her parachute; it partially collapses midway during descent.)
“See where it stops floating slowly, and then changes speed. So you want to
figure out why it’s doing that.”
Teacher comments and questions can help shift a student’s analytical perspective and
way of looking so that it includes broadening or zooming out attention in order to assess
overall product performance when needed, and narrowly focusing or zooming in attention
in order to conduct detailed critiques of specific subsystems and explore their connections
to other parts of a device.

Matrix Pattern H. Haphazard or Linear vs. Managed and Iterative Designing

THE STRATEGY OF REVISING AND ITERATING Beginning designers design in hap-
hazard ways, working at random on whatever problems emerge, or they treat design as a set
of strategies to be done once in linear order. Informed designers do design as an iterative
process, while improving ideas and prototypes based on feedback and cycling back to upgrade
their understanding of the problem. They manage their time and resources strategically and
use design strategies multiple times in any order, as needed, in a systematic way.
Some beginning designers move haphazardly from one emerging problem to the
next, acting on impulse and in ways that seem unplanned and random. When asked at
any point in their designing what strategies they might use next, they are able to iden-
tify few if any procedures to further their work. Beginning designers also follow steps
to produce a finished product “as though designing is a serial/linear process,” where it-
eration “is not in their process model” (Newstetter & McCracken, 2001, p. 68) or is
described by them as a poor use of time (Adams & Fralick, 2010). The mistaken idea
of design as a linear process can be conveyed through the graphics of design models
employed by authors and school textbooks (Figure 4).
Informed designers agree with statements about how “Design is iteration” and disagree
with statements about how “Good designers get it right the first time”
(Mosborg et al., 2005). They are aware of the ambiguities inherent in complex engineering

FIGURE 4. An example of a published design model that depicts engineering design as a linear process.
Cited by Kimbell (1994, p. 334).
770 Crismond & Adams

design tasks: that neither the problem nor the goals are well defined and that there are
many ways to frame a problem, multiple plausible solutions, and no stopping rules other
than a designer’s judgment (Cross, 1992; Goel & Pirolli, 1992; Grant, 1992). In actual
practice, designers cannot work in a linear fashion, since practices that often appear as a
final step in design process models, such as evaluation (Kimbell, 1994) or communication
(Mosborg et al., 2005), occur repeatedly throughout the process. Overall, experienced de-
signers combine problem structuring and solving in an iterative process (Adams et al.,
2003; Ennis & Gyeszly, 1991; Lawson & Dorst, 2009), self-monitor their progress, flexi-
bly adapt to new insights (Adams, 2001), chunk strategies together to reach viable design
goals (Atman et al., 2007), and strategically and opportunistically deviate from a step-by-
step design process as needed (Ball & Ormerod, 1995).
Iterative design, represented in design process models as feedback loops or check-and-
reflect points within and across subtasks, is considered an “integral feature” and “natural
attribute of design competency” (Adams et al., 2003). It has been described as involving
multiple analysis-evaluation-synthesis cycles where an understanding of the task simulta-
neously coevolves with developing a solution (Archer, 1979; Lawson & Dorst, 2009) as
“cycles of proposal, testing, and modification of an evolving design” (Smith & Tjandra,
1998), global and local iterative cycles ( Jin & Chusilp, 2005) and goal-directed transition
behaviors (Adams & Atman, 2000). From our own experiences, we refer to iterations as
“another pass,” “the next version,” or even “starting over.” For it to work well, iteration in
design must engage meaningful learning, where improved solutions grow out of an evolv-
ing understanding of the problem, and new insights and information get continually as-
similated into a designer’s understanding of the design task (Adams, 2001; Adams et al.,
2003; Schön, 1983; Ullman, Herling, & Sinton, 1996; see also Patterns A and I).
Iterations may occur spontaneously, such as reacting to a particular dilemma or oppor-
tunity, strategically, such as planning to revisit and review prior design decisions about the
problem, or as part of an overall design approach. The outcomes of iterative designing can
be modifications to a problem, solution, or design plan, or simultaneous coupled iterations
(Adams, 2001). For each adjustment made, the designer must not only analyze the effects
of the change but also reevaluate the design task. Iterative problem scoping cycles (see
Pattern A) result in reformulating problem requirements and modifying and adapting to
changing perceptions of the design problem or constraint migration (Chandresekaran,
1992), which has been associated with the development of more creative solutions (Akin,
1994; Akin & Lin, 1995). Solution revision cycles (see also Patterns C, E, F, G, and H)
may occur because a solution failed to work or satisfy one or more design requirement, or
new requirements emerged during the evaluation of a solution (Braha & Maimon, 1997;
McGinnis & Ullman, 1992).
The use of iteration strategies constitutes effective design practice (Adams et al., 2003)
and supports learning by allowing the designer to continually revisit and reflect upon each
aspect of the design task (Braha & Maimon, 1997; Bryson, Bereiter, Scardamalia, & Joram,
1991). Doing iterative designing has been found to play a significant role in the quality of
final design solutions and involves “incrementally and simultaneously advancing upon both
a representation of the problem and a final solution” (Adams et al., 2003, p. 286). In a com-
parative study where engineering freshmen, seniors, and practicing professionals were
asked to do the conceptual design of a playground, increased time spent iterating (Adams
et al., 2003; Adams, 2001) was associated with higher quality design solutions to a statisti-
cally significant degree.
The Informed Design Teaching & Learning Matrix 771

Design iterations mark a journey from an under-specified starting point to an elusive

target goal (Goel, 1989; Hybs & Gero, 1992). At any point along this journey, as designers
attempt to make sense of information about the problem, generate plausible solutions,
and evaluate the feasibility and desirability of solutions, they uncover new complexities or
opportunities and gain new insight that may uncover hidden assumptions, missing infor-
mation, or contradictions that need to be resolved. Teachers who have their students per-
form a design model’s steps only once, who fail to explain why those strategies are relevant,
or who neglect to emphasize that they might be altered or done in a different order, can
leave students thinking that design steps are recipes or mere rituals (McCormick et al.,
1994, p. 38) and believing that designing is a single-pass, linear process (as in Figure 4).
The following teaching techniques aim to help students develop as informed designers
who use iteration strategies that support continual learning while revisiting and reflecting
upon each aspect of the design task (Adams et al., 2003).
Teaching strategies Understanding the role that the build-troubleshoot-improve cycle
plays in iterative design and gaining proficiency in planning and managing project time
are fundamental and achievable practices that students can learn. A critical and perhaps
threshold concept is developing an awareness of and tolerance for ambiguity in design
(Adams et al., 2003; Cross, 2001; Daly, Adams, & Bodner, 2012), as well as associated
strategies for reducing or preserving ambiguity (Lande & Leifer, 2010). The following are
ways for teachers to support students in becoming proficient in doing well-managed and
iterative designing.
Design storyboards Teachers can collect evidence of students’ haphazard or systematic
designing by having them create a design storyboard, a graphic that holds a
series of sketches or digital snapshots, from early designs to interim prototypes to final
product. Such displays can be accompanied by short verbal descriptions and explanations
of design decisions and aim to “document the story of how a challenge was met over time
. . . each frame displaying the latest solution to the challenge” (Sadler et al., 2000).
Project and time management Teachers can help students become skillful at doing
design-based project management by setting deadlines for them that “provide staging posts
which focus attention on the decisions that need to be made” (Barlex & Wright, 1998,
p.163). These staging posts can include establishing deadlines for sketches, requesting
weekly progress reports, conducting class reviews of prototypes, and, perhaps most impor-
tantly, giving final project presentations to classmates or invited professionals who can act as
outside judges (GTRC, 2004c). Since design has no inherent stopping rules (Goel & Pirro-
li, 1992), designers can perform too many revisions or be ineffective while doing iterative
designing (Adams, 2002), as seen when overly motivated beginning designers endlessly
tweak their design plans and prototypes. Such designers can benefit from being coached in
“freezing a design,” where a time is set after which changes to the design cannot be made.
Flexibly adapting the strategies found in a design process model to meet the needs of the
moment should be encouraged, but not expected of all students since it is a hallmark of de-
sign expertise. The Layers of Necessity Model (Wedman & Tessmer, 1991) articulates how
instructional designers adjust their use of the ADDIE instructional design model (Smith &
Ragan, 2005) based on the time remaining before a project deadline is reached.
Instruction and scaffolding for systematic design Atman and Bursic (1996) found that
simply having undergraduate engineers read a textbook’s description of the design process
favorably impacted the quality of students’ design work and resulted in them doing more
transitions and using more design strategies. Case studies can depict experts modeling a
772 Crismond & Adams

variety of strategies in design thinking in various contexts (Barlex & Wright, 1998), which
can help students realize the power and utility of iterative design. Teachers themselves can
illustrate the role that iteration plays in design or any other design strategy through direct
instruction, offering comments on student projects, or modeling such behavior using a
flawed prototype or product. In one freshman year engineering course at Purdue where it-
erative designing was consistently scaffolded across multiple design tasks, there was a sig-
nificant pre- and post-difference in the increase in importance that students attributed to
the role iteration plays among a set of 23 design strategies and activities (Adams & Fral-
ick, 2010).
Risk-taking and iteration Some students can be overly cautious when designing and
need support in taking risks with their design ideas. In a meta-analysis of studies that ex-
amined the risk-taking tendencies of males and females in a variety of contexts, large gen-
der differences were noted when intellectual risk-taking was involved (Byrnes, Miller, &
Schafer, 1999, p. 378). Kimbell describes research that showed notable differences in the
design work of male and female U.K. design and technology students, with girls outper-
forming boys in reflective tasks like investigating and evaluating ideas while boys were
better at idea generation and development (Kimbell & Stables, 2008). Kimbell also
noted that girls seemed to take fewer risks with their plans and ideas than boys
(GTRC, 2004d). Design instructors have encouraged risk-taking by offering students
case studies and personal anecdotes where accepting and building upon failure proved
a viable pathway for improving design ideas. A well-publicized motto of the product
design firm IDEO builds upon this idea: “Fail often in order to succeed sooner”
(Kelley, 2001, p. 232).

Matrix Pattern I. Tacit vs. Reflective Design Thinking

THE STRATEGY OF REFLECTING ON PROCESS Beginning designers do tacit designing
when they think and act with little self-reflection and do little monitoring of their own or others’
actions. Informed designers practice reflective thinking by keeping tabs on their own and others’
design work in a metacognitive way and reviewing their processes and products once they have
completed their work.
Beginners do design tacitly when they act with little or no awareness of what they are
doing, do not articulate what knowledge they know or need to know to further their inves-
tigations, and pay scant attention to the progress they make, obstacles they encounter, or
design values that influence their decisions. They can fail to review steps they have taken or
to examine the assumptions underlying their framing of the design problem. They leave
their knowledge of their designs and design process implicit and unarticulated, which can
limit their ability to transfer knowledge they have accumulated to new situations.
Metacognition is the knowledge and monitoring of oneself and others, tasks done, and
strategies and understandings used (Flavell, 1979). For students, it involves the “ability to
stands outside themselves and look in at their own designing” (Kimbell & Perry, 2001),
follow their own and others’ actions in order to coordinate the switching of strategies
(Sternberg, 1997), and revisit assumptions when faced with evidence that their assump-
tions have become problematic (Perkins, 1995, pp. 218–220). At its core, metacognition
in design involves a kind of reflection in action and self-monitoring that provides feed-
back to improve both process and product. It entails the epistemic belief that understand-
ings change with experience (Flavell, 1979, p. 907) but can be as difficult to learn as core
science and engineering concepts and design thinking itself (Flowers et al., 1986, p. 21).
The Informed Design Teaching & Learning Matrix 773

Metacognitive thought is closely linked to design decision making (Kimbell & Perry,
2001) and is associated with higher levels of design performance and product quality
(Adams & Atman, 2000). Informed designers reflect on the viability of the plans they
propose and the lessons learned from past design efforts – both of which are manifesta-
tions of their own mental models of the system they are developing (Elmer, 2002). These
ideas are common themes in studies of how designers engage in reflective practice and
learn through design (Adams et al., 2003; Dorst & Lawson, 2009; Schön, 1984; Valken-
burg, 1998).
Teaching strategies Early research of children designers’ computer programming in
Logo reported improvements in students’ reflectivity and metacognitive thinking (Clem-
ent & Gullo, 1984), although such gains were limited to the contexts in which students
had originally worked (Pea & Kurland, 1984). Middle school students who used Georgia
Tech’s Learning by Design™ curriculum were found to perform metacognitive thinking
better than students in control classes who did no designing (Kolodner et al., 2003). The
following are some of the approaches that have been created to support students’ design-
based reflective thinking.
Design diaries and portfolios Design portfolios can play a variety of roles in scaffolding
students in their design work, even though some students maintain that they find keeping
design diaries, journals, and portfolios distasteful, and do so only to enable their instruc-
tors to assess their work (Welch & Barlex, 2004). The paper-and-pencil Design Diary in
the Learning by Design™ curriculum provided students with single-page worksheets that
supported them in reflecting on what they knew and needed to learn while generating
questions in the Problem Understanding sheet (see Pattern A), reviewing what they
learned from early investigations in the Messing About section (see Pattern D), and
combing through results of fair-test experiments (see Pattern F) in the Design Tests pages
(Kolodner et al., 2004). Some portfolio systems act as a sketchbook to support creative
idea production and reflection, or as a job bag to aid students in scheduling and monitor-
ing the progress of design work; others help students showcase their ideas for final presen-
tations (Welch, Barlex, & Taylor, 2005). Portfolios can also help students recall and evalu-
ate key events in their individual and group design work, construct explanations for design
decisions they have made, and articulate their values in engineering practice (Dunsmore,
Turns, & Yellin, 2011, p. 338). Keeping portfolios can help learners describe details of
their own process (Sobek & Jain, 2007), construct and articulate their own philosophy of
design (Hirsch & McKenna, 2008), and make evidence-based claims about how they are
prepared for future practice (Turns et al., 2010).
Compare and contrast design cases Numerous groups have engaged students in review-
ing transcripts, videos, or samples of their own or others’ design work to support reflective
thinking in design. Students have done design tasks reported in published research, com-
pared and contrasted their design practices with those used in the reported studies, and
then reflected upon and discussed differences and similarities (Turns et al., 2003). Stu-
dents can achieve highly insightful observations and reflections on what strategies and be-
haviors contribute to design success and failure by reviewing videos showing other design
teams at work (e.g., video segments from the PBS show Design Squad; Purzer, 2010) or
watching clips of another team doing the same design task that the students have just
completed (see GTRC, 2004e). Students’ understanding of design has improved after
coding verbal protocols of individuals designing a playground for various design behav-
iors, and then later engaging in inter-rater reliability discussions (Scott et al., 2001) and
774 Crismond & Adams

critiquing protocol-analysis derived timelines that depict individual designers’ processes

(Atman, Deibel, & Borgford-Parnell, 2009). These approaches can lead to rich discus-
sions regarding problem framing and formulation, differences in values that drive design
decisions, and successes and challenges in doing collaborative and cooperative work in
teams.
Computer-supported structured reflections Computer technologies provide a number of
scaffolds that can enhance an individual’s or group’s reflective thinking about designing.
These include (a) electronic versions of structured diaries that offer process prompts and
pose structured questions to help students attend to particular aspects of their design work
(e.g., Turns et al., 1997); (b) synchronous and asynchronous online systems that support
learning communities in engaging in reflective social discourse (Lin, Hmelo, Kinzer, &
Secules, 1999); (c) integrated assessment and learning systems that elicit student reflection
in the context of design experiences, while supporting formative and summative assess-
ment of professional design skills (Davis et al., 2011); and (d) multiple-perspective process
displays that visually depict processes used from different viewpoints (Schauble, Ragha-
van, & Glaser, 1993).
Providing students practice in identifying their own design strategies may enhance their
design-oriented metacognitive thinking. An NSF-funded Design Compass software has
been used to support students, grades 8–16, in coding their own or others’ actions while
designing (Crismond, Hayes, & Danahy, 2010). The Design Compass scaffolds users in
doing three basic functions: logging what design strategies they are using and when; ar-
chiving digital snapshots of drawings and movies of prototype tests; and viewing a histo-
gram the compass generates that displays the aggregate times spent using various strategies
during one or more design sessions. The Design Compass enables students to conduct more
data-driven conversations about their design work with their design instructors. Such data
can help students reflect on the effectiveness of their design strategy use and answer the
question, “How might we have allocated our project time differently among these design
strategies to improve our product and process?”

DISCUSSION OF CONTRIBUTIONS

The Informed Design Teaching and Learning Matrix is the product of a scholarship of
integration for engineering education that links a broad-based and multidisciplinary
scholarship of design cognition with an emerging scholarship of teaching and learning in
engineering design. Some of the challenges faced by this scholarship of integration effort
include (1) simplifying the scale and complexity of the cross-disciplinary design research
landscape, (2) formulating learning goals and gathering design strategies that have been
used to help designers learn to design, and (3) fashioning a use-inspired tool to enable
teachers to develop their own Design PCK and help students become informed designers.
Below, we summarize the key points related to the theoretical and practical contributions
of this effort and note limitations that open up opportunities for further research and de-
velopment.
We see two major audiences for this paper and its scholarship of integration approach.
For researchers, the Matrix contributes to educational theory the notion of informed de-
sign and key performance dimensions that students may achieve during their formal edu-
cation in K–16 settings. The Matrix also acts as a framework for enabling researchers to
(a) situate existing research findings related to engineering design cognition and studies of
the impact of various instructional methods on student design learning; (b) identify gaps
The Informed Design Teaching & Learning Matrix 775

in the field’s current research base; and (c) connect and situate results from future engi-
neering education research as well as validate and improve the current Matrix and educa-
tional theory of informed design.
For teachers, the Matrix may be helpful in building their own Design PCK and im-
proving their classroom practice when using engineering design activities. The Matrix
helps do this by directing teachers’ attention to common design misconceptions and habits
of mind of beginning designers, suggesting performances that students might achieve as
informed designers, and then compiling for teachers learning goals linked to instructional
strategies that are relevant to the patterns noted in the Matrix.

Implications for Educational Theory Building and Research

Bruner (1964) and Reighluth (1999) distinguish between two kinds of educational theo-
ries: ones that are mainly descriptive and deal with learning and development, and others
that are more prescriptive and offer guidelines on what ideas and skills should be taught
(i.e., learning goals) and suggest ways to achieve those goals effectively and efficiently (i.e.,
teaching strategies and pedagogy). In the following paragraphs, we describe how the Ma-
trix acts as an emergent educational theory and offer suggestions for improving engineer-
ing education.
The Matrix rests upon the notion of informed design and its collection of key perfor-
mance dimensions. When placed into service as a research framework, the Matrix can sit-
uate existing studies, as illustrated in the Unpacking section of this paper, while pointing
to gaps in the current research literature. For instance, studies regarding problem framing
have been done in elementary grades (Roth, 1995) and undergraduate settings, but not
middle school or high school venues. The Matrix as a research framework can also reveal
opportunities to study the relationships between predictor variables related to design be-
havior (single or combined patterns) and measureable outcomes of students’ design work
(e.g., performance of final products) and thinking (e.g., explanations for design decisions).
It can then provide a theory-based structure onto which new studies may be hung.
Although the Matrix’s starting points and end points may need further refinement
through empirical validation studies, and additional research will more finely differentiate
the Matrix’s current two-step progression statements, it still represents a crucial first step
towards developing a comprehensive conjectural model of K–16 design learning progres-
sions (NRC, 2007). Such work would require developing suites of psychometric instru-
ments that can reliably measure students’ use of design strategies over large tracks of time
and across different instructional contexts. Longitudinal studies that follow students’
learning of engineering design could reveal, hone, and refine detailed learning progression
statements that would make “informed teaching” with design activities more possible
for all.
The Matrix may also be combined with other theoretical frameworks to give rise to
new research questions, including “How is development of informed designer practices
related to students’ educational experience, motivation, self-efficacy, or persistence in
STEM fields?” Basic research could validate or refute causal claims linking aspects of de-
sign practice with the quality of students’ design products, thinking, and learning. Devel-
opmental studies could identify critical thresholds and investigate the applicability of de-
sign patterns across a wide age range from childhood to early adulthood.
This emergent instructional theory of informed design provides guidelines on ways to
improve students’ design learning. Here, the emphasis shifts from explaining students’
776 Crismond & Adams

design capabilities to offering a palette of viable learning goals and effective teaching
strategies, from which formative and summative assessments can be devised. This theory
may stimulate new research questions on pedagogical and teacher professional develop-
ment issues, such as “Which teaching practices best support specific elements of design
learning?” “What workshop and other experiences support teachers in developing their
own in-depth Design PCK?”

Implications for Practice: Using the Matrix in the Classroom

Efforts at improving teaching practice are often disconnected from educational research
and theories (Sowder, 2000; Arbaugh et al., 2010). The Matrix attempts to bridge this gap
by referencing and compiling research studies into a theory-based framework for re-
searchers and a planning tool and observation guide for teachers. Central to the Matrix’s
practical contributions is its aim to help teachers see their students’ design work from the
perspectives of an experienced design researcher, design educator, and a cognitive and
learning science researcher. In the following paragraphs, we provide three examples of
practical applications from professional development workshops for teachers and from
high school and college contexts for students. These illustrate how the Matrix can (1) pro-
vide teachers with a first-generation representation of Design PCK to help them con-
struct an evolving set of Design PCK understandings and schemas; (2) help teachers and
students monitor students’ evolving design skills, concepts, and dispositions; (3) broaden
the range of strategies that teachers use when using short- or long-term design projects;
and (4) improve students’ understanding of engineering design and ability to assess their
own design practices and work.
Teachers in professional development workshops and graduate courses The Ma-
trix has been used to build teachers’ awareness of Design PCK in recent professional de-
velopment workshops involving in-service elementary teachers, middle school technology
education and high school engineering teachers (e.g., Crismond & Peterie, 2011). At
these venues, teachers learned about design misconceptions and differences between be-
ginning and informed designers, compared and contrasted the practices of scientific in-
quiry and engineering design, and aligned new national science standards (NAS, 2012) to
the Matrix’ learning goals and instructional approaches.
A series of three graduate education STEM methods courses were taught at City Col-
lege of New York from 2011 to 2012 (Crismond & Adams, 2012), where in-service
teachers from New York City in the first semester learned about hands-on materials and
pedagogy related to scientific inquiry, and then in the second semester focused on doing
and using engineering design tasks with children. Drafts of this Matrix paper were the
main reading of the second course, where teachers for their final projects redesigned and
implemented published, design-based curricular materials with their students. The final
course in the series, Teacher as Designer, had teachers apply what they learned about engi-
neering design to solving instructional and other school-based problems they typically
face in their work (e.g., redesigning space in a classroom or the school building, designing
layouts of pages or handouts for students, developing alternative summative assessments
to show what students are learning). Teachers used a rubric that was based on the first
three columns of the Matrix with an additional column where teachers could cite evidence
to support their assessment of students’ engineering design practices. The instructor also
used the same rubric when rating the design practices teachers used during the course and
in completing their final projects.
The Informed Design Teaching & Learning Matrix 777

As an example of this work, a grades 4–5 science cluster teacher recently used the Ma-
trix after developing initial lesson plans for an instructional unit from the NSF-funded
curriculum series, Engineering Is Elementary (Museum of Science, 2009). This curriculum
employs a five-step design process model of Ask, Imagine, Plan, Create, and Improve.
The Catching the Wind: Designing Windmills module asks students to redesign a blade
configuration for a model windmill so that the model generates the maximum torque.
The teacher first developed a two-page lesson plan and then, in consultation with author
Crismond, reviewed the Matrix with reference to this plan. An analysis of the recorded
and transcribed collaborative session revealed the following sequence of activities: (1) the
teacher explained to Crismond his lesson plan and key planning decisions; (2) Crismond
gave an overview of key points from the Matrix of Informed Design; (3) both aligned the
five strategies from the EiE curriculum’s design process model to those in the Matrix (e.g.,
Ask mapped to Patterns A and B, Imagine mapped to Pattern C, etc.); (4) the teacher
identified objectives from the Matrix’s learning goals column to include in a revised lesson
plan (e.g., a revised emphasis on using design as a context for doing controlled experi-
ments (Pattern F) and troubleshooting (Pattern G)); and (5) the teacher selected tech-
niques from the Matrix’s teaching strategies column that he felt met his students’ learning
needs. For a final project, the teacher implemented the lessons and collected formative as-
sessment data that focused on students’ troubleshooting thinking and that helped him to
make adjustments to daily instruction.
High school students The Matrix has been used with high school engineering and
physics students to do direct instruction about the differences between beginning and in-
formed designers and to support class discussions of those behaviors (Crismond & Peterie,
2011). Students rated themselves along the beginning-informed design continuum for
each of the Matrix’s patterns and provided evidence for those ratings based on their recall
of recently completed design project work (Figure 5). These activities not only provided
formative assessment data that helped the instructor better understand students’ grasp of
specific design strategies and their concepts of informed designing, but also afforded stu-
dents with an opportunity to “self-evaluate and reflect” (McTighe & Wiggins, 2004, p.
214) upon their own growth towards becoming informed designers.
College students In a first-year undergraduate engineering course, author Adams
used the Matrix to scaffold design learning as her students worked in teams on increas-
ingly complex and large-scale design tasks. She presented the Matrix and repeatedly re-
ferred to its language and ideas to help students understand how design is different from
closed-ended engineering problem solving and self-assess their own growth as informed
designers. Students also used the Matrix to help them critique and predict the strengths
and weaknesses of other design teams seen in video segments from the PBS program De-
sign Squad (see also Purzer, 2010). In class, students discussed their observations of what
they saw as effective and ineffective design strategy use and mapped these to the Matrix’s
descriptions of beginning and informed designing.
Another study, also conducted in a first-year engineering program, suggests that the
Matrix has promise as a tool for direct instruction and student self-assessment. A pre- and
post-test survey was administered to 115 engineering freshmen, who were asked to rank
from a list of 23 design activities what they felt were the most and least critical activities for
producing high-quality designs (Adams & Fralick, 2010). By semester’s end, there was a
significant shift in awareness of the ambiguity inherent in design tasks and the need for
adopting an iterative approach to design (Pattern H). In the pre-activity survey, students
778 Crismond & Adams

Students’ Recalled Examples of Their Own Behavior as…

Matrix Pattern
…Beginning Designer …Informed Designer
A. Solve In our project, only one step was done [Our teacher] told us information about
Problems Too before testing, which was building. parachutes and I did not make a design
Soon vs. Wait to We could have looked thru the until I understood everything I could
Learn More problems and verify what we needed about a parachute.
to do.
… … …
H. Single Design For the paper car activity, as a class, On our parachute design, we had to go
Cycle vs. no one did any redesign in their through the design process cycle about
Iterative project to make it better besides small four times before getting our final
Designing tweaks to the same design. projects.

FIGURE 5. Modified Matrix table filled with students’ own examples of acting as beginning or informed
designers. In this table, the language associated with each Matrix pattern (column 1) was modified to make
it more accessible to students.

were more likely to identify iteration as one of the least important design activities for
producing solutions of high quality, citing iteration as an “inefficient use of time,” a “waste
of time,” and how good teams “shouldn’t have to iterate much.” Students’ post-activity re-
sponses in effect moved iteration from the least important to one of the most important
design activities. Students wrote of how iteration “continuously makes the design better”
and is an “extremely useful process that allows you to re-look at different aspects of your
design and decide what to improve on.” One student explained how “having a plan is im-
portant, perhaps vital, but so is being flexible with the plan and being able to adapt to the
current circumstances of the project.” There were also significant shifts in students’ recog-
nition of the centrality of problem formulation strategies (Patterns A, B, and H) such as
understanding the problem, gathering information, and identifying constraints (e.g., “an
insufficient problem statement can derail a project and can cause delays later in the
project”).

LIMITATIONS OF THE MATRIX

While this scholarship of integration effort is provocative in articulating a new language

for design learning and teaching and is powerful in providing direction to the next
phase of crucial work, it has its limitations. First, while developing the Matrix, the au-
thors made difficult decisions regarding the scope of the scholarship of integration ef-
fort. To manage complexity and feasibility, we have left a number of substantial chal-
lenges for future work. One involves reviewing and integrating what might be included
in an informed designer’s understanding of the nature of engineering design (Lewis &
Zuga, 2005, p. 55). This understanding has analogues to the concept in science educa-
tion of the nature of science (Lederman et al., 2002) and in engineering education of
the nature of engineering (e.g., Karatas et al., 2011). The social aspects of design
(Bucciarelli, 1996, 2002), including the challenges of helping students develop their
abilities to collaborate and cooperate in design teams and learn through their interac-
tions with their peers, also have not been articulated in this version of the framework.
The role of communication and social interactions in designing – whether it is within
or across teams, disciplines, cultures, or broader stakeholder audiences – is not ad-
dressed. In addition, more idiosyncratic, though critical, issues related to learning to use
tools, work with various materials, and fabricate prototypes – tasks that typically do
The Informed Design Teaching & Learning Matrix 779

have a place in undergraduate cornerstone engineering courses (e.g., West, Flowers, &
Gilmore, 1990) – have not been included in this version of the Matrix.
The scope of the Matrix has, by design, been constrained to K–16 instructional set-
tings. Its primary audiences include K–12 STEM educators, engineering educators at the
undergraduate level, and potential early career professionals or graduate students. How-
ever, much of the research represented in this paper does not address the special concerns
or learning issues of K–2 students. This limitation represents not only a gap in the field of
engineering education but also a significant opportunity since, for example, it is quite un-
clear whether these younger students can frame design problems effectively or use ab-
stractions found in systems thinking to understand how everyday products work. Since
the Matrix spans childhood education to early adult education, future studies would need
to target issues that bridge different learners in different contexts. Also, it is quite possible
that some K–16 students have considerable design experience and have engaged in deep
reflection such that their learning trajectories may extend beyond that of informed de-
signing. While this does not suggest that the information contained in the Matrix is in-
correct, it does highlight the need for research to support adjusting Matrix end points
closer to expertise. Finally, the Matrix was designed to help teachers reflect on and devel-
op their own Design PCK but does not represent or fully articulate what makes up any
teacher’s Design PCK. Such an endeavor would require dedicated studies on what K–16
teachers understand about design knowing and learning, and this shapes their instruc-
tional approaches using design tasks.

CONCLUSIONS

The outcomes of this scholarship of integration process include the articulation of the
idea of informed design upon which the Matrix is based. The Matrix table acts as a guide to
help teachers identify, diagnose, and explain some of the highly ineffective design habits
of students. It also aims to help teachers formulate pragmatic learning goals and compile a
suite of teaching activities and techniques to use or adapt as needed. Critically, the Matrix
is the place from which teachers can formulate their own formative assessment tasks, and
it can help them implement evidence-based adjustments to their day-to-day instruction.
The Matrix aims to be a first-generation construct of the Design PCK that teachers need
to know and develop to be effective users of design tasks with their students.
With these complementary tools in hand, an emergent instructional theory for teach-
ing informed design was proposed, and the use of the Matrix as a conceptual framework
for placing and locating disparate and separate research findings was also suggested. The
Matrix may also serve as a place where new findings from studies on the effectiveness of
the teaching remedies and the results from longitudinal studies that follow students’ learn-
ing of engineering design over large tracks of time can be integrated. Also, descriptions of
beginning designers and the steps they follow towards becoming informed designers
could be revised, refuted, or validated. Such data could lead to the honing and refining of a
series of detailed learning progressions statements for engineering design that will make
effective teaching with design activities a more achievable goal for teachers.
Helping support informed teaching with engineering design activities is an ultimate
goal of this work. This would include helping teachers to (a) look for and notice inef-
ficient behaviors and habits of mind of beginning designers, (b) select realistic learn-
ing objectives that aim to improve particular design behaviors or address one or more
780 Crismond & Adams

of the seven performance dimensions of informed design, (c) create viable formative
assessments tasks to assess students’ growth in engineering design practice, and (d)
coach students in using the Matrix themselves to guide their design actions and sup-
port meaningful reflective practice. Such practices can represent a significant shift
from seeing instruction in engineering design as teaching projects to creating design-
based learning experiences that provide students with opportunities to engage in “ap-
proximations of practice” (Grossman et al., 2009) and even experiment with provi-
sional identities as future designers and engineers.

HOW A MATRIX LEARNS

The Informed Design Teaching and Learning Matrix, as with any building (see Brand,
1994), can learn and adapt through feedback. Comments and questions regarding this
paper and its contents may be sent to David Crismond, City College of New York, 138th
St. & Convent Ave., NAC Building 6/207b, New York, NY 10031, or emailed to
dcrismond@ccny.cuny.edu. Feedback might include (1) applications, how have you used
the Matrix in your classrooms or in teacher professional development settings; (2) feed-
back, what worked well and poorly when you read or used the Matrix; (3) gaps, what seems
to be missing in the Matrix’s current collection of patterns and misconceptions and for-
mulations of starting points and endpoints; and (4) teaching techniques, what goals and
strategies of instruction reported in the literature might be included in the Matrix.

ACKNOWLEDGEMENTS

Insights for this paper grew out of work on an NSF-funded teacher professional develop-
ment project, Design in the Classroom (ESI 99-86854), which supports teachers’ use of de-
sign tasks with their students, an NSF-funded project on the continuum of design exper-
tise (ROLE 01-25547), and an NSF CAREER grant exploring cross-disciplinary
approaches to thinking and learning (EEP-0748005), and an NSF IEECI collaborative
project (08-36008 and 08-35949) that led to the creation of the Design Compass. The au-
thors wish to thank all of those who have provided feedback as the Matrix developed, in-
cluding those at talks at national conferences at ASEE, ITEA, and AERA. Special thanks
go to Craig Adams, David Barlex, Terry Bennett, Woodie Flowers, Ed Goldman, Mor-
gan Hynes, Richard Kimbell, Matt Peterie, Doug Steinhoff, and Jennifer Turns for feed-
back and suggestions for improving this work.

REFERENCES

Adams, J. L. (1986). Conceptual blockbusting: A guide to better ideas. Reading, MA: Perseus
Books.
Adams, R. S. (2001). Cognitive processes in iterative design behavior. Unpublished doctoral dis-
sertation. Seattle, WA: University of Washington.
Adams, R. S. (2002). “Understanding design iteration: Representations from an empirical
study.” In D. Durling & J. Shackleton (Eds.), Common ground: Proceedings of the Design
Research Society International Conference at Brunel University (pp. 1151–1161). Stafford-
shire University Press: UK.
The Informed Design Teaching & Learning Matrix 781

Adams, R. S., & Atman, C. J. (2000). Characterizing engineering student design processes:
An illustration of iteration. Proceedings of the Annual Meeting of the American Society
of Engineering Education Conference, Session 2330. St. Louis, MO.
Adams, R. S., Beltz, N., Mann, L., & Wilson, D. (2010). Exploring student differences in
formulating cross-disciplinary sustainability problems. International Journal of Engineering
Education, 26(2), 324–338.
Adams, R. S., & Fralick, B. (2010). Work in progress: A conceptions of design instrument as
an assessment tool. Proceedings of the Frontiers in Education Conference, Washington DC.
Adams, R. S., Turns, J., & Atman, C.J. (2003). Educating effective engineering designers: The
role of reflective practice. Design Studies, 24(3), 275–294.
Akin, Ö. (1994). Creativity in design. Performance Improvement Quarterly, 7(3), 9–21.
Akin, Ö., & Akin, C. (1996). Frames of reference in architectural design: Analysing the
hyperacclamation (A-h-a-!). Design Studies, 17(4), 341–361.
Akin, Ö., & Lin, C. (1996). Design protocol data and novel design decisions. In N. Cross, H.
Christiaans, & K. Dorst (Eds.), Analysing design activity (pp. 35–63). Chichester, England:
John Wiley & Sons, Ltd.
Alexander, C. (1964). Notes on the synthesis of form. Cambridge, MA: Harvard University Press.
Alexander, C. (1979). A timeless way of building. New York: Oxford University Press.
American Association for the Advancement of Science [AAAS]. (1993). Benchmarks for sci-
ence literacy. New York: Oxford University Press.
American Association for the Advancement of Science [AAAS]. (2001). Atlas of science litera-
cy. Washington, DC: National Science Teachers Association.
Andrews, D. H., & Goodson, L. A. (1980). A comparative analysis of models of instructional
design. Journal of Instructional Development 3(4), 2–16.
Anning, A. (1997). Drawing Out Ideas: Graphicacy and young children. International Journal
of Technology and Design Education 7(3), 219–239.
Arbaugh, F., Herbel-Eisenmann, B., Ramirez, N., Knuth, E., Kranendonk, H., & Quander, J.
R. (2010). Linking research & practice: The NCTM research agenda conference report. Wash-
ington, DC: National Council of Teachers of Mathematics.
Archer, L. B. (1965). Systematic method for designers. London: Council of Industrial Design.
Archer, L. B. (1979). Design as a discipline. Design Studies 1(1), 17–20.
Association for the Promotion and Advancement of Science Education [APASE]. (1991).
Engineering for children: Structures. Vancouver, Canada: APASE.
Atman, C. J., Adams, R. S., Mosborg, S., Cardella, M. E., Turns, J., & Saleem, J. (2007). En-
gineering design processes: A comparison of students and expert practitioners. Journal of
Engineering Education 96(4), 359–379.
Atman, C. J., & Bursic, K. M. (1996). Teaching engineering design: Can reading a textbook
make a difference? Research in Engineering Design 8(4), 240–250.
Atman, C. J., Chimka, K. M., Bursic, K. M., & Nachtmann, H. L. (1999). A comparison of
freshman and senior engineering design processes. Design Studies 20(2), 131–152.
Atman, C. J., Deibel, K. & Borgford-Parnell, J. (2009). The Process of Engineering Design:
A Comparison of Three Representations. In Proceedings of the International Conference on
Engineering Design (ICED), Stanford, CA, August.
Axton, T. R., Doverspike, D., Park, S. R., & Barrett, G. V. (1997). A model of the informa-
tion-processing and cognitive ability requirements for mechanical troubleshooting. Inter-
national Journal of Cognitive Ergonomics 1(3), 245–266.
Ball, L. J., & Christensen, B. T. (2009). Analogical reasoning and mental simulation in design:
Two strategies linked to uncertainty resolution. Design Studies 30(2), 169–186.
782 Crismond & Adams

Ball, L. J., & Ormerod, T. C. (1995). Structured and opportunistic processing in design: A
critical discussion. International Journal of Human-Computer Studies 43(1), 131–151.
Bamberger, J. (2003). The development of intuitive musical understanding: A natural experi-
ment. Psychology of Music 31(1), 7–36.
Bamberger, J. & Schön, D. (1983). Learning as reflective conversation with materials: Notes
from work in progress. Art Education 36(2), 68–73.
Barlex, D. (1995). Nuffield design and technology student’s book. Harlow: Longman.
Barlex, D. (2000). Young foresight: Handbook for teachers and mentors. London: Software Pro-
duction Enterprises.
Barlex, D. (2004). PIES. Design and Technology News & Views 9, 8–9. Retrieved November
21, 2006, from http://www.tep.org.uk.
Barlex, D., & Givens, N. P. (1995). The Nuffield approach to the teaching of mechanisms at
key stage 3. IDATER ’95, 48–51.
Barlex, D., & Wright, R. (1998). Using the internet as an information gathering tool for the
design and technology curriculum. IDATER ’98, 160–168.
Bartunek, J. M. (2007). Academic-Practitioner Collaboration Need Not Require Joint or
Relevant Research: Toward a Relational Scholarship of Integration. Academy of Manage-
ment Journal, 50(6), 1323–1333.
Baya, V., & Leifer, L. J. (1994). A study of the information handling behavior of designers
during conceptual design. Design theory and methodology-DTM’94 (pp. 153–160). New
York: American Society of Mechanical Engineers.
Benenson, G., & Neujahr, J. (2002). Stuff that works! Mechanisms and other systems. Ports-
mouth, NH: Heinemann.
Bereiter, C., & Scardamalia, M. (1993). Surpassing ourselves: An inquiry into the nature and im-
plications of expertise. LaSalle, IL: Open Court.
Bereiter, C., & Scardamalia, M. (2006). Education for the knowledge age: Design-centered
models of teaching and instruction. In P. A. Alexander & P. H. Winne (Eds.), Handbook of
educational psychology (2nd ed.; pp. 695–713). Mahwah, NJ: Lawrence Erlbaum.
Booth, W. C., Colomb, G. G., & Williams, J. M. (2008). The craft of research, (3rd.; Series:
(CGWEP) Chicago Guides to Writing, Editing, and Publishing. Chicago, IL: University
of Chicago Press.
Boyer, E. L. (1990). Scholarship reconsidered: Priorities of the professoriate. Princeton, NJ: Carne-
gie Foundation for the Advancement of Teaching.
Braha, D., & Maimon, O. (1997). The design process: Properties, paradigms, and structures.
IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 27(2),
146–166.
Brand, S. (1994). How buildings learn: What happens after they’re built. New York:
Viking.
Brereton, M. F., Cannon, D. M., Mabogunje, A., & Leifer, L. J. (1996). Collaboration in de-
sign teams: How social interaction shapes the product. In N. Cross, H. Christiaans, & K.
Dorst (Eds.), Analysing design activity (pp. 319–341). Chichester, England: John Wiley &
Sons Ltd.
Bransford, J. D., Brown, A. L., & Cocking, R. R. (1999). How people learn: Brain, mind, expe-
rience and school. Washington, DC: National Academies Press.
Bransford, J. D., Franks, J. J., Vye, N. J., & Sherwood, R. D. (1989). New approaches to in-
struction: Because wisdom can’t be told. In S. Vosniadou & A. Ortony (Eds.), Similarity
and analogical reasoning (pp. 470–497). Cambridge, UK: Cambridge University Press.
The Informed Design Teaching & Learning Matrix 783

Broadbent, G. (1984). The development of design methods. In N. Cross (Ed.),

Developments in design methodology (pp. 337–346). Chichester, UK: Wiley.
Brown, T. (2009). Change by design: How design thinking transforms organizations and inspires
innovation. New York: HarperCollins.
Bryson, M., Bereiter, C., Scardamalia, M., & Joram, E. (1991). Going beyond the problem as
given: Problem solving in expert and novice writers. In R. J. S. P. A. Frensch (Ed.), Com-
plex problem solving: Principles and mechanisms (pp. 61–84). Hillsdale, NJ: Lawrence Erl-
baum.
Bruer, J. T. (1993). Schools of thought: How the politics of literacy shape thinking in the classroom.
Cambridge, MA: MIT Press.
Bucciarelli, L. (1984). Reflective practice in engineering design. Design Studies 5(3), 185–190.
Bucciarelli, L. (2002). Between thought and object in engineering design. Design Studies
23(3), 219–231.
Bucciarelli, L. (2003). Engineering Philosophy, 1-75. Delft, The Netherlands: DUP Satellite
Press.
Buchanan, R. (1995). Wicked problems in design thinking. In V. Margolin & R.
Buchanan (Eds.), The idea of design: A Design Issues reader (pp. 3–20). Cambridge, MA:
MIT Press.
Burghardt, D., & Hacker, M. (2004). Informed design: A contemporary approach to design
pedagogy as a core process in technology. The Technology Teacher 63(1), 6–8.
Bursic, K. M., & Atman, C. J. (1997). Information gathering: A critical step for quality in the
design process. Quality Management Journal 4(4), 60–75.
Bybee, R. (2002). Scientific inquiry, student learning, and the science curriculum. In R. Bybee
(Ed.), Learning science and the science of learning (pp. 25–35). Arlington, VA: NSTA Press.
Byrnes, J. P., Miller, D. C., & Schafer, W. D. (1999). Gender differences in risk taking: A me-
ta-analysis. Psychological Bulletin 125(3), 367–383.
Cardella, M. E., Atman, C. J., & Adams, R. S. (2006). Mapping between design activities and
external representations for engineering student designers. Design Studies 27(1), 5–24.
Chandresekaran, B. (1992). Design problem solving: A task analysis. In M. Green (Ed.),
Knowledge aided design (pp. 25–46). London: Academic Press.
Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics
problems by experts and novices, Cognitive Science, 5, 121–152.
Christiaans, H., & Dorst, K. (1992). Cognitive models in industrial design engineering. De-
sign Theory and Methodology 42, 131–140.
Churchman, C. W. (1967). Wicked problems. Management Science 14(4), 141–142.
Clement, D. H., & Gullo, D. F. (1984). Effects of computer programming on young chil-
dren’s cognition. Journal of Educational Psychology 76(6), 1051–1058.
Cline, H. F., & Mandinach, E. B. (2000). The corruption of a research design: A case study of
a curriculum innovation project. In A. E. Kelly & R. A. Lesh (Eds.), Handbook for research
design in mathematics and science education (pp. 169–189). Amsterdam: Elsevier.
Committee on Engineering Design. (1961). Report on engineering design. Journal of Engi-
neering Education 51(8), 645–660.
Committee on K-12 Engineering Education. (2009). In L. Katehi, G. Pearson, & M. Feder
(Eds.), Engineering in K-12 education. Washington DC: National Academy of Engineer-
ing and National Research Council.
Constable, H. (1994). A study of aspects of design and technology capability at key stage 1
and 2. IDATER ’94, 9–14.
784 Crismond & Adams

Coyle, H. P. (2001). Electromagnets teachers guide: A supplemental curriculum for middle school
physical science. Dubuque, IA: Kendall/Hunt.
Craig, D. L. (2001). Stalking homo faber: A comparison of research strategies for studying de-
sign behavior. In C. Eastman, M. McCracken, & W. Newstetter (Eds.), Design knowing
and learning: Cognition in design education (pp. 13–36). Amsterdam: Elsevier.
Crismond, D. (1997). Investigate-and-redesign tasks as a context for learning and doing science
and technolog y. Unpublished doctoral dissertation. Cambridge, MA:
Harvard Graduate School of Education.
Crismond, D. (2001). Learning and using science and technology ideas when doing investi-
gate-and-redesign tasks: A study of naive, novice and expert designers doing constrained
and scaffolded design work. Journal of Research in Science Teaching 38(7), 791–820.
Crismond, D. (2005). Contrasting Behaviors of Beginner Versus Informed Designers: Pre-
sentation of Evidence for a Framework of ‘Informed Designing’. Paper for Annual Con-
ference of the Association for Educational Communications and Technology. Orlando,
FL, October 20.
Crismond, D. (2008). Case Studies of Diagnostic Reasoning’s Role in Engineering Design.
Paper for ASEE Annual Conference & Exposition, Pittsburgh, PA, June 22–25.
Crismond, D. (2011). Scaffolding strategies that integrate engineering design and scientific
inquiry in project-based learning environments. In M. Barak & M. Hacker (Eds.), Foster-
ing human development through engineering and technology education. Rotterdam, Nether-
lands: Sense Publishers.
Crismond, D. & Adams, R. (2012). Developing Design Pedagogical Content Knowledge in
K–8 Teachers: Report on an Extended Professional Development Experience. Paper
presented at the 2nd P-12 Engineering and Design Education Research Summit.
Washington, DC.
Crismond, D., Camp, P. J., Ryan, M., & Kolodner, J. L. (2001). Design rules of thumb—
Connecting science and design. Special session 27.50. Annual Conference of the
American Educational Research Association. Seattle, WA.
Crismond, D. P., Howland, J., & Jonassen, D. (2011). Designing with technology. In J. How-
land, D. H. Jonassen, & R. M. Marra (Eds.), Meaningful learning with technology (4th ed.;
pp. 72–90). Upper Saddle River, NJ: Merrill Prentice Hall.
Crismond, D., Hynes, M., & Donahy, E. (2010). The Design Compass: A computer tool for
scaffolding students’ metacognition and discussions about their engineering design process.
Paper presented at the AAAI Spring Symposium on Using Electronic Tangibles to Pro-
mote Learning: Design and Evaluation, Palo Alto, CA, March 22–24.
Crismond, D., Kolodner, J. L., Fasse, B., Gray, J., & Holbrook, J. (1999). Learning by Design’s
professional development and research agenda. Invited NSF Pre-Session Paper presented
at the International Technology Education Association’s 61th Annual Conference, India-
napolis, IN.
Crismond, D. & Peterie, M. (2011). Tools and materials for developing design-based reflec-
tive thinking in high school engineering students and enhancing discussions between
teacher and student team. Workshop presented at the ASEE Annual Conference & Ex-
position. Vancouver, BC: June 25.
Cross, N. (1992). A history of design methodology. In M. J. de Vries, N. Cross, & D. P.
Grant (Eds.) Design methodology and relationships with science (pp. 15–27). The Nether-
lands: Kluwer Academic Publishers.
Cross, N. (1999). Design research: A disciplined conversation. Design Issues 15(2), 5–10.
The Informed Design Teaching & Learning Matrix 785

Cross, N. (2000). Engineering design methods: Strategies for product design (3rd ed.). New York:
John Wiley & Sons.
Cross, N. (2001a). Design cognition: Results from protocol and other empirical studies of de-
sign activity. In C. Eastman, M. McCracken, & W. Newstetter (Eds.) Design knowing and
learning: Cognition in design education (pp. 79–103). Amsterdam: Elsevier.
Cross, N. (2001b). Designerly ways of knowing: Design discipline versus design science. De-
sign Issues 17(3), 49–55.
Cross, N. & Cross, A. C. (1998). Expertise in engineering design. Research in Engineering De-
sign 10, 141–149.
Dalrymple, O., Sears, D., & Evangelou, D. (2010). Evaluating the motivational and learning
potential of an instructional practice for use with first year engineering students. Proceed-
ings of the Annual Meeting of the American Society for Engineering Education, Louisville.
Daly, S. R., Adams, R.S., & Bodner, G. (2012). What does it mean to design? A qualitative
investigation of design professionals’ experiences. Journal of Engineering Education, 101(2).
Daly, S. R., Christian, J.L., Yilmaz, S., Seifer, C.M. & Gonzalez, R. (2011). Teaching Design
Ideation. Proceedings of the Annual Meeting of the American Society of Engineering Education
Conference, June.
Daly, S. R., Yilmaz, S., Seifert, C. M., & Gonzalez, R. (2010). Cognitive heuristic use in engi-
neering design ideation. Proceedings of the Annual Meeting of the American Society of Engi-
neering Education Conference, June.
Dauphinée D., & Martin, J. B. (2000). Breaking down the walls: Thoughts on the scholarship
of integration. Academic Medicine 75(9), 881–886.
Davis, D., Beyerlein, S., Thompson, P., McCormack, J., Harrison, O., Trevisan, M., Gerlick,
R., & Howe, S. (2011). IDEALS: A Model for Integrating Engineering Design Profes-
sional Skills Assessment and Learning Proceedings of the Annual Meeting of the American
Society of Engineering Education Conference, Vancouver, CA.
De Bono, E. (1970). Lateral thinking: Creativity step by step. New York: Harper & Row.
De Lisi, R. & Golbeck, S. L. (1999). Implication of Piagetian Theory for Peer Learning. In
A. M. O’Donnell & A. King (Eds.), Cognitive perspectives on peer learning (pp. 3–37).
Mahwah, NJ: Lawrence Erlbaum Associates.
Derringer, P. (1996). Knowledge gets distilled in the Invention Machine. Mass High Tech
14(23), July 22.
Diaz, J., & Carter, D. A. (1999). The Elements of Pop-Up: A pop-up book for aspiring paper engi-
neers. New York: Little Simon.
Dreyfus, H., & Dreyfus, S. (2005). Expertise in real world contexts. Organization Studies
26(5), 779–792.
Dubberly, H. (2004). How do you design? A compendium of models. San Francisco: Dubberly
Design Office.
Dufresne, R. J., Gerace, W. J., Hardiman, P. T. & Mestre, J. P. (1992). Constraining novices to
perform expertiselike problem analyses: Effects on schema acquisition. Journal of the
Learning Sciences 2(3), 307–331.
Duncan, R. G., & Hmelo-Silver, C. (2009). Editorial: Learning progressions: Aligning cur-
riculum, instruction, and assessment. Journal of Research in Science Teaching 46(6), 606–
609.
Dunsmore, K., Turns, J. & Yellin, J. M. (2011). Looking toward the real world: Student con-
ceptions of engineering. Journal of Engineering Education 100(2): 329–348.
Dym, C. & Little, P. (2004). Engineering design: A project-based introduction (2nd ed.). New
York: John Wiley & Sons.
786 Crismond & Adams

Eastman, C., W. Newstetter, & M. McCracken (Eds). (2001). Design knowing and learning:
Cognition in design education. Amsterdam: Elsevier.
Educational Development Center [EDC]. (1999). Model windmill: A video case study of an ex-
tended investigation in technology and science. Watertown, MA: Educational Development
Center, Inc.
Egan, B. (1999). Children talking about designing: how do young children perceive the func-
tions/uses of drawing as part of the design process? IDATER’99, 79–83.
Elio, R. & Scharf, P. B. (1990). Modeling novice-to-expert shifts in problem solving strategy
and knowledge organization. Cognitive Science 14(4), 579–639.
Elmer, R. (2002). Meta-cognition and design and technology education. The Journal of Design
and Technology Education 7(1), 19–25.
Ennis, C. W. & Gyeszly, S. W. (1991). Protocol analysis of the engineering systems design
approach. Research in Engineering Design 3(1), 15–22.
Erickson, J., & Lehrer, R. (1998). The evolution of critical standards as students design hy-
permedia documents. Journal of the Learning Sciences 7(3–4), 351–386.
Figueiredo, A. D. (2008). Toward an Epistemology of Engineering. In D. Goldberg & N.
McCarthy (Eds.), Proceedings Workshop on Philosophy & Engineering [WPE 2008] (pp. 94–
95). London: Royal Engineering Academy.
Fischer, K. W. (1980). A theory of cognitive development: The control and construction of
hierarchies of skills. Psychological Review 87(6), 477–531.
Fischer, K. W. (2006). Dynamic Cycles of Cognitive and Brain Development: Measuring
Growth in Mind, Brain, and Education. In A. M. Battro & K. W. Fischer (Eds.), The edu-
cated brain. Cambridge, UK: Cambridge University Press.
Fischer, K. W. & Bidell, T. R. (1998). Dynamic development of psychological structures in
action and thought. In R. M. Lerner & W. Damon (Eds.), Handbook of child psychology:
Volume 1. Theoretical models of human development (5th ed.) pp. 467–561. New York: Wiley.
Fish, J., & Scrivener, S. A. (1990). Amplifying the mind’s eye: Sketching and visual cognition.
Leonardo 23(1), 117–126.
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-de-
velopmental inquiry. American Psychologist 34(10), 906–911.
Fleer, M. (2000). An early childhood research agenda: Voices from the field. DETYA, Canberra.
Flowers, L., Hayes, J. R., Carey, L., Schriver, K., & Stratman, J. (1986). Detection, diagnosis,
and the strategies of revision. College Composition and Communication 37(1), 16–55.
Fortus, D., Dershimer, R. C., Krajcik, J., Marx, R. W., & Mamlok-Naaman, R. (2004). De-
sign-based science and student learning. Journal of Research in Science Teaching, 41(10),
1081–1110.
Foshay, W. R., Silber, K. H., & Steinicki, M. (2003). Writing training materials that work: How
to train anyone to do anything. Hoboken, NJ: John Wiley & Sons.
Frye, E. (1997). Engineering problem solving for mathematics, science and technology education.
Hanover, NJ: Trustees of Dartmouth College.
Garcia, J., Sinfield, J., Yadav, A., & Adams, R. (2012). Learning through entrepreneurially ori-
ented case-based instruction. International Journal of Engineering Education 28(2), 448–457.
Garner, S. (1989). Drawing and designing: Exploration and manipulation through two-di-
mensional modeling. IDATER’89, 43–50.
Georgia Tech Research Corporation [GTRC]. (2004a). Design in the classroom: Supporting
creativity in kids. Downloaded on March 12, 2012 and viewed from 2:43–4:19 at http://
designintheclassroom.com/designProcess/teachStrat/creativityKids.html.
The Informed Design Teaching & Learning Matrix 787

Georgia Tech Research Corporation [GTRC]. (2004b). Design in the classroom: Day 2 –
Identify Key Variables, MOVIE 2, 3:25–3:36. Downloaded on March 12, 2012 and
viewed at http://www.designintheclassroom.com/designTasks/parachute/parTimeline/
techEd2.html.
Georgia Tech Research Corporation [GTRC]. (2004c). Design in the classroom: Authentic
assessment. Downloaded on March 12, 2012 and viewed at http://designintheclassroom.
com/designProcess/assessment/authenticAssess.html.
Georgia Tech Research Corporation [GTRC]. (2004d). Design in the classroom: Gender
and design. Downloaded on March 12, 2012 and viewed at http://designintheclassroom.
com/designProcess/designControversies/gender.html.
Georgia Tech Research Corporation [GTRC]. (2004e). Design in the classroom: Developing
the “seeing” of designer roles in students. Downloaded on March 12, 2012 and viewed at
http://designintheclassroom.com/designProcess/designControversies/gender.html.
Georgia Tech Research Corporation [GTRC]. (2010). Project-Based Inquiry Science: Diving
into science. Armonk, NY: It’s About Time, Herff Jones Educational Division.
Gero, J. S. (1999). Preface. In J. S. Gero & B. Tversky (Eds.), Visual and spatial reasoning in de-
sign. Sydney: Key Centre of Design Computing and Cognition.
Gero, J. S. (2011). Fixation and commitment while designing and its measurement. Journal of
Creative Behavior 45(2), 108–115.
Gero, J. S., & Kannengiesser, U. (2004). The situated function-behavior-structure framework.
Design Studies 25(4), 373–391.
Gil, N., Tommelein, I. D. & Beckman, S. (2004). Postponing design processes in unpredict-
able environments. Research in Engineering Design 15(3), 139–154.
Goals Committee. (1968). Goals of engineering education: Final report of the goals committee.
Washington, DC: American Society for Engineering Education.
Goldman, E. (2002). Chair design. TIES Magazine, 27–31. Retrieved August 17, 2006 from
http://www.tiesmagazine.org/archives/sep_2002/index.html.
Goldschmidt, G. (1991). The dialectics of sketching. Creativity Research Journal 4(2), 123–
143.
Goldstein, W. M., & Hogarth, R. M. (1997). Judgment and decision research: Some history
context. In W. M. Goldstein & R. M. Hogarth (Eds.), Research on judgment and decision
making: Currents, connections, and controversies (pp. 3–65). Cambridge, UK: Cambridge
University Press.
Goel, V. (1989). Design within information-processing theory: The design problem space. AI
Magazine, 10(1), 19–35.
Goel, V., & Pirolli, P. (1992). The structure of design spaces. Cognitive Science 16(3), 395–429.
Goodwin, C. (1994). Professional Vision. American Anthropologist 96(3), 606–633.
Grant, D. P. (1992). Housing location for low income residents: An architectural case
study of simulating conflicts of interest and generating compromise proposals. In M. J.
de Vries, N. Cross, & D. P. Grant (Eds.) Design methodology and relationships with science
(pp. 63–101). The Netherlands: Kluwer Academic Publishers.
Gregory, S. (1966). The design method. London: Butterworth.
Grossman, P., Compton, C., Igra, D., Ronfeldt, M., Shahan, E., & Williamson, P. W. (2009).
Teaching practice: A cross-professional perspective. Teachers College Record 111(9), 2055–
2100.
Gustafson, B., MacDonald, D., & Gentilini, S. (2007). Using talking and drawing to design:
Elementary children collaborating with university industrial design students. Journal of
Technology Education 19(1), 19–34.
788 Crismond & Adams

Hacker, M., & Burghardt, D. (2004). Technology education: Learning by design. Upper Saddle
River, NJ: Pearson Prentice-Hall.
Harlen, W. (2001). Primary science: Taking the plunge. Portsmouth, NH: Heinemann.
Harris, J.G. (Moderator and Organizer), E.M. DeLoatch, W.R. Grogan, I.C. Peden, & J.R.
Whinnery. (1994). Journal of Engineering Education Roundtable: Reflections on the
Grinter Report. Journal of Engineering Education 83(1), 69–94.
Harrison, S., & Minneman, S. (1996). A bike in hand: A study of 3-D objects in design. In N.
Cross, H. Christiaans, & K. Dorst (Eds.), Analysing design activity (pp. 417–436). Chich-
ester, England: John Wiley & Sons, Ltd.
Hauser, J., & D. Clausing. (1988). The house of quality. Harvard Business Review 66(3), 63–
73.
Hawkins, D. (2002). The informed vision: Essays on learning and human nature. New York:
Agathon Press.
Hawkins, J. (1990). Gender differences and problem-solving with novel technological objects.
Invited talk, Educational Testing Service, Princeton, NJ.
Hayes, J.R. (1989). The complete problem solver. Hillsdale, NJ: Lawrence Erlbaum.
Heiser, J., Tversky, B., & Silverman, M. (2004). Sketches for and from collaboration. In J. S.
Gero, B. Tversky & T. Knight (Eds.), Visual and spatial reasoning in design III (pp. 69–78).
Sydney: Key Centre of Design Computation and Cognition, Univ. of Sydney.
Hirsch, P. L., & McKenna, A. F. (2008). Using reflection to promote teamwork understand-
ing in engineering design education. International Journal of Engineering Education 24(2),
377–385.
Hofmeyer, A., Newton, M., & Scott, C. (2007). Valuing the scholarship of integration and
the scholarship of application in the academy for health science scholars: Recommended
methods. Health Research Policy and Systems 29(5), 5.
Hybs, I., & Gero, J. S. (1992). An evolutionary process model of design. Design Studies 13(3),
273–290.
Hynes, M. (2010). Middle-school Teachers’ Understanding and Teaching of the Engineering
Design Process: A Look at Subject Matter and Pedagogical Content Knowledge. Interna-
tional Journal of Technology and Design Education 21(3), 307–320.
International Technology Education Association. (2000). Standards for technological literacy.
Reston, VA: Author.
Jacobson, C., & Lehrer, R. (2000). Teacher appropriation and student learning of
geometry through design. Journal for Research in Mathematics Education 31(1), 71–88.
Jansson, D. G., & Smith, S. M. (1991). Design fixation. Design Studies 12(1), 3–11.
Jin, Y., & Cusilp, P. (2006). Study of mental iteration in different design situations. Design
Studies 27(1), 25–55.
Johansson, F. (2006). Medici effect: What elephants and epidemics can teach us about innovation.
Boston, MA: Harvard Business School.
Johnson, S. D. (1988). Cognitive analysis of expert and novice troubleshooting performance.
Performance Improvement Quarterly 1(3), 38–54.
Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured
problem-solving learning outcomes. Educational Technology Research & Development
45(1), 65–94.
Jonassen, D. H. (2003). Learning to solve problems: An instructional design guide. San Francisco:
Pfeiffer.
Jonassen, D. H., & Grabinger, S. R. (1990). Analyzing and selecting instructional strategies
and tactics. Performance Improvement Quarterly 3(2), 29–47.
The Informed Design Teaching & Learning Matrix 789

Jonassen, D. H., & Hung, W. (2006). Learning to troubleshoot: A new theory-based design
architecture. Educational Psychology Review 18(1): 77–114.
Jonassen, D. H., & Hung, W. (2008). All problems are not equal: Implications for problem-
based learning. Interdisciplinary Journal of Problem-based Learning, 2(2): 6–28.
Jonassen, D. H., Strobel, J. & Lee, C. B. (2006). Everyday problem solving in engineering:
Lessons for engineering educators. Journal of Engineering Education 95(2), 139–151.
Jones, F. D. (Ed.). (1930). Ingenious mechanisms for designers and inventors. New York: Indus-
trial Press.
Jones, J. C. (1992). Design methods. New York: Van Nostrand Reinhold.
Jonson, B. (2005). Design ideation: the conceptual sketch in the digital age. Design Studies
26(6), 613–624.
Kafai, Y. B., Ching, C. C., & Marshall, S. (1997). Children as designers of educational multi-
media software. Computers and Education 29(2/3), 117–126.
Kanter, D. (2010). Doing the project and learning the content: Designing project-based sci-
ence curricula for meaningful understanding. Science Education, 94(3), 525–551.
Karatas, F. O., Mickos, A., & Bodner, G. M. (2011). Sixth-grade students’ views of the nature
of engineering and images of engineers. Journal of Science Education and Technology 20(2),
123–135.
Kavakli, M., & Gero, J. (2003). Difference between expert and novice designers: An experi-
mental study. In U. Lindemann (Ed.), Human behaviour in design (pp. 42–51). Berlin:
Springer-Verlag.
Kegan, R. (1982). The evolving self: Problem and process in human development. Cambridge,
MA: Harvard University Press.
Kelley, T. (2001). The art of innovation: Lessons in creativity from IDEO, America’s leading design
firm. New York: Currency Books.
Kezar, A. (2002). Rising to our professional imperative: The need for the scholarship of inte-
gration. The PEN, Post-secondary Education Network Newsletter.
Kezar, A. (2009). Synthesis of scholarship on change in higher education. Paper presented at
the Mobilizing STEM Education for a Sustainable Future conference.
Kimbell, R. (1994). Using advanced technology for teaching, learning, and assessment. In D.
Blandow & M. J. Dyrenfurth (Eds.) Technology education in school and industry: Emerging
didactics for human resource development (pp. 331–342). Berlin: Springer-Verlag.
Kimbell, R. (2004). Ideas and ideation. The Journal of Design and Technology Education 9(3),
136–137.
Kimbell, R., & Perry, D. (2001). Design and technology in the knowledge economy.
London: Engineering Council.
Kimbell, R., & Stables, K. (2008). Researching design learning: Issues and findings from two de-
cades of research and development. Lexington, KY: Springer.
Kimbell, R., K. Stables, K., & Green, R. (1996). Understanding practice in design and technolo-
gy. Philadelphia, PA: Open University Press.
Kimbell, R., Stables, K., Wheeler, T., Wozniak, A., & Kelly, V. (1991). The assessment of per-
formance in design and technology. London: Schools Examination and Assessment Council.
Kohn, N. W., & Smith, S. M. (2009). Partly versus completely out of your mind: Effects of in-
cubation and distraction on resolving fixation. Journal of Creative Behavior 43(2), 102–118.
Kolodner, J. L. (1993). Case-based reasoning. San Mateo, CA: Morgan Kaufman.
Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J., Puntembakar, S.,
& Ryan, M. (2003). Problem-based learning meets case-based reasoning in the
790 Crismond & Adams

middle-school science classroom: Putting Learning by Design™ into practice. Journal of

the Learning Sciences 12(4), 495–547.
Kolodner, J. L., Camp, P. J., Crismond, D., Fasse, B., Gray, J., Holbrook, J., & Ryan, M.
(2004). Promoting deep science learning through case-based reasoning: Rituals and prac-
tices in Learning by Design classrooms. In N.M. Seel and S. Dykstra (Eds.), Curriculum,
plans and processes of instructional design: International perspectives. Mahwah, NJ: Lawrence
Erlbaum.
Kolodner, J. L., Crismond, D., Gray, J., Holbrook, J., & Puntembakar, S. (1998). Learning by
Design from theory to practice. Proceedings of the International Conference of the Learning
Sciences, 16–22.
Kolodner, J. L., & Wills, L. M. (1996). Powers of observation in creative design. Design Stud-
ies 17(4), 385–416.
Kuffner, T. A., & Ullman, D. G. (1990). The information requests of mechanical design engi-
neers. In J.R. Rinderle (Ed.), Proceedings of the Design Theory and Methodology Conference
(pp. 167–174). American Society of Mechanical Engineers.
Lande, M., & Leifer, L. (2010). Incubating Engineers and Hatching Design Thinkers: Me-
chanical Engineering Students Learning Design with Ambidextrous Ways of Thinking.
Proceedings of the Annual American Society for Engineering Education Conference, Louisville,
KY.
LaPorte, J. E., & Sanders, M. E. (1995). Technology, science, mathematics connection activities.
Lake Forest, IL: Glencoe.
Larkin, J. H., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Models of competence in
solving physics problems. Cognitive Science 4(4), 317–345.
Lawson, B. (1979). Cognitive strategies in architectural design. Ergonomics 22(1),
59–68.
Lawson, B., & Dorst, K. (2009). Design expertise. Oxford, UK: Architectural Press.
Lederman, N., Abd-El-Khalick, F., Bell, R., & Schwartz, R. (2002). Views of nature of sci-
ence questionnaire: Toward valid and meaningful assessment of learners’ conceptions of
nature of science. Journal of Research in Science Teaching 39(6), 497–521.
Lehrer, R., & Schauble, L. (2009). Invited comment: Images of learning, images of progress.
Journal of Research in Science Teaching 46(6), 731–735.
Lemons, D., Carberry, A., Swan, C., & Rogers, C. (2010). The benefit of model building in
teaching engineering design. Design Studies 31(3), 288–309.
Lewin, D. (1979). On the place of design in engineering. Design Studies 1(2), 113–117.
Lin, X. D., Hmelo, C. E., Kinzer, C. K., & Secules, T. J. (1999). Designing technology to sup-
port reflection. Educational Technology Research & Development 47(3), 43–62.
Linder, B., & Flowers, W. C. (2001). Integrating engineering science and design: A defini-
tion and discussion. International Journal of Engineering Education 17(4–5), 436–439.
Linsey, J. S. & Viswanathan, V. K. (2010). Innovation skills for tomorrow’s sustainable design-
ers. International Journal of Engineering Education, Special Issue of the Harvey Mudd Design
Education Workshop, 451–461.
Loewenstein, J., Thompson, L., & Gentner, D. (1999). Analogical encoding facilitates
knowledge transfer in negotiation. Psychonomic Bulletin & Review 6(4), 586–597.
Logan, G. D., & Radcliffe, D. F. (1998). Artefacting in a cross-discipline design team. Pro-
ceedings of DETC98. 1998 ASME Design Engineering Technical Conference, Atlanta,
GA.
The Informed Design Teaching & Learning Matrix 791

MacDonald, D., & Gustafson, B. (2004). The role of design drawing among children en-
gaged in a parachute building activity. Journal of Technology Education 16(1),
55–71.
MacPherson, R. T. (1998). Factors affecting technological troubleshooting skills. Journal of
Industrial Teacher Education 35(4), 1–18.
Mankoff, S. P., Brander, C., Ferrone, S., & Marincola, F. M. (2004). Lost in translation: Ob-
stacles in translational medicine. Journal of Translational Medicine 2(14).
doi:10.1186/1479-5876-2-14.
Mann, C. R. (1918). A study of engineering education. New York: Carnegie Foundation for the
Advancement of Teaching.
Mann, R. W. (1981). Engineering design education: U.S. – Retrospective and contemporary.
Journal of Mechanical Design 103(4), 696–701.
Martin, J., Adams, R., & Turns, J. (2002). Who listens to whom? A citation analysis of recent
papers on engineering design education. Proceedings of the Annual American Society for En-
gineering Education Conference, Montreal, June, Session 2325.
Matchett, E. (1968). Control of thought in creative work. Chartered Mechanical Engineer
14(4).
McCormick, R. (1993). Design education and science: Practical implications. In M. J. deVr-
ies, N. Cross, & D. P. Grant (Eds.), Design methodology and relationships with science (pp.
309–319). Boston: Kulwer Academic Publisher.
McCormick, R. (1994). Learning through apprenticeship. In D. Blandow & M. J. Dyren-
furth (Eds.), Technology education in school and industry: Emerging didactics for human re-
source development (pp. 331–342). Berlin: Springer-Verlag.
McCormick, R., Murphy, P., & Davidson, M. (1994). Design and technology as revelation
and ritual. IDATER ’94, 38–42.
McDonough, W., & Braungart, M. (2002). Cradle to cradle: Remaking the way we make things.
New York: North Point Press.
McGinnis, B. D. & Ullman, D. G. (1992). The evolution of commitments in the design of a
component. Journal of Mechanical Design, 114(1), 1–7.
McKenna, A., Linsenmeier, R., & Glucksberg, M. (2008). Characterizing Computational
Adaptive Expertise. Proceedings of the Annual Meeting of the American Society of Engineering
Education Conference, Pittsburgh.
McKim. R. H. (1980). Experiences in visual thinking (2nd ed.) Monterey, CA: Brooks/Cole
Publishing Company.
McLennan, J. F. (2004). The philosophy of sustainable design. Kansas City: Ecotone.
McNeill, T., Gero, J. S. & Warren, J. (1998). Understanding conceptual electronic design
using protocol analysis. Research in Engineering Design 10(3), 129–140.
McTighe, J. & Wiggins, G. (2004). Understanding by design: Professional development work-
book. Alexandria, VA: Association for Supervision and Curriculum Development.
Miller, C. (1995). Learning through designing: Connecting theory with hardware in engi-
neering education. Unpublished doctoral thesis. Cambridge, MA: MIT.
Milne, A., & Leifer, L. (1999). The ecology of innovation in engineering design. Proceedings
of the International Conference on Engineering Design. Munich, Germany, August.
Mioduser, D., Venezky, R. L., & Gong, B. (1996). Students’ perceptions and designs of simple
control systems. Computers in Human Behavior 12(3), 363–388.
Molina, A., Al-Ashaab, A. H., Ellis, T. I. A., Younger, R. I. M., & Bell, R. (1995). A review of
computer-aided simultaneous engineering systems. Research in Engineering Design 7(1),
38–63.
792 Crismond & Adams

Moorman, K., & Ram, A. (1994). A model of creative understanding. In Proceedings of the
12th National Conference on Artificial Intelligence: Seattle, WA: AAAI-94.
Morris, N. M., & Rouse, W. B. (1985). Review and evaluation of empirical research in trou-
bleshooting. Human Factors 27(5), 503–530.
Mosborg, S., Adams, R., Kim, R., Atman, C. J., Turns, J., & Cardella, M. (2005). Concep-
tions of the engineering design process: An expert study of advanced practicing profes-
sionals. Proceedings of the Annual Meeting of the American Society of Engineering Education
Conference, Portland, OR.
Museum of Science. (2009). Catching the wind: Designing windmills. Boston, MA: Museum
of Science.
National Academy of Engineering. (2004). The engineer of 2020: Visions of engineering in the
new century. Washington, DC: The National Academies Press.
National Academy of Engineering. (2005). Educating the engineer of 2020: Adapting
engineering education to the new century. Washington, DC: The National Academies Press.
National Academy of Sciences. (2010). Standards for K–12 engineering education? Washing-
ton, DC: The National Academies Press.
National Academy of Sciences. (2012). A framework for K–12 science education: Practices, cross-
cutting concepts, and core ideas. Washington, DC: National Academies Press.
National Research Council [NRC]. (1996). National science education standards. Washington,
DC: National Academy Press.
National Research Council [NRC]. (2007). Taking science to school: Learning and teaching sci-
ence in grades K-8. Washington, DC: National Academy Press.
National Research Council [NRC]. (2011). Promising practices in undergraduate science, tech-
nology, engineering, and mathematics education: Summary of two workshops. Natalie Nielsen,
Rapporteur. Planning Committee on Evidence on Selected Innovations in Undergraduate
STEM Education. Board on Science Education, Division of Behavioral and Social Sci-
ences and Education. Washington, DC: The National Academies Press.
Nelson, H. & Stolterman, E. (2003). The design way: Intentional change in an unpredictable
world. New Jersey: Educational Technology Publications.
Newstetter, W. C., & McCracken, W. M. (2001). Novice conceptions of design: Implications
for the design of learning environments. In C. Eastman, M. McCracken, & W. Newstetter
(Eds.), Design knowing and learning: Cognition in design education (pp. 63–77). Amster-
dam: Elsevier.
Noble, D. F. (1979). America by design: Science, technology, and the rise of corporate capitalism.
New York: Alfred A. Knopf.
Norman, D. (1996). Cognitive engineering. In C.T. Mitchell (Ed.), New thinking in design:
Conversations on theory and practice. New York: Van Nostrand Reinhold.
Ogle, D. (1986). A teaching model that develops active reading of expository text. The Read-
ing Teacher 39(6), 564–570.
Otto, K. N. & Wood, K. L. (1998). Product evolution: A reverse engineering and redesign
methology. Research in Engineering Design 10(4), 226–243.
Pahl, G. & Beitz, W. (1995). Engineering design: A systematic approach. London: Springer-Ver-
lag.
Paulus, P. B., Kohn, N. W., & Arditti, L. E. (2011). Effects of quantity and quality instruc-
tions on brainstorming. Journal of Creative Behavior 45(1), 38–46.
Pea, R. D., & Kurland, D. M. (1984). On the cognitive effects of learning computer program-
ming. New Ideas in Psychology 2(2), 137–168.
The Informed Design Teaching & Learning Matrix 793

Pembridge, J., & Paretti, M. (2010). The current state of capstone design pedagogy. Proceed-
ings of the Annual Meeting of the American Society of Engineering Education Conference, Lou-
isville, KY.
Perkins, D. N. (1995). Outsmarting IQ: The emerging science of learnable intelligence. New York:
Free Press.
Perkins, D. N. (1997). Creativity’s camel: The role of analogy in invention. In T. B. Ward, S. M.
Smith, & J. Vaid (Eds.), Creative thought: An investigation of conceptual structures and process-
es (pp. 523–538). Washington, DC: American Psychological Assoc.
Perkins, D. N., Crismond, D., Simmons, B., & Unger, C. (1995). Inside understanding. In D.
N. Perkins, J. M. Schwartz, M. M. West, & M. S. Wiske (Eds.), Software goes to school (pp.
70–87). New York: Oxford University Press.
Petrosino, A. J. (1998). The use of reflection and revision in hands-on experimental activities by
at-risk children. Unpublished doctoral dissertation. Nashville, TN: Vanderbilt University.
Petroski, H. (1993). Failure as source of engineering judgment: Case of John Roebling. Jour-
nal of Performance of Constructed Facilities 7(1), 46–58.
Pietroforte, R. (1998). Civil engineering education through case studies of failure. Journal of
Performance of Constructed Facilities 12(2), 51–55.
Powell, J.A. (1987). Is design a trivial pursuit? Design Studies 8(4), 187–206.
Puntambekar, S., & Kolodner, J. L. (2005). Toward implementing distributed scaffolding:
Helping students learn science from design. Journal of Research in Science Teaching 42(2),
185–217.
Purcell, A. T., & Gero, J. S. (1996). Design and other types of fixation. Design Studies 17(4),
363–383.
Purzer, S. (2010). The MERIT kit: Methods for evaluating roles and interactions in teams.
Proceedings of the Annual Meeting of the American Society of Engineering Education Confer-
ence, AC 2010-1446. Louisville, KY.
Radcliffe, D. & Lee, T. (1989). Design methods used by undergraduate engineering students.
Design Studies 10(4), 199–207.
Rasmussen, J. (1984). Strategies for state identification and diagnosis in supervisory control
tasks, and design of computer-based support systems. Advances in Man-Machine Systems
Research, Volume 1, 139–193. JAI Press.
Rasmussen, J. (1993). Diagnostic reasoning in action. IEEE Transactions on Systems, Man and
Cybernetics 23(4), 981–992.
Reigeluth, C. M. (1999). What is instructional-design theory and how is it changing? In C.
M. Reigeluth (Ed.), Instructional-Design Theories and Models: A new paradigm of instruc-
tional theory, Volume II (pp. 5–29). Mahwah, NJ: Lawrence Erlbaum Associates.
Rendon-Herrero, O. (1993). Including failure case studies in civil engineering courses. Journal
of Performance of Constructed Facilities 7(3), 181–185.
Rhodes, P. (1998). Abundance of information: How do designers use information? IDATER
’98, 132–140.
Rittel, W.W.J., & Webber, M. M. (1984). Planning problems are wicked problems. In N.
Cross (Ed.), Developments in design methodology. Chichester: John Wiley & Sons.
Roozenburg, N. F. M. & Cross, N. G. (1991). Models of the design process: integrating across
the disciplines. Design Studies 12(4), 215–220.
Roth, W.-M. (1995a). From ‘wiggly structures’ to ‘unshaky towers’: Problem framing, solu-
tion finding, and negotiation of courses of action during a civil engineering unit for ele-
mentary students. Research in Science Education 25(4), 365–381.
794 Crismond & Adams

Roth, W.-M. (1995b). Inventors, copycats, and everyone else: The emergence of shared re-
sources and practices as defining aspects of classroom cultures. Journal of Research in Science
Teaching 30(2), 127–152.
Rowland, G. (1992). What do instructional designers actually do? An initial investigation of
expert practice. Performance Improvement Quarterly 5(2), 65–86.
Sachs, A. (1999). Stuckness in the design studio. Design Studies 20(2), 195–209.
Sadler, P. M., Coyle, H. P., & Schwartz, M. (2000). Engineering competitions in the middle
school classroom: Key elements in developing effective design challenges. Journal of the
Learning Sciences 9(3), 299–327.
Sandborn, P., Myers, J., Barron, T., & McCarthy, M. (2009). Using teardown analysis as a ve-
hicle to teach electronic systems manufacturing cost modeling. International Journal of
Engineering Education, 25(1), 42–52.
Sanders, E. B. (2006). Design research in 2006. Design Research Quarterly 1(1), 1–8.
Schauble, L., Klopfer, L. E., & Raghavan, K. (1991). Students’ transition from an engineering
model to a science model of experimentation. Journal of Research in Science Teaching 28(9),
859–882.
Schauble, L., Raghavan, K., & Glaser, R. (1993). The discover and reflection notation: A
graphical trace for supporting self-regulation in computer-based laboratories. In S. P.
Lajoie & S. J. Derry (Eds.), Computers as cognitive tools (pp. 319–337). Hillsdale, NJ: Law-
rence Erlbaum.
Schön, D. (1983). The reflective practitioner: How professionals think in action. New York: Basic
Books.
Schön, D. (1984). Problems, frames and perspective on designing. Design Studies 5(3), 132–
136.
Schön, D. (1987). Educating the reflective practitioner: Toward a new design for teaching and
learning in the professions. San Francisco: Jossey-Bass.
Schön, D. (1995). Knowing in Action: The New Scholarship Requires a New Epistemology.
Change 27(6), 27–34.
Scott, C., Atman, C. J. & Turns, J. (2001). Mastering design concepts through the coding of
design. Paper presented at the Annual Meeting of the American Educational Research
Association. Seattle, WA.
Seely, B. E. (1999). The other re-engineering of engineering education, 1900–1965. Journal of
Engineering Education 88(3), 285–294.
Shah, J. J., & Vargas-Hernandez, N. (2003). Metrics for measuring ideation effectiveness.
Design Studies 24(2), 111–134.
Shah, J. J., Smith, S. M., Vargas-Hernandez, N., Gerkens, D. R., & Wulan, M. (2003). Em-
pirical studies of design ideation: Alignment of design experiments with lab experiments.
Proceedings of DETC 2003: ASME 2003 International Conference on Design Theory and
Methodology. Sept 2–6, 2004, Chicago, IL.
Shafir, E., Simonson, I., & Tversky, A. (1997). Reason-based choice. In W. M. Goldstein &
R. M. Hogarth (Eds.), Research on judgment and decision making: Currents, connections, and
controversies. Cambridge, UK: Cambridge University Press.
Sheppard, S., & Jenison, R. (1997). Examples of freshman design education. International
Journal of Engineering Education 13(4), 248–261.
Sheppard, S. D., Macatangay, K., Colby, A., & Sullivan, W. (2008). Educating engineers: De-
signing for the future of the field. San Francisco: Jossey-Bass.
Shulman, L. S. (1986). Those who can understand: Knowledge growth in teaching. Educa-
tional Researcher 15(2), 4–14.
The Informed Design Teaching & Learning Matrix 795

Simon, H. (1984). The structure of ill-structured problems. In N. Cross (Ed.), Developments

in design methodology. Chichester: John Wiley & Sons.
Smith, P. L., & Ragan, T. J. (2005). Instructional design. Hoboken, NJ: J Wiley & Sons.
Smith, R. P., & Tjandra, P. (1998). Experimental observation of iteration in engineering de-
sign. Research in Engineering Design 10(2), 107–117.
Smith, S. M. (1995). Fixation, incubation, and insight in memory, problem solving, and cre-
ativity. In S. M. Smith, T. B. Ward, & R. A. Finke (Eds.), The creative cognition approach
(pp. 135–155). Cambridge: MIT Press.
Sobek, II, D. K., & Jain, V. K. (2007). Process Factors Affecting Design Quality: A Virtual
Design of Experiments Approach. Journal of Mechanical Design 129(5), 483–490.
Society for the Promotion of Engineering Education [SPEE]. (1930). Report of the investiga-
tion of engineering education 1923–1929. Pittsburgh, PA.
Songer, N. B., Kelcey, B., & Gotwals, A. W. (2009). How and When Does Complex Reason-
ing Occur? Empirically Driven Development of a Learning Progression Focused on
Complex Reasoning about Biodiversity. Journal of Research in Science Teaching 46(6), 610–
631.
Spinks, N., Silburn, N., & Birchall, D. (2006). Educating engineers for the 21st century: The in-
dustry view. Oxfordshire, UK: Royal Academy of London.
Stables, K., & Rogers, M. (2001). Reflective and literate boys: Can design and technology
make a difference? IDATER’01, 124–129.
Stacey, M. K., Eckert, C. M., & Earl, C. F. (2009). From Ronchamp by sledge: On the prag-
matics of object references. In J. McDonnell and P. Lloyd (Eds.), About designing:
Analysing design meetings (pp. 360–379). Leiden, Netherlands: CRC Press.
Sternberg, R. (1998). Metacognition, abilities, and developing expertise: What makes an ex-
pert student? Instructional Science 26(1–2), 127–140.
Strobel, J., & Pan, R. (2011). Compound problem solving: Insights from the workplace for
engineering education. Journal of Professional Issues in Engineering Education & Practice
137(4), 215–222.
Suwa, M., & Tversky, B. (1997). What do architects and students perceive in their design
sketches? A protocol analysis. Design Studies 18(4), 385–403.
Terpenny, R. S., & Goff, C. J. (2005). Characterizing engineering student design processes:
An illustration of iteration. Proceedings of the Annual Meeting of the American Society of En-
gineering Education Conference, Session 3255. June 12–15, Portland, OR.
Turns, J. (1998). Students’ use of functional, behavioral, and structural terms to describe arti-
facts during design. Proceedings of the Annual Meeting of the American Society of Engineering
Education Conference.
Turns, J., Adams, R. S., Linse, A., & Atman, C. J. (2003). Bridging from research to teaching
in undergraduate engineering design education. International Journal of Engineering Edu-
cation 20(3), 379–390.
Turns, J., Adams, R. S., Cardella, M., Atman, C. J., Martin, J., & Newman, J. (2006). Tackling
the research-to-practice challenge in engineering design education: Making the invisible
visible. International Journal of Engineering Education 22(3), 598–606.
Turns, J., Sattler, B., & Kilgore, D. (2010). Disciplinary knowledge, identity, and navigation:
The contributions of portfolio construction. Proceedings of the International Society of the
Learning Sciences (pp. 818–825). Chicago, June 30.
Valkenburg, R. (1998). The reflective practice of design teams. Design Studies 19(3), 249–
271.
796 Crismond & Adams

Visser, W. (1995). Reuse of knowledge: empirical studies. In M. Veloso & A. Aamodt (Eds.),
Case-based reasoning: Research and development. First International Conference, ICCBR-95
(pp. 335–346). Sesimbra, Portugal, October 1995. Berlin: Springer.
Visser, W. (2006). The cognitive artifacts of designing. Mahwah, NJ: Lawrence
Erlbaum.
Ullman, D.G. (1997). The mechanical design process. Boston: McGraw Hill.
Ullman, D. G., Herling, D., & Sinton, A. (1996). Analysis of protocol data to identify prod-
uct information evolution and decision making process. In N. Cross, H. Christiaans, & K.
Dorst (Eds.), Analysing design activity (pp. 169–186). Chichester: John Wiley & Sons.
Ullman, D. G., Wood, S., & Craig, D. (1990). The importance of drawing in the mechanical
design process. Computers & Graphics 14(2), 263–274.
Ulrich, K.T., & Eppinger, S. D. (1995). Product design and development. New York: McGraw-
Hill.
Urban, G.L., & Hauser, J. R. (1995). Design and marketing of new products (2nd ed.) Engle-
wood Cliffs, NJ: Prentice Hall.
Vargas-Hernandez, N., Shah, J. J., Smith, S. M. (2010). Understanding design ideation
mechanisms through multilevel aligned empirical studies, Design Studies 31(4), 382–410.
Visser, W. (1996). Two functions of analogical reasoning in design: A cognitive-psychology
approach, Design Studies 17(4), 417–434.
Visser, W. (2009). The function of gesture in an architectural design meeting. In J. McDon-
nell & P. Lloyd (Eds.), R. Luck, F. Reid, & N. Cross (Assoc. Eds.), About designing:
Analysing design meetings (pp. 269–284). London: Taylor & Francis.
Wedman, J., & Tessmer, M. (1991). Adapting instructional design to project circumstance:
The layers of necessity model. Educational Technology 31(7), 48–52.
Welch, M. (1998). Students’ use of three-dimensional modeling while designing and making
a solution to a technological problem. International Journal of Technology and Design Edu-
cation 8(3), 241–260.
Welch, M., & Barlex, D. (2004). Portfolios in design and technology education: Investigating
differing views. In E. W. L. Norman, D. Spendlove, P. Grover, & A. Mitchell (Eds.),
DATA International Research Conference 2004 (pp. 193–197).
Welch, M., Barlex, D., & Lim, H. S. (2000). Sketching: Friend or foe to the novice designer?
International Journal of Technology and Design Education 10(2), 125–148.
Welch, M., Barlex, D., & Taylor, K. (2005). I don’t enjoy making the folder: Secondary stu-
dents’ views of portfolios in technology education. DATA International Research Conference
2005.
Welch, M., & Lim, H. S. (1999). Teaching sketching and its effects on the solutions produced
by novice designers. IDATER‘99, 188–194.
West, H., Flowers, W., & Gilmore, D. (1990). Hands-on design in engineering education:
Learning by doing what? Journal of Engineering Education 80(5), 560–564.
White, B., & Frederiksen, J. (1993). Causal models and intermediate abstractions: A missing
link for successful science education? In R. Glaser (Ed.), Advances in instructional psycholo-
gy (pp. 177–252). Hillsdale, NJ: Lawrence Erlbaum.
Whitehead, A. N. (1929). The aims of education. New York: Macmillan.
Wild, P. J., & McMahon, C. (2010). A diary study of information needs and document usage
in the engineering domain. Design Studies 31(1), 46–73.
Wills, L.M., & Kolodner, J. L. (1994). Explaining serendipitous recognition in design. 16th
Annual Conference of the Cognitive Science Society, 940–945. Atlanta, GA.
The Informed Design Teaching & Learning Matrix 797

Woods, D. R., & Crowe, C. M. (1984). Characteristics of engineering students in their first
two years. Engineering Education 74(5), 289–295.
Woolnough, B. E. (1991). Practical science as a holistic activity. In B. E. Woolnough (Ed.),
Practical Science (pp. 181–188). Philadelphia, PA: Open University Press.
Yilmaz, S., & Seifert, C. M. (2011). Creativity through design heuristics: A case study of ex-
pert product design. Design Studies 32(4), 384–415.
Zubrowski, B. (2002a). Integrating science into design technology projects: Using a standard
model in the design process. Journal of Technology Education 13(2), 48–67.
Zubrowski, B. (2002b). Design It! Farmingdale, NY: Kelvin.
Zubrowski, B. (2009). Explorations and meaning making in the learning of science: Innovations
in science education and technology. Berlin: Springer-Verlag.

AUTHORS

David P. Crismond is an associate professor in the School of Education at City Col-

lege, City University of New York, 138th St. & Convent Ave. NAC 6/207b, New York,
NY 10031; dcrismond@ccny.cuny.edu.

Robin S. Adams is an associate professor in the School of Engineering Education and

an affiliate of Environmental and Ecological Engineering at Purdue University, 1233
Armstrong Hall, 701 West Stadium Avenue, West Lafayette, IN 47907-2045; rsadams@
purdue.edu,