54 IEEE TRANSACTIONS ON EDUCATION, VOL. 56, NO.

1, FEBRUARY 2013

A One-Year Introductory Robotics Curriculum

for Computer Science Upperclassmen
Nikolaus Correll, Member, IEEE, Rowan Wing, and David Coleman

Abstract—This paper describes a one-year introductory robotics in the long run, specific “robotics” classes are needed that focus
course sequence focusing on computational aspects of robotics on the interface between the computational and physical world
for third- and fourth-year students. The key challenges this cur- and let students experience real-world dynamics, uncertainty,
riculum addresses are scalability, i.e., how to teach a robotics class
with a limited amount of hardware to a large audience,student
and complexity firsthand.
assessment, i.e., how to assess the students’ success on robotic Teaching such a class poses considerable obstacles, which
design and programming assignments, and depth versus breadth, this paper identifies and addresses.
i.e., how to down-select content from the interdisciplinary field 1) Scalability: Classes that involve complex robotic systems,
of robotics to computer science students. This is achieved by i.e., those that go beyond simple toy platforms and in-
combining simulation-based laboratory assignments, which can volve processing of rich sensor information with reason-
be conducted anywhere and anytime, with compatible hardware
devices that allow a seamless transition from simulation to real able computational power, are difficult to scale. Acquiring,
hardware, and a focus on performance-based assessment with maintaining, and explaining the hardware becomes a bot-
an open-ended final project/competition. Content learning and tleneck, which limits class size at those few institutions that
retention is assessed for a subset of students who successfully went can afford a number of complex robotic systems.
through the proposed curriculum. All class materials as well as 2) Assessment: Only a subset of the course material of
hardware—in particular, a low-cost, highly articulated robotic
arm developed for teaching advanced robotics concepts—are
a robotics curriculum can be assessed conceptually or
open-source and available online. mathematically. Demonstrating a thorough understanding
of an algorithm requires implementation. Unlike purely
Index Terms—Computer science education, educational robots.
computational algorithms, which can be assessed auto-
matically using test-input, authentic assessment scenarios
I. INTRODUCTION are difficult to create for a robotics class.
3) Complexity: Robotic systems require a large number of in-
R OBOTS are computers that are extended by sensing, actu-
ation, and communication capabilities. Similarly, devices
that were previously limited to actuation or sensing alone, such
dependent modules (vision, locomotion, planning, etc.) to
function. As it is not possible to teach all of the required
background comprehensively in one class, finding the right
as vacuum cleaners, lawnmowers, cars, or everyday gadgets and
trade-off between providing ready-made modules, and of-
household appliances, are moving toward becoming robots by
fering students the possibility to explore the basics of the
increasing their sensing and computation capabilities, eventu-
respective fields while ensuring some deep learning, is a
ally leading to a paradigm shift from “personal computers” to
challenging problem.
“personal robots” [1]. This viewpoint positions “robotics” not as
This paper describes a two-semester robotics course designed
a new discipline, but as an extension of computer science and
for upperclassmen, which addresses the issues of scalability,
its subdisciplines including artificial intelligence, natural lan-
assessment, and complexity. All teaching materials, including
guage processing, computer vision, software engineering, secu-
transcripts of lectures, screencasts, software, and hardware
rity, and human–computer interaction, among others.
models, are available online1.
The key challenges of a robotic system—both from an educa-
tional and a research perspective—are: 1) that the dynamics and
uncertainty of the real world require a shift from deterministic to
A. Related Work
probabilistic reasoning, and (2) that robots are systems-of-sys-
tems often consisting of hundreds of distributed, heterogeneous
There exists a series of scholarly assessed robotics curricula
computing elements. While these challenges might require cur-
at the high school [2], freshmen/sophomore [3]–[6], and un-
riculum updates to almost all computer science subdisciplines
dergraduate degrees/minor levels [7]–[10]. A challenge com-
monly addressed in robotics education is how to blend elec-
Manuscript received May 21, 2012; accepted August 21, 2012. Date of pub- trical and mechanical engineering contents with computer sci-
lication October 19, 2012; date of current version January 30, 2013. This work
was supported by the NSF under Award No. 1150223 and the Engineering Ex-
ence [11]–[13]. The work described here proposes a curriculum
cellence Fund of the University of Colorado at Boulder. at the upperclassmen level that makes electrical and mechanical
The authors are with the Department of Computer Science, University engineering aspects of robotics transparent and exclusively fo-
of Colorado at Boulder, Boulder, CO 80309-0430 USA (e-mail: niko-
laus.correll@colorado.edu; rowan.wing@colorado.edu; david.coleman@
cuses on computational challenges, in a similar fashion to that
colorado.edu). of [8]. Like other classes, this curriculum takes advantage of
Color versions of one or more of the figures in this paper are available online the benefits of a practice-integrated curriculum [3], integrating
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/TE.2012.2220774 1http://correll.cs.colorado.edu

0018-9359/$31.00 © 2012 IEEE

CORRELL et al.: ONE-YEAR INTRODUCTORY ROBOTICS CURRICULUM FOR COMPUTER SCIENCE UPPERCLASSMEN 55

actuators are accelerometer, microphones, and speakers.) This

configuration allows for instruction in basic robotic concepts
ranging from odometry, to collision avoidance, to EKF-based
SLAM using visual markers.
The e-Puck can be programmed using the Webots simulated
environment, shown in Fig. 1, via cross compilation or using the
remote controlled option. In the latter case, all sensor readings,
Fig. 1. Real and simulated instantiations of the “Ratslife” environment including camera, odometry, and distance sensors, are trans-
designed by Cyberbotics S.A. Two e-Puck robots compete on reaching four ferred to the simulator via Bluetooth. In turn, all actuator com-
chargers in the environment. Whereas simple wall-following might solve the mands are sent from the simulator to the e-Puck robot. This al-
game, optimal solutions require simultaneous localization and mapping as well
as strategic planning. lows students to drive the robot using the computational capa-
bilities of a desktop machine, which is important when doing
image processing and SLAM, and to hot-swap simulated and
competition as a driver for deep learning [8], [14]–[17], and per- real robots. For example, students can design an experiment to
formance-based assessment, which measures student learning collect odometry data, collect the data in simulation for a large
through hands-on, product-centered tasks [18]. number of trials, and then verify the observed behavior using a
There exists also a series of textbooks providing an outline few trials involving a real robot without changing any code.
for one- and two-semester courses. Most available resources The class meets twice a week, once for a lecture (75 min)
address specific topics at the graduate level, for example kine- and once for a lab session (150 min). Whereas lectures involve
matics and dynamics [19], [20], motion planning [21], [22], the entire class, labs are offered at multiple times to keep group
or probabilistic localization and mapping [23]. At least two sizes to a maximum of 20 students, which can be supported by
textbooks specifically address broader computational princi- a single teaching assistant (TA). A midterm exam replaces one
ples [24], [25], but are limited to high-level content for the lecture in the middle of the curriculum. During the lectures, stu-
sake of providing as broad an overview of the field as possible, dents are exposed to the theoretical basis for the core algorithms
and thus are not suitable for undergraduates. Literature at the of the course. The necessary mathematics are presented within
lower undergraduate level, e.g., [26], however, mostly focuses a conceptual context, and connections are drawn between the
on simple behavioral robotics as well as foundations of mecha- abstract aspects of the algorithms and the consequences for em-
tronic systems and does not touch on deeper computational bodied systems. The laboratory portion of the course supports
topics. the theoretical topics of the lectures and provides a hands-on op-
The work described here, together with teaching materials portunity for implementation. Students are encouraged to work
provided online, specifically addresses computer science upper- cooperatively to transfer their knowledge between theory, sim-
classmen and bridges the gap between freshmen/sophomore and ulation, and ultimately the physical robotic system.
graduate courses. The course content is divided into three primary components,
II. INTRODUCTION TO ROBOTICS each lasting approximately a month. The first part of the course
introduces students to the background they need for the re-
The first semester of the course focuses on kinematics of mainder of the semester. Students are introduced to the field of
wheeled robots and their localization and navigation in un- robotics and the variability of current research and applications.
known environments and is similar in style to [8]. The lecture They learn the basics of kinematics and locomotion, familiarize
starts from first principles and Bayes’ rule and leads to the themselves with the Webots simulation environment, and begin
extended Kalman filter (EKF) and EKF-based simultaneous to plan how they will approach the Ratslife maze task. During
localization and mapping (SLAM), two of the breakthrough re- the second part of the course, they get deeper into the details
sults in control theory and AI of the last century. The curriculum of sensing and actuation. Students experiment with different
is motivated by a robotic maze competition “Ratslife”2 [27] forms of sensing, including distance sensors, odometry, and
developed by Cyberbotics S.A., in which two robots compete vision, and how each may be used to facilitate movement
over recharging at four charging stations that are hidden in and localization within an environment. The third part of the
a maze. The chargers provide enough energy for the robot course exposes them to more advanced topics involved in robot
to continue for another 90 s, but require 120 s to recharge. navigation. Students experiment with how error propagation
The contest therefore encourages a strategy of mapping the can be quantized and contained using Kalman filtering and
chargers and systematically moving back and forth between SLAM. The remaining few weeks of the course are devoted
them. This process is facilitated by 48 unique markers on the to integrating and assessing concepts learned. Students form
walls of the environment that can be recognized with basic teams and develop a comprehensive controller for the Ratslife
image processing techniques. The real-world environment, maze and participate in debates on various aspects of robotics.
shown in Fig. 1, is made from LEGO bricks (instructions During the first component, mathematical concepts are solidi-
available online) and requires two e-Puck robots [28]. The fied using written homework assignments. The primary deliver-
e-Puck is equipped with two stepper motors in a differential able from the lab is a systematic measurement and histogram of
drive configuration and an array of infrared distance sensors, a the odometry error with respect to translation (moving straight
camera, and a Bluetooth connection. (Additional sensors and for 10 cm) and rotation (turning by 90 and moving straight)
2http://www.ratslife.org of the simulated and actual e-Puck. Laboratory assignments are
56 IEEE TRANSACTIONS ON EDUCATION, VOL. 56, NO. 1, FEBRUARY 2013

initially scaffolded, and the students are supported through the group collaborates on a single controller for the Ratslife maze.
use of prefabricated environments. As students become more This final project gives students an opportunity to revisit each
comfortable, these supports are slowly removed, and students element of the course and synthesize their knowledge. The
are prepared for the next component of the course. final day of laboratory is a competition between the controllers
The deliverables during the second part of the course are im- developed by the groups, both in the simulated Webots world
plementations of topics discussed in class. First, students will and on the real-world e-Puck in a LEGO maze. The difference
implement a simple path-planning algorithm such as Dijkstra’s in performance on these two platforms helps to highlight the
or A* on the Ratslife grid, using a simulated GPS node that challenges inherent in embodied robotics. The students are also
provides accurate positioning information—an instance of pro- required to present and defend their design choice and reflect
viding scaffolding to the task. Second, students will create a on the performance of their controller on the two platforms
controller that allows for obstacle avoidance behavior. Neces- during the competition.
sary concepts including proximal (Braitenberg-style) and distal The last two weeks of lectures are devoted to class debates.
(rule-based) controllers are explicitly provided in the instruc- Student teams prepare pro and contra arguments for a statement
tions for the lab. The function of both controllers is verified by of current technical or societal concern. Sample topics include
the TA, but can also be assessed by experimental data collected Robots putting humans out of work is a risk that needs to be mit-
by the students, e.g., the average time spent in collision avoid- igated; Robots should not have the capability to autonomously
ance for proximal and distal controllers in a distance learning discharge weapons/drive around in cities (autonomous cars);
setting. The third lab consists of exploring various filters for or Robots need to be made from components other than links,
smoothing and edge detection, as well as a rule-based algorithm joints, and gears in order to reach the agility of people. The
to detect patterns using the (simulated) camera of the E-Puck students are instructed to make as much use as possible of tech-
robot. The algorithm, already implemented in the basic con- nical arguments that are grounded in the course materials and in
troller that is provided to the students, filters images in the en- additional literature researched by the students. For example,
vironment for colors. The lab’s deliverable comprises system- students can use the inherent uncertainty of sensors to argue
atic measurements of range and bearing estimates of three or for or against enabling robots to use deadly weapons. Similarly,
more features. Here, range is proportional to the width of the students relate the importance and impact of current develop-
feature, whereas bearing is proportional to its slope. An addi- ments in robotics to earlier inventions that led to industrializa-
tional homework requires the students to reason about the differ- tion, when considering the risk of robots putting humans out of
ences between ultrasound (time of flight), infrared (intensity), work.
and laser scanners (phase shift), as well as identifying various
convolution Kernels. The last lab for this section of the course III. ADVANCED ROBOTICS
consists of an implementation of a simplified line-fitting algo-
rithm using three of the infrared distance sensors of the e-Puck The second semester of the course shifts the focus from
robot and a least-squares solution. mobility to manipulation and systematically extends the con-
During the third component of the course, students formalize cepts taught in the first semester. Students are exposed to
their observations made on basic sensing and actuation, using state-of-the-art search and path-finding algorithms such as
Bayes’ rule and the Gaussian distribution. They begin by Rapidly-exploring Random Trees (RRT), computationally
developing a mathematical understanding of error propagation more efficient solutions to the SLAM problem (Graph-based
and then move on to methods for controlling this propagation, SLAM), advanced feature recognition algorithms such as
eventually leading to the Kalman filter and EKF-based SLAM. SIFT and SURF, and feature recognition in 3-D point clouds.
Although both of these topics would warrant a class on their Students will work on projects addressing a grand challenge
own, e.g., based on [23], complexity is reduced by constructing application in robotics using a 7-DOF arm that closely models
the EKF solution from the error propagation concepts al- the software tool-chain of state-of-the-art platforms such as
ready derived for odometry, line fitting, and feature detection. the PR2 robot and that allows seamless transition to the Cla-
EKF-based SLAM is then derived by treating the positions of mArm3, an open-source dexterous manipulator developed by
features as additional variables in the EKF. previous iterations of the Advanced Robotics class, shown
The deliverables for the third component are a homework that in Fig. 2. Students not only obtain an overview of advanced
lets students derive error propagation for simple examples with computer science topics in robotics, but also become prepared
and without integrating previous error and a lab in which stu- for industry challenges that make use of similar tools. Instead
dents apply error propagation to odometry based on the data col- of an exam, emphasis is put on the ability to transfer expert
lected in previous labs. To properly implement the Kalman filter knowledge acquired during project work to the rest of the class,
or SLAM from scratch is a multiweek project and is not required via weekly design reviews and a conference-paper-style final
report. Sample projects that can be worked on by multiple
in this class. Instead, students are provided with a MATLAB
iterations of the class include a team of robots implementing
implementation of the Kalman filter that they can explore and
gardening tasks [29] or robots assembling other robots from
include in their final project.
“Cubelets” [30].
The final project gives students an opportunity to share
The class meets for 50 min three times a week. Meetings are
what they have learned with the other students and engage in
divided into a lecture, design review, and lab. In this format,
a competition with each other. During the final few weeks in
the laboratory section, students form into groups, and each 3http://correll.cs.colorado.edu/clam
CORRELL et al.: ONE-YEAR INTRODUCTORY ROBOTICS CURRICULUM FOR COMPUTER SCIENCE UPPERCLASSMEN 57

analogy with stiffer spring constants corresponding to observa-

tions with lower variance is very helpful.
Lectures are concluded with a tutorial on how to write a
research paper, which reiterates the scientific method and the
experimental approach that the students were prepared for
throughout the labs, and a review of statistical significance
tests, focusing on the -test (for testing whether a distribution
is Gaussian), Student’s t-test (for comparing two Gaussian
distributions), the Wilcoxon–Ranked Sum test (for comparing
two nonparametric distributions), and testing statistical sig-
nificance of true-false tests using Fisher’s “Lady tasting tea
example” [41].
Class deliverables consist of design reviews and lab reports.
Design reviews are solicited from a subset of the student
groups one week in advance; these consist of 10–20-min-long
Fig. 2. (a) “ClamArm” designed by students of the Advanced Robotics class presentations on their current work and plans. Students are
over multiple iterations of the class. (b) RGB-D sensor point cloud superim-
posed on 3-D model of the arm (bottom inset) in Rviz. encouraged to explain and justify the methods they are using
in order to provide an additional learning experience for the
rest of the class [29]. Three multiweek lab assignments are
a class supports up to 20 students without the need for addi- geared toward preparing the students to work with ROS, the
tional lab sessions. Design reviews are substituted for labs for visualization (RViz) and simulation (Gazebo) environments,
the first few weeks to get students up to speed with the robot op- and to perform systematic experiments. First, students com-
erating system (ROS). Unlike “Introduction to Robotics,” the plete a tutorial on ROS message passing and are asked to
advanced class is open to graduate students who will need to conduct experiments to measure the average time of message
prepare for each lecture using the online materials from the intro passing and service execution. This lab not only requires the
class. Consequently, lectures follow the structure of “Introduc- students to obtain a certain level of proficiency with the ROS
tion to Robotics” and systematically complement its content. environment, but also motivates them to conduct systematic
The lectures begin by complementing the simple forward experimentation, as the message passing time turns out to be
kinematics of differential drive systems with Ackerman steering log-normal distributed. A second lab introduces the OMPL
following the treatment by [22] and formalizes the treatment of planning framework that is used within ROS. Students will
multilink manipulators using the Denavit–Hartenberg Scheme, need to provide the average time it takes to calculate a path as
following [19]. Building up on inverse kinematics of two-link well as its average length using a manipulation task involving a
mechanisms, the class introduces inverse kinematics of mul- simulated PR2 robot. The final lab consists of WillowGarage’s
tilink arms using analytical techniques for a three-link arm tabletop manipulation tutorial, which requires the students to
following an example from [19] and then introduces the ve- work through a complete perception-action loop, starting from
locity Jacobian, leading to the damped least-square method for ICP matching of objects from point-cloud data, to path-plan-
numerical solutions. Dijkstra’s algorithm and A* are comple- ning, to a prestored grasp routine if the object is recognized,
mented by sampling-based planners ranging from RRT to PRM, or executing a grasp heuristic otherwise. The final deliverable
highlighting implementation-relevant issues following [31]. for this lab is a flowchart illustrating the interrelation of the
Simple convolution-based preprocessing methods and simple different components.
feature-detection algorithms known from the intro class are In lieu of a final exam, students present their (group) project
used as a segue to explain scale-invariant feature transforms during the last week of class and hand in a written report in IEEE
including SIFT [32] and SURF [33]. These algorithms can double-column format. Grading of the project is treated much
provide robust features for both iterative close point (ICP) like a review of a research paper, emphasizing a clear descrip-
matching and RGB-D SLAM, which is explained following tion of the work actually being done and sound experimental
the outline provided in [34] and extends simple line-fitting validation of claims made. As not all projects lend themselves
algorithms from the “Introduction to Robotics” class as well to hypothesis-based research, students also have the option to
as generalizing the feature-based EKF-based approach used provide a final report in tutorial or survey form.
in Ratslife. SIFT features are also used to motivate visual
servoing, following the tutorial in [35] and provide further IV. RESULTS
insight into velocity Jacobians. Grasping is then introduced, The results presented here include an assessment of learning
building up from Coulomb’s friction law to develop the con- and of the retention of specific topics in both classes, as well as
cept of task-relevant wrench spaces following [36] and [37]. the efficiency of class debates.
The EKF-based SLAM method from the intro class is now
contrasted with graph-based SLAM, which is particularly A. Learning and Retention
relevant for the previously introduced RGB-D SLAM [34]. For both classes, the students were surveyed using 10 ques-
Based on [38], graph-based SLAM is formulated as a max- tions in the form of “one-answer-per-row” matrices. The test
imum-likelihood estimation problem, and various numerical was administered half a year after the final exam. Each matrix
solution methods [39], [40] are discussed. Here, the spring-mass surveyed the students’ subjective perception of their confidence
58 IEEE TRANSACTIONS ON EDUCATION, VOL. 56, NO. 1, FEBRUARY 2013

TABLE I For both classes, subjective confidence consistently increases

SUBJECTIVE ASSESSMENT OF CONFIDENCE LEVEL IN SPECIFIC TOPICS FROM 0 from “Before” to “After” class and exceeds “Get basic idea”
(“NOTHING”) TO 4 (“VERY CONFIDENT”), BEFORE THE INTRODUCTORY CLASS, for all topics in both classes for topics about which students
AFTER THE CLASS, AND 6 MONTHS AFTER THE CLASS.
knew “Nothing” or “Close to nothing.” The relative variance
can be understood as a measure of knowledge diversity in class.
For all topics and both classes, relative variance decreases from
“Before” to “After,” suggesting students arriving at a common
ground. Subjective retention after six months (only surveyed for
the intro class) remains above “Before” levels for all topics, but
consistently decreases from the level perceived right after class.
A consistent increase of relative variance suggests a diversifi-
cation of students into those who retain the knowledge, perhaps
because they remained actively engaged in the subject matter,
and those who feel that they have begun to forget content.

B. Class Debates
In the intro class, two full lectures were reserved for class
TABLE II
SUBJECTIVE ASSESSMENT OF CONFIDENCE LEVEL IN SPECIFIC TOPICS IN debates. This section presents representative results, taken from
“ADVANCED ROBOTICS” FROM 0 (“NOTHING”) TO 4 (“VERY CONFIDENT”), Fall 2010, of the perception of the students of the debate format.
BEFORE THE CLASS AND AFTER THE CLASS. Students were asked to respond to a series of statements using
the terms “Strongly disagree,” “Disagree,” “Neutral,” “Agree,”
and “Strongly Agree.” Two solicitations via e-mail led to a re-
sponse rate of 18/26 or around 70%. The questions were targeted
at shedding light on three specific aspects: relevance of debates
for the engineering profession, learning experience, and overall
format. Results are summarized in Table III.
While there is agreement that debates help to prepare students
for the engineering profession by improving presentation skills,
preparing engineers to think about questions posed by society,
and reflecting up-to-date topics, the debates seem to have little
effect on changing the students’ actual opinions on a topic (only
level with respect to a certain topic “Before class,” “After class,” two students responded positively). Students are also undecided
and, for the introductory course, “Now” (row labels), choosing about whether the debates helped them to better understand the
from “Nothing,” “Close to nothing,” “Get basic idea,” “Con- technical content of the class. Yet, students find the debate con-
fident,” and “Very confident” (column labels). This format al- cept important enough that they would like to keep it rather than
lowed the students to directly compare their responses. The test a more in-depth treatment of the technical content of the class,
was anonymous. and they disagree that debates should be given less time in class.
The intro class survey consists of 35 undergraduate students, However, students are undecided whether debates are important
31 male and four female, 27 of whom are majoring in Com- enough to merit early inclusion in the curriculum or to be part
puter Science, three in Aerospace Engineering, two in Mathe- of every class.
matics, one in Applied Math, one in Physics, and one in Engi- Concerning the overall format, students find that discussion
neering Physics. Twenty-one students responded to the survey time was too short when allotting 10 min per position and 15 min
after three solicitations. The advanced class survey consists of for discussion and rebuttal. Also, students tend to agree that de-
14 students, six of whom are undergraduate students, and eight bates are an opportunity to decompress (“relaxing”), which is
of whom are first-year Master’s and Ph.D. students. Three of desirable as this period of class coincides with wrapping up the
the undergraduates are B.S. Computer Science students, one course project.
Physics, one Math, and one Electrical Engineering. Five of the
undergraduates have also taken the intro class a semester ear- V. DISCUSSION
lier. Seven graduate students are Computer Science students, Although drastically facilitating the preparation of the class
and one is an Electrical Engineering student. Ten students re- and increasing its scalability when compared to shared-platform
sponded to the survey after three solicitations. courses such as [7], [8], and [29], working with simulation tools
Specific topics as well as results for confidence on a scale is not without challenges. Changes to the IT infrastructure that
from 0–4 (“Nothing” to “Very Confident”) before the class, after are beyond the control of the instructor, or updates of related
the class and, for the intro class, and half a year after the final software, e.g., the Linux operating system or general-purpose
exam are shown in Tables I and II. Values represent the mean software libraries that the class environment depends on, have
and the relative variance (variance/mean). No statistical test was drastically lowered the efficiency of the tools in every iteration
performed because the distributions are independent due to test of the class in one way or another. Providing the students with
design (students ranked their confidence levels for a specific a virtual machine on which all the required software is installed
topic within a single questionnaire). has somewhat alleviated those problems, but issues remain for
CORRELL et al.: ONE-YEAR INTRODUCTORY ROBOTICS CURRICULUM FOR COMPUTER SCIENCE UPPERCLASSMEN 59

TABLE III
STUDENT REACTIONS TO THE DEBATE FORMAT IN FALL 2010.

a few students whose hardware was not sufficiently capable This lack of depth might make them expendable in a course
or compatible. For example, missing hardware acceleration for with already compressed content, given the lack of time and the
graphics and problems with USB connections to connect to the rather limited contribution of the debates to the understanding
real platforms via the virtual machine were recurrent problems of the course work. A potential remedy could be to add more
for some. structure to the assignment by providing one or two key pa-
Performance-based assessment has put the burden on the stu- pers for each topic that provide technical content to support one
dents to produce evidence of their having accomplished the cur- argument or the other, as well as pointing out relevant lecture
ricular goals. Having the students present systematically col- slides that debaters should recall to the class to support a spe-
lected experimental data instead of simply demonstrating code cific argument.
or solving analytical problems does not only allow the teacher to
quickly assess the students’ performance, but also requires the VI. CONCLUSION
students to master a systems problem that ranges from under- The focus on simulation and tools that facilitate transfer from
standing and implementing the math to dealing with the (simu- simulation to real hardware allows a large number of students
lated) challenges of real sensors and actuators including noise, to be reached with limited hardware resources. Making experi-
bandwidth, and range. This form of assessment therefore not mentation on hardware contingent on successful completion of
only allows students to demonstrate a broader volume of content simulation experiments also dramatically lowers the burden of
knowledge, but also forces them to make connections between verifying systems implemented by the students. Together, these
ideas and place their learning in context. Performance-based as- features allow easy scaling of the class for a large number of
sessments also have the added benefit of providing richer infor- students and might facilitate a distance learning environment in
mation to the teacher on the learning needs of the students. As which the bottleneck is coming to campus. In this way, robotics
a result, they are an excellent formative as well as summative education can become more accessible for rural communities
assessment, which allows both the instructor and TA to address and low-budget programs.
specific challenges in a just-in-time fashion. Both parts of the class utilize performance-based assessment,
The complexity of the topics remains a major challenge. The consisting of labs, final projects, and, in the intro class, de-
majority of the students are “confident” or “very confident” bates to measure student achievement. The laboratory deliver-
only in the most basic topics, but in complex topics such as the ables provide the teacher with clear snapshots of student under-
EKF and EKF-based SLAM, they only “get the basic idea” or standing throughout the class. This formative assessment allows
are “confident.” This poses the question whether a curriculum the teacher to differentiate instruction to address areas of weak-
should forgo these topics altogether for the sake of making ness in student understanding. By using the task itself to natu-
students “very confident” in basic topics. The authors believe, rally point out areas of incomplete understanding, students have
however, that “getting the basic idea” is an important step in the opportunity to fortify their learning in these areas. This leads
preparing the students for an informed choice on whether to not only to higher success rates in the course, but also to a high
pursue graduate-level work in these topics, while also providing level of learning and retention.
an opportunity for more advanced students to master the topic Because it is comprehensive in nature, the proposed cur-
by embracing the open-ended competition challenge, which riculum contains content that cannot possibly be understood
was consistently observed by the instructors for one or two deeply by the entire student population. This is mitigated by
students per class. providing the students with open-ended problems in the form
Having to present in class, hearing and arguing about a posi- of a competition/final project, which provide an opportunity
tion completely opposite to one’s own, and thinking about the for deep learning to those that have the time and interest. Yet, it
broader impact of science and technology are important skills remains debatable whether to “get the basic idea” is sufficient,
that should be encouraged early on. These benefits are under- or whether this time should have been spent to produce more
standable as such by a majority of the students. A drawback of “very confident” students in basic topics. This paper defends
the debates in their current form is their lack of technical depth. the position that robotics is a systems science that requires the
60 IEEE TRANSACTIONS ON EDUCATION, VOL. 56, NO. 1, FEBRUARY 2013

engineer to understand the basic ideas of every (computational) [27] O. Michel and F. Rohrer, “The rat’s life benchmark: Competing cog-
component of a robot as well as its limitations. Therefore, nitive robots,” in Proc. 8th PerMIS, New York, NY, 2008, pp. 43–49.
[28] F. Mondada, M. Bonani, X. Raemy, J. Pugh, C. Cianci, A. Klaptocz, S.
a broad foundation, such as that provided by the proposed Magnenat, J.-C. Zufferey, D. Floriano, and A. Martinoli, “The e-Puck, a
curriculum at the undergraduate level, is helpful in providing robot designed for education in engineering,” in Proc. 9th Conf. Auton.
the context for follow-up graduate-level classes focusing on Robot Syst. Competitions, 2009, pp. 59–65.
[29] N. Correll and D. Rus, “Peer-to-peer learning in robotics education:
dynamics, probabilistic robotics, or computer vision. Lessons from a challenge project class,” Comput. Educ. J., vol. 1, no.
3, pp. 60–66, 2010.
[30] E. Schweikardt and M. Gross, “roBlocks: A robotic construction kit
REFERENCES for mathematics and science education,” in Proc. 8th ICMI, 2006, pp.
72–75.
[1] D. Rus, “Teaching robotics everywhere,” IEEE Robot. Autom. Mag., [31] I. Sucan and L. Kavraki, “On the implementation of single-query sam-
vol. 13, no. 1, pp. 15–94, Mar. 2006. pling-based motion planners,” in Proc. IEEE Int. Conf. Robot. Autom.,
[2] N. Ayanian, J. Keller, D. Cappelleri, and V. Kumar, “Development of a 2010, pp. 2005–2011.
successful open-ended robotics desing course at the high school level,” [32] D. Lowe, “Distinctive image features from scale-invariant keypoints,”
Comput. Educ. J., vol. 1, no. 3, pp. 21–31, 2010. Int. J. Comput. Vision, vol. 60, no. 2, pp. 91–110, 2004.
[3] M. Yim, K. Kuchenbecker, J. Bassani, P. Arratia, V. Kumar, J. Fiene, [33] H. Bay, A. Ess, T. Tuytelaars, and L. V. Gool, “SURF: Speeded up
and J. Lukes, “A practice-integrated undergradate curriculum in me- robust features,” Comput. Vision Image Understanding, vol. 110, no.
chanical engineering,” in Proc. ASEE Conf. Expo., Pittsburgh, PA, Jun. 3, pp. 346–359, 2008.
2008, Paper AC 2008-2230. [34] P. Henry, M. Krainin, E. Herbst, X. Ren, and D. Fox, “RGB-D map-
[4] D. Spencer and N. Jaksic, “A multidisciplinary robotics learning en- ping: Using depth cameras for dense 3D modeling of indoor environ-
vironment: What mindstorms and DARPA urban challenge have in ments,” Int. J. Robot. Res., vol. 31, no. 5, pp. 647–663, Apr. 2012.
common,” Comput. Educ. J., vol. 1, no. 3, pp. 32–40, 2010. [35] F. Chaumette and S. Hutchinson, “Visual servo control part I: Basic
[5] C. Berry, “Mobile robotics: A tool for application-based integration approaches,” IEEE Robot. Autom. Mag., vol. 13, no. 4, pp. 82–90, Dec.
of multidisciplinary undergraduate concepts and research,” Comput. 2006.
Educ. J., vol. 1, no. 3, pp. 67–80, 2010. [36] A. Miller and P. Allen, “Graspit! A versatile simulator for robotic
[6] D. Cappelleri, “A novel lab and project-based learning introductory grasping,” IEEE Robot. Autom. Mag., vol. 11, no. 4, pp. 110–122,
robotics course,” Comput. Educ. J., vol. 1, no. 3, pp. 81–91, 2010. Dec. 2004.
[7] A. Soto, P. Espinace, and R. Mitnik, “A mobile robotics course for [37] M. Ciocarlie, K. Hsiao, G. Jones, S. Chitta, R. Rusu, and I. Sucan, “To-
undergraduate students in computer science,” in Proc. IEEE 3rd Latin wards reliable grasping and manipulation in household environments,”
Amer. Robot. Symp., Oct. , pp. 187–192. in Proc. Int. Symp. Exp. Robot., 2010.
[8] J.-F. Lalonde, C. Bartley, and I. Nourbakhsh, in Proc. ICRA, May 2006, [38] G. Grisetti, R. Kuemmerle, C. Stachniss, and W. Burgard, “A tutorial
pp. 345–350. on graph-based SLAM,” IEEE Intell. Transp. Syst. Mag., vol. 2, no. 4,
[9] M. Boutell, C. Berry, D. Fisher, and S. Chenoweth, “A multidisci- pp. 31–43, Winter, 2010.
plinary robotics minor,” Comput. Educ. J., vol. 1, no. 3, pp. 102–111, [39] E. Olson, J. Leonard, and S. Teller, “Fast iterative alignment of pose
2010. graphs with poor initial estimates,” in Proc. Int. Conf. Robot. Autom.,
[10] T. Padir, M. Gennert, G. Fischer, W. Michalson, and E. Cobb, “Im- 2006, pp. 2262–2269.
plementation of an undergraduate robotics engineering curriculum,” [40] G. Grisetti, C. Stachniss, S. Grzonka, and W. Burgard, “A tree param-
Comput. Educ. J., vol. 1, no. 3, pp. 92–101, 2010. eterization for efficiently computing maximum likelihood maps using
[11] S. Bruder and K. Wedeward, “Robotics in the classroom,” IEEE Robot. gradient descent,” in Proc. Robot., Sci. Syst., 2007.
Autom. Mag., vol. 10, no. 3, pp. 25–29, Sep. 2003. [41] D. Basu, “Randomization analysis of experimental data: The fisher ran-
[12] K. Nagai, “Learning while doing: Practical robotics education,” IEEE domization test,” J. Amer. Statist. Assoc., vol. 75, no. 371, pp. 575–582,
Robot. Autom. Mag., vol. 8, no. 2, pp. 39–43, Jun. 2001. 1980.
[13] J. Piepmeier, B. Bishop, and K. Knowles, “Modern robotics engi-
neering instruction,” IEEE Robot. Autom. Mag., vol. 10, no. 2, pp. Nikolaus Correll (M’04) received the Master’s degree in electrical engineering
33–37, Jun. 2003. from ETH Zurich, Zurich, Switzerland, in 2003, and the Ph.D. degree in com-
[14] R. Murphy, “Competing for a robotics education,” IEEE Robot. Autom. puter science from EPFL, Lausanne, Switzerland, in 2007.
Mag., vol. 8, no. 2, pp. 44–55, Jun. 2001. He is an Assistant Professor in computer science with the University of
[15] P. Fiorini and D. Kragic, “Education by competition,” IEEE Robot. Colorado, Boulder. He teaches “Introduction to Robotics” and “Advanced
Autom. Magazine, vol. 13, no. 3, pp. 6–6, Sep. 2006. Robotics” to third- and fourth-year students. He worked as a Post-Doctoral
[16] M. Chew, S. Demidenko, C. Messom, and G. Gupta, “Robotics com- Associate with MIT CSAIL, Cambridge, MA. His research interests are
petitions in engineering education,” in Proc. Int. Conf. Auton. Robots multirobot and swarming systems.
Agents, Wellington, New Zealand, Feb. 2009, pp. 624–627.
[17] J. Grimes and J. Seng, “Robotics competition: Providing structure, flex-
ibility and an extensive learning experience,” in Proc. Frontiers Edu-
cation Conf., Saratoga Springs, NY, Oct. 2008, pp. F4C-9–F4C-13.
[18] R. Linn, E. Baker, and S. Dubar, “Complex, performance-based assess- Rowan Wing received the Bachelor of Arts and Sciences degree from St. John’s
ment: Expectations and validation criteria,” Educ. Res., vol. 20, no. 8, College, Santa Fe, NM, and the Master’s degree in education from the Univer-
pp. 15–21, 1991. sity of Colorado, Boulder, and is currently pursuing the Ph.D. degree in com-
[19] J. Craig, Introduction to Robotics: Mechanics and Control, 3rd ed. puter science and cognitive neuroscience at the University of Colorado.
Upper Saddle River, NJ: Prentice Hall, 2004. He taught mathematics for 12 years at the secondary and post-secondary
[20] M. Spong, S. Hutchinson, and M. Vidyasagar, Robot Modeling and level and taught the laboratory component of “Introduction to Robotics” at the
Control, 1st ed. Hoboken, NJ: Wiley, 2005. University of Colorado. His research interests include multiagent systems and
[21] H. Choset, K. Lynch, S. Hutchinson, G. Kantor, W. Burgard, L. swarm cognition.
Kavraki, and S. Thrun, Principles of Robot Motion: Theory, Algo-
rithms, and Implementations. Cambridge, MA: Bradford, 2005.
[22] S. LaValle, Planning Algorithms. Cambridge, U.K.: Cambridge
Univ. Press, 2006. David Coleman received the B.S. degree in mechanical engineering with a
[23] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics. Cam- minor in computer science from the Georgia Institute of Technology, Atlanta,
bridge, MA: MIT Press, 2005. in 2010, and is currently pursuing the Ph.D. degree in computer science at the
[24] R. Siegwart, I. Nourbakhsh, and D. Scaramuzza, Introduction to Au- University of Colorado, Boulder.
tonomous Mobile Robots, 2nd ed. Cambridge, MA: MIT Press, 2011. He has been a Teaching Assistant (TA) for several “Intro to Programming”
[25] G. Dudek and M. Jenkin, Computational Principles of Mobile courses and assisted in preparing the Advanced Robotics curriculum at the Uni-
Robotics, 2nd ed. Cambridge, U.K.: Cambridge Univ. Press, 2010. versity of Colorado. His research interests include robotic arm manipulation,
[26] M. Mataric, The Robotics Primer. Cambridge, MA: MIT Press, 2007. grasping, perception, and neural networks.