CS M213A / ECE M202A (Fall 2024)

Module #1: Course Introduction

Mani Srivastava
mbs@ucla.edu
Networked & Embedded Systems Lab
CS & ECE Departments
UCLA

Copyright (c) 2024

 Course Staff

• Instructor: Mani Srivastava

‣ Professor in CS and ECE Departments
‣ Background
- Originally from India
- Ph.D. from U.C. Berkeley (1992)
- 4.5 years at Bell Labs Research
- At UCLA since 1996
- Also, Amazon Scholar @ AWS AI Labs
- Research broadly in learning-enabled IoT and human-cyber-physical systems
‣ O ce: 6730-E Boelter Hall / Lab: 1762 Boelter Hall
‣ O ce Hours: Tuesday 10-11AM and Friday 4-5PM on Zoom + by appointment

‣ Zoom URL: See Piazza

‣ Email: mbs@ucla.edu
- please use direct message on Piazza instead of email
- if you do email please put “M202A” or “M213A” as subject
‣ URL: https://www.ee.ucla.edu/mani-srivastava/

2
ffi
ffi
Enrollment and Prerequisites

• Enrollment
‣ Prioritize (i) MENG IoT Track Students, (ii) PhD students working for embedded
systems faculty in ECE and CS, (iii) Other ECE and CS students, (iv) The rest.
‣ Most PTEs already handed out (MENG and PHD students)
‣ Another 4-6 PTEs split equally between ECE and CS, based on waiting list position
after con rmation of interest in the class
‣ The class capacity will not be increased
‣ Many students drop out as there is a lot of over-enrollment

• Prerequisites
‣ The course material should be accessible to anyone with the typical Computer
Engineering coursework
‣ You should have done coursework in computer organization and data structures /
algorithms
‣ Your should be very comfortable with programming
- both systems programming (e.g. C/C++) and application programming (e.g. Python)
‣ It would be highly bene cial if you have done an undergraduate level ML course

3
fi
fi
 Class Protocols

• An in-person course - expected to attend lectures in the classroom

‣ Some lectures will be over Zoom or asynchronous, due to travel con icts
‣ Three con icts: Thu Oct 3, Thu Oct 10, and Tue Oct 15

• Lectures are intended to be interactive and so you are encouraged to interrupt

me if you have a question

• I record the lectures (laptop screen plus audio) for students who may
occasionally need it due to medical, conference, or such reasons
‣ Your voice might be incidentally captured and become a part of the video
‣ YouTube playlist link will be shared on Piazza after the rst lecture
‣ I reserve the right to stop posting the videos if I see low attendance or
consistent skipping of lectures by students

• O ce hours: Tuesday 10-11AM and Friday 4-5PM on Zoom for now

4
ffi
fl
fi
fl
About the course…
Embedded, Wearable Mobile Planetary-scale Cloud
Devices Devices Network Services
Embedded, Wearable Mobile Planetary-scale Cloud
Devices Devices Network Services

-
-

Systems for Digitizing

and Connecting Our
Personal, Social, and
Physical Spaces
What are Embedded Systems?

[Wikipedia]
An embedded system is a computer system with a
dedicated function within a larger mechanical or electrical
system, often with real-time computing constraints.

• General-purpose computers: exible, wide range of needs, physical world

interaction not mandatory

• What are some examples of embedded systems?

• What is real-time constraint? What are other challenges besides real-time?

• Major trends: networked, miniature, human-coupled, AI-enabled, multi-tenant,

raising privacy concerns, and getting hacked!

• This course: embedded systems that are networked

(or networked systems that are embedded) 8
fl
Examples of Networked Embedded Systems

9
Related Concept: Internet of Things (IoT)

• Wikipedia: “interconnection of uniquely identi able

embedded computing devices within the existing
Internet infrastructure”

• Edge of the Internet now dominated by “things”

and not traditional computers

• Advanced connectivity of devices, systems, services

‣ Spans many protocols and PHYs

• Sense, transport, analyze, infer, decide, actuate…

• Wide variety of applications

‣ Health, buildings, transportation, vehicles etc.

• Unique challenges and concerns

‣ security, privacy, con guration

10
fi
fi
IoT Examples

11
From Passive IoT to Active IoT

12
Related Concept: Ubiquitous or Pervasive Computing

Ubiquitous computing (or "ubicomp") is a

concept in software engineering and
computer science where computing is
m a d e to a p p e ar a n y t i m e a n d
everywhere. This paradigm is also
described as per vasive computing,
ambient intelligence or “everyware".

13
 Tactical (CPS)
Related Concept: Cyber-Physical Systems Edge: A Huma

State
Infer Decide

• Perception-Cognition-Action Loops
State

• Integration of computation, networking, Infer Decide

sensing, actuation, and physical processes

Cloud
• Builds upon embedded systems (Distribute, Store,
Sharing, Storage,
‣ Computers and software in “things” Process)
Analysis, Control

• Challenge: combining modeling of physical

processes and their dynamics with modeling of

actuator

actuator
sensor

sensor
computation and its dynamics

actuator

actuator
sensor

sensor
‣ di erential equations, stochastic processes,
etc. vs. algorithms and programs

14
http://cyberphysicalsystems.org
ff
 Not just CPS, but Human-Cyber-Physical Systems
Human Cyber‐Physical Systems (h‐CPS)
CPS that operate in concert with humans

Driver Assistance in Cars Fly-by-wire Cockpit

Interfaces UAVs with Human
Operators

© Rethink Robotics

Robotic Surgery & Semi-Autonomous Manufacturing

Medicine 2

15
Sanjit Seshia, Berkeley
Related Concepts: Edge Computing, Fog Computing
• Loosely speaking: extend cloud closer to devices
producing and consuming the data

• Wikipedia
‣ Edge Computing is a distributed computing paradigm
which brings computation and data storage closer to the
location where it is needed, to improve response times
and save bandwidth.
‣ Fog Computing is architecture that uses edge devices
to carry out a substantial amount of computation, Doing all processing & storage in the cloud is impractical
storage, communication locally and routed over the (scalability, latency, privacy, security etc.)
internet backbone.

16
Why the interest?
New Instruments for
Pervasive Awareness and Knowledge Discovery

High-frequency high-dimensional real-time sensor data streams

revealing previously unobservable through e ective analytics.

workplace
analytics

mPerf
18
ff
 From Knowledge Discovery to
PS-IoT System Evolution
Just-in-Time Adaptive Interventions and Control
CPS-IoT System Evolution

Measurements & Events Inferences & Interventions

Measurements & Events Inferences & Interventions

Sensor Sensor Interventions

Interventions
Markers
Markers Predictors
Predictors
(manual,
(manual, autonomous)
autonomous)
Data Data

mHealth: Digital Biomarkers + Just-in-Time Adaptive Interventions 2

mHealth: Digital Biomarkers + Just-in-Time Adaptive Interventions

19
Cyber-Physical and IoT Systems
CPS-IoT System Evolution
CPS-IoT System Evolution
Measurements & Events Inferences & Interventions
Measurements & Events Inferences & Interventions

Sensor Interventions
Markers Predictors (manual, autonomous)
Data

mHealth: Digital Biomarkers + Just-in-Time Adaptive Interventions 2

20
Eventually: Pervasive Autonomy

21
 Distributed or Logically Centralized?

Logically Centralized Control a

proach to Coordination and Control
App App App App App App

Control Platform Control Platform

Network Network

ficult to scale, reason & do global planning/optimization with

Centralized Control and Planning – Swarms can accom
pp needs to solve a distributed system problem
d in other computer and networking systems
22
vehttp://platformlab.stanford.edu/Seminar%20Talks/Big-Control-Expedition-pres-PL-retreat-June2016-v0.1.pdf
decided not to focus on P2P control and coordination
 Big Control: Infrastructure for Collaborative Device Swarms

Fast Local Autonomous + Slow BCP

Logically Centralized
Internals: Distribu
cations. BCP will solve problems such as distributed data management, data fusion, and behavior planning,
and it will offer simplified frameworks such as a declarative query language for behavior planning.
Figure 1 shows the overall environment for •
Big Control Platform (BCP) Requirements
managing device swarms and the architecture of
BCP. BCP comprises three layers. The lowest
Package
Tracking
Applications
Commute
Management
Disaster
Relief
Large
Scale
Mapping
layer is responsible for distributed state manage-• Real time sensing •
Big Control Platform (BCP)
ment and communication, including communica- – Location, velocity, trajectory
tion with swarm devices and high-performance no- – Environmental Stable
Trajectory parameters
Active
Adaptive
Optimal
Deep
Reinforcmt
Sensing
App among
tifications App theAppBCP elements. The middle – Streaming Tracking
data
Scheduling Learning

layer provides two modules that

Global View will underlie
Historic & all ap- High volume
Declarative Planning
noisy data
Global Views of Swarm
Data
Data Ingestion, Reposi-
plications. The first module isSimulation
responsible for•dataHistoricPlanning
data in
& Control
the store Inference Engine tories
Big Control Platform
ingestion and fusion, and includes an inference en-
Data Huge data setsDistributed State Management,
gine to build consistent global views from the noisy Notifications
data provided by swarm devices. Thetime •
second mod- Global view or state
Real time Real •
control ule is a Network function for Big – Most
planning engine, whosesensing Real-
accurate state of devices in the Real-
physical
Time space Time
Control applications is similar to that of a relational Actuation Sensing

database for Big Data applications. It will allow ag- – Assessment of the physical space
gregate behavioral goals to be specified a declara- ComputationallyWired and veryWireless
intensiveNetworks
tive language, and it will then compute detailed• de-Real time control •
vice behaviors to implement those goals. The third – A revised plan (actions) to devices:
layer uses these facilities to create domain-specific location, velocity, trajectory, what to
BCP requires servicesaddressing
such as activemany tough
sensing and deep reinforce- sense, … Devices

challenges
ment learning. Low latency computationally intensive with
Figure 1: System overview for managing collaborative device
predictability
BCP will execute on large clusters of servers in swarms.
23
both edge and core datacenters. In order to handle
http://platformlab.stanford.edu/Seminar%20Talks/Big-Control-Expedition-pres-PL-retreat-June2016-v0.1.pdf
A Rich Ecosystem of Interlinked Technologies
Cloud, Network, Mobile, Embedded….

Analytics
Sharing
Storage
 New Technologies Leading to “AIoT”
A Nexus of Enabling Technologies Leading to “AIoT”

Rich Sensors & Actuators Deep Learning Hardware Accelerators

4

25
 Specialized Mass-market
Sensors “Universal” Sensors

+ +
Low-dimensionality High-dimensionality
Structured Unstructured
Data Data

+ +
Low-cost High-cost
First-Principles Data-Driven
Mechanistic Models AI/ML Models

26
 Complex Inferences from Simple Sensors
Complex Inferences from Simple Sensors

Interacting with wearable devices via on-body tapping

https://arxiv.org/pdf/1707.03502.pdf

Human activity & behavior recognition

Accurate estimation of 3D motion trajectory

27
 Reduced Cost of Sensing
Reduced Cost of Sensing

Lidar-like dense depth by fusing cameras with small, cheap, low-power mmWave radars

+ =

28
 Richer and Deeper Awareness

Richer and Deeper Awareness

• 3D-imaging-guided RF-vibrometry
•‣ Association:
3D-imaging-guided RF-vibrometry between internal state and object
establish correspondence
‣ Accuracy: provides
‣ Association: prior forcorrespondence
establish estimation of intrinsic statesinternal
between of objects
state and object
‣ Latency: avoids
‣ Accuracy: exhaustive
provides search
prior over all possible
for estimation distances/angles
of intrinsic to locate vibration
states of objects
‣ Latency: avoids exhaustive search over all possible distances/angles to locate vibration
8

29
 Common Three-tier IoT Systems:
Device-Gateway-Cloud (DGC)
Embedded
Gateway
Device
Physical World

Embedded Gateway

Embedded
Device

Wearable
Device

Implantable
Device Storage, Learning, Inferencing,
Mobile Gateway Decisions, Personalization, Interventions

PC Gateway

Sense, Actuate, Relay, Sense, Archive, Process, 30

Bu er, Process, UI Bu er, Process, UI Disseminate
ff
ff
Implementations across the SWaP Continuum
Implementations across the SWaP Continuum

Multiple processor cores, 32/64b, Arm A / Intel Atom, Single processor, 16/32b, Arm M or low-end RiscV,
~1 GHz / several GB RAM / many GB Flash, embedded GPU, ~10-100 MHz / kB RAM / MB Flash, simple accelerators < 1W, ~
Multiple watts, ~ 100cc, ~ 100 gm; carried, or plugged in; Linux 10 cc, ~ 10 gm; wearable, battery or harvested; RTOS

Models can be many MB to few GB Models in ~100 kB

Off-loading to Cloud Higher-end IoT Devices Ultra Resource Constrained IoT Devices

31
 Another Enabler: The “Maker” Culture

• Digital desktop fabrication + small scale, rapid contract manufacturing

(Shenzhen)

• Successful crowdfunding now pre-requisite for VC funding of consumer

hardware

• Encourages rush to “working prototype”, worry about DFM, robustness,

security later

• HW becoming like SW, but not always in a good way

‣ Compare to Web development practices
‣ Driven by user/application considerations, not technology per se
‣ Iterative, prototype-driven development
‣ Enable application (not technology) experts to be productive quickly

• What are the right platform abstractions to get robust and secure design?
32
Hartmann @ UCB (http://iot.stanford.edu/workshop14/SITP-8-11-14-Hartmann.pdf)
Desktop Fabrication

Laser Cutter 3D Printer PCB Milling

33
Modular & Open Hardware,
Rapid Fabrication Services

https://www.adafruit.com http://www.ohwr.org http://opencores.org

34
http://www.sunstone.com https://www.ponoko.com
Course Content & Logistics
Foundational Issues in Embedded Computing

• Platforms (hw, sw)

• Connectivity

• Power/Energy

• Time (throughput, latency, absolute time)

• Location (and other context)

• Intelligence

• Security

• Privacy

• Safety

• Human-coupling

36
Experience with Various Tools and Technologies

37
Syllabus

Module Topic

1 Embedded Hardware

2 Organizing Software

3 Scheduling Computation

4 Sensing and Perception

5 Embedded Communication

6 Time Synchronization

7 Localization

8 Trustworthiness (security, privacy, safety)

9 Emerging Topics
38
Course Components

Individual (≤4)
Assignments Concepts, Programming, Literature/Web
40%

Individual or teams of up to 3
Project Analysis, Design, Experiment… 60%
(Report Website + Video Demo/Talk)

39
 Grading Algorithm

• Some components of the course will be graded on a numerical scale while others
directly on a letter grade scale (A+=4.3, A=4, A-=3.7, … D-=0.7, F=0)
• Expectation for letter grade:
‣ A+/A/A- = correct / high-quality; B+/B/B- = some- aws / acceptable; C+/C/C- =
signi cant- aws / below-average-quality; D+/D/C- = mostly-incorrect / unacceptable-
quality; F = did not make a serious attempt
• Numerical scores will be mapped to letter grade as follows
‣ Scale to 100
‣ If median<80, normalize so that highest=100 and median=80
‣ Mapping: scores ≥95 = A, ≥90=A-; ≥85=B+, ≥80=B, and so on.
• Course grade
‣ Compute weighted course score on scale of 4
‣ Convert to letter grade using mid-points between adjacent letter grades as thresholds
‣ A+=(4,4.3], A=[3.85,4], A-=[3.5,3.85), etc.
‣ thresholds may be selective nudged lower at instructor’s discretion to account for
unusual clustering
• S grade means letter grade of B or better [UCLA rules]
• Historical: Median/Mean course grade between B+ and A-

40
fi
fl
fl
 Course Project

• Dig deep into a focus area on your own as lectures would provide a “broad” coverage
• Aim for something concrete and tangible, even if minor
‣ Simulation, analysis, software/hardware design, tools, application...
‣ Literature surveys not acceptable
• Project topics
‣ Some topics will be provided as exemplars
‣ Discuss project ideas with me
‣ While project may be motivated by your prior research or work, it must be a distinct topic and
e ort whose output you must be willing to publicly disseminate
- you may not reuse work already being done or planned for a thesis or another project
- you may not collaborate with other researchers in your group
‣ Topics will have to be approved by me and I reserve the right to veto any project proposal
• While teams are allowed, it is with the understanding that if a partner drops the course
then the remaining member(s) would need to nish the project, and other than giving
extra time the instructor cannot help
• What should be your goal?
‣ Something useful or cool -risky ideas leading to negative results are ne
‣ The key is to keep the project simple, and focused
‣ Aim for high quality!

41
ff
fi
fi
Thinking about Project Ideas: Heilmeier Catechism

• George H. Heilmeier, a former DARPA director (1975-1977), crafted a set of

questions known as the "Heilmeier Catechism" that anyone proposing a research
project or product development e ort should be able to answer

• What are you trying to do? Articulate your objectives using absolutely no jargon.
• How is it done today, and what are the limits of current practice?
• What is new in your approach and why do you think it will be successful?
• Who cares? If you are successful, what difference will it make?
• What are the risks?
• How much will it cost?
• How long will it take?
• What are the mid-term and nal “exams” to check for success?

• Of course not all are relevant to a course project setting…

42
fi
ff
Project Timeline

• Week 2-3: Mandatory initial discussion with instructor about project ideas [2%]
• Document and summarize meeting via a direct non-anonymous message on Piazza to
instructor
• Week 3-4: Mandatory second discussion with instructor about project ideas 3%]
• Document and summarize meeting via a direct non-anonymous message on Piazza to
instructor
• Week 5: Initial project repo and website ready on GitHub [5%]
• Submit project title, team information, GitHub repo URL, and GitHub website URL via web form
• Website must have initial sections for goals, speci c aims,
• Week 7: Midterm checkpoint [20%]
• 5-min max in-class presentation [80%]
• Website updated and also add initial text for data, tools, methods, and results [20%]
• Finals Week: Submission [70%]
• 15 min max Zoom presentation plus any demo [80%]
• Website nalized [20%]
• Grading will consider
• problem and approach: di culty (relative to team size), novelty
• execution: architecture, analysis, evaluation, e ort, elegance
• presentation: quality, organization, discussion

43
fi
ffi
ff
fi
 More Project Guidelines

• The project should re ect a serious e ort to go beyond the course material
‣ Obtain additional sources from the net, journals, or books.
• All prior work, published or not, public domain or proprietary, should be fully credited
‣ "my o cemate, Joe Schmo, says ...", "This section of code is modi ed from XXX gotten from
YYY” etc.
• Do not build software or hardware from scratch
‣ Your project will not be evaluated on the basis of how much e ort you put into it, but rather on
how e ective your work is.
‣ Go to the net or commercial software and nd something to build on.
• Learn to use the relevant tools and languages
‣ At least to the level of pro ciency required to make your point
‣ Get the compiler, simulator, design environment, and install it
• If you are already engaged in relevant work, leverage it.
• You may work in groups of up to 3
‣ I will expect the ambitious-ness of the project to be proportional to the group size
‣ I think optimal group size is 2
• You need to deal with inter-personal issues in the group yourself
‣ Learning how to work in a team is important professionally
• Project report must describe what each team member did
‣ Part of the score will depend on team e ort and part on individual e ort

44
Source: Prof. Lee @ Berkeley, Prof. Kastner @ UCSD
ffi
ff
fl
fi
ff
ff
fi
ff
fi
ff
Course on the Web

• What is BruinLearn?
‣ UCLA’s on-line platform for courses

• We will not use BruinLearn as it doesn’t t well for the needs of this course.
‣ However, I have posted an announcement there with links to class resources

45
fi
Course on the Web

• Class material and communication on Piazza

‣ Link: piazza.com/ucla/fall2024/ecem202acsm213a/home
‣ Enrollment in Piazza is automatic by BruinLearn
‣ Links to assets on Google Drive (various les) and YouTube (lecture videos)
‣ Access to material (including videos) from prior o erings

• Submissions via Gradescope

‣ Most class work submitted via Gradescope https://www.gradescope.com
‣ Log in using UCLA SSO as Graderscope syncs with UCLA roster
‣ Software as zip or other les as speci ed
‣ Written answers as properly typeset PDF documents
- No hardcopy submissions accepted and they will not be graded
- Likewise, handwritten-and-scanned submissions will not be graded

• GitHub for projects

46
fi
fi
fi
ff
No Textbook!
Computers

• You need to have access to a computer (ideally a laptop) where you have
administrator privileges in order to install software.

• The computer must have at least one and ideally two USB 3.0 USB
ports (either built-in or via adapters) and suitable adapters/cables/hub to
allow you to connect to hardware devices with micro-USB ports.

• While we will try to provide instructions for all the three major OSs (Linux, Mac
OS X, and Windows), please note that some of the tools are much better
supported or easier to install under Unix-like OSs (Linux, Mac OS X).
‣ No ChromeBooks - get a real computer :-)
‣ If you are on Windows then it may become necessary to run a Linux VM (using
VirtualBox or VMWare ) or Container (Docker) con gured to access the USB
ports and with the laptop provisioned with plenty of RAM (16GB or more)
‣ Windows now has a Linux subsystem, which might work as well
‣ If on a Mac, it is strongly recommended to install the Homebrew package
manager
48
fi
Smartphones

• In order to run certain applications that interact with hardware via Bluetooth
Low Energy (BLE) as well as for use in your project, you would bene t from
having an Android or iOS smartphone with Bluetooth Low Energy (BLE).
‣ Most phones from recent years will have BLE, but please double check,
particularly if you've an Android smartphone with an old version of the OS

49

fi
Hardware Platforms for Coursework

• You may need a hardware platform for use in the course

Embedded
Gateway
Device

Embedded
Device

Wearable
Device

Implantable
Device
Mobile PC 50
 Arduino Nano 33 BLE Sense

• Nordic’s nRF52840 with Cortex-M4 (part of nRF52 series)

‣ BLE and Bluetooth® client as well as host device
‣ nRF52840 has IEEE 802.15.4 radio too
• Many sensors…
• Several OS choices
‣ Arduino Core on top of mBed RTOS
‣ Zephyr OS
• Embedded AI using TinyML (TensorFlow Lite)
• Can be purchased from many on-line stores
‣ Will need a micro-USB cable to connect to computer
‣ Suggest soldering headers (a version comes with
51
unsoldered headers)
Lots of Other Options

• Alternatives that are based on the same mbed OS based software stack
‣ Arduino Nano RP2040 Connect
- RP2040 processor (dual Cortex-M0+)
- Bluetooth and WiFi
- IMU, Microphone, Temperature

‣ Arduino Nano 33 BLE

- like Nano 33 BLE Sense but has only IMU

‣ Seeed Studio XIAO nRF52840 Sense or, Seeed Studio XIAO RP2040
- need further vetting…

• And others using other software stacks

‣ Raspberry Pico 2 (RP2350), Raspberry Pi 5 + AI Hat, ESP32, etc.

52
Hardware Platforms for Coursework

• And you may need a gateway device that you can program: iOS or Android
Smartphone, or a computer with Bluetooth (BLE), WiFI, and USB

Embedded
Gateway
Device

Embedded
Device

Wearable
Device

Implantable
Device
Mobile PC 53
Cheating & Plagiarism

• What is cheating & plagiarism?

‣ Acting dishonestly, practicing fraud
‣ Using other people’s writings or ideas without permission
- E.g. from other students, other sources such as web sites, solutions from elsewhere
- Note that it doesn’t have to be literal copying – stealing ideas but presenting in a di erent
style is still cheating and plagiarism.
‣ Individual assignments must be done individually, team assignments must be done
by the team members only
‣ While it is okay to clarify from others as to what a question “means”, it is in general
not okay to discuss or share solution approaches, and never okay to share actual
solutions
• Person aiding in cheating & plagiarism is equally guilty as the bene ciary, and will be
similarly penalized
• University policy will be followed by reporting to Dean’s o ce
• For more information:
‣ UCLA’s Academic Integrity Policy: https://deanofstudents.ucla.edu/academic-
integrity
‣ Interactive tutorial: https://guides.library.ucla.edu/bruin-success/

54
ffi
fi
ff
Post any more questions on Piazza!!!
55