RPCN course, 2024

PERCEPTION OVERVIEW &

INERTIAL NAVIGATION 1/3
VILLE LEHTOLA
DR. SC.
ASSISTANT PROFESSOR
UNIVERSITY OF TWENTE
https://chat.openai.com/chat
chatGPT 16.01.2023
HAVE YOU BEEN IN TOUCH WITH ROBOT PERCEPTION?
Did you fly a drone? What sensors could you identify from your drone? Did you use some
other system with robotic perception?

All answers anonymous! (1-2 mins to respond)

We will return to these answers also on later lectures

https://giphy.com/gifs/1k0YYUlIOYK2QA6HVK

4
ROBOT PERCEPTION

• Know your sensors: IMU, LIDAR, vision, GNSS, others

• Know your coordinate frames and transformations: how to transform the measurements so that they are useful?

• Where is robot perception needed?

• “Chaotic”/natural environments

• Where is robot perception not needed?

• Structured, controlled environments

• There will be application examples throughout the course, but especially towards the end
PERCEPTION TECHNIQUES

• Can robots sense like humans or even better?

 PERCEPTION TECHNIQUES
A. Vision and Depth sensing: techniques for measuring the 1. Sight
distance to objects in the environment using sensors such as a) Active vision: laser scanning
stereo cameras, lidar, or structured light.. b) Passive vision: photogrammetric computer vision
B. Auditory perception & Speech recognition: techniques for c) Two eyes: stereo-vision
analyzing and interpreting auditory data from microphones and
other sensors. 2. Hearing
C. Tactile perception: techniques for analyzing and interpreting
touch data from tactile sensors. 3. Touch
D. Gravity sensing with IMU and accelerometers
E. Sensor fusion & Multi-modal perception: techniques for 4. Balance (Vestibular sense)
Balance
combining data from multiple sensors to improve perception and 1. IMU

understanding of the environment.

F. Self-localization and SLAM: techniques for determining the 5. Situational awareness (Cognition about the environment)
robot's position and orientation within its environment. Also
GNSS
 SENSORS

• Sensing, gathering information about the

environment
• Acquiring data

• Much will be covered but not everything! See

the list -->

Sensors introduced Sensor types

in the course covered in
RPCN course

Springer handbook of robotics, p.90

ROBOTS (THIS COURSE) VS HUMANS

Active vision: laser scanning

Sight Passive vision: photogrammetric computer vision
Two eyes: stereo-vision

No Hearing, No touching

Balance: Inertial Measurement Unit (IMU)

Ontologies, Representations, Cognition about the environment,

Situational awareness Spatial representations about the environment, SLAM (self-
localization and mapping)
INERTIAL NAVIGATION
1/3
VILLE LEHTOLA
DR. SC.
ASSISTANT PROFESSOR
UNIVERSITY OF TWENTE
MOTIVATION:
WHERE TO MOUNT THE IMU ON AN AUTONOMOUS CAR?
A) On the roof

B) Rear wheel axis

C) Center of mass

D) Anywhere
LEARNING OBJECTIVES & CONTENT

• Learning objectives:

• 1 - Design sensor systems and select techniques

• E.g., What are IMU measurements used for?
• 6 - Evaluate errors
• E.g., What errors can be countered by calibration?

• Content:

• Sensor (IMU=inertial measurement unit), sensor properties (IMU fundamentals), IMU basics, error sources
READING MATERIAL

• “Fundamentals of Inertial Navigation, Satellite-based Positioning and their Integration” written by Aboelmagd Noureldin,
Tashfeen B. Karamat, Jacques Georgy, published by Springer-Verlag Berlin Heidelberg 2013.
• Look for the course canvas page for the book link in e-library.
• We asked for the permission to reuse the materials of the book on this course from the first author.

14
INERTIAL MEASUREMENT UNIT (IMU)
• IMU is a composition of three accelerometers, which are perpendicular to each other, and
three gyroscopes, which are also perpendicular to each other.
• Accelerometers measure acceleration.
• Gyroscopes measure angular velocity.

Fundamentals of Inertial Navigation, Satellite-based Positioning and

15
their Integration, Noureldin et al. 2013
INERTIAL MEASUREMENT UNIT (IMU), CONT’D
• Acceleration and angular velocity measurements are used to estimate the position, the
velocity and the attitude (Mechanization is on lecture 3).

• There are two IMU types:

• Gimbaled
• Strapdown

16
GIMBALED VS. STRAPDOWN

Characteristics Gimbaled systems Strapdown systems

Size Bigger Relatively small
Weight Heavy Relatively small
Performance Superior performance High accuracy
Robustness Low immunity to shocks Highly reliable, immune to
and vibrations shocks and vibrations

17 Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
ACCELEROMETER
𝑑𝑑

• Hooke’s law

𝐹𝐹 = 𝑘𝑘𝑘𝑘

Spring constant k
Displacement d
Force F

• Newton’s second law

𝐹𝐹 = 𝑚𝑚𝑚𝑚
𝑘𝑘
𝑎𝑎 = 𝑑𝑑
𝑚𝑚

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
18
 Does an accelerometer measure gravity?

ACCELEROMETER, CONT’D

• Hooke’s law with gravity

𝐹𝐹 = 𝑘𝑘𝑘𝑘 − 𝑚𝑚𝑚𝑚

• Newton’s second law

z
𝐹𝐹 = 𝑚𝑚𝑚𝑚

𝑘𝑘
𝑎𝑎 = 𝑑𝑑 − 𝑔𝑔
𝑚𝑚

Fundamentals of Inertial Navigation, Satellite-based Positioning and

19
their Integration, Noureldin et al. 2013
 No, when the IMU is at rest, it measures the
reaction to gravity, i.e., the specific force issued by
ACCELEROMETER, CONT’D the spring

• Hooke’s law with gravity

𝐹𝐹 = 𝑘𝑘𝑘𝑘 − 𝑚𝑚𝑚𝑚

• Newton’s second law

z
𝐹𝐹 = 𝑚𝑚𝑚𝑚

𝑘𝑘
𝑎𝑎 = 𝑑𝑑 − 𝑔𝑔
𝑚𝑚

• Acceleration is zero, 𝑎𝑎 = 0, right?

• But the measurement yields

9.81
𝑚𝑚 𝑔𝑔⃑ = (0,0, −𝑔𝑔) −𝑔𝑔⃑ = (0,0, 𝑔𝑔)
𝑠𝑠𝑠
20
ACCELEROMETER, CONT’D

𝑘𝑘
𝑎𝑎 = 𝑑𝑑 − 𝑔𝑔 = 0
𝑚𝑚

• While the measurement yields

𝑚𝑚
f = 𝑎𝑎 − 𝑔𝑔 = 9.81 z
𝑠𝑠𝑠

Where
• f is the specific force
• a is the acceleration with respect to the inertial
frame
• g is the gravitational acceleration minus
centripedal force +9.81 𝑚𝑚/𝑠𝑠 2

𝑔𝑔⃑ = (0,0, −𝑔𝑔) −𝑔𝑔⃑ = (0,0, 𝑔𝑔)

21
ACCELEROMETER, CONT’D

• Thought experiment: accelerometer in a vacuum (g for near Earth’s surface)

𝑎𝑎 = 𝑔𝑔

• Specific force measured by the accelerometer

𝑓𝑓 = 𝑎𝑎 − 𝑔𝑔 = 0
GYROSCOPE BASICS
We make the gyroscope spin with a high angular
velocity 𝜔𝜔. This spinning motion creates a significant
angular momentum 𝐿𝐿

𝐿𝐿 = 𝐼𝐼𝜔𝜔

where 𝐼𝐼 is the moment of inertia of the gyroscope.

When an external force or torque is applied (like a
change in the orientation of the device it's mounted
on), this results in a measurable change in 𝜔𝜔, which
corresponds to the external influence.
https://www.surveyinggroup.com/what-is-gyroscope-and-how-does-it-work/
GYROSCOPE BASICS, CONT’D

This is what is measured!

GYROSCOPE BASICS, CONT’D
Role of Moment of Inertia, I, in Gyroscopes
• Gyroscope Quality
• Larger I leads to larger angular momentum L
• Benefits:
• Increased stability and resistance to orientation
changes
• Reduced error and drift in measurements
• Design Trade-offs
• Increased Size & Weight: Higher I typically requires more
mass or larger dimensions
• Application Consideration: Balance between I and 𝜔𝜔 for
optimal performance without excessive bulk
• Other: hallenges related to the need of both controlling
𝜔𝜔 and measuring it for external changes
https://www.surveyinggroup.com/what-is-gyroscope-and-how-does-it-work/
 What is the assumption on

BASICS OF INTEGRATION acceleration used in the equation?

(discuss/chat with your neighbor if
able, 1 min)
𝑡𝑡1
Δ𝑣𝑣 𝑡𝑡 = 𝑣𝑣 𝑡𝑡1 − 𝑣𝑣 𝑡𝑡0 = � 𝑎𝑎 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑡𝑡0

26
NAVIGATION IN ONE DIMENSION (1D)

• Acceleration a, velocity v, and position x are expressed with scalar values

• Motion is linear and rotation is not allowed
• Lets assume that the initial position and the initial velocity are known. It follows that
𝑡𝑡
𝑣𝑣 𝑡𝑡 = � 𝑎𝑎 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑡𝑡0
= 𝑎𝑎𝑎𝑎 + 𝑣𝑣0
𝑡𝑡
𝑥𝑥 𝑡𝑡 = � 𝑣𝑣 𝑡𝑡 𝑑𝑑𝑑𝑑
𝑡𝑡0
𝑡𝑡
=� 𝑎𝑎𝑡𝑡 + 𝑣𝑣0 𝑑𝑑𝑑𝑑
𝑡𝑡0

= 𝑎𝑎𝑡𝑡 2 + 𝑣𝑣0 𝑡𝑡 + 𝑥𝑥0

A dragster accelerating really fast. Image from GIPHY

27
NAVIGATION IN TWO DIMENSIONS (2D)
• Two accelerometers and one gyroscope
• Translation and rotation

https://gph.is/2IB6UuF

Fundamentals of Inertial Navigation, Satellite-based Positioning and

28
their Integration, Noureldin et al. 2013
NAVIGATION IN TWO DIMENSIONS (2D), CONT’D

• The accelerometers measure accelerations in x and y directions

• We can express these accelerations in N and E directions, for
example, by

𝑎𝑎𝐸𝐸 = 𝑎𝑎𝑥𝑥 cos 𝐴𝐴 + 𝑎𝑎𝑦𝑦 sin 𝐴𝐴

𝑎𝑎𝑁𝑁 = 𝑎𝑎𝑥𝑥 sin 𝐴𝐴 + 𝑎𝑎𝑦𝑦 cos 𝐴𝐴

• Transformation between two bases

• Matrix form
𝑎𝑎𝐸𝐸 cos 𝐴𝐴 sin 𝐴𝐴 𝑎𝑎𝑥𝑥
=
𝑎𝑎𝑁𝑁 − sin 𝐴𝐴 cos 𝐴𝐴 𝑎𝑎𝑦𝑦

29
Rotation matrix
NAVIGATION IN TWO DIMENSIONS (2D), CONT’D

• Velocity

• Position

• Azimuth angle

Angular velocity

??
30
NAVIGATION IN TWO DIMENSIONS (2D), CONT’D

• Velocity Note that in reality 𝑎𝑎 = 𝑎𝑎(𝑡𝑡) and

this time-dependence leads into a
classical path integral! How do we
then know 𝑎𝑎(𝑡𝑡) for long integration
• Position times?!

• Azimuth angle
a path

Angular velocity

Initial angle
31
NAVIGATION IN TWO DIMENSIONS (2D), CONT’D

• Velocity Solution: Still assume that a is

constant but integrate only with
very small time steps, e.g. 0.01 s
=> Piece-wise constant function
• Position approximation

Note: IMU measurement

frequency is important!
• Azimuth a drifted path

a path
drift

32
NAVIGATION IN THREE DIMENSIONS (3D)

• Three accelerometers + three gyroscopes

• We study three-dimensional (3D) positioning in this course

https://giphy.com/gifs/1k0YYUlIOYK2QA6HVK

Measurements Mechanization output

33
Fundamentals of Inertial Navigation, Satellite-based Positioning and
their Integration, Noureldin et al. 2013
NAVIGATION IN THREE DIMENSIONS (3D), CONT’D

• Three accelerometers + three gyroscopes

• We study three-dimensional (3D) positioning in this course
EXERCISE 1

• What is the measured acceleration of a stationary IMU?

• a_x = … , a_y = …. , a_z = ….

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
EXERCISE 1, ANSWER

• What is the measured acceleration of a stationary IMU?

• a_x = … , a_y = …. , a_z = ….

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
EXERCISE 2

• What is the measured acceleration if there is rotation around x axis?

• a_x = … , a_y = …. , a_z = ….

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
EXERCISE 2, ANSWER

• What is the measured acceleration if there is rotation around x axis?

• a_x = … , a_y = …. , a_z = ….

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
EXERCISE 3, DEMO

• If there is another rotation around y, what is the measured acceleration

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
 EXERCISE EARTH, ANGULAR VELOCITIES
• What is the measured angular rate in these three IMUs?

Equator Arbitrary North

position pole
ω_x
ω_y
ω_z
EXERCISE EARTH, ANSWER

• What is the measured angular rate in these three IMUs?

Equator Arbitrary North

position* pole
ω_x 0 0 0
ω_y ω_e cos(ω_e) 0
ω_z 0 sin(ω_e) ω_e

(*): See the book!

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
EXERCISE EARTH, AFTERMATH

• Note: if the direction of the true north is unknown, but the z axis is still pointing upwards, we may write

𝜔𝜔𝑥𝑥𝑥𝑥 = √(𝜔𝜔𝑥𝑥2 + 𝜔𝜔𝑦𝑦2 ) = 𝑐𝑐𝑐𝑐𝑐𝑐 𝜔𝜔𝑒𝑒

𝜔𝜔𝑧𝑧 = sin 𝜔𝜔𝑒𝑒

• .. Since it is not known how the cos component is divided between x and y axes. This holds because

sin2 𝛼𝛼 + cos 2 𝛼𝛼 = 1

Note that in the calibration exercise, the direction of true north is treated as unknown!
IMU ERRORS

• Basics
• IMU errors
• Coordinate system
• Mechanization
• Error model
WHY DOES IMU INTEGRATION FAIL?
(.. even if we use very small time steps like discussed)

• Integral over
• IMU systematic errors
• IMU random errors

http://gph.is/2htT515 (IMU integration failing may cause this!)

IMU SYSTEMATIC ERRORS

1. Bias offset
2. Scale factor error
3. Non-linearity
4. Scale factor sign asymmetry
5. Dead zone
6. Quantization error
7. Non-orthogonality error
8. Misalignment error
IMU RANDOM ERRORS

1. Run to run bias offset

2. Bias drift
3. Scale factor instability
4. White noise
BIAS OFFSET

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
SCALE FACTOR ERROR

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
NON-LINEARITY

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
SCALE FACTOR SIGN ASYMMETRY

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
DEAD ZONE

E.g. for very weak

accelerations

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
QUANTIZATION ERROR

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
NON-ORTHOGONALITY

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
MISALIGNMENT

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
RUN TO RUN BIAS OFFSET

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
WHITE NOISE

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
ANGULAR RATE ERRORS
This is what we really get
This is what we would like to have!
from the measurement!

The index notation

is introduced on the
next lecture!
ACCELEROMETER ERRORS
ACCELEROMETER ERRORS, A MORE COMPLEX MODEL

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
THOUGHT EXERCISE

• What type of errors can be calibrated away?

• Random errors? Systematic errors? Both?

Image from GIPHY

THOUGHT EXERCISE

• What type of errors can be calibrated away?

• Systematic errors are consistent, repeatable errors that follow
some predictable pattern.
• Random errors are caused by unpredictable or uncontrollable
variables in the environment or system.

Systematic errors can be calibrated away. Random errors

depend on the quality of the IMU!

Image from GIPHY

IMU GRADES
Performance Strategic grade Navigation grade Tactical grade Commercial grade

Positional error 30-100 m/h .5 m/s 5-10 m/s Large variation

Gyroscope drift 0.0001°–0.001° / h <0.01°/h 1–10°/h 0.1°/s

Gyroscope random walk - <0.002°/ sqrt(h) 0.05-<0.02°/sqrt(h) Several°/ sqrt(h)

Accelerometer bias 0.1–1 <100 µg 1–5 mg 100–1,000 µg

Application Submarines General Navigation, Mapping (Integrated Low-cost navigation,

Ballistic missiles Geo-reference with GPS) Antilock, etc.

The IMU’s grade affects the

Fundamentals of Inertial Navigation, Satellite-based Positioning and
magnitude of random errors! their Integration, Noureldin et al. 2013
IMU GRADES, CONT’D

• Measurement frequency == Bandwidth

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
CALIBRATING BIAS AND SCALE, ACCELEROMETER

bias scale Note: assume z-axis parallel to

gravity vector

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
 CALIBRATING BIAS AND SCALE, GYROSCOPE

• Place the IMU at rest:

• Write the equations for solving the bias:

Fundamentals of Inertial Navigation, Satellite-based Positioning and

their Integration, Noureldin et al. 2013
 Like the lecture?

CALIBRATION

• Turntable is used for Commercial calibration,

see the image 

• We will return to the calibration of bias and

scale in the incoming IMU practical!

Fundamentals of Inertial Navigation,

Satellite-based Positioning and their
Integration, Noureldin et al. 2013
SUMMARY

• Sensor (IMU=inertial measurement unit), Sensor properties

• Gyroscope
• Accelerometer

• IMU basics
• Integration
• Measuring gravity indirectly

• Error sources
• Stochastic errors such as white noise
• Systematic errors such as bias
QUESTIONS

• What does an IMU measure?

• What biases may an IMU have? How do these affect the measurements?

• Pedestrian dead reckoning (PDR) uses accelerometers and gyroscopes to estimate speed and heading. What are the limitations
of PDR?