Robotics 2

Introduction to Control

Prof. Alessandro De Luca

 What do we mean by robot control?

n different level of definitions may be given to robot control

n successfully complete a task or work program
n accurate execution of a motion trajectory
n zeroing a positioning error
⇒ control system unit has a hierarchical internal structure
high-level control
more more

... ...
intelligence precision
direct control

§ different time scales in the various control levels: lowest ≤ 1 ms,

higher levels up to seconds
§ different but cooperating models, objectives, methods

Robotics 2 2
 Evaluation of control performance
n quality of execution in nominal conditions
n velocity/speed of task completion
n accuracy/repeatability (in static and dynamic terms)
n energy requirements
⇒ improvements also thanks to models (software!)
§ robustness in perturbed/uncertain conditions
§ adaptation to changing environments
§ high repeatability despite disturbances, changes of parameters,
uncertainties, modeling errors
⇒ can be improved by a generalized use of feedback,
using more sensor information
⇒ learn through repeated robot trials/human experience

Robotics 2 3
 Static positioning
accuracy and repeatability

poor accuracy poor accuracy

poor repeatability good repeatability

good accuracy good accuracy

poor repeatability good repeatability

what about “dynamic” accuracy on (test or selected) motion trajectories?

Robotics 2 4
 Basic control schemes

control robot environment

open-loop action
command

control robot environment

perception

combination of closed-loop commands

feedforward and
feedback commands METHODS MODELS

Robotics 2 5
 Control schemes and uncertainty
n feedback control
n insensitivity to mild disturbances, small variations of parameters,
and different initial conditions
n robust control
n tolerates relatively large uncertainties of known range
n adaptive control
n improves performance online, adapting the control law to unknown
range of uncertainties and/or large (but slow) parameter variations
n AI-based (intelligent) control
n performance improved based on trials/experience: LEARNING
n autonomous/supervised search of internal structure for capturing
system behavior: SELF-ORGANIZING
n Data-driven models: (Deep) Neural Networks, Gaussian Processes
uncertainty on parametric values IDENTIFICATION
… on the system structure ...
Robotics 2 6
 Limits in control of industrial robots -1

n from a functional viewpoint

n “closed” control architectures, relatively difficult to interface with
external programs and sensing devices for hard real-time operation
n need of some expertise for programming and handling of exceptions

⇒ introducing easy/more intuitive user (multi-modal) interfaces

n at the higher level
n open-loop task command generation
⇒ exteroceptive sensory feedback absent or very loose, with low
capability of autonomous reasoning
n at the intermediate level
n limited consideration of advanced kinematic and dynamic issues
⇒ e.g., singularity robustness: solved on a case-by-case basis
⇒ task redundancy: no automatic use of the extra degrees of freedom

Robotics 2 7
 Limits in control of industrial robots -2

§ at the lower (direct) level

reduced execution speed (“control bandwidth”)
n

⇒ typically, heavy mechanical structures

n reduced dynamic accuracy on fast motion trajectories

⇒ standard: use of kinematic control + PID only

n problems with dry friction and backlash at the joints

n compliance in the robot structure

⇒ flexible transmissions
now desired (belts, harmonic drives, long shafts)
for safe
physical ⇒ large structures or relatively lightweight links
Human-Robot
Interaction
need to include better dynamic models and model-based control laws
handled, e.g., using direct-drive actuators or online friction compensation
Robotics 2 8
 Example of robot positioning
n low damped vibrations due to joint elasticity

video

without modeling
and explicit
control of
joint elasticity

n 6R KUKA KR-15/2 robot (235 kg), with 15 kg payload

Robotics 2 9
 Advanced robot control laws
n deeper mathematical/physical analysis and modeling of
robot components (model-based approach)
n schemes using various control loops at different/multiple
hierarchical levels (feedback) and with additional sensors
n visual servoing
n force/torque sensors for interaction control
n ...
n “new” methods
n integration of (open-loop/feedforward) motion planning and
feedback control aspects (e.g., sensor-based planning)
n fast (sensor-based) re-planning
n model predictive control (with preview)
n learning (iterative, by imitation, skill transfer, ...)
n …
Robotics 2 10
 Example of visual-based control
n human-obstacle collision avoidance
video

n 3R SoftArm prototype with McKibben actuators (Univ. of Pisa)

using repulsive force field built from stereo camera information
Robotics 2 11
 Functional structure of a control unit
sensor measurements

task fault detection, vision

program
SENSORS:
optical encoders,
trajectory vision, distance, velocity tachos,
planning contact force strain gauges,
joint or wrist
direct control position, F/T sensors, IMUs,
algorithms velocity tactile sensors,
micro-switches,
current
actuators range/depth sensors,
laser, CCD/CMOS and
stereo cameras,
robot
proprioceptive sensors RGB-D cameras,
…
environment
exteroceptive sensors
(also “virtual” ones, i.e., model-based)
Robotics 2 12
 Functional structure of a control unit
programming languages

task Java, Lisp, expert- and

program dedicated
rule-based systems
programming
languages
trajectory
Matlab, C++, Python Task-
planning
Object-
Robot-
direct control Oriented
Assembler (PICs), C, C++
algorithms

actuators
T-O: insert P1 into H5
O-O: move APPR frame #13
robot
R-O: rotate joint 3 by -45°

environment
often “addressed” using the manual TEACH BOX
in conventional industrial robots
Robotics 2 13
 Functional structure of a control unit
modeling issues

task modeling of tasks

program (with AI reasoning)

trajectory geometric and kinematic models

planning coordinate transformations

direct control dynamic-based methods

algorithms (nonlinear) torque control

actuators
(electrical and mechanical)
dynamic models
robot
structured and unstructured
environment world modeling (and acquisition)

Robotics 2 14
 Industrial robot programming languages

§ ABB Rapid

§ COMAU PDL2

§ FANUC Karel

§ KUKA KRL

§ MITSUBISHI MELFA

§ UNIVERSAL ROBOTS RoboDK

§…
Robotics 2 15
 Robot control/research software
(last updated in April 2025)

§ a (partial) list of open source robot software

§ for simulation and/or real-time control
§ for interfacing with devices and sensors
§ research oriented
Player/Stage playerstage.sourceforge.net ⇒ github.com/rtv/stage
§ Stage: in origin, a networked Linux/MacOS X robotics server acting as
abstraction layer to support a variety of hardware ⇒ now a 2(.5)D
mobile robot standalone simulation environment
§ Gazebo: 3D robot simulator (ODE physics engine and OpenGL
rendering), now an independent project ⇒ gazebosim.org
CoppeliaSim (was V-REP; edu version available) www.coppeliarobotics.com
§ each object/model controlled via an embedded script, a plugin, a ROS
node, a remote API client, or a custom solution
§ controllers written in C/C++, Python, Java, Matlab, ...
Robotics 2 16
 Robot control/research software (cont’d)

Robotics System Toolbox (license for Sapienza)

§ tools/algorithms for simulation of kinematics, dynamics, trajectory planning,
control of serial manipulators, mobile robots and humanoids
§ library of robots, scene and map creation, Gazebo interface …
QUT Robot Academy petercorke.com
§ free software, including Robotics Toolbox (release 10) and Machine Vision
Toolbox (release 4) for MATLAB (open source on github) à now in Python
ROS (Robot Operating System) ros.org
§ middleware with hardware abstraction, device drivers, libraries, visualizers,
message-passing, package management (ROS 2: Linux, Windows, MacOS)
§ “nodes”: executable code (in Python, C++) running with a publish/subscribe
communication style
§ drivers, tools, state-of-the-art algorithms … (all open source)
PyRobotics (Python API) pypi.org/project/pyRobotics (v1.8 in 2015)
Robotics 2 17
 Robot control/research software

Robotics 2 18
 OROCOS control software
§ OROCOS (Open RObot COntrol Software) orocos.org
• open-source, portable C++ libraries for robot control
• Real-Time Toolkit (for Linux, MacOS X, Windows Visual Studio)
• supports CORBA for distributed network computing and ROS interface
• (user-defined) application libraries

⇒ github
Robotics 2 19
 Example application using OROCOS
video

multi-sensor fusion for multi-robot manipulation

in a human populated environment (KU Leuven)
Robotics 2 20
 Summarizing …
n to improve performance of robot controllers
1. more complete modeling (kinematics and dynamics)
2. introduction of feedback throughout all hierarchical levels
§ dynamic control at low level allows in principle
1. much higher accuracy on generic motion trajectories
2. larger velocity in task execution with same accuracy
§ interplay between control, mechanics, electronics
1. able to control accurately also lightweight/compliant robots
2. full utilization of task-related redundancy
3. smart mechanical design can reduce control efforts (e.g.,
closed kinematic chains simplifying robot inertia matrix)
4. actuators with higher dynamic performance (e.g., direct drives)
and/or including controlled variable stiffness
advanced applications should justify additional costs
(e.g., laser cutting with 10g accelerations, safe human-robot interaction)
Robotics 2 21
 Benefits of model-based control
n trajectory tracking task: comparison between standard
industrial and new model-based controller
video
standard new model-based
industrial controller controller
three squares in:

y
horizontal
plane

vertical
front
z plane

vertical
sagittal
plane
x

Robotics 2 22
 Robot learning by imitation
n learning from human motion primitives (imitation)
n motion refinement by kinesthetic teaching (with impedance control)

video

@TUM, Munich (D. Lee, C. Ott), for the EU SAPHARI project

Robotics 2 23
 Using visual or depth sensor feedback
video
n robust visual or depth (Kinect)
feedback for motion tracking

n collision avoidance schemes

(here, redundancy w.r.t. an E-E task)
video
Robotics 2 24
 Panoramic view of control laws
n problems & methods for robot manipulators that will be considered
(control command is always a joint torque, if not else specified)
definition
joint Cartesian task
type of of error
space space space
task
PD, PID, PD with visual
servoing
regulation gravity compensation, gravity
(kinematic
iterative learning compensation scheme)
free
motion feedback linearization,
trajectory inverse dynamics + PD, feedback
tracking passivity-based control, linearization
robust/adaptive control
impedance
contact control hybrid
(with variants),
motion - admittance
force-velocity
control
(with force exchange) control
(kinematic scheme)
Robotics 2 25
 Dynamic or kinematic control laws
n torque-controlled robots
n issue current commands ! = !! (with #! = $" !! ) to drive the
(electrical) motors, based on information on the dynamic model
n often, a low-level (analog) current loop is present to enforce the
execution of the desired command
n may use a torque measure ## (by joint torque sensors) to do the
same, in case of joint/transmission elasticity (with ## = $(' − )))
n best suited for high dynamic performance and ‘transparent’ control
of interaction forces
n position/motion-controlled robots
n issue kinematic commands: velocity )̇ = )̇ ! , acceleration )̈ = )̈ ! ,
or their integrated/micro-interpolated version ) = )!
n references for a low-level direct loop at high frequency ("! ≅ 400 -.!)

Robotics 2 26
 Torque- vs. position-controlled robots
%" or
&" (! )! %̇ % &
task/trajectory torque task
planner controller
actuation robot ∫ kinematics
including feedforward
– e.g., (! (&)
sensing
dynamic control

%" or
&" %̇ ! ( %̇ % &
task/trajectory high-level low-level actuation task
planner controller controller " → robot ∫ kinematics
including feedforward
– e.g., $̇ ! (&)

kinematic control sensing

n both modes may be present even in the same robotic system

Robotics 2 27
 HRI in industrial settings
non-collaborative robots:
safety fences are required to
prevent harming human operators

collaborative robots:
allow human workers to
stand in their proximity and
work together on the same task
Main robot safety standards
ISO 10218-1/2:2011
ISO/TS 15066:2016

Robotics 2 28
 Human-Robot Interaction taxonomy
n cognitive (cHRI) vs. physical (pHRI) Human-Robot Interaction
n cHRI models of humans, of robots, and of the interaction itself
n dialog-based, intention- and activity-based, simulation-theoretic models

B. Mutlu, N. Roy, S. Sabanovic: Ch. 71, Springer Handbook of Robotics, 2016

Robotics 2 29
 Human-Robot Interaction taxonomy
n pHRI planned and controlled robot behaviors: 3-layer architecture

Safety
lightweight mechanical design collision detection
compliance at robot joints and safe reaction

Coexistence
robot and human sharing collision avoidance
the same workspace no need of physical contact

Collaboration
contactless, e.g., gestures with intentional contact and
or voice commands coordinated exchange of forces

A. De Luca, F. Flacco: IEEE BioRob Conference, 2012

Robotics 2 30
 Human-Robot Collaboration
n the different possible levels of pHRI are represented also within
ISO safety standards (from safe coexistence to safe collaboration)

V. Villani et al.: Mechatronics, 2018 video

for more info & material on safe pHRI: http://www.diag.uniroma1.it/deluca/pHRI.php
Robotics 2 31