Professor Bruno Siciliano
Professor Oussama Khatib
Author Peter Corke
Foreword
Preface
Contents
Nomenclature
1
Introduction
1.1
Robots, Jobs and Ethics
1.2
About the Book
1.2.1 MATLAB Software and the Toolboxes
1.2.2 Notation, Conventions and Organization
1.2.3 Audience and Prerequisites
1.2.4 Learning with the Book
1.2.5 Teaching with the Book
1.2.6 Outline
Further Reading
Part I Foundations
2
Representing Position and Orientation
2.1 Working in Two Dimensions (2D)
2.1.1
Orientation in 2-Dimensions
2.1.1.1
Orthonormal Rotation Matrix
2.1.1.2 Matrix Exponential
2.1.2 Pose in 2-Dimensions
2.1.2.1
Homogeneous Transformation Matrix
2.1.2.2 Centers of Rotation
2.1.2.3 Twists in 2D
2.2
Working in Three Dimensions (3D)
2.2.1 Orientation in 3-Dimensions
2.2.1.1 Orthonormal Rotation Matrix
2.2.1.2 Three- Angle Representations
2.2.1.3 Singularities and Gimbal Lock
2.2.1.4 Two Vector Representation
2.2.1.5 Rotation about an Arbitrary Vector
2.2.1.6 Matrix Exponentials
2.2.1.7 Unit Quaternions
2.2.2 Pose in 3-Dimensions
2.2.2.1 Homogeneous Transformation Matrix
2.2.2.2 Vector-Quaternion Pair
2.2.2.3 Twists
2.3
Advanced Topics
2.3.1
Normalization
2.3.2
Understanding the Exponential Mapping
2.3.3
More About Twists
2.3.4 Dual Quaternions
2.3.5 Configuration Space
2.4
Using the Toolbox
2.5
Wrapping Up
Further Reading
Exercises
3 Time and Motion
3.1
Time-Varying Pose
3.1.1
Derivative of Pose
3.1.2 Transforming Spatial Velocities
3.1.3 Incremental Rotation
3.1.4 Incremental Rigid-Body Motion
3.2
Accelerating Bodies and Reference Frames
3.2.1 Dynamics of Moving Bodies
3.2.2 Transforming Forces and Torques
3.2.3 Inertial Reference Frame
3.3
Creating Time-Varying Pose
3.3.1 Smooth One-Dimensional Trajectories
3.3.2 Multi-Dimensional Trajectories
3.3.3 Multi-Segment Trajectories
3.3.4 Interpolation of Orientation in 3D
3.3.4.1 Direction of Rotation
3.3.5 Cartesian Motion in 3D
3.4
Application: Inertial Navigation
l3.4.1 Gyroscopes
3.4.1.1 How Gyroscopes Work
3.4.1.2 Estimating Orientation
3.4.2 Accelerometers
3.4.2.1 How Accelerometers Work
3.4.2.2 Estimating Pose and Body Acceleration
3.4.3 Magnetometers
3.4.3.1 How Magnetometers Work
3.4.3.2 Estimating Heading
3.4.4 Sensor Fusion
3.5
Wrapping Up
Further Reading
Exercises
Part II Mobile Robots
II
Mobile Robots
4
Mobile Robot Vehicles
4.1
Wheeled Mobile Robots
4.1.1 Car-Like Mobile Robots
4.1.1.1 Moving to a Point
4.1.1.2 Following a Line
4.1.1.3 Following a Trajectory
4.1.1.4 Moving to a Pose
4.1.2 Differentially-Steered Vehicle
4.1.3 Omnidirectional Vehicle
4.2
Flying Robots
4.3
Advanced Topics
4.3.1 Nonholonomic and Under-Actuated Systems
4.4Wrapping Up
Further Reading
Toolbox and MATLAB Notes
Exercises
Chapter
5 Navigation
5.1
Reactive Navigation
l5.1.1 Braitenberg Vehicles
l5.1.2 Simple Automata
5.2
Map-Based Planning
5.2.1 Distance Transform
5.2.2 D*
5.2.3 Introduction to Roadmap Methods
5.2.4 Probabilistic Roadmap Method (PRM)
5.2.5 Lattice Planner
5.2.6 Rapidly-Exploring Random Tree (RRT)
5.3
Wrapping Up
Further Reading
Resources
MATLAB Notes
Exercises
Chapter
6
Localization
6.1
Dead Reckoning
6.1.1 Modeling the Vehicle
6.1.2 Estimating Pose
6.2
Localizing with a Map
6.3
Creating a Map
6.4
Localization and Mapping
6.5
Rao-Blackwellized SLAM
6.6
Pose Graph SLAM
6.7
Sequential Monte-Carlo Localization
6.8
Application: Scanning Laser Rangefinder
Laser Odometry
Laser-Based Map Building
Laser-Based Localization
6.9
Wrapping Up
Further Reading
Toolbox and MATLAB Notes
Exercises
Part III Arm-Type Robots
III
Arm-Type Robots
7
Robot Arm Kinematics
7.1
Forward Kinematics
7.1.1 2-Dimensional (Planar) Robotic Arms
7.1.2 3-Dimensional Robotic Arms
l7.1.2.1 Denavit-Hartenberg Parameters
l7.1.2.2 Product of Exponentials
l7.1.2.3 6-Axis Industrial Robot
7.2
Inverse Kinematics
7.2.1 2-Dimensional (Planar) Robotic Arms
7.2.1.1 Closed-Form Solution
7.2.1.2 Numerical Solution
7.2.2 3-Dimensional Robotic Arms
7.2.2.1Closed-Form Solution
7.2.2.2 Numerical Solution
7.2.2.3 Under-Actuated Manipulator
l7.2.2.4 Redundant Manipulator
7.3
Trajectories
7.3.1 Joint-Space Motion
7.3.2 Cartesian Motion
7.3.3 Kinematics in Simulink
7.3.4 Motion through a Singularity
7.3.5 Configuration Change
7.4
Advanced Topics
7.4.1 Joint Angle Offsets
7.4.2 Determining Denavit-Hartenberg Parameters
7.4.3 Modified Denavit-Hartenberg Parameters
7.5
Applications
7.5.1 Writing on a Surface
7.5.2 A Simple Walking Robot
Kinematics
Motion of One Leg
Motion of Four Legs
7.6
Wrapping Up
Further Reading
MATLAB and Toolbox Notes
Exercises
Chapter
8
Manipulator Velocity
8.1
Manipulator Jacobian
8.1.1 Jacobian in the World Coordinate Frame
8.1.2 Jacobian in the End-Effector Coordinate Frame
8.1.3 Analytical Jacobian
8.2
Jacobian Condition and Manipulability
l8.2.1 Jacobian Singularities
l8.2.2 Manipulability
8.3
Resolved-Rate Motion Control
8.3.1 Jacobian Singularity
8.4
Under- and Over-Actuated Manipulators
8.4.1 Jacobian for Under-Actuated Robot
8.4.2 Jacobian for Over-Actuated Robot
8.5
Force Relationships
8.5.1 Transforming Wrenches to Joint Space
8.5.2 Force Ellipsoids
8.6
Inverse Kinematics: a General Numerical Approach
l8.6.1 Numerical Inverse Kinematics
8.7
Advanced Topics
8.7.1 Computing the Manipulator Jacobian Using Twists
8.8 Wrapping Up
Further Reading
MATLAB and Toolbox Notes
Exercises
9
Dynamics and Control
9.1
Independent Joint Control
9.1.1 Actuators
9.1.2 Friction
9.1.3 Effect of the Link Mass
9.1.4 Gearbox
9.1.5 Modeling the Robot Joint
9.1.6 Velocity Control Loop
9.1.7 Position Control Loop
9.1.8 Independent Joint Control Summary
9.2
Rigid-Body Equations of Motion
9.2.1 Gravity Term
9.2.2 Inertia Matrix
9.2.3 Coriolis Matrix
9.2.4 Friction
9.2.5 Effect of Payload
9.2.6 Base Force
9.2.7 Dynamic Manipulability
9.3
Forward Dynamics
9.4
Rigid-Body Dynamics Compensation
9.4.1 Feedforward Control
9.4.2 Computed Torque Control
l9.4.3 Operational Space Control
9.5
Applications
9.5.1 Series-Elastic Actuator (SEA)
9.6
Wrapping Up
Further Reading
Exercises
Part IV Computer Vision
IV
Computer Vision
10
Light and Color
10.1
Spectral Representation of Light
10.1.1 Absorption
10.1.2 Reflectance
10.1.3 Luminance
10.2 Color
10.2.1The Human Eye
10.2.2 Measuring Color
10.2.3 Reproducing Colors
10.2.4 Chromaticity Space
10.2.5 Color Names
10.2.6 Other Color and Chromaticity Spaces
10.2.7 Transforming between Different Primaries
10.2.8 What Is White?
10.3
Advanced Topics
10.3.1 Color Temperature
10.3.2 Color Constancy
10.3.3 White Balancing
10.3.4 Color Change Due to Absorption
10.3.5 Dichromatic Reflectance
10.3.6 Gamma
10.4
Application: Color Image
10.4.1 Comparing Color Spaces
10.4.2 Shadow Removal
10.5
Wrapping Up
Further Reading
Data Sources
Exercises
11 Image Formation
11.1
Perspective Camera
11.1.1 Perspective Projection
11.1.2 Modeling a Perspective Camera
11.1.3 Discrete Image Plane
11.1.4 Camera Matrix
11.1.5 Projecting Points
11.1.6 Lens Distortion
11.2
Camera Calibration
11.2.1 Homogeneous Transformation Approach
11.2.2 Decomposing the Camera Calibration Matrix
11.2.3 Pose Estimation
11.2.4 Camera Calibration Toolbox
11.3
Wide Field-of-View Imaging
11.3.1 Fisheye Lens Camera
11.3.2 Catadioptric Camera
11.3.3 Spherical Camera
11.4
Unified Imaging
11.4.1 Mapping Wide-Angle Images to the Sphere
11.4.2 Mapping from the Sphere to a Perspective Image
11.5
Novel Cameras
11.5.1 Multi-Camera Arrays
11.5.2 Light-Field Cameras
11.6
Advanced Topics
11.6.1 Projecting 3D Lines and Quadrics
11.6.2 Nonperspective Cameras
11.7
Wrapping Up
Further Reading and Resources
Toolbox Notes
Exercises
Chapter
12
Images and Image Processing
12.1
Obtaining an Image
12.1.1 Images from Files
12.1.2 Images from an Attached Camera
12.1.3 Images from a Movie File
12.1.4 Images from the Web
12.1.5 Images from Maps
12.1.6 Images from Code
12.2
Image Histograms
12.3
Monadic Operations
12.4
Diadic Operations
12.5
Spatial Operations
12.5.1 Linear Spatial Filtering
12.5.1.1 Smoothing
12.5.1.2 Boundary Effects
12.5.1.3 Edge Detection
12.5.2 Template Matching
12.5.2.1 Nonparameteric Local Transforms
12.5.3 Nonlinear Operations
12.6
Mathematical Morphology
12.6.1 Noise Removal
12.6.2 Boundary Detection
12.6.3 Hit or Miss Transform
12.6.4 Distance Transform
12.7
Shape Changing
12.7.1 Cropping
12.7.2 Image Resizing
12.7.3 Image Pyramids
12.7.4 Image Warping
12.8
Wrapping Up
Further Reading
Sources of Image Data
MATLAB Notes
General Software Tools
Exercises
Chapter
13
Image Feature Extraction
13.1
Region Features
13.1.1 Classification
13.1.1.1 Grey-Level Classification
13.1.1.2 Color Classification
13.1.2 Representation
13.1.2.1 Graph-Based Segmentation
13.1.3 Description
13.1.3.1 Bounding Boxes
13.1.3.2 Moments
13.1.3.3 Blob Features
13.1.3.4 Shape from Moments
13.1.3.5 Shape from Perimeter
13.1.3.6 Character Recognition
13.1.4 Summary
13.2
Line Features
13.2.1 Summary
13.3
Point Features
13.3.1 Classical Corner Detectors
13.3.2 Scale-Space Corner Detectors
13.3.2.1 Scale-Space Point Feature
13.4
Wrapping Up
MATLAB Notes
Further Reading
Exercises
Chapter
14
Using Multiple Images
14.1
Feature Correspondence
14.2
Geometry of Multiple Views
14.2.1 The Fundamental Matrix
14.2.2 The Essential Matrix
14.2.3 Estimating the Fundamental Matrix from Real Image Data
14.2.4 Planar Homography
14.3
Stereo Vision
14.3.1 Sparse Stereo
14.3.2 Dense Stereo Matching
14.3.2.1 Stereo Failure Modes
14.3.3 Peak Refinement
14.3.4 Cleaning up and Reconstruction
14.3.5 3D Texture Mapped Display
14.3.6 Anaglyphs
14.3.7 Image Rectification
14.4
Bundle Adjustment
14.5
Point Clouds
14.5.1 Fitting a Plane
14.5.2 Matching Two Sets of Points
14.6
Structured Light
14.7
Applications
14.7.1 Perspective Correction
14.7.2 Mosaicing
14.7.3 Image Matching and Retrieval
14.7.4 Visual Odometry
14.8
Wrapping Up
MATLAB and Toolbox Notes
Further Reading
Resources
Exercises
Part V Robotics, Vision and Control
V
Robotics, Vision and Control
15
Vision-Based Control
15.1
Position-Based Visual Servoing
15.2
Image-Based Visual Servoing
15.2.1 Camera and Image Motion
15.2.2 Controlling Feature Motion
15.2.3 Estimating Feature Depth
15.2.4 Performance Issues
15.3
Using Other Image Features
15.3.1 Line Features
15.3.2 Circle Features
15.3.3 Photometric Features
15.4
Wrapping Up
Further Reading
Exercises
16
Advanced Visual Servoing
16.1
XY/Z-Partitioned IBVS
16.2
IBVS Using Polar Coordinates
16.3
IBVS for a Spherical Camera
16.4
Applications
16.4.1 Arm-Type Robot
16.4.2 Mobile Robot
16.4.2.1 Holonomic Mobile Robot
16.4.2.2 Nonholonomic Mobile Robot
16.4.3 Aerial Robot
16.5
Wrapping Up
Further Reading
Resources
Exercises
Appendices
A Installing the Toolboxes
B
Linear Algebra Refresher
B.1
Vectors
B.2
Matrices
B.2.1 Square Matrices
B.2.2 Nonsquare and Singular Matrices
C
Geometry
C.1
Euclidean Geometry
C.1.1 Points
C.1.2 Lines
C.1.2.1
Lines in 2D
C.1.2.2 Lines in 3D and Plücker Coordinates
C.1.3 Planes
C.1.4 Ellipses and Ellipsoids
C.1.4.1 Properties
C.1.4.2 Drawing an Ellipse
C.1.4.3 Fitting an Ellipse to Data
From a Set of Interior Points
From a Set of Boundary Points
C.2
Homogeneous Coordinates
C.2.1 Two Dimensions
C.2.1.1 Conics
C.2.2 Three Dimensions
C.2.2.1 Lines
C.2.2.2 Planes
C.2.2.3 Quadrics
C.3
Geometric Transformations
D
Lie Groups and Algebras
E
Linearization, Jacobians and Hessians
F
Solving Systems of Equations
F.1
Linear Problems
F.1.1 Nonhomogeneous Systems
F.1.2 Homogeneous Systems
F.1.2
Homogeneous Systems
F.2.1 Finding Roots
F.2.2 Nonlinear Minimization
F.2.3 Nonlinear Least Squares Minimization
Numerical Issues
F.2.4 Sparse Nonlinear Least Squares
State Vector
Inherent Structure
Large Scale Problems
Anchoring
G
Gaussian Random Variables
H
Kalman Filter
H.1
Linear Systems – Kalman Filter
H.2
Nonlinear Systems – Extended Kalman Filter
I
Graphs
Appendix
J
Peak Finding
Bibliography
Index
A
H
B
I
C
J
K
L
D
E
M
G
N
T
P
U
V
R
W
S
Y
A
B
C
E
F
D
G
H
I
J
K
L
M
N
Q
O
R
P
S
T
V
X, Y, Z
U
Symbols
A
B
C
D
E
F
G
I
H
J
K
L
M
N
O
P
R
Q
S
T
U
V
W
Y
Z
X