Section I.

INTRODUCTION

Chapter 1. What is Robotics?

1 What is a robot?

It might come as a surprise that it is actually tricky to define the word "robot." Contrast the idea of a
science fiction android with a remote control flying drone. The android (appears) to think, feel, and
move with the intellect of a human, while the drone simply executes the command of its operator.
Yet, most people are in agreement that they can be rightfully called "robots."

There are some necessary conditions for something to be a robot. A robot is a machine that:

 Moves in the physical world,

 Senses its environment and/or itself, and

 Uses computational elements to generate its movement.

However, these are not sufficient for qualifying a machine as a robot. For example, your washing
machine (if built in the last 10 years) will likely have all of these characteristics. A washing machine
certainly moves; it senses the settings of its controls, the weight of your laundry, the incoming water
temperature, possibly the dirtiness of your laundry, and so on; and will adjust its cycle speeds and
duration accordingly using computation circuits. But there seems to be something missing here, since
few people would refer to their washing machine as a robot! Similarly, your (relatively modern) car
performs fuel injection, anti-lock braking, cruise control, and airbag deployment using several
computers and sensors to monitor fuel efficiency, tire pressure, velocity, seat occupancy, etc. Yet, we
are not ready to call it a robot just yet[1].

Let us propose a couple of other possible criteria to call a system a robot. What about if a robot were
required to:

 Exhibit autonomy or automation, and

 Exhibit apparently intelligent behaviour?

These provide a sharper boundary, as they would disqualify the washing machine and non-self-driving
car from consideration as robots. But, there exist many robots that are not autonomous, such as the
remote-controlled drone mentioned above, or surgical robots under surgeon control, like the Intuitive
Surgical Da Vinci robot.

An intelligence criterion is also difficult to apply because since it is challenging to define "intelligence"
without delving into a philosophical minefield! By using the phrase "apparently intelligent", we
sidestep the issue by assuming a human judge. But what is agreed upon as "intelligent" may change
from year to year; compared to those devices in the 1950's, our modern washing machines and cars
are actually quite smart! Perhaps as the control and artificial intelligence technology used in robots
become more widely adopted, the line dividing robot and non-robot machines will become blurrier
and blurrier... until the term "robot" has lost its meaning.

Overall, it may be a pointless exercise to extract a precise definition of a robot. In any case, the
layperson's "I know a robot when I see one" should suffice for the purposes of this book.

[1]
: Presumably, by time of publication, self-driving cars are not yet widely commercially available.
2 How to develop a robot

A roboticist is a thinker, not a tinkerer. Although many students begin tinkering with robots at a
relatively young age, this process is not usually the best way to develop a robot that performs a task
well. Robotics is a more deliberate way of reaching the end goal that is informed through decades of
prior research, analysis, and practical experience. One way to define it would be as follows:

Robotics: the study of systematic, principled techniques to aid in the development of robots that
perform desired functions.

Although robots are some of the most complex machines in the world, there is a logic to how they
should be developed. A good roboticist will follow this logic whilst using any available techniques at
his/her disposal.

Specifically, the recipe for developing an intelligent robot must follow these major steps:

1. Fabrication: Design and fabricate a mechanism with sensing, actuation, and computing
capabilities to fulfill the intended task.

2. Measurement: Develop measurement apparatus(es) and a testing protocol to observe the

3. Calibration: Use measurements to calibrate (or learn) a model of the mechanism's dynamics,
actuation, and sensing.

4. Control: Develop and validate a control sub-system that maintains system stability and
provides medium-level functionality for planning.

5. Knowledge representation: Decide upon a knowledge representation to be shared between

6. Perception: Develop and evaluate a perception sub-system that produces knowledge relevant
to the task (robot state, maps, object identities)

7. Planning: Implement and test a planning sub-system that generates feasible trajectories to
accomplish high-level tasks.

8. Supervisor: Develop a high-level supervisor that schedules tasks, provides a user interface,
etc.

9. Testing and evaluation: Test the entire system in the field (on the real task in the real world,
and perhaps with human end-users).

It is important to note that these steps are almost never followed linearly, because robot design is a
cyclic process of trial and error. Any robotics project will incur many, many design cycles over its
lifetime. For example, unless the team is limited to an established robot platform, or purchasing off-
the-shelf parts, redesigns of the mechanism usually occur after steps 3, 4, and 6. Mechanical redesigns
may also occur after planning tests to make task execution more successful. On the software side, new
knowledge, perception, and planning requirements are bound to arise as tasks are tested more
thoroughly. After testing with end-users, it is not uncommon to go all the way "back to the drawing
board" to build a new mechanism! A wise roboticist will develop their later components to rapidly
accommodate minor mechanical changes.
Buying an established robot platform can greatly speed up development time by shortcutting steps 1-
4, but many vendors sell fairly raw hardware (requiring a rehash of steps 2-4). Also, there may be
locked-in decisions that prevent certain functionality to be implemented later. To use a robot as a
haptic device, make sure its motor controllers provide high-rate feedback and a force control mode!
To have your robot navigate outdoors, make sure that it has a laser sensor or stereo vision rather than
a structured light sensor! This makes it very important to examine the technical specs of a robot —
even the most mundane details — with a fine-toothed comb before making a purchase.

The theory, mathematics, and algorithms that are discussed in this book are designed to facilitate the
development of functional robots. For example, it is unnecessary to go to "square one" for every
component of a robot, as long as that component is deeply understood. A good roboticist will
understand which off-the-shelf techniques apply, and where. However, there is no substitute for real-
world testing!

Testing is one of the most painful parts of robotics, but ultimately one of its most satisfying. Although
a given technique might be theoretically beautiful, it will have been largely tested in the lab, making
certain idealizations of the real world, or for a slightly different use case. The bulk of time spent doing
robotics work is usually tweaking, tuning, and testing these tried-and-true techniques until they fit for
the problem at hand. But once the technique is validated by thorough field testing, its merits are
unimpeachable!

In summary, a good roboticist:

 Understands the robot development process.

 "Minds the gaps:" understands how their sub-system works with other components of the
project.

 Is not afraid of testing.

 Clearly communicates the assumptions, performance measures, and limitations of their sub-
system.

 Understands the assumptions made by any technique before employing it successfully in

 Is aware of classical and state-of-the-art techniques.

 Is up-to-date on current design tools, hardware, software, algorithms, and programming

3 Abstractions

System (n). A thing of such complexity that a human can only begin to comprehend it using cartoons.

This statement, although tongue-in-cheek, is actually fairly accurate. A system is composed of many
interacting parts, each of which is individually complex, and it would take ages to understand deeply
how everything works and interacts. The "cartoons" referred to above are known (especially in the
computing world) as abstractions. An abstraction is a "black box" with inputs and outputs that
performs some understandable function, which lets you ignore, to some extent, what goes on "under
the hood".

For example, to a first approximation, the basic function of a car is to turn on when a key is turned in
the ignition, steer when the steering wheel is turned, accelerate when the accelerator is pressed, and
decelerate when the brake is pressed. This is the implied contract that the abstraction of a car makes
with to a user; it makes little difference what type of material the tires are made of, the amount of
wear on the brake pads, the function of the differential, whether the engine's fuel air mixture is
controlled by computer or carburetor, whether the car's chassis was welded by robot or by human, or
even whether the motor is an electric or combustion engine. It can generally be assumed that driving
the car will take you from point A to point B, assuming that A and B are connected by roads and that
we know how to operate a car. But a car is an incredibly complex machine, comprised of tens of
thousands of mechanical parts, and modern cars have millions of lines of computer code. We don't
have to understand each of these little components; the car just works as a transportation device. This
is the cartoon we tell ourselves, and most of the time the car operates just as the cartoon tells us.

Ah, you say, but what if the car is out of gas, or a tire is flat, or a family of bears has broken into your
car and set up a den in the back seat? Well, these areas in which the cartoon "breaks down". This is
also known as a leaky abstraction, because some factors that were unspecified in our abstract,
assumed contract have "leaked" into the black box to cause unexpected or undesirable behavior. A
famous software engineering principle states that "all abstractions are, to some extent, leaky". This
certainly holds true for robotics as well.

The development of a robot will certainly generate many interacting components, each of which
performs a complex mechanical or software function. Due to this complexity, multiple engineers will
work on multiple components, and the only way to tractably comprehend the entire system is through
abstraction. A good engineer will properly label their component and summarize its function in an
easy to understand manner. E.g., the perception module gives the identity and location of all of the
interesting things in view of the robot's cameras; the planning module waits for an instruction from a
user and then plans a motion to act as requested; the controller module executes a current motion,
and updates the motion whenever the planner requests it; the UI module shows the user the state of
the robot, performs speech recognition, and translates words into recognized commands. Moreover,
she will document the components' specifications in a way that other engineers can use it in a
predictable manner.

An important function of abstractions is that they let us swap implementations of what goes on
"under the hood". We will describe several IK solvers, motion planners, trajectory optimizers, state
estimators, and object recognizers. All of them, although they may perform the same intended
function, will differ in terms of performance characteristics (e.g., for algorithms, computational
complexity, failure rates, network usage; for hardware, speed, strength, noise, and power
consumption). Different versions of an algorithm's code will also be swappable, and may have vastly
different performance characteristics. For example, 3D mapping on a GPU is orders of magnitude
faster than doing it on a CPU. It is a general rule that there will be many potential implementations of
a given robot component, and more will be developed every year. A good architect of a robotic system
will need to research and understand the state-of-the-art and plan how the system is developed to
optimize the trade-offs between performance, resource usage, and implementation difficulty.

A major goal of this book is to aid in the process of systems engineering by:

1. Defining standard abstractions used in robotics

2. Describing where these abstractions leak, why they leak, and what can be done to plug the
leaks to the extent possible

3. Presenting multiple implementations, and explain their performance characteristics