Downloaded 50 times
![1.1. Python Overview 3
1.1.2 Preview of a Python Program
As a simple introduction, Code Fragment 1.1 presents a Python program that com-
putes the grade-point average (GPA) for a student based on letter grades that are
entered by a user. Many of the techniques demonstrated in this example will be
discussed in the remainder of this chapter. At this point, we draw attention to a few
high-level issues, for readers who are new to Python as a programming language.
Python’s syntax relies heavily on the use of whitespace. Individual statements
are typically concluded with a newline character, although a command can extend
to another line, either with a concluding backslash character (), or if an opening
delimiter has not yet been closed, such as the { character in defining value map.
Whitespace is also key in delimiting the bodies of control structures in Python.
Specifically, a block of code is indented to designate it as the body of a control
structure, and nested control structures use increasing amounts of indentation. In
Code Fragment 1.1, the body of the while loop consists of the subsequent 8 lines,
including a nested conditional structure.
Comments are annotations provided for human readers, yet ignored by the
Python interpreter. The primary syntax for comments in Python is based on use
of the # character, which designates the remainder of the line as a comment.
print( Welcome to the GPA calculator. )
print( Please enter all your letter grades, one per line. )
print( Enter a blank line to designate the end. )
# map from letter grade to point value
points = { A+ :4.0, A :4.0, A- :3.67, B+ :3.33, B :3.0, B- :2.67,
C+ :2.33, C :2.0, C :1.67, D+ :1.33, D :1.0, F :0.0}
num courses = 0
total points = 0
done = False
while not done:
grade = input( ) # read line from user
if grade == : # empty line was entered
done = True
elif grade not in points: # unrecognized grade entered
print(Unknown grade {0} being ignored.format(grade))
else:
num courses += 1
total points += points[grade]
if num courses 0: # avoid division by zero
print( Your GPA is {0:.3} .format(total points / num courses))
Code Fragment 1.1: A Python program that computes a grade-point average (GPA).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-25-320.jpg)
![1.2. Objects in Python 9
Sequence Types: The list, tuple, and str Classes
The list, tuple, and str classes are sequence types in Python, representing a col-
lection of values in which the order is significant. The list class is the most general,
representing a sequence of arbitrary objects (akin to an “array” in other languages).
The tuple class is an immutable version of the list class, benefiting from a stream-
lined internal representation. The str class is specially designed for representing
an immutable sequence of text characters. We note that Python does not have a
separate class for characters; they are just strings with length one.
The list Class
A list instance stores a sequence of objects. A list is a referential structure, as it
technically stores a sequence of references to its elements (see Figure 1.4). El-
ements of a list may be arbitrary objects (including the None object). Lists are
array-based sequences and are zero-indexed, thus a list of length n has elements
indexed from 0 to n−1 inclusive. Lists are perhaps the most used container type in
Python and they will be extremely central to our study of data structures and algo-
rithms. They have many valuable behaviors, including the ability to dynamically
expand and contract their capacities as needed. In this chapter, we will discuss only
the most basic properties of lists. We revisit the inner working of all of Python’s
sequence types as the focus of Chapter 5.
Python uses the characters [ ] as delimiters for a list literal, with [ ] itself being
an empty list. As another example, [ red , green , blue ] is a list containing
three string instances. The contents of a list literal need not be expressed as literals;
if identifiers a and b have been established, then syntax [a, b] is legitimate.
The list() constructor produces an empty list by default. However, the construc-
tor will accept any parameter that is of an iterable type. We will discuss iteration
further in Section 1.8, but examples of iterable types include all of the standard con-
tainer types (e.g., strings, list, tuples, sets, dictionaries). For example, the syntax
list( hello ) produces a list of individual characters, [ h , e , l , l , o ].
Because an existing list is itself iterable, the syntax backup = list(data) can be
used to construct a new list instance referencing the same contents as the original.
3 4 5 6 7
0 1 2 10
9
8
primes:
13 19 23 29 31
7
5
3
2 11 17
Figure 1.4: Python’s internal representation of a list of integers, instantiated as
prime = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31]. The implicit indices of the ele-
ments are shown below each entry.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-31-320.jpg)
![10 Chapter 1. Python Primer
The tuple Class
The tuple class provides an immutable version of a sequence, and therefore its
instances have an internal representation that may be more streamlined than that of
a list. While Python uses the [ ] characters to delimit a list, parentheses delimit a
tuple, with () being an empty tuple. There is one important subtlety. To express
a tuple of length one as a literal, a comma must be placed after the element, but
within the parentheses. For example, (17,) is a one-element tuple. The reason for
this requirement is that, without the trailing comma, the expression (17) is viewed
as a simple parenthesized numeric expression.
The str Class
Python’s str class is specifically designed to efficiently represent an immutable
sequence of characters, based upon the Unicode international character set. Strings
have a more compact internal representation than the referential lists and tuples, as
portrayed in Figure 1.5.
0
A
S M P L E
3 4 5
1 2
Figure 1.5: A Python string, which is an indexed sequence of characters.
String literals can be enclosed in single quotes, as in hello , or double
quotes, as in hello. This choice is convenient, especially when using an-
other of the quotation characters as an actual character in the sequence, as in
Don t worry. Alternatively, the quote delimiter can be designated using a
backslash as a so-called escape character, as in Don t worry . Because the
backslash has this purpose, the backslash must itself be escaped to occur as a natu-
ral character of the string literal, as in C:Python , for a string that would be
displayed as C:Python. Other commonly escaped characters are n for newline
and t for tab. Unicode characters can be included, such as 20u20AC for the
string 20 .
Python also supports using the delimiter or to begin and end a string
literal. The advantage of such triple-quoted strings is that newline characters can
be embedded naturally (rather than escaped as n). This can greatly improve the
readability of long, multiline strings in source code. For example, at the beginning
of Code Fragment 1.1, rather than use separate print statements for each line of
introductory output, we can use a single print statement, as follows:
print(”””Welcome to the GPA calculator.
Please enter all your letter grades, one per line.
Enter a blank line to designate the end.”””)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-32-320.jpg)
![1.2. Objects in Python 11
The set and frozenset Classes
Python’s set class represents the mathematical notion of a set, namely a collection
of elements, without duplicates, and without an inherent order to those elements.
The major advantage of using a set, as opposed to a list, is that it has a highly
optimized method for checking whether a specific element is contained in the set.
This is based on a data structure known as a hash table (which will be the primary
topic of Chapter 10). However, there are two important restrictions due to the
algorithmic underpinnings. The first is that the set does not maintain the elements
in any particular order. The second is that only instances of immutable types can be
added to a Python set. Therefore, objects such as integers, floating-point numbers,
and character strings are eligible to be elements of a set. It is possible to maintain a
set of tuples, but not a set of lists or a set of sets, as lists and sets are mutable. The
frozenset class is an immutable form of the set type, so it is legal to have a set of
frozensets.
Python uses curly braces { and } as delimiters for a set, for example, as {17}
or { red , green , blue }. The exception to this rule is that { } does not
represent an empty set; for historical reasons, it represents an empty dictionary
(see next paragraph). Instead, the constructor syntax set() produces an empty set.
If an iterable parameter is sent to the constructor, then the set of distinct elements
is produced. For example, set( hello ) produces { h , e , l , o }.
The dict Class
Python’s dict class represents a dictionary, or mapping, from a set of distinct keys
to associated values. For example, a dictionary might map from unique student ID
numbers, to larger student records (such as the student’s name, address, and course
grades). Python implements a dict using an almost identical approach to that of a
set, but with storage of the associated values.
A dictionary literal also uses curly braces, and because dictionaries were intro-
duced in Python prior to sets, the literal form { } produces an empty dictionary.
A nonempty dictionary is expressed using a comma-separated series of key:value
pairs. For example, the dictionary { ga : Irish , de : German } maps
ga to Irish and de to German .
The constructor for the dict class accepts an existing mapping as a parameter,
in which case it creates a new dictionary with identical associations as the existing
one. Alternatively, the constructor accepts a sequence of key-value pairs as a pa-
rameter, as in dict(pairs) with pairs = [( ga , Irish ), ( de , German )].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-33-320.jpg)
![14 Chapter 1. Python Primer
Python carefully extends the semantics of // and % to cases where one or both
operands are negative. For the sake of notation, let us assume that variables n
and m represent respectively the dividend and divisor of a quotient n
m, and that
q = n // m and r = n % m. Python guarantees that q m + r will equal n. We
already saw an example of this identity with positive operands, as 6∗4+ 3 = 27.
When the divisor m is positive, Python further guarantees that 0 ≤ r m. As
a consequence, we find that −27 // 4 evaluates to −7 and −27 % 4 evaluates
to 1, as (−7)∗4+ 1 = −27. When the divisor is negative, Python guarantees that
m r ≤ 0. As an example, 27 // −4 is −7 and 27 % −4 is −1, satisfying the
identity 27 = (−7)∗(−4)+(−1).
The conventions for the // and % operators are even extended to floating-
point operands, with the expression q = n // m being the integral floor of the
quotient, and r = n % m being the “remainder” to ensure that q m + r equals
n. For example, 8.2 // 3.14 evaluates to 2.0 and 8.2 % 3.14 evaluates to 1.92, as
2.0∗3.14+1.92 = 8.2.
Bitwise Operators
Python provides the following bitwise operators for integers:
∼ bitwise complement (prefix unary operator)
bitwise and
| bitwise or
ˆ bitwise exclusive-or
shift bits left, filling in with zeros
shift bits right, filling in with sign bit
Sequence Operators
Each of Python’s built-in sequence types (str, tuple, and list) support the following
operator syntaxes:
s[j] element at index j
s[start:stop] slice including indices [start,stop)
s[start:stop:step] slice including indices start, start + step,
start + 2 step, . . . , up to but not equalling or stop
s + t concatenation of sequences
k s shorthand for s + s + s + ... (k times)
val in s containment check
val not in s non-containment check
Python relies on zero-indexing of sequences, thus a sequence of length n has ele-
ments indexed from 0 to n− 1 inclusive. Python also supports the use of negative
indices, which denote a distance from the end of the sequence; index −1 denotes
the last element, index −2 the second to last, and so on. Python uses a slicing](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-36-320.jpg)
![1.3. Expressions, Operators, and Precedence 15
notation to describe subsequences of a sequence. Slices are described as half-open
intervals, with a start index that is included, and a stop index that is excluded. For
example, the syntax data[3:8] denotes a subsequence including the five indices:
3,4,5,6,7. An optional “step” value, possibly negative, can be indicated as a third
parameter of the slice. If a start index or stop index is omitted in the slicing nota-
tion, it is presumed to designate the respective extreme of the original sequence.
Because lists are mutable, the syntax s[j] = val can be used to replace an ele-
ment at a given index. Lists also support a syntax, del s[j], that removes the desig-
nated element from the list. Slice notation can also be used to replace or delete a
sublist.
The notation val in s can be used for any of the sequences to see if there is an
element equivalent to val in the sequence. For strings, this syntax can be used to
check for a single character or for a larger substring, as with amp in example .
All sequences define comparison operations based on lexicographic order, per-
forming an element by element comparison until the first difference is found. For
example, [5, 6, 9] [5, 7] because of the entries at index 1. Therefore, the follow-
ing operations are supported by sequence types:
s == t equivalent (element by element)
s != t not equivalent
s t lexicographically less than
s = t lexicographically less than or equal to
s t lexicographically greater than
s = t lexicographically greater than or equal to
Operators for Sets and Dictionaries
Sets and frozensets support the following operators:
key in s containment check
key not in s non-containment check
s1 == s2 s1 is equivalent to s2
s1 != s2 s1 is not equivalent to s2
s1 = s2 s1 is subset of s2
s1 s2 s1 is proper subset of s2
s1 = s2 s1 is superset of s2
s1 s2 s1 is proper superset of s2
s1 | s2 the union of s1 and s2
s1 s2 the intersection of s1 and s2
s1 − s2 the set of elements in s1 but not s2
s1 ˆ s2 the set of elements in precisely one of s1 or s2
Note well that sets do not guarantee a particular order of their elements, so the
comparison operators, such as , are not lexicographic; rather, they are based on
the mathematical notion of a subset. As a result, the comparison operators define](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-37-320.jpg)
![16 Chapter 1. Python Primer
a partial order, but not a total order, as disjoint sets are neither “less than,” “equal
to,” or “greater than” each other. Sets also support many fundamental behaviors
through named methods (e.g., add, remove); we will explore their functionality
more fully in Chapter 10.
Dictionaries, like sets, do not maintain a well-defined order on their elements.
Furthermore, the concept of a subset is not typically meaningful for dictionaries, so
the dict class does not support operators such as . Dictionaries support the notion
of equivalence, with d1 == d2 if the two dictionaries contain the same set of key-
value pairs. The most widely used behavior of dictionaries is accessing a value
associated with a particular key k with the indexing syntax, d[k]. The supported
operators are as follows:
d[key] value associated with given key
d[key] = value set (or reset) the value associated with given key
del d[key] remove key and its associated value from dictionary
key in d containment check
key not in d non-containment check
d1 == d2 d1 is equivalent to d2
d1 != d2 d1 is not equivalent to d2
Dictionaries also support many useful behaviors through named methods, which
we explore more fully in Chapter 10.
Extended Assignment Operators
Python supports an extended assignment operator for most binary operators, for
example, allowing a syntax such as count += 5. By default, this is a shorthand for
the more verbose count = count + 5. For an immutable type, such as a number or
a string, one should not presume that this syntax changes the value of the existing
object, but instead that it will reassign the identifier to a newly constructed value.
(See discussion of Figure 1.3.) However, it is possible for a type to redefine such
semantics to mutate the object, as the list class does for the += operator.
alpha = [1, 2, 3]
beta = alpha # an alias for alpha
beta += [4, 5] # extends the original list with two more elements
beta = beta + [6, 7] # reassigns beta to a new list [1, 2, 3, 4, 5, 6, 7]
print(alpha) # will be [1, 2, 3, 4, 5]
This example demonstrates the subtle difference between the list semantics for the
syntax beta += foo versus beta = beta + foo.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-38-320.jpg)
![1.3. Expressions, Operators, and Precedence 17
1.3.1 Compound Expressions and Operator Precedence
Programming languages must have clear rules for the order in which compound
expressions, such as 5 + 2 3, are evaluated. The formal order of precedence
for operators in Python is given in Table 1.3. Operators in a category with higher
precedence will be evaluated before those with lower precedence, unless the expres-
sion is otherwise parenthesized. Therefore, we see that Python gives precedence to
multiplication over addition, and therefore evaluates the expression 5 + 2 3 as
5 + (2 3), with value 11, but the parenthesized expression (5 + 2) 3 evalu-
ates to value 21. Operators within a category are typically evaluated from left to
right, thus 5 − 2 + 3 has value 6. Exceptions to this rule include that unary oper-
ators and exponentiation are evaluated from right to left.
Python allows a chained assignment, such as x = y = 0, to assign multiple
identifiers to the rightmost value. Python also allows the chaining of comparison
operators. For example, the expression 1 = x + y = 10 is evaluated as the
compound (1 = x + y) and (x + y = 10), but without computing the inter-
mediate value x + y twice.
Operator Precedence
Type Symbols
1 member access expr.member
2
function/method calls expr(...)
container subscripts/slices expr[...]
3 exponentiation
4 unary operators +expr, −expr, ˜expr
5 multiplication, division , /, //, %
6 addition, subtraction +, −
7 bitwise shifting ,
8 bitwise-and
9 bitwise-xor ˆ
10 bitwise-or |
11
comparisons is, is not, ==, !=, , =, , =
containment in, not in
12 logical-not not expr
13 logical-and and
14 logical-or or
15 conditional val1 if cond else val2
16 assignments =, +=, −=, =, etc.
Table 1.3: Operator precedence in Python, with categories ordered from highest
precedence to lowest precedence. When stated, we use expr to denote a literal,
identifier, or result of a previously evaluated expression. All operators without
explicit mention of expr are binary operators, with syntax expr1 operator expr2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-39-320.jpg)
![20 Chapter 1. Python Primer
1.4.2 Loops
Python offers two distinct looping constructs. A while loop allows general repeti-
tion based upon the repeated testing of a Boolean condition. A for loop provides
convenient iteration of values from a defined series (such as characters of a string,
elements of a list, or numbers within a given range). We discuss both forms in this
section.
While Loops
The syntax for a while loop in Python is as follows:
while condition:
body
As with an if statement, condition can be an arbitrary Boolean expression, and
body can be an arbitrary block of code (including nested control structures). The
execution of a while loop begins with a test of the Boolean condition. If that condi-
tion evaluates to True, the body of the loop is performed. After each execution of
the body, the loop condition is retested, and if it evaluates to True, another iteration
of the body is performed. When the conditional test evaluates to False (assuming
it ever does), the loop is exited and the flow of control continues just beyond the
body of the loop.
As an example, here is a loop that advances an index through a sequence of
characters until finding an entry with value X or reaching the end of the sequence.
j = 0
while j len(data) and data[j] != X :
j += 1
The len function, which we will introduce in Section 1.5.2, returns the length of a
sequence such as a list or string. The correctness of this loop relies on the short-
circuiting behavior of the and operator, as described on page 12. We intention-
ally test j len(data) to ensure that j is a valid index, prior to accessing element
data[j]. Had we written that compound condition with the opposite order, the eval-
uation of data[j] would eventually raise an IndexError when X is not found. (See
Section 1.7 for discussion of exceptions.)
As written, when this loop terminates, variable j’s value will be the index of
the leftmost occurrence of X , if found, or otherwise the length of the sequence
(which is recognizable as an invalid index to indicate failure of the search). It is
worth noting that this code behaves correctly, even in the special case when the list
is empty, as the condition j len(data) will initially fail and the body of the loop
will never be executed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-42-320.jpg)
![1.4. Control Flow 21
For Loops
Python’s for-loop syntax is a more convenient alternative to a while loop when
iterating through a series of elements. The for-loop syntax can be used on any
type of iterable structure, such as a list, tuple str, set, dict, or file (we will discuss
iterators more formally in Section 1.8). Its general syntax appears as follows.
for element in iterable:
body # body may refer to element as an identifier
For readers familiar with Java, the semantics of Python’s for loop is similar to the
“for each” loop style introduced in Java 1.5.
As an instructive example of such a loop, we consider the task of computing
the sum of a list of numbers. (Admittedly, Python has a built-in function, sum, for
this purpose.) We perform the calculation with a for loop as follows, assuming that
data identifies the list:
total = 0
for val in data:
total += val # note use of the loop variable, val
The loop body executes once for each element of the data sequence, with the iden-
tifier, val, from the for-loop syntax assigned at the beginning of each pass to a
respective element. It is worth noting that val is treated as a standard identifier. If
the element of the original data happens to be mutable, the val identifier can be
used to invoke its methods. But a reassignment of identifier val to a new value has
no affect on the original data, nor on the next iteration of the loop.
As a second classic example, we consider the task of finding the maximum
value in a list of elements (again, admitting that Python’s built-in max function
already provides this support). If we can assume that the list, data, has at least one
element, we could implement this task as follows:
biggest = data[0] # as we assume nonempty list
for val in data:
if val biggest:
biggest = val
Although we could accomplish both of the above tasks with a while loop, the
for-loop syntax had an advantage of simplicity, as there is no need to manage an
explicit index into the list nor to author a Boolean loop condition. Furthermore, we
can use a for loop in cases for which a while loop does not apply, such as when
iterating through a collection, such as a set, that does not support any direct form
of indexing.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-43-320.jpg)
![22 Chapter 1. Python Primer
Index-Based For Loops
The simplicity of a standard for loop over the elements of a list is wonderful; how-
ever, one limitation of that form is that we do not know where an element resides
within the sequence. In some applications, we need knowledge of the index of an
element within the sequence. For example, suppose that we want to know where
the maximum element in a list resides.
Rather than directly looping over the elements of the list in that case, we prefer
to loop over all possible indices of the list. For this purpose, Python provides
a built-in class named range that generates integer sequences. (We will discuss
generators in Section 1.8.) In simplest form, the syntax range(n) generates the
series of n values from 0 to n − 1. Conveniently, these are precisely the series of
valid indices into a sequence of length n. Therefore, a standard Python idiom for
looping through the series of indices of a data sequence uses a syntax,
for j in range(len(data)):
In this case, identifier j is not an element of the data—it is an integer. But the
expression data[j] can be used to retrieve the respective element. For example, we
can find the index of the maximum element of a list as follows:
big index = 0
for j in range(len(data)):
if data[j] data[big index]:
big index = j
Break and Continue Statements
Python supports a break statement that immediately terminate a while or for loop
when executed within its body. More formally, if applied within nested control
structures, it causes the termination of the most immediately enclosing loop. As
a typical example, here is code that determines whether a target value occurs in a
data set:
found = False
for item in data:
if item == target:
found = True
break
Python also supports a continue statement that causes the current iteration of a
loop body to stop, but with subsequent passes of the loop proceeding as expected.
We recommend that the break and continue statements be used sparingly. Yet,
there are situations in which these commands can be effectively used to avoid in-
troducing overly complex logical conditions.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-44-320.jpg)
![1.5. Functions 25
These assignment statements establish identifier data as an alias for grades and
target as a name for the string literal A . (See Figure 1.7.)
...
str
A
data target
grades
list
Figure 1.7: A portrayal of parameter passing in Python, for the function call
count(grades, A ). Identifiers data and target are formal parameters defined
within the local scope of the count function.
The communication of a return value from the function back to the caller is
similarly implemented as an assignment. Therefore, with our sample invocation of
prizes = count(grades, A ), the identifier prizes in the caller’s scope is assigned
to the object that is identified as n in the return statement within our function body.
An advantage to Python’s mechanism for passing information to and from a
function is that objects are not copied. This ensures that the invocation of a function
is efficient, even in a case where a parameter or return value is a complex object.
Mutable Parameters
Python’s parameter passing model has additional implications when a parameter is
a mutable object. Because the formal parameter is an alias for the actual parameter,
the body of the function may interact with the object in ways that change its state.
Considering again our sample invocation of the count function, if the body of the
function executes the command data.append( F ), the new entry is added to the
end of the list identified as data within the function, which is one and the same as
the list known to the caller as grades. As an aside, we note that reassigning a new
value to a formal parameter with a function body, such as by setting data = [ ],
does not alter the actual parameter; such a reassignment simply breaks the alias.
Our hypothetical example of a count method that appends a new element to a
list lacks common sense. There is no reason to expect such a behavior, and it would
be quite a poor design to have such an unexpected effect on the parameter. There
are, however, many legitimate cases in which a function may be designed (and
clearly documented) to modify the state of a parameter. As a concrete example,
we present the following implementation of a method named scale that’s primary
purpose is to multiply all entries of a numeric data set by a given factor.
def scale(data, factor):
for j in range(len(data)):
data[j] = factor](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-47-320.jpg)
![26 Chapter 1. Python Primer
Default Parameter Values
Python provides means for functions to support more than one possible calling
signature. Such a function is said to be polymorphic (which is Greek for “many
forms”). Most notably, functions can declare one or more default values for pa-
rameters, thereby allowing the caller to invoke a function with varying numbers of
actual parameters. As an artificial example, if a function is declared with signature
def foo(a, b=15, c=27):
there are three parameters, the last two of which offer default values. A caller is
welcome to send three actual parameters, as in foo(4, 12, 8), in which case the de-
fault values are not used. If, on the other hand, the caller only sends one parameter,
foo(4), the function will execute with parameters values a=4, b=15, c=27. If a
caller sends two parameters, they are assumed to be the first two, with the third be-
ing the default. Thus, foo(8, 20) executes with a=8, b=20, c=27. However, it is
illegal to define a function with a signature such as bar(a, b=15, c) with b having
a default value, yet not the subsequent c; if a default parameter value is present for
one parameter, it must be present for all further parameters.
As a more motivating example for the use of a default parameter, we revisit
the task of computing a student’s GPA (see Code Fragment 1.1). Rather than as-
sume direct input and output with the console, we prefer to design a function that
computes and returns a GPA. Our original implementation uses a fixed mapping
from each letter grade (such as a B−) to a corresponding point value (such as
2.67). While that point system is somewhat common, it may not agree with the
system used by all schools. (For example, some may assign an A+ grade a value
higher than 4.0.) Therefore, we design a compute gpa function, given in Code
Fragment 1.2, which allows the caller to specify a custom mapping from grades to
values, while offering the standard point system as a default.
def compute gpa(grades, points={ A+ :4.0, A :4.0, A- :3.67, B+ :3.33,
B :3.0, B- :2.67, C+ :2.33, C :2.0,
C :1.67, D+ :1.33, D :1.0, F :0.0}):
num courses = 0
total points = 0
for g in grades:
if g in points: # a recognizable grade
num courses += 1
total points += points[g]
return total points / num courses
Code Fragment 1.2: A function that computes a student’s GPA with a point value
system that can be customized as an optional parameter.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-48-320.jpg)
![1.6. Simple Input and Output 31
When reading a numeric value from the user, a programmer must use the input
function to get the string of characters, and then use the int or float syntax to
construct the numeric value that character string represents. That is, if a call to
response = input() reports that the user entered the characters, 2013 , the syntax
int(response) could be used to produce the integer value 2013. It is quite common
to combine these operations with a syntax such as
year = int(input( In what year were you born? ))
if we assume that the user will enter an appropriate response. (In Section 1.7 we
discuss error handling in such a situation.)
Because input returns a string as its result, use of that function can be combined
with the existing functionality of the string class, as described in Appendix A. For
example, if the user enters multiple pieces of information on the same line, it is
common to call the split method on the result, as in
reply = input( Enter x and y, separated by spaces: )
pieces = reply.split( ) # returns a list of strings, as separated by spaces
x = float(pieces[0])
y = float(pieces[1])
A Sample Program
Here is a simple, but complete, program that demonstrates the use of the input
and print functions. The tools for formatting the final output is discussed in Ap-
pendix A.
age = int(input( Enter your age in years: ))
max heart rate = 206.9 − (0.67 age) # as per Med Sci Sports Exerc.
target = 0.65 max heart rate
print( Your target fat-burning heart rate is , target)
1.6.2 Files
Files are typically accessed in Python beginning with a call to a built-in function,
named open, that returns a proxy for interactions with the underlying file. For
example, the command, fp = open( sample.txt ), attempts to open a file named
sample.txt, returning a proxy that allows read-only access to the text file.
The open function accepts an optional second parameter that determines the
access mode. The default mode is r for reading. Other common modes are w
for writing to the file (causing any existing file with that name to be overwritten),
or a for appending to the end of an existing file. Although we focus on use of
text files, it is possible to work with binary files, using access modes such as rb
or wb .](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-53-320.jpg)
![34 Chapter 1. Python Primer
Sending the wrong number, type, or value of parameters to a function is another
common cause for an exception. For example, a call to abs( hello ) will raise a
TypeError because the parameter is not numeric, and a call to abs(3, 5) will raise
a TypeError because one parameter is expected. A ValueError is typically raised
when the correct number and type of parameters are sent, but a value is illegitimate
for the context of the function. For example, the int constructor accepts a string,
as with int( 137 ), but a ValueError is raised if that string does not represent an
integer, as with int( 3.14 ) or int( hello ).
Python’s sequence types (e.g., list, tuple, and str) raise an IndexError when
syntax such as data[k] is used with an integer k that is not a valid index for the given
sequence (as described in Section 1.2.3). Sets and dictionaries raise a KeyError
when an attempt is made to access a nonexistent element.
1.7.1 Raising an Exception
An exception is thrown by executing the raise statement, with an appropriate in-
stance of an exception class as an argument that designates the problem. For exam-
ple, if a function for computing a square root is sent a negative value as a parameter,
it can raise an exception with the command:
raise ValueError( x cannot be negative )
This syntax raises a newly created instance of the ValueError class, with the error
message serving as a parameter to the constructor. If this exception is not caught
within the body of the function, the execution of the function immediately ceases
and the exception is propagated to the calling context (and possibly beyond).
When checking the validity of parameters sent to a function, it is customary
to first verify that a parameter is of an appropriate type, and then to verify that it
has an appropriate value. For example, the sqrt function in Python’s math library
performs error-checking that might be implemented as follows:
def sqrt(x):
if not isinstance(x, (int, float)):
raise TypeError( x must be numeric )
elif x 0:
raise ValueError( x cannot be negative )
# do the real work here...
Checking the type of an object can be performed at run-time using the built-in
function, isinstance. In simplest form, isinstance(obj, cls) returns True if object,
obj, is an instance of class, cls, or any subclass of that type. In the above example, a
more general form is used with a tuple of allowable types indicated with the second
parameter. After confirming that the parameter is numeric, the function enforces
an expectation that the number be nonnegative, raising a ValueError otherwise.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-56-320.jpg)
![1.7. Exception Handling 35
How much error-checking to perform within a function is a matter of debate.
Checking the type and value of each parameter demands additional execution time
and, if taken to an extreme, seems counter to the nature of Python. Consider the
built-in sum function, which computes a sum of a collection of numbers. An im-
plementation with rigorous error-checking might be written as follows:
def sum(values):
if not isinstance(values, collections.Iterable):
raise TypeError( parameter must be an iterable type )
total = 0
for v in values:
if not isinstance(v, (int, float)):
raise TypeError( elements must be numeric )
total = total+ v
return total
The abstract base class, collections.Iterable, includes all of Python’s iterable con-
tainers types that guarantee support for the for-loop syntax (e.g., list, tuple, set);
we discuss iterables in Section 1.8, and the use of modules, such as collections, in
Section 1.11. Within the body of the for loop, each element is verified as numeric
before being added to the total. A far more direct and clear implementation of this
function can be written as follows:
def sum(values):
total = 0
for v in values:
total = total + v
return total
Interestingly, this simple implementation performs exactly like Python’s built-in
version of the function. Even without the explicit checks, appropriate exceptions
are raised naturally by the code. In particular, if values is not an iterable type, the
attempt to use the for-loop syntax raises a TypeError reporting that the object is not
iterable. In the case when a user sends an iterable type that includes a nonnumer-
ical element, such as sum([3.14, oops ]), a TypeError is naturally raised by the
evaluation of expression total + v. The error message
unsupported operand type(s) for +: ’float’ and ’str’
should be sufficiently informative to the caller. Perhaps slightly less obvious is the
error that results from sum([ alpha , beta ]). It will technically report a failed
attempt to add an int and str, due to the initial evaluation of total + alpha ,
when total has been initialized to 0.
In the remainder of this book, we tend to favor the simpler implementations
in the interest of clean presentation, performing minimal error-checking in most
situations.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-57-320.jpg)
![1.8. Iterators and Generators 39
1.8 Iterators and Generators
In Section 1.4.2, we introduced the for-loop syntax beginning as:
for element in iterable:
and we noted that there are many types of objects in Python that qualify as being
iterable. Basic container types, such as list, tuple, and set, qualify as iterable types.
Furthermore, a string can produce an iteration of its characters, a dictionary can
produce an iteration of its keys, and a file can produce an iteration of its lines. User-
defined types may also support iteration. In Python, the mechanism for iteration is
based upon the following conventions:
• An iterator is an object that manages an iteration through a series of values. If
variable, i, identifies an iterator object, then each call to the built-in function,
next(i), produces a subsequent element from the underlying series, with a
StopIteration exception raised to indicate that there are no further elements.
• An iterable is an object, obj, that produces an iterator via the syntax iter(obj).
By these definitions, an instance of a list is an iterable, but not itself an iterator.
With data = [1, 2, 4, 8], it is not legal to call next(data). However, an iterator
object can be produced with syntax, i = iter(data), and then each subsequent call
to next(i) will return an element of that list. The for-loop syntax in Python simply
automates this process, creating an iterator for the give iterable, and then repeatedly
calling for the next element until catching the StopIteration exception.
More generally, it is possible to create multiple iterators based upon the same
iterable object, with each iterator maintaining its own state of progress. However,
iterators typically maintain their state with indirect reference back to the original
collection of elements. For example, calling iter(data) on a list instance produces
an instance of the list iterator class. That iterator does not store its own copy of the
list of elements. Instead, it maintains a current index into the original list, represent-
ing the next element to be reported. Therefore, if the contents of the original list
are modified after the iterator is constructed, but before the iteration is complete,
the iterator will be reporting the updated contents of the list.
Python also supports functions and classes that produce an implicit iterable se-
ries of values, that is, without constructing a data structure to store all of its values
at once. For example, the call range(1000000) does not return a list of numbers; it
returns a range object that is iterable. This object generates the million values one
at a time, and only as needed. Such a lazy evaluation technique has great advan-
tage. In the case of range, it allows a loop of the form, for j in range(1000000):,
to execute without setting aside memory for storing one million values. Also, if
such a loop were to be interrupted in some fashion, no time will have been spent
computing unused values of the range.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-61-320.jpg)
![40 Chapter 1. Python Primer
We see lazy evaluation used in many of Python’s libraries. For example, the
dictionary class supports methods keys(), values(), and items(), which respec-
tively produce a “view” of all keys, values, or (key,value) pairs within a dictionary.
None of these methods produces an explicit list of results. Instead, the views that
are produced are iterable objects based upon the actual contents of the dictionary.
An explicit list of values from such an iteration can be immediately constructed by
calling the list class constructor with the iteration as a parameter. For example, the
syntax list(range(1000)) produces a list instance with values from 0 to 999, while
the syntax list(d.values()) produces a list that has elements based upon the current
values of dictionary d. We can similarly construct a tuple or set instance based
upon a given iterable.
Generators
In Section 2.3.4, we will explain how to define a class whose instances serve as
iterators. However, the most convenient technique for creating iterators in Python
is through the use of generators. A generator is implemented with a syntax that
is very similar to a function, but instead of returning values, a yield statement is
executed to indicate each element of the series. As an example, consider the goal
of determining all factors of a positive integer. For example, the number 100 has
factors 1, 2, 4, 5, 10, 20, 25, 50, 100. A traditional function might produce and
return a list containing all factors, implemented as:
def factors(n): # traditional function that computes factors
results = [ ] # store factors in a new list
for k in range(1,n+1):
if n % k == 0: # divides evenly, thus k is a factor
results.append(k) # add k to the list of factors
return results # return the entire list
In contrast, an implementation of a generator for computing those factors could be
implemented as follows:
def factors(n): # generator that computes factors
for k in range(1,n+1):
if n % k == 0: # divides evenly, thus k is a factor
yield k # yield this factor as next result
Notice use of the keyword yield rather than return to indicate a result. This indi-
cates to Python that we are defining a generator, rather than a traditional function. It
is illegal to combine yield and return statements in the same implementation, other
than a zero-argument return statement to cause a generator to end its execution. If
a programmer writes a loop such as for factor in factors(100):, an instance of our
generator is created. For each iteration of the loop, Python executes our procedure](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-62-320.jpg)
![1.9. Additional Python Conveniences 43
1.9.2 Comprehension Syntax
A very common programming task is to produce one series of values based upon
the processing of another series. Often, this task can be accomplished quite simply
in Python using what is known as a comprehension syntax. We begin by demon-
strating list comprehension, as this was the first form to be supported by Python.
Its general form is as follows:
[ expression for value in iterable if condition ]
We note that both expression and condition may depend on value, and that the
if-clause is optional. The evaluation of the comprehension is logically equivalent
to the following traditional control structure for computing a resulting list:
result = [ ]
for value in iterable:
if condition:
result.append(expression)
As a concrete example, a list of the squares of the numbers from 1 to n, that is
[1,4,9,16,25,... ,n2], can be created by traditional means as follows:
squares = [ ]
for k in range(1, n+1):
squares.append(k k)
With list comprehension, this logic is expressed as follows:
squares = [k k for k in range(1, n+1)]
As a second example, Section 1.8 introduced the goal of producing a list of factors
for an integer n. That task is accomplished with the following list comprehension:
factors = [k for k in range(1,n+1) if n % k == 0]
Python supports similar comprehension syntaxes that respectively produce a
set, generator, or dictionary. We compare those syntaxes using our example for
producing the squares of numbers.
[ k k for k in range(1, n+1) ] list comprehension
{ k k for k in range(1, n+1) } set comprehension
( k k for k in range(1, n+1) ) generator comprehension
{ k : k k for k in range(1, n+1) } dictionary comprehension
The generator syntax is particularly attractive when results do not need to be stored
in memory. For example, to compute the sum of the first n squares, the genera-
tor syntax, total = sum(k k for k in range(1, n+1)), is preferred to the use of an
explicitly instantiated list comprehension as the parameter.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-65-320.jpg)
![44 Chapter 1. Python Primer
1.9.3 Packing and Unpacking of Sequences
Python provides two additional conveniences involving the treatment of tuples and
other sequence types. The first is rather cosmetic. If a series of comma-separated
expressions are given in a larger context, they will be treated as a single tuple, even
if no enclosing parentheses are provided. For example, the assignment
data = 2, 4, 6, 8
results in identifier, data, being assigned to the tuple (2, 4, 6, 8). This behavior
is called automatic packing of a tuple. One common use of packing in Python is
when returning multiple values from a function. If the body of a function executes
the command,
return x, y
it will be formally returning a single object that is the tuple (x, y).
As a dual to the packing behavior, Python can automatically unpack a se-
quence, allowing one to assign a series of individual identifiers to the elements
of sequence. As an example, we can write
a, b, c, d = range(7, 11)
which has the effect of assigning a=7, b=8, c=9, and d=10, as those are the four
values in the sequence returned by the call to range. For this syntax, the right-hand
side expression can be any iterable type, as long as the number of variables on the
left-hand side is the same as the number of elements in the iteration.
This technique can be used to unpack tuples returned by a function. For exam-
ple, the built-in function, divmod(a, b), returns the pair of values (a // b, a % b)
associated with an integer division. Although the caller can consider the return
value to be a single tuple, it is possible to write
quotient, remainder = divmod(a, b)
to separately identify the two entries of the returned tuple. This syntax can also be
used in the context of a for loop, when iterating over a sequence of iterables, as in
for x, y in [ (7, 2), (5, 8), (6, 4) ]:
In this example, there will be three iterations of the loop. During the first pass, x=7
and y=2, and so on. This style of loop is quite commonly used to iterate through
key-value pairs that are returned by the items() method of the dict class, as in:
for k, v in mapping.items():](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-66-320.jpg)
![50 Chapter 1. Python Primer
next number in a sequence based upon one or more past numbers that it has gen-
erated. Indeed, a simple yet popular pseudo-random number generator chooses its
next number based solely on the most recently chosen number and some additional
parameters using the following formula.
next = (a*current + b) % n;
where a, b, and n are appropriately chosen integers. Python uses a more advanced
technique known as a Mersenne twister. It turns out that the sequences generated
by these techniques can be proven to be statistically uniform, which is usually
good enough for most applications requiring random numbers, such as games. For
applications, such as computer security settings, where one needs unpredictable
random sequences, this kind of formula should not be used. Instead, one should
ideally sample from a source that is actually random, such as radio static coming
from outer space.
Since the next number in a pseudo-random generator is determined by the pre-
vious number(s), such a generator always needs a place to start, which is called its
seed. The sequence of numbers generated for a given seed will always be the same.
One common trick to get a different sequence each time a program is run is to use
a seed that will be different for each run. For example, we could use some timed
input from a user or the current system time in milliseconds.
Python’s random module provides support for pseudo-random number gener-
ation by defining a Random class; instances of that class serve as generators with
independent state. This allows different aspects of a program to rely on their own
pseudo-random number generator, so that calls to one generator do not affect the
sequence of numbers produced by another. For convenience, all of the methods
supported by the Random class are also supported as stand-alone functions of the
random module (essentially using a single generator instance for all top-level calls).
Syntax Description
seed(hashable)
Initializes the pseudo-random number generator
based upon the hash value of the parameter
random()
Returns a pseudo-random floating-point
value in the interval [0.0,1.0).
randint(a,b)
Returns a pseudo-random integer
in the closed interval [a,b].
randrange(start, stop, step)
Returns a pseudo-random integer in the standard
Python range indicated by the parameters.
choice(seq)
Returns an element of the given sequence
chosen pseudo-randomly.
shuffle(seq)
Reorders the elements of the given
sequence pseudo-randomly.
Table 1.8: Methods supported by instances of the Random class, and as top-level
functions of the random module.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-72-320.jpg)
![1.12. Exercises 51
1.12 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-1.1 Write a short Python function, is multiple(n, m), that takes two integer
values and returns True if n is a multiple of m, that is, n = mi for some
integer i, and False otherwise.
R-1.2 Write a short Python function, is even(k), that takes an integer value and
returns True if k is even, and False otherwise. However, your function
cannot use the multiplication, modulo, or division operators.
R-1.3 Write a short Python function, minmax(data), that takes a sequence of
one or more numbers, and returns the smallest and largest numbers, in the
form of a tuple of length two. Do not use the built-in functions min or
max in implementing your solution.
R-1.4 Write a short Python function that takes a positive integer n and returns
the sum of the squares of all the positive integers smaller than n.
R-1.5 Give a single command that computes the sum from Exercise R-1.4, rely-
ing on Python’s comprehension syntax and the built-in sum function.
R-1.6 Write a short Python function that takes a positive integer n and returns
the sum of the squares of all the odd positive integers smaller than n.
R-1.7 Give a single command that computes the sum from Exercise R-1.6, rely-
ing on Python’s comprehension syntax and the built-in sum function.
R-1.8 Python allows negative integers to be used as indices into a sequence,
such as a string. If string s has length n, and expression s[k] is used for in-
dex −n ≤ k 0, what is the equivalent index j ≥ 0 such that s[j] references
the same element?
R-1.9 What parameters should be sent to the range constructor, to produce a
range with values 50, 60, 70, 80?
R-1.10 What parameters should be sent to the range constructor, to produce a
range with values 8, 6, 4, 2, 0, −2, −4, −6, −8?
R-1.11 Demonstrate how to use Python’s list comprehension syntax to produce
the list [1, 2, 4, 8, 16, 32, 64, 128, 256].
R-1.12 Python’s random module includes a function choice(data) that returns a
random element from a non-empty sequence. The random module in-
cludes a more basic function randrange, with parameterization similar to
the built-in range function, that return a random choice from the given
range. Using only the randrange function, implement your own version
of the choice function.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-73-320.jpg)
![52 Chapter 1. Python Primer
Creativity
C-1.13 Write a pseudo-code description of a function that reverses a list of n
integers, so that the numbers are listed in the opposite order than they
were before, and compare this method to an equivalent Python function
for doing the same thing.
C-1.14 Write a short Python function that takes a sequence of integer values and
determines if there is a distinct pair of numbers in the sequence whose
product is odd.
C-1.15 Write a Python function that takes a sequence of numbers and determines
if all the numbers are different from each other (that is, they are distinct).
C-1.16 In our implementation of the scale function (page 25), the body of the loop
executes the command data[j] = factor. We have discussed that numeric
types are immutable, and that use of the = operator in this context causes
the creation of a new instance (not the mutation of an existing instance).
How is it still possible, then, that our implementation of scale changes the
actual parameter sent by the caller?
C-1.17 Had we implemented the scale function (page 25) as follows, does it work
properly?
def scale(data, factor):
for val in data:
val = factor
Explain why or why not.
C-1.18 Demonstrate how to use Python’s list comprehension syntax to produce
the list [0, 2, 6, 12, 20, 30, 42, 56, 72, 90].
C-1.19 Demonstrate how to use Python’s list comprehension syntax to produce
the list [ a , b , c , ..., z ], but without having to type all 26 such
characters literally.
C-1.20 Python’s random module includes a function shuffle(data) that accepts a
list of elements and randomly reorders the elements so that each possi-
ble order occurs with equal probability. The random module includes a
more basic function randint(a, b) that returns a uniformly random integer
from a to b (including both endpoints). Using only the randint function,
implement your own version of the shuffle function.
C-1.21 Write a Python program that repeatedly reads lines from standard input
until an EOFError is raised, and then outputs those lines in reverse order
(a user can indicate end of input by typing ctrl-D).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-74-320.jpg)
![1.12. Exercises 53
C-1.22 Write a short Python program that takes two arrays a and b of length n
storing int values, and returns the dot product of a and b. That is, it returns
an array c of length n such that c[i] = a[i]·b[i], for i = 0,...,n−1.
C-1.23 Give an example of a Python code fragment that attempts to write an ele-
ment to a list based on an index that may be out of bounds. If that index
is out of bounds, the program should catch the exception that results, and
print the following error message:
“Don’t try buffer overflow attacks in Python!”
C-1.24 Write a short Python function that counts the number of vowels in a given
character string.
C-1.25 Write a short Python function that takes a string s, representing a sentence,
and returns a copy of the string with all punctuation removed. For exam-
ple, if given the string Let s try, Mike., this function would return
Lets try Mike.
C-1.26 Write a short program that takes as input three integers, a, b, and c, from
the console and determines if they can be used in a correct arithmetic
formula (in the given order), like “a+b = c,” “a = b−c,” or “a∗b = c.”
C-1.27 In Section 1.8, we provided three different implementations of a generator
that computes factors of a given integer. The third of those implementa-
tions, from page 41, was the most efficient, but we noted that it did not
yield the factors in increasing order. Modify the generator so that it reports
factors in increasing order, while maintaining its general performance ad-
vantages.
C-1.28 The p-norm of a vector v = (v1,v2,...,vn) in n-dimensional space is de-
fined as
v =
p
vp
1 +vp
2 +···+vp
n.
For the special case of p = 2, this results in the traditional Euclidean
norm, which represents the length of the vector. For example, the Eu-
clidean norm of a two-dimensional vector with coordinates (4,3) has a
Euclidean norm of
√
42 +32 =
√
16+9 =
√
25 = 5. Give an implemen-
tation of a function named norm such that norm(v, p) returns the p-norm
value of v and norm(v) returns the Euclidean norm of v. You may assume
that v is a list of numbers.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-75-320.jpg)
![Chapter Notes 55
Chapter Notes
The official Python Web site (http://www.python.org) has a wealth of information, in-
cluding a tutorial and full documentation of the built-in functions, classes, and standard
modules. The Python interpreter is itself a useful reference, as the interactive command
help(foo) provides documentation for any function, class, or module that foo identifies.
Books providing an introduction to programming in Python include titles authored by
Campbell et al. [22], Cedar [25], Dawson [32], Goldwasser and Letscher [43], Lutz [72],
Perkovic [82], and Zelle [105]. More complete reference books on Python include titles by
Beazley [12], and Summerfield [91].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-77-320.jpg)
![64 Chapter 2. Object-Oriented Programming
2.2.2 Pseudo-Code
As an intermediate step before the implementation of a design, programmers are
often asked to describe algorithms in a way that is intended for human eyes only.
Such descriptions are called pseudo-code. Pseudo-code is not a computer program,
but is more structured than usual prose. It is a mixture of natural language and
high-level programming constructs that describe the main ideas behind a generic
implementation of a data structure or algorithm. Because pseudo-code is designed
for a human reader, not a computer, we can communicate high-level ideas, without
being burdened with low-level implementation details. At the same time, we should
not gloss over important steps. Like many forms of human communication, finding
the right balance is an important skill that is refined through practice.
In this book, we rely on a pseudo-code style that we hope will be evident to
Python programmers, yet with a mix of mathematical notations and English prose.
For example, we might use the phrase “indicate an error” rather than a formal raise
statement. Following conventions of Python, we rely on indentation to indicate
the extent of control structures and on an indexing notation in which entries of a
sequence A with length n are indexed from A[0] to A[n− 1]. However, we choose
to enclose comments within curly braces { like these } in our pseudo-code, rather
than using Python’s # character.
2.2.3 Coding Style and Documentation
Programs should be made easy to read and understand. Good programmers should
therefore be mindful of their coding style, and develop a style that communicates
the important aspects of a program’s design for both humans and computers. Con-
ventions for coding style tend to vary between different programming communities.
The official Style Guide for Python Code is available online at
http://www.python.org/dev/peps/pep-0008/
The main principles that we adopt are as follows:
• Python code blocks are typically indented by 4 spaces. However, to avoid
having our code fragments overrun the book’s margins, we use 2 spaces for
each level of indentation. It is strongly recommended that tabs be avoided, as
tabs are displayed with differing widths across systems, and tabs and spaces
are not viewed as identical by the Python interpreter. Many Python-aware
editors will automatically replace tabs with an appropriate number of spaces.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-86-320.jpg)
![66 Chapter 2. Object-Oriented Programming
Documentation
Python provides integrated support for embedding formal documentation directly
in source code using a mechanism known as a docstring. Formally, any string literal
that appears as the first statement within the body of a module, class, or function
(including a member function of a class) will be considered to be a docstring. By
convention, those string literals should be delimited within triple quotes (”””). As
an example, our version of the scale function from page 25 could be documented
as follows:
def scale(data, factor):
”””Multiply all entries of numeric data list by the given factor.”””
for j in range(len(data)):
data[j] = factor
It is common to use the triple-quoted string delimiter for a docstring, even when
the string fits on a single line, as in the above example. More detailed docstrings
should begin with a single line that summarizes the purpose, followed by a blank
line, and then further details. For example, we might more clearly document the
scale function as follows:
def scale(data, factor):
”””Multiply all entries of numeric data list by the given factor.
data an instance of any mutable sequence type (such as a list)
containing numeric elements
factor a number that serves as the multiplicative factor for scaling
”””
for j in range(len(data)):
data[j] = factor
A docstring is stored as a field of the module, function, or class in which it
is declared. It serves as documentation and can be retrieved in a variety of ways.
For example, the command help(x), within the Python interpreter, produces the
documentation associated with the identified object x. An external tool named
pydoc is distributed with Python and can be used to generate formal documentation
as text or as a Web page. Guidelines for authoring useful docstrings are available
at:
http://www.python.org/dev/peps/pep-0257/
In this book, we will try to present docstrings when space allows. Omitted
docstrings can be found in the online version of our source code.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-88-320.jpg)
![2.3. Class Definitions 73
Testing the Class
In Code Fragment 2.3, we demonstrate some basic usage of the CreditCard class,
inserting three cards into a list named wallet. We use loops to make some charges
and payments, and use various accessors to print results to the console.
These tests are enclosed within a conditional, if name == __main__ :,
so that they can be embedded in the source code with the class definition. Using
the terminology of Section 2.2.4, these tests provide method coverage, as each of
the methods is called at least once, but it does not provide statement coverage, as
there is never a case in which a charge is rejected due to the credit limit. This
is not a particular advanced from of testing as the output of the given tests must
be manually audited in order to determine whether the class behaved as expected.
Python has tools for more formal testing (see discussion of the unittest module
in Section 2.2.4), so that resulting values can be automatically compared to the
predicted outcomes, with output generated only when an error is detected.
53 if name == __main__ :
54 wallet = [ ]
55 wallet.append(CreditCard( John Bowman , California Savings ,
56 5391 0375 9387 5309 , 2500) )
57 wallet.append(CreditCard( John Bowman , California Federal ,
58 3485 0399 3395 1954 , 3500) )
59 wallet.append(CreditCard( John Bowman , California Finance ,
60 5391 0375 9387 5309 , 5000) )
61
62 for val in range(1, 17):
63 wallet[0].charge(val)
64 wallet[1].charge(2 val)
65 wallet[2].charge(3 val)
66
67 for c in range(3):
68 print( Customer = , wallet[c].get customer())
69 print( Bank = , wallet[c].get bank())
70 print( Account = , wallet[c].get account())
71 print( Limit = , wallet[c].get limit())
72 print( Balance = , wallet[c].get balance())
73 while wallet[c].get balance( ) 100:
74 wallet[c].make payment(100)
75 print( New balance = , wallet[c].get balance())
76 print()
Code Fragment 2.3: Testing the CreditCard class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-95-320.jpg)
![2.3. Class Definitions 75
Common Syntax Special Method Form
a + b a. add (b); alternatively b. radd (a)
a − b a. sub (b); alternatively b. rsub (a)
a b a. mul (b); alternatively b. rmul (a)
a / b a. truediv (b); alternatively b. rtruediv (a)
a // b a. floordiv (b); alternatively b. rfloordiv (a)
a % b a. mod (b); alternatively b. rmod (a)
a b a. pow (b); alternatively b. rpow (a)
a b a. lshift (b); alternatively b. rlshift (a)
a b a. rshift (b); alternatively b. rrshift (a)
a b a. and (b); alternatively b. rand (a)
a ˆ b a. xor (b); alternatively b. rxor (a)
a | b a. or (b); alternatively b. ror (a)
a += b a. iadd (b)
a −= b a. isub (b)
a = b a. imul (b)
... ...
+a a. pos ()
−a a. neg ()
˜a a. invert ()
abs(a) a. abs ()
a b a. lt (b)
a = b a. le (b)
a b a. gt (b)
a = b a. ge (b)
a == b a. eq (b)
a != b a. ne (b)
v in a a. contains (v)
a[k] a. getitem (k)
a[k] = v a. setitem (k,v)
del a[k] a. delitem (k)
a(arg1, arg2, ...) a. call (arg1, arg2, ...)
len(a) a. len ()
hash(a) a. hash ()
iter(a) a. iter ()
next(a) a. next ()
bool(a) a. bool ()
float(a) a. float ()
int(a) a. int ()
repr(a) a. repr ()
reversed(a) a. reversed ()
str(a) a. str ()
Table 2.1: Overloaded operations, implemented with Python’s special methods.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-97-320.jpg)
![2.3. Class Definitions 77
2.3.3 Example: Multidimensional Vector Class
To demonstrate the use of operator overloading via special methods, we provide
an implementation of a Vector class, representing the coordinates of a vector in a
multidimensional space. For example, in a three-dimensional space, we might wish
to represent a vector with coordinates 5,−2, 3 . Although it might be tempting to
directly use a Python list to represent those coordinates, a list does not provide an
appropriate abstraction for a geometric vector. In particular, if using lists, the ex-
pression [5, −2, 3] + [1, 4, 2] results in the list [5, −2, 3, 1, 4, 2]. When working
with vectors, if u = 5,−2, 3 and v = 1, 4, 2 , one would expect the expression,
u + v, to return a three-dimensional vector with coordinates 6, 2, 5 .
We therefore define a Vector class, in Code Fragment 2.4, that provides a better
abstraction for the notion of a geometric vector. Internally, our vector relies upon
an instance of a list, named coords, as its storage mechanism. By keeping the
internal list encapsulated, we can enforce the desired public interface for instances
of our class. A demonstration of supported behaviors includes the following:
v = Vector(5) # construct five-dimensional 0, 0, 0, 0, 0
v[1] = 23 # 0, 23, 0, 0, 0 (based on use of setitem )
v[−1] = 45 # 0, 23, 0, 0, 45 (also via setitem )
print(v[4]) # print 45 (via getitem )
u = v + v # 0, 46, 0, 0, 90 (via add )
print(u) # print 0, 46, 0, 0, 90
total = 0
for entry in v: # implicit iteration via len and getitem
total += entry
We implement many of the behaviors by trivially invoking a similar behavior
on the underlying list of coordinates. However, our implementation of add
is customized. Assuming the two operands are vectors with the same length, this
method creates a new vector and sets the coordinates of the new vector to be equal
to the respective sum of the operands’ elements.
It is interesting to note that the class definition, as given in Code Fragment 2.4,
automatically supports the syntax u = v + [5, 3, 10, −2, 1], resulting in a new
vector that is the element-by-element “sum” of the first vector and the list in-
stance. This is a result of Python’s polymorphism. Literally, “polymorphism”
means “many forms.” Although it is tempting to think of the other parameter of
our add method as another Vector instance, we never declared it as such.
Within the body, the only behaviors we rely on for parameter other is that it sup-
ports len(other) and access to other[j]. Therefore, our code executes when the
right-hand operand is a list of numbers (with matching length).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-99-320.jpg)
![78 Chapter 2. Object-Oriented Programming
1 class Vector:
2 ”””Represent a vector in a multidimensional space.”””
3
4 def init (self, d):
5 ”””Create d-dimensional vector of zeros.”””
6 self. coords = [0] d
7
8 def len (self):
9 ”””Return the dimension of the vector.”””
10 return len(self. coords)
11
12 def getitem (self, j):
13 ”””Return jth coordinate of vector.”””
14 return self. coords[j]
15
16 def setitem (self, j, val):
17 ”””Set jth coordinate of vector to given value.”””
18 self. coords[j] = val
19
20 def add (self, other):
21 ”””Return sum of two vectors.”””
22 if len(self) != len(other): # relies on len method
23 raise ValueError( dimensions must agree )
24 result = Vector(len(self)) # start with vector of zeros
25 for j in range(len(self)):
26 result[j] = self[j] + other[j]
27 return result
28
29 def eq (self, other):
30 ”””Return True if vector has same coordinates as other.”””
31 return self. coords == other. coords
32
33 def ne (self, other):
34 ”””Return True if vector differs from other.”””
35 return not self == other # rely on existing eq definition
36
37 def str (self):
38 ”””Produce string representation of vector.”””
39 return + str(self. coords)[1:−1] + # adapt list representation
Code Fragment 2.4: Definition of a simple Vector class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-100-320.jpg)
![2.3. Class Definitions 79
2.3.4 Iterators
Iteration is an important concept in the design of data structures. We introduced
Python’s mechanism for iteration in Section 1.8. In short, an iterator for a collec-
tion provides one key behavior: It supports a special method named next that
returns the next element of the collection, if any, or raises a StopIteration exception
to indicate that there are no further elements.
Fortunately, it is rare to have to directly implement an iterator class. Our pre-
ferred approach is the use of the generator syntax (also described in Section 1.8),
which automatically produces an iterator of yielded values.
Python also helps by providing an automatic iterator implementation for any
class that defines both len and getitem . To provide an instructive exam-
ple of a low-level iterator, Code Fragment 2.5 demonstrates just such an iterator
class that works on any collection that supports both len and getitem .
This class can be instantiated as SequenceIterator(data). It operates by keeping an
internal reference to the data sequence, as well as a current index into the sequence.
Each time next is called, the index is incremented, until reaching the end of
the sequence.
1 class SequenceIterator:
2 ”””An iterator for any of Python s sequence types.”””
3
4 def init (self, sequence):
5 ”””Create an iterator for the given sequence.”””
6 self. seq = sequence # keep a reference to the underlying data
7 self. k = −1 # will increment to 0 on first call to next
8
9 def next (self):
10 ”””Return the next element, or else raise StopIteration error.”””
11 self. k += 1 # advance to next index
12 if self. k len(self. seq):
13 return(self. seq[self. k]) # return the data element
14 else:
15 raise StopIteration( ) # there are no more elements
16
17 def iter (self):
18 ”””By convention, an iterator must return itself as an iterator.”””
19 return self
Code Fragment 2.5: An iterator class for any sequence type.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-101-320.jpg)
![80 Chapter 2. Object-Oriented Programming
2.3.5 Example: Range Class
As the final example for this section, we develop our own implementation of a
class that mimics Python’s built-in range class. Before introducing our class, we
discuss the history of the built-in version. Prior to Python 3 being released, range
was implemented as a function, and it returned a list instance with elements in
the specified range. For example, range(2, 10, 2) returned the list [2, 4, 6, 8].
However, a typical use of the function was to support a for-loop syntax, such as
for k in range(10000000). Unfortunately, this caused the instantiation and initial-
ization of a list with the range of numbers. That was an unnecessarily expensive
step, in terms of both time and memory usage.
The mechanism used to support ranges in Python 3 is entirely different (to be
fair, the “new” behavior existed in Python 2 under the name xrange). It uses a
strategy known as lazy evaluation. Rather than creating a new list instance, range
is a class that can effectively represent the desired range of elements without ever
storing them explicitly in memory. To better explore the built-in range class, we
recommend that you create an instance as r = range(8, 140, 5). The result is a
relatively lightweight object, an instance of the range class, that has only a few
behaviors. The syntax len(r) will report the number of elements that are in the
given range (27, in our example). A range also supports the getitem method,
so that syntax r[15] reports the sixteenth element in the range (as r[0] is the first
element). Because the class supports both len and getitem , it inherits
automatic support for iteration (see Section 2.3.4), which is why it is possible to
execute a for loop over a range.
At this point, we are ready to demonstrate our own version of such a class. Code
Fragment 2.6 provides a class we name Range (so as to clearly differentiate it from
built-in range). The biggest challenge in the implementation is properly computing
the number of elements that belong in the range, given the parameters sent by the
caller when constructing a range. By computing that value in the constructor, and
storing it as self. length, it becomes trivial to return it from the len method. To
properly implement a call to getitem (k), we simply take the starting value of
the range plus k times the step size (i.e., for k=0, we return the start value). There
are a few subtleties worth examining in the code:
• To properly support optional parameters, we rely on the technique described
on page 27, when discussing a functional version of range.
• We compute the number of elements in the range as
max(0, (stop − start + step − 1) // step)
It is worth testing this formula for both positive and negative step sizes.
• The getitem method properly supports negative indices by converting
an index −k to len(self)−k before computing the result.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-102-320.jpg)
![94 Chapter 2. Object-Oriented Programming
1 from abc import ABCMeta, abstractmethod # need these definitions
2
3 class Sequence(metaclass=ABCMeta):
4 ”””Our own version of collections.Sequence abstract base class.”””
5
6 @abstractmethod
7 def len (self):
8 ”””Return the length of the sequence.”””
9
10 @abstractmethod
11 def getitem (self, j):
12 ”””Return the element at index j of the sequence.”””
13
14 def contains (self, val):
15 ”””Return True if val found in the sequence; False otherwise.”””
16 for j in range(len(self)):
17 if self[j] == val: # found match
18 return True
19 return False
20
21 def index(self, val):
22 ”””Return leftmost index at which val is found (or raise ValueError).”””
23 for j in range(len(self)):
24 if self[j] == val: # leftmost match
25 return j
26 raise ValueError( value not in sequence ) # never found a match
27
28 def count(self, val):
29 ”””Return the number of elements equal to given value.”””
30 k = 0
31 for j in range(len(self)):
32 if self[j] == val: # found a match
33 k += 1
34 return k
Code Fragment 2.14: An abstract base class akin to collections.Sequence.
This implementation relies on two advanced Python techniques. The first is that
we declare the ABCMeta class of the abc module as a metaclass of our Sequence
class. A metaclass is different from a superclass, in that it provides a template for
the class definition itself. Specifically, the ABCMeta declaration assures that the
constructor for the class raises an error.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-116-320.jpg)
![2.4. Inheritance 95
The second advanced technique is the use of the @abstractmethod decorator
immediately before the len and getitem methods are declared. That de-
clares these two particular methods to be abstract, meaning that we do not provide
an implementation within our Sequence base class, but that we expect any concrete
subclasses to support those two methods. Python enforces this expectation, by dis-
allowing instantiation for any subclass that does not override the abstract methods
with concrete implementations.
The rest of the Sequence class definition provides tangible implementations for
other behaviors, under the assumption that the abstract len and getitem
methods will exist in a concrete subclass. If you carefully examine the source code,
the implementations of methods contains , index, and count do not rely on any
assumption about the self instances, other than that syntax len(self) and self[j] are
supported (by special methods len and getitem , respectively). Support
for iteration is automatic as well, as described in Section 2.3.4.
In the remainder of this book, we omit the formality of using the abc module.
If we need an “abstract” base class, we simply document the expectation that sub-
classes provide assumed functionality, without technical declaration of the methods
as abstract. But we will make use of the wonderful abstract base classes that are
defined within the collections module (such as Sequence). To use such a class, we
need only rely on standard inheritance techniques.
For example, our Range class, from Code Fragment 2.6 of Section 2.3.5, is an
example of a class that supports the len and getitem methods. But that
class does not support methods count or index. Had we originally declared it with
Sequence as a superclass, then it would also inherit the count and index methods.
The syntax for such a declaration would begin as:
class Range(collections.Sequence):
Finally, we emphasize that if a subclass provides its own implementation of
an inherited behaviors from a base class, the new definition overrides the inherited
one. This technique can be used when we have the ability to provide a more effi-
cient implementation for a behavior than is achieved by the generic approach. As
an example, the general implementation of contains for a sequence is based
on a loop used to search for the desired value. For our Range class, there is an
opportunity for a more efficient determination of containment. For example, it
is evident that the expression, 100000 in Range(0, 2000000, 100), should evalu-
ate to True, even without examining the individual elements of the range, because
the range starts with zero, has an increment of 100, and goes until 2 million; it
must include 100000, as that is a multiple of 100 that is between the start and
stop values. Exercise C-2.27 explores the goal of providing an implementation of
Range. contains that avoids the use of a (time-consuming) loop.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-117-320.jpg)
![102 Chapter 2. Object-Oriented Programming
red
green
blue
color
52
163
169
list list
warmtones palette
red
green
blue
color
43
124
249
Figure 2.10: A shallow copy of a list of colors.
This is a better situation than our first attempt, as we can legitimately add
or remove elements from palette without affecting warmtones. However, if we
edit a color instance from the palette list, we effectively change the contents of
warmtones. Although palette and warmtones are distinct lists, there remains indi-
rect aliasing, for example, with palette[0] and warmtones[0] as aliases for the same
color instance.
We prefer that palette be what is known as a deep copy of warmtones. In a
deep copy, the new copy references its own copies of those objects referenced by
the original version. (See Figure 2.11.)
blue
color
52
163
169
list
red
green
blue
color
43
124
249 red
green
blue
color
52
163
169
list
red
green
blue
color
43
124
249
warmtones palette
red
green
Figure 2.11: A deep copy of a list of colors.
Python’s copy Module
To create a deep copy, we could populate our list by explicitly making copies of
the original color instances, but this requires that we know how to make copies of
colors (rather than aliasing). Python provides a very convenient module, named
copy, that can produce both shallow copies and deep copies of arbitrary objects.
This module supports two functions: the copy function creates a shallow copy
of its argument, and the deepcopy function creates a deep copy of its argument.
After importing the module, we may create a deep copy for our example, as shown
in Figure 2.11, using the command:
palette = copy.deepcopy(warmtones)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-124-320.jpg)
![104 Chapter 2. Object-Oriented Programming
R-2.11 In Section 2.3.3, we note that our Vector class supports a syntax such as
v = u + [5, 3, 10, −2, 1], in which the sum of a vector and list returns
a new vector. However, the syntax v = [5, 3, 10, −2, 1] + u is illegal.
Explain how the Vector class definition can be revised so that this syntax
generates a new vector.
R-2.12 Implement the mul method for the Vector class of Section 2.3.3, so
that the expression v 3 returns a new vector with coordinates that are 3
times the respective coordinates of v.
R-2.13 Exercise R-2.12 asks for an implementation of mul , for the Vector
class of Section 2.3.3, to provide support for the syntax v 3. Implement
the rmul method, to provide additional support for syntax 3 v.
R-2.14 Implement the mul method for the Vector class of Section 2.3.3, so
that the expression u v returns a scalar that represents the dot product of
the vectors, that is, ∑d
i=1 ui ·vi.
R-2.15 The Vector class of Section 2.3.3 provides a constructor that takes an in-
teger d, and produces a d-dimensional vector with all coordinates equal to
0. Another convenient form for creating a new vector would be to send the
constructor a parameter that is some iterable type representing a sequence
of numbers, and to create a vector with dimension equal to the length of
that sequence and coordinates equal to the sequence values. For example,
Vector([4, 7, 5]) would produce a three-dimensional vector with coordi-
nates 4, 7, 5. Modify the constructor so that either of these forms is
acceptable; that is, if a single integer is sent, it produces a vector of that
dimension with all zeros, but if a sequence of numbers is provided, it pro-
duces a vector with coordinates based on that sequence.
R-2.16 Our Range class, from Section 2.3.5, relies on the formula
max(0, (stop − start + step − 1) // step)
to compute the number of elements in the range. It is not immediately ev-
ident why this formula provides the correct calculation, even if assuming
a positive step size. Justify this formula, in your own words.
R-2.17 Draw a class inheritance diagram for the following set of classes:
• Class Goat extends object and adds an instance variable tail and
methods milk() and jump().
• Class Pig extends object and adds an instance variable nose and
methods eat(food) and wallow().
• Class Horse extends object and adds instance variables height and
color, and methods run() and jump().
• Class Racer extends Horse and adds a method race().
• Class Equestrian extends Horse, adding an instance variable weight
and methods trot() and is trained().](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-126-320.jpg)
![108 Chapter 2. Object-Oriented Programming
Chapter Notes
For a broad overview of developments in computer science and engineering, we refer the
reader to The Computer Science and Engineering Handbook [96]. For more information
about the Therac-25 incident, please see the paper by Leveson and Turner [69].
The reader interested in studying object-oriented programming further, is referred to
the books by Booch [17], Budd [20], and Liskov and Guttag [71]. Liskov and Guttag
also provide a nice discussion of abstract data types, as does the survey paper by Cardelli
and Wegner [23] and the book chapter by Demurjian [33] in the The Computer Science
and Engineering Handbook [96]. Design patterns are described in the book by Gamma et
al. [41].
Books with specific focus on object-oriented programming in Python include those
by Goldwasser and Letscher [43] at the introductory level, and by Phillips [83] at a more
advanced level,](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-130-320.jpg)
![3.3. Asymptotic Analysis 123
3.3 Asymptotic Analysis
In algorithm analysis, we focus on the growth rate of the running time as a function
of the input size n, taking a “big-picture” approach. For example, it is often enough
just to know that the running time of an algorithm grows proportionally to n.
We analyze algorithms using a mathematical notation for functions that disre-
gards constant factors. Namely, we characterize the running times of algorithms
by using functions that map the size of the input, n, to values that correspond to
the main factor that determines the growth rate in terms of n. This approach re-
flects that each basic step in a pseudo-code description or a high-level language
implementation may correspond to a small number of primitive operations. Thus,
we can perform an analysis of an algorithm by estimating the number of primitive
operations executed up to a constant factor, rather than getting bogged down in
language-specific or hardware-specific analysis of the exact number of operations
that execute on the computer.
As a tangible example, we revisit the goal of finding the largest element of a
Python list; we first used this example when introducing for loops on page 21 of
Section 1.4.2. Code Fragment 3.1 presents a function named find max for this task.
1 def find max(data):
2 ”””Return the maximum element from a nonempty Python list.”””
3 biggest = data[0] # The initial value to beat
4 for val in data: # For each value:
5 if val biggest # if it is greater than the best so far,
6 biggest = val # we have found a new best (so far)
7 return biggest # When loop ends, biggest is the max
Code Fragment 3.1: A function that returns the maximum value of a Python list.
This is a classic example of an algorithm with a running time that grows pro-
portional to n, as the loop executes once for each data element, with some fixed
number of primitive operations executing for each pass. In the remainder of this
section, we provide a framework to formalize this claim.
3.3.1 The “Big-Oh” Notation
Let f(n) and g(n) be functions mapping positive integers to positive real numbers.
We say that f(n) is O(g(n)) if there is a real constant c 0 and an integer constant
n0 ≥ 1 such that
f(n) ≤ cg(n), for n ≥ n0.
This definition is often referred to as the “big-Oh” notation, for it is sometimes pro-
nounced as “f(n) is big-Oh of g(n).” Figure 3.5 illustrates the general definition.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-145-320.jpg)
![130 Chapter 3. Algorithm Analysis
If we must draw a line between efficient and inefficient algorithms, therefore,
it is natural to make this distinction be that between those algorithms running in
polynomial time and those running in exponential time. That is, make the distinc-
tion between algorithms with a running time that is O(nc), for some constant c 1,
and those with a running time that is O(bn), for some constant b 1. Like so many
notions we have discussed in this section, this too should be taken with a “grain of
salt,” for an algorithm running in O(n100) time should probably not be considered
“efficient.” Even so, the distinction between polynomial-time and exponential-time
algorithms is considered a robust measure of tractability.
3.3.3 Examples of Algorithm Analysis
Now that we have the big-Oh notation for doing algorithm analysis, let us give some
examples by characterizing the running time of some simple algorithms using this
notation. Moreover, in keeping with our earlier promise, we illustrate below how
each of the seven functions given earlier in this chapter can be used to characterize
the running time of an example algorithm.
Rather than use pseudo-code in this section, we give complete Python imple-
mentations for our examples. We use Python’s list class as the natural representa-
tion for an “array” of values. In Chapter 5, we will fully explore the underpinnings
of Python’s list class, and the efficiency of the various behaviors that it supports. In
this section, we rely on just a few of its behaviors, discussing their efficiencies as
introduced.
Constant-Time Operations
Given an instance, named data, of the Python list class, a call to the function,
len(data), is evaluated in constant time. This is a very simple algorithm because
the list class maintains, for each list, an instance variable that records the current
length of the list. This allows it to immediately report that length, rather than take
time to iteratively count each of the elements in the list. Using asymptotic notation,
we say that this function runs in O(1) time; that is, the running time of this function
is independent of the length, n, of the list.
Another central behavior of Python’s list class is that it allows access to an arbi-
trary element of the list using syntax, data[j], for integer index j. Because Python’s
lists are implemented as array-based sequences, references to a list’s elements are
stored in a consecutive block of memory. The jth element of the list can be found,
not by iterating through the list one element at a time, but by validating the index,
and using it as an offset into the underlying array. In turn, computer hardware sup-
ports constant-time access to an element based on its memory address. Therefore,
we say that the expression data[j] is evaluated in O(1) time for a Python list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-152-320.jpg)
![3.3. Asymptotic Analysis 131
Revisiting the Problem of Finding the Maximum of a Sequence
For our next example, we revisit the find max algorithm, given in Code Frag-
ment 3.1 on page 123, for finding the largest value in a sequence. Proposition 3.7
on page 125 claimed an O(n) run-time for the find max algorithm. Consistent with
our earlier analysis of syntax data[0], the initialization uses O(1) time. The loop
executes n times, and within each iteration, it performs one comparison and possi-
bly one assignment statement (as well as maintenance of the loop variable). Finally,
we note that the mechanism for enacting a return statement in Python uses O(1)
time. Combining these steps, we have that the find max function runs in O(n) time.
Further Analysis of the Maximum-Finding Algorithm
A more interesting question about find max is how many times we might update
the current “biggest” value. In the worst case, if the data is given to us in increasing
order, the biggest value is reassigned n − 1 times. But what if the input is given
to us in random order, with all orders equally likely; what would be the expected
number of times we update the biggest value in this case? To answer this question,
note that we update the current biggest in an iteration of the loop only if the current
element is bigger than all the elements that precede it. If the sequence is given to
us in random order, the probability that the jth element is the largest of the first j
elements is 1/j (assuming uniqueness). Hence, the expected number of times we
update the biggest (including initialization) is Hn = ∑n
j=1 1/j, which is known as
the nth Harmonic number. It turns out (see Proposition B.16) that Hn is O(logn).
Therefore, the expected number of times the biggest value is updated by find max
on a randomly ordered sequence is O(logn).
Prefix Averages
The next problem we consider is computing what are known as prefix averages
of a sequence of numbers. Namely, given a sequence S consisting of n num-
bers, we want to compute a sequence A such that A[ j] is the average of elements
S[0],...,S[ j], for j = 0,...,n−1, that is,
A[ j] =
∑
j
i=0 S[i]
j +1
.
Computing prefix averages has many applications in economics and statistics. For
example, given the year-by-year returns of a mutual fund, ordered from recent to
past, an investor will typically want to see the fund’s average annual returns for the
most recent year, the most recent three years, the most recent five years, and so on.
Likewise, given a stream of daily Web usage logs, a Web site manager may wish
to track average usage trends over various time periods. We analyze three different
implementations that solve this problem but with rather different running times.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-153-320.jpg)
![132 Chapter 3. Algorithm Analysis
A Quadratic-Time Algorithm
Our first algorithm for computing prefix averages, named prefix average1, is shown
in Code Fragment 3.2. It computes every element of A separately, using an inner
loop to compute the partial sum.
1 def prefix average1(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 for j in range(n):
6 total = 0 # begin computing S[0] + ... + S[j]
7 for i in range(j + 1):
8 total += S[i]
9 A[j] = total / (j+1) # record the average
10 return A
Code Fragment 3.2: Algorithm prefix average1.
In order to analyze the prefix average1 algorithm, we consider the various steps
that are executed.
• The statement, n = len(S), executes in constant time, as described at the
beginning of Section 3.3.3.
• The statement, A = [0] n, causes the creation and initialization of a Python
list with length n, and with all entries equal to zero. This uses a constant
number of primitive operations per element, and thus runs in O(n) time.
• There are two nested for loops, which are controlled, respectively, by coun-
ters j and i. The body of the outer loop, controlled by counter j, is ex-
ecuted n times, for j = 0,...,n − 1. Therefore, statements total = 0 and
A[j] = total / (j+1) are executed n times each. This implies that these two
statements, plus the management of counter j in the range, contribute a num-
ber of primitive operations proportional to n, that is, O(n) time.
• The body of the inner loop, which is controlled by counter i, is executed j+1
times, depending on the current value of the outer loop counter j. Thus, state-
ment total += S[i], in the inner loop, is executed 1+ 2+ 3+ ··· + n times.
By recalling Proposition 3.3, we know that 1+2+3+···+n = n(n+1)/2,
which implies that the statement in the inner loop contributes O(n2) time.
A similar argument can be done for the primitive operations associated with
maintaining counter i, which also take O(n2) time.
The running time of implementation prefix average1 is given by the sum of three
terms. The first and the second terms are O(n), and the third term is O(n2). By a
simple application of Proposition 3.9, the running time of prefix average1 is O(n2).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-154-320.jpg)
![3.3. Asymptotic Analysis 133
Our second implementation for computing prefix averages, prefix average2, is
presented in Code Fragment 3.3.
1 def prefix average2(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 for j in range(n):
6 A[j] = sum(S[0:j+1]) / (j+1) # record the average
7 return A
Code Fragment 3.3: Algorithm prefix average2.
This approach is essentially the same high-level algorithm as in prefix average1,
but we have replaced the inner loop by using the single expression sum(S[0:j+1])
to compute the partial sum, S[0]+··· +S[ j]. While the use of that function greatly
simplifies the presentation of the algorithm, it is worth asking how it affects the
efficiency. Asymptotically, this implementation is no better. Even though the ex-
pression, sum(S[0:j+1]), seems like a single command, it is a function call and
an evaluation of that function takes O(j + 1) time in this context. Technically, the
computation of the slice, S[0:j+1], also uses O(j + 1) time, as it constructs a new
list instance for storage. So the running time of prefix average2 is still dominated
by a series of steps that take time proportional to 1+2+3+···+n, and thus O(n2).
A Linear-Time Algorithm
Our final algorithm, prefix averages3, is given in Code Fragment 3.4. Just as with
our first two algorithms, we are interested in computing, for each j, the prefix sum
S[0] + S[1] + ··· + S[ j], denoted as total in our code, so that we can then compute
the prefix average A[j] =total / (j + 1). However, there is a key difference that
results in much greater efficiency.
1 def prefix average3(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 total = 0 # compute prefix sum as S[0] + S[1] + ...
6 for j in range(n):
7 total += S[j] # update prefix sum to include S[j]
8 A[j] = total / (j+1) # compute average based on current sum
9 return A
Code Fragment 3.4: Algorithm prefix average3.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-155-320.jpg)
![134 Chapter 3. Algorithm Analysis
In our first two algorithms, the prefix sum is computed anew for each value of j.
That contributed O(j) time for each j, leading to the quadratic behavior. In algo-
rithm prefix average3, we maintain the current prefix sum dynamically, effectively
computing S[0]+S[1]+··· +S[ j] as total + S[j], where value total is equal to the
sum S[0] + S[1] + ··· + S[ j − 1] computed by the previous pass of the loop over j.
The analysis of the running time of algorithm prefix average3 follows:
• Initializing variables n and total uses O(1) time.
• Initializing the list A uses O(n) time.
• There is a single for loop, which is controlled by counter j. The maintenance
of that counter by the range iterator contributes a total of O(n) time.
• The body of the loop is executed n times, for j = 0,...,n − 1. Thus, state-
ments total += S[j] and A[j] = total / (j+1) are executed n times each.
Since each of these statements uses O(1) time per iteration, their overall
contribution is O(n) time.
The running time of algorithm prefix average3 is given by the sum of the four
terms. The first is O(1) and the remaining three are O(n). By a simple application
of Proposition 3.9, the running time of prefix average3 is O(n), which is much
better than the quadratic time of algorithms prefix average1 and prefix average2.
Three-Way Set Disjointness
Suppose we are given three sequences of numbers, A, B, and C. We will assume
that no individual sequence contains duplicate values, but that there may be some
numbers that are in two or three of the sequences. The three-way set disjointness
problem is to determine if the intersection of the three sequences is empty, namely,
that there is no element x such that x ∈ A, x ∈ B, and x ∈ C. A simple Python
function to determine this property is given in Code Fragment 3.5.
1 def disjoint1(A, B, C):
2 ”””Return True if there is no element common to all three lists.”””
3 for a in A:
4 for b in B:
5 for c in C:
6 if a == b == c:
7 return False # we found a common value
8 return True # if we reach this, sets are disjoint
Code Fragment 3.5: Algorithm disjoint1 for testing three-way set disjointness.
This simple algorithm loops through each possible triple of values from the
three sets to see if those values are equivalent. If each of the original sets has size
n, then the worst-case running time of this function is O(n3).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-156-320.jpg)
![136 Chapter 3. Algorithm Analysis
1 def unique1(S):
2 ”””Return True if there are no duplicate elements in sequence S.”””
3 for j in range(len(S)):
4 for k in range(j+1, len(S)):
5 if S[j] == S[k]:
6 return False # found duplicate pair
7 return True # if we reach this, elements were unique
Code Fragment 3.7: Algorithm unique1 for testing element uniqueness.
those pairs refer to elements that are equivalent to each other. It does this using two
nested for loops, such that the first iteration of the outer loop causes n−1 iterations
of the inner loop, the second iteration of the outer loop causes n − 2 iterations of
the inner loop, and so on. Thus, the worst-case running time of this function is
proportional to
(n−1)+(n−2)+···+2+1,
which we recognize as the familiar O(n2) summation from Proposition 3.3.
Using Sorting as a Problem-Solving Tool
An even better algorithm for the element uniqueness problem is based on using
sorting as a problem-solving tool. In this case, by sorting the sequence of elements,
we are guaranteed that any duplicate elements will be placed next to each other.
Thus, to determine if there are any duplicates, all we need to do is perform a sin-
gle pass over the sorted sequence, looking for consecutive duplicates. A Python
implementation of this algorithm is as follows:
1 def unique2(S):
2 ”””Return True if there are no duplicate elements in sequence S.”””
3 temp = sorted(S) # create a sorted copy of S
4 for j in range(1, len(temp)):
5 if S[j−1] == S[j]:
6 return False # found duplicate pair
7 return True # if we reach this, elements were unique
Code Fragment 3.8: Algorithm unique2 for testing element uniqueness.
The built-in function, sorted, as described in Section 1.5.2, produces a copy of
the original list with elements in sorted order. It guarantees a worst-case running
time of O(nlogn); see Chapter 12 for a discussion of common sorting algorithms.
Once the data is sorted, the subsequent loop runs in O(n) time, and so the entire
unique2 algorithm runs in O(nlogn) time.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-158-320.jpg)
![140 Chapter 3. Algorithm Analysis
Loop Invariants
The final justification technique we discuss in this section is the loop invariant. To
prove some statement L about a loop is correct, define L in terms of a series of
smaller statements L0,L1,...,Lk, where:
1. The initial claim, L0, is true before the loop begins.
2. If Lj−1 is true before iteration j, then Lj will be true after iteration j.
3. The final statement, Lk, implies the desired statement L to be true.
Let us give a simple example of using a loop-invariant argument to justify the
correctness of an algorithm. In particular, we use a loop invariant to justify that
the function, find (see Code Fragment 3.9), finds the smallest index at which ele-
ment val occurs in sequence S.
1 def find(S, val):
2 ”””Return index j such that S[j] == val, or -1 if no such element.”””
3 n = len(S)
4 j = 0
5 while j n:
6 if S[j] == val:
7 return j # a match was found at index j
8 j += 1
9 return −1
Code Fragment 3.9: Algorithm for finding the first index at which a given element
occurs in a Python list.
To show that find is correct, we inductively define a series of statements, Lj,
that lead to the correctness of our algorithm. Specifically, we claim the following
is true at the beginning of iteration j of the while loop:
Lj: val is not equal to any of the first j elements of S.
This claim is true at the beginning of the first iteration of the loop, because j is 0
and there are no elements among the first 0 in S (this kind of a trivially true claim
is said to hold vacuously). In iteration j, we compare element val to element S[ j]
and return the index j if these two elements are equivalent, which is clearly correct
and completes the algorithm in this case. If the two elements val and S[ j] are not
equal, then we have found one more element not equal to val and we increment
the index j. Thus, the claim Lj will be true for this new value of j; hence, it is
true at the beginning of the next iteration. If the while loop terminates without
ever returning an index in S, then we have j = n. That is, Ln is true—there are no
elements of S equal to val. Therefore, the algorithm correctly returns −1 to indicate
that val is not in S.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-162-320.jpg)
![3.5. Exercises 143
1 def example1(S):
2 ”””Return the sum of the elements in sequence S.”””
3 n = len(S)
4 total = 0
5 for j in range(n): # loop from 0 to n-1
6 total += S[j]
7 return total
8
9 def example2(S):
10 ”””Return the sum of the elements with even index in sequence S.”””
11 n = len(S)
12 total = 0
13 for j in range(0, n, 2): # note the increment of 2
14 total += S[j]
15 return total
16
17 def example3(S):
18 ”””Return the sum of the prefix sums of sequence S.”””
19 n = len(S)
20 total = 0
21 for j in range(n): # loop from 0 to n-1
22 for k in range(1+j): # loop from 0 to j
23 total += S[k]
24 return total
25
26 def example4(S):
27 ”””Return the sum of the prefix sums of sequence S.”””
28 n = len(S)
29 prefix = 0
30 total = 0
31 for j in range(n):
32 prefix += S[j]
33 total += prefix
34 return total
35
36 def example5(A, B): # assume that A and B have equal length
37 ”””Return the number of elements in B equal to the sum of prefix sums in A.”””
38 n = len(A)
39 count = 0
40 for i in range(n): # loop from 0 to n-1
41 total = 0
42 for j in range(n): # loop from 0 to n-1
43 for k in range(1+j): # loop from 0 to j
44 total += A[k]
45 if B[i] == total:
46 count += 1
47 return count
Code Fragment 3.10: Some sample algorithms for analysis.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-165-320.jpg)
![144 Chapter 3. Algorithm Analysis
R-3.32 Given an n-element sequence S, Algorithm D calls Algorithm E on each
element S[i]. Algorithm E runs in O(i) time when it is called on element
S[i]. What is the worst-case running time of Algorithm D?
R-3.33 Al and Bob are arguing about their algorithms. Al claims his O(nlogn)-
time method is always faster than Bob’s O(n2)-time method. To settle the
issue, they perform a set of experiments. To Al’s dismay, they find that if
n 100, the O(n2)-time algorithm runs faster, and only when n ≥ 100 is
the O(nlogn)-time one better. Explain how this is possible.
R-3.34 There is a well-known city (which will go nameless here) whose inhabi-
tants have the reputation of enjoying a meal only if that meal is the best
they have ever experienced in their life. Otherwise, they hate it. Assum-
ing meal quality is distributed uniformly across a person’s life, describe
the expected number of times inhabitants of this city are happy with their
meals?
Creativity
C-3.35 Assuming it is possible to sort n numbers in O(nlogn) time, show that it
is possible to solve the three-way set disjointness problem in O(nlogn)
time.
C-3.36 Describe an efficient algorithm for finding the ten largest elements in a
sequence of size n. What is the running time of your algorithm?
C-3.37 Give an example of a positive function f(n) such that f(n) is neither O(n)
nor Ω(n).
C-3.38 Show that ∑n
i=1 i2 is O(n3).
C-3.39 Show that ∑n
i=1 i/2i 2. (Hint: Try to bound this sum term by term with
a geometric progression.)
C-3.40 Show that logb f(n) is Θ(log f(n)) if b 1 is a constant.
C-3.41 Describe an algorithm for finding both the minimum and maximum of n
numbers using fewer than 3n/2 comparisons. (Hint: First, construct a
group of candidate minimums and a group of candidate maximums.)
C-3.42 Bob built a Web site and gave the URL only to his n friends, which he
numbered from 1 to n. He told friend number i that he/she can visit the
Web site at most i times. Now Bob has a counter, C, keeping track of the
total number of visits to the site (but not the identities of who visits). What
is the minimum value for C such that Bob can know that one of his friends
has visited his/her maximum allowed number of times?
C-3.43 Draw a visual justification of Proposition 3.3 analogous to that of Fig-
ure 3.3(b) for the case when n is odd.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-166-320.jpg)
![3.5. Exercises 145
C-3.44 Communication security is extremely important in computer networks,
and one way many network protocols achieve security is to encrypt mes-
sages. Typical cryptographic schemes for the secure transmission of mes-
sages over such networks are based on the fact that no efficient algorithms
are known for factoring large integers. Hence, if we can represent a secret
message by a large prime number p, we can transmit, over the network,
the number r = p·q, where q p is another large prime number that acts
as the encryption key. An eavesdropper who obtains the transmitted num-
ber r on the network would have to factor r in order to figure out the secret
message p.
Using factoring to figure out a message is very difficult without knowing
the encryption key q. To understand why, consider the following naive
factoring algorithm:
for p in range(2,r):
if r % p == 0: # if p divides r
return The secret message is p!
a. Suppose that the eavesdropper uses the above algorithm and has a
computer that can carry out in 1 microsecond (1 millionth of a sec-
ond) a division between two integers of up to 100 bits each. Give an
estimate of the time that it will take in the worst case to decipher the
secret message p if the transmitted message r has 100 bits.
b. What is the worst-case time complexity of the above algorithm?
Since the input to the algorithm is just one large number r, assume
that the input size n is the number of bytes needed to store r, that is,
n = (log2 r)/8+1, and that each division takes time O(n).
C-3.45 A sequence S contains n − 1 unique integers in the range [0,n − 1], that
is, there is one number from this range that is not in S. Design an O(n)-
time algorithm for finding that number. You are only allowed to use O(1)
additional space besides the sequence S itself.
C-3.46 Al says he can prove that all sheep in a flock are the same color:
Base case: One sheep. It is clearly the same color as itself.
Induction step: A flock of n sheep. Take a sheep, a, out. The remaining
n − 1 are all the same color by induction. Now put sheep a back in and
take out a different sheep, b. By induction, the n− 1 sheep (now with a)
are all the same color. Therefore, all the sheep in the flock are the same
color. What is wrong with Al’s “justification”?
C-3.47 Let S be a set of n lines in the plane such that no two are parallel and
no three meet in the same point. Show, by induction, that the lines in S
determine Θ(n2) intersection points.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-167-320.jpg)
![146 Chapter 3. Algorithm Analysis
C-3.48 Consider the following “justification” that the Fibonacci function, F(n)
(see Proposition 3.20) is O(n):
Base case (n ≤ 2): F(1) = 1 and F(2) = 2.
Induction step (n 2): Assume claim true for n n. Consider n. F(n) =
F(n−2)+F(n−1). By induction, F(n−2) is O(n−2) and F(n−1) is
O(n−1). Then, F(n) is O((n−2)+(n−1)), by the identity presented in
Exercise R-3.11. Therefore, F(n) is O(n).
What is wrong with this “justification”?
C-3.49 Consider the Fibonacci function, F(n) (see Proposition 3.20). Show by
induction that F(n) is Ω((3/2)n).
C-3.50 Let p(x) be a polynomial of degree n, that is, p(x) = ∑n
i=0 aixi.
(a) Describe a simple O(n2)-time algorithm for computing p(x).
(b) Describe an O(nlogn)-time algorithm for computing p(x), based upon
a more efficient calculation of xi.
(c) Now consider a rewriting of p(x) as
p(x) = a0 +x(a1 +x(a2 +x(a3 +···+x(an−1 +xan)···))),
which is known as Horner’s method. Using the big-Oh notation, charac-
terize the number of arithmetic operations this method executes.
C-3.51 Show that the summation ∑n
i=1 logi is O(nlogn).
C-3.52 Show that the summation ∑n
i=1 logi is Ω(nlogn).
C-3.53 An evil king has n bottles of wine, and a spy has just poisoned one of
them. Unfortunately, they do not know which one it is. The poison is very
deadly; just one drop diluted even a billion to one will still kill. Even so,
it takes a full month for the poison to take effect. Design a scheme for
determining exactly which one of the wine bottles was poisoned in just
one month’s time while expending O(logn) taste testers.
C-3.54 A sequence S contains n integers taken from the interval [0,4n], with repe-
titions allowed. Describe an efficient algorithm for determining an integer
value k that occurs the most often in S. What is the running time of your
algorithm?
Projects
P-3.55 Perform an experimental analysis of the three algorithms prefix average1,
prefix average2, and prefix average3, from Section 3.3.3. Visualize their
running times as a function of the input size with a log-log chart.
P-3.56 Perform an experimental analysis that compares the relative running times
of the functions shown in Code Fragment 3.10.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-168-320.jpg)
![Chapter Notes 147
P-3.57 Perform experimental analysis to test the hypothesis that Python’s sorted
method runs in O(nlogn) time on average.
P-3.58 For each of the three algorithms, unique1, unique2, and unique3, which
solve the element uniqueness problem, perform an experimental analysis
to determine the largest value of n such that the given algorithm runs in
one minute or less.
Chapter Notes
The big-Oh notation has prompted several comments about its proper use [19, 49, 63].
Knuth [64, 63] defines it using the notation f(n) = O(g(n)), but says this “equality” is only
“one way.” We have chosen to take a more standard view of equality and view the big-Oh
notation as a set, following Brassard [19]. The reader interested in studying average-case
analysis is referred to the book chapter by Vitter and Flajolet [101]. For some additional
mathematical tools, please refer to Appendix B.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-169-320.jpg)
![4.1. Illustrative Examples 155
4.1.3 Binary Search
In this section, we describe a classic recursive algorithm, binary search, that is used
to efficiently locate a target value within a sorted sequence of n elements. This is
among the most important of computer algorithms, and it is the reason that we so
often store data in sorted order (as in Figure 4.4).
37
5
0 1 2 3 4 6 7 8 9 10 11 12 13 14 15
9
2 4 5 7 8 12 14 17 19 22 25 27 28 33
Figure 4.4: Values stored in sorted order within an indexable sequence, such as a
Python list. The numbers at top are the indices.
When the sequence is unsorted, the standard approach to search for a target
value is to use a loop to examine every element, until either finding the target or
exhausting the data set. This is known as the sequential search algorithm. This
algorithm runs in O(n) time (i.e., linear time) since every element is inspected in
the worst case.
When the sequence is sorted and indexable, there is a much more efficient
algorithm. (For intuition, think about how you would accomplish this task by
hand!) For any index j, we know that all the values stored at indices 0,..., j − 1
are less than or equal to the value at index j, and all the values stored at indices
j +1,...,n−1 are greater than or equal to that at index j. This observation allows
us to quickly “home in” on a search target using a variant of the children’s game
“high-low.” We call an element of the sequence a candidate if, at the current stage
of the search, we cannot rule out that this item matches the target. The algorithm
maintains two parameters, low and high, such that all the candidate entries have
index at least low and at most high. Initially, low = 0 and high = n− 1. We then
compare the target value to the median candidate, that is, the item data[mid] with
index
mid = (low +high)/2.
We consider three cases:
• If the target equals data[mid], then we have found the item we are looking
for, and the search terminates successfully.
• If target data[mid], then we recur on the first half of the sequence, that is,
on the interval of indices from low to mid−1.
• If target data[mid], then we recur on the second half of the sequence, that
is, on the interval of indices from mid+1 to high.
An unsuccessful search occurs if low high, as the interval [low,high] is empty.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-177-320.jpg)
![156 Chapter 4. Recursion
This algorithm is known as binary search. We give a Python implementation
in Code Fragment 4.3, and an illustration of the execution of the algorithm in Fig-
ure 4.5. Whereas sequential search runs in O(n) time, the more efficient binary
search runs in O(logn) time. This is a significant improvement, given that if n
is one billion, logn is only 30. (We defer our formal analysis of binary search’s
running time to Proposition 4.2 in Section 4.2.)
1 def binary search(data, target, low, high):
2 ”””Return True if target is found in indicated portion of a Python list.
3
4 The search only considers the portion from data[low] to data[high] inclusive.
5 ”””
6 if low high:
7 return False # interval is empty; no match
8 else:
9 mid = (low + high) // 2
10 if target == data[mid]: # found a match
11 return True
12 elif target data[mid]:
13 # recur on the portion left of the middle
14 return binary search(data, target, low, mid − 1)
15 else:
16 # recur on the portion right of the middle
17 return binary search(data, target, mid + 1, high)
Code Fragment 4.3: An implementation of the binary search algorithm.
mid
high
high
low
low mid
low mid
low=mid=high
high
14 19 22 25 27 28 33 37
6 7 8 9 10 11 12 13 14 15
7
5
4
2 9
8
9
2 4 5 7 8 12 14 17
37
33
28
27
25
22
19
9
2 4 5 7 8 12 14 17 19 22 25 27 28 33 37
19 22 25 27 28 33 37
5
0 1 2 3 4
17
14
12
9
2 4 5 7 8 12 17
Figure 4.5: Example of a binary search for target value 22.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-178-320.jpg)
![4.1. Illustrative Examples 159
Python’s os Module
To provide a Python implementation of a recursive algorithm for computing disk
usage, we rely on Python’s os module, which provides robust tools for interacting
with the operating system during the execution of a program. This is an extensive
library, but we will only need the following four functions:
• os.path.getsize(path)
Return the immediate disk usage (measured in bytes) for the file or directory
that is identified by the string path (e.g., /user/rt/courses).
• os.path.isdir(path)
Return True if entry designated by string path is a directory; False otherwise.
• os.listdir(path)
Return a list of strings that are the names of all entries within a directory
designated by string path. In our sample file system, if the parameter is
/user/rt/courses, this returns the list [ cs016 , cs252 ].
• os.path.join(path, filename)
Compose the path string and filename string using an appropriate operating
system separator between the two (e.g., the / character for a Unix/Linux
system, and the character for Windows). Return the string that represents
the full path to the file.
Python Implementation
With use of the os module, we now convert the algorithm from Code Fragment 4.4
into the Python implementation of Code Fragment 4.5.
1 import os
2
3 def disk usage(path):
4 ”””Return the number of bytes used by a file/folder and any descendents.”””
5 total = os.path.getsize(path) # account for direct usage
6 if os.path.isdir(path): # if this is a directory,
7 for filename in os.listdir(path): # then for each child:
8 childpath = os.path.join(path, filename) # compose full path to child
9 total += disk usage(childpath) # add child’s usage to total
10
11 print ( {0:7} .format(total), path) # descriptive output (optional)
12 return total # return the grand total
Code Fragment 4.5: A recursive function for reporting disk usage of a file system.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-181-320.jpg)
![4.3. Recursion Run Amok 165
4.3 Recursion Run Amok
Although recursion is a very powerful tool, it can easily be misused in various ways.
In this section, we examine several problems in which a poorly implemented recur-
sion causes drastic inefficiency, and we discuss some strategies for recognizing and
avoid such pitfalls.
We begin by revisiting the element uniqueness problem, defined on page 135
of Section 3.3.3. We can use the following recursive formulation to determine if
all n elements of a sequence are unique. As a base case, when n = 1, the elements
are trivially unique. For n ≥ 2, the elements are unique if and only if the first n−1
elements are unique, the last n−1 items are unique, and the first and last elements
are different (as that is the only pair that was not already checked as a subcase). A
recursive implementation based on this idea is given in Code Fragment 4.6, named
unique3 (to differentiate it from unique1 and unique2 from Chapter 3).
1 def unique3(S, start, stop):
2 ”””Return True if there are no duplicate elements in slice S[start:stop].”””
3 if stop − start = 1: return True # at most one item
4 elif not unique(S, start, stop−1): return False # first part has duplicate
5 elif not unique(S, start+1, stop): return False # second part has duplicate
6 else: return S[start] != S[stop−1] # do first and last differ?
Code Fragment 4.6: Recursive unique3 for testing element uniqueness.
Unfortunately, this is a terribly inefficient use of recursion. The nonrecursive
part of each call uses O(1) time, so the overall running time will be proportional to
the total number of recursive invocations. To analyze the problem, we let n denote
the number of entries under consideration, that is, let n= stop − start.
If n = 1, then the running time of unique3 is O(1), since there are no recursive
calls for this case. In the general case, the important observation is that a single call
to unique3 for a problem of size n may result in two recursive calls on problems of
size n − 1. Those two calls with size n − 1 could in turn result in four calls (two
each) with a range of size n − 2, and thus eight calls with size n − 3 and so on.
Thus, in the worst case, the total number of function calls is given by the geometric
summation
1+2+4+···+2n−1
,
which is equal to 2n − 1 by Proposition 3.5. Thus, the running time of function
unique3 is O(2n). This is an incredibly inefficient function for solving the ele-
ment uniqueness problem. Its inefficiency comes not from the fact that it uses
recursion—it comes from the fact that it uses recursion poorly, which is something
we address in Exercise C-4.11.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-187-320.jpg)
![170 Chapter 4. Recursion
A recursive algorithm for computing the sum of a sequence of numbers based
on this intuition is implemented in Code Fragment 4.9.
1 def linear sum(S, n):
2 ”””Return the sum of the first n numbers of sequence S.”””
3 if n == 0:
4 return 0
5 else:
6 return linear sum(S, n−1) + S[n−1]
Code Fragment 4.9: Summing the elements of a sequence using linear recursion.
A recursion trace of the linear sum function for a small example is given in
Figure 4.10. For an input of size n, the linear sum algorithm makes n+1 function
calls. Hence, it will take O(n) time, because it spends a constant amount of time
performing the nonrecursive part of each call. Moreover, we can also see that the
memory space used by the algorithm (in addition to the sequence S) is also O(n), as
we use a constant amount of memory space for each of the n+1 activation records
in the trace at the time we make the final recursive call (with n = 0).
return 15 + S[4] = 15 + 8 = 23
linear sum(S, 5)
linear sum(S, 4)
linear sum(S, 3)
linear sum(S, 2)
linear sum(S, 1)
linear sum(S, 0)
return 0
return 0 + S[0] = 0 + 4 = 4
return 4 + S[1] = 4 + 3 = 7
return 7 + S[2] = 7 + 6 = 13
return 13 + S[3] = 13 + 2 = 15
Figure 4.10: Recursion trace for an execution of linear sum(S, 5) with input pa-
rameter S = [4, 3, 6, 2, 8].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-192-320.jpg)
![4.4. Further Examples of Recursion 171
Reversing a Sequence with Recursion
Next, let us consider the problem of reversing the n elements of a sequence, S, so
that the first element becomes the last, the second element becomes second to the
last, and so on. We can solve this problem using linear recursion, by observing that
the reversal of a sequence can be achieved by swapping the first and last elements
and then recursively reversing the remaining elements. We present an implemen-
tation of this algorithm in Code Fragment 4.10, using the convention that the first
time we call this algorithm we do so as reverse(S, 0, len(S)).
1 def reverse(S, start, stop):
2 ”””Reverse elements in implicit slice S[start:stop].”””
3 if start stop − 1: # if at least 2 elements:
4 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last
5 reverse(S, start+1, stop−1) # recur on rest
Code Fragment 4.10: Reversing the elements of a sequence using linear recursion.
Note that there are two implicit base case scenarios: When start == stop, the
implicit range is empty, and when start == stop−1, the implicit range has only
one element. In either of these cases, there is no need for action, as a sequence
with zero elements or one element is trivially equal to its reversal. When otherwise
invoking recursion, we are guaranteed to make progress towards a base case, as
the difference, stop−start, decreases by two with each call (see Figure 4.11). If n
is even, we will eventually reach the start == stop case, and if n is odd, we will
eventually reach the start == stop − 1 case.
The above argument implies that the recursive algorithm of Code Fragment 4.10
is guaranteed to terminate after a total of 1+
n
2
recursive calls. Since each call
involves a constant amount of work, the entire process runs in O(n) time.
6
3 6 2 8 9 5
5 9 8 2 6 3 4
5 3 6 2 8 9 4
5 9 8 2 6 3 4
5 9 6 2 8 3 4
5
0 1 2 3 4
4
Figure 4.11: A trace of the recursion for reversing a sequence. The shaded portion
has yet to be reversed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-193-320.jpg)
![174 Chapter 4. Recursion
4.4.2 Binary Recursion
When a function makes two recursive calls, we say that it uses binary recursion.
We have already seen several examples of binary recursion, most notably when
drawing the English ruler (Section 4.1.2), or in the bad fibonacci function of Sec-
tion 4.3. As another application of binary recursion, let us revisit the problem of
summing the n elements of a sequence, S, of numbers. Computing the sum of one
or zero elements is trivial. With two or more elements, we can recursively com-
pute the sum of the first half, and the sum of the second half, and add these sums
together. Our implementation of such an algorithm, in Code Fragment 4.13, is
initially invoked as binary sum(A, 0, len(A)).
1 def binary sum(S, start, stop):
2 ”””Return the sum of the numbers in implicit slice S[start:stop].”””
3 if start = stop: # zero elements in slice
4 return 0
5 elif start == stop−1: # one element in slice
6 return S[start]
7 else: # two or more elements in slice
8 mid = (start + stop) // 2
9 return binary sum(S, start, mid) + binary sum(S, mid, stop)
Code Fragment 4.13: Summing the elements of a sequence using binary recursion.
To analyze algorithm binary sum, we consider, for simplicity, the case where
n is a power of two. Figure 4.13 shows the recursion trace of an execution of
binary sum(0, 8). We label each box with the values of parameters start:stop
for that call. The size of the range is divided in half at each recursive call, and
so the depth of the recursion is 1 + log2 n. Therefore, binary sum uses O(logn)
amount of additional space, which is a big improvement over the O(n) space used
by the linear sum function of Code Fragment 4.9. However, the running time of
binary sum is O(n), as there are 2n−1 function calls, each requiring constant time.
0:1 1:2 2:3 4:5 6:7 7:8
3:4 5:6
0:2 4:6 6:8
2:4
0:4 4:8
0:8
Figure 4.13: Recursion trace for the execution of binary sum(0, 8).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-196-320.jpg)
![4.6. Eliminating Tail Recursion 179
1 def binary search iterative(data, target):
2 ”””Return True if target is found in the given Python list.”””
3 low = 0
4 high = len(data)−1
5 while low = high:
6 mid = (low + high) // 2
7 if target == data[mid]: # found a match
8 return True
9 elif target data[mid]:
10 high = mid − 1 # only consider values left of mid
11 else:
12 low = mid + 1 # only consider values right of mid
13 return False # loop ended without success
Code Fragment 4.15: A nonrecursive implementation of binary search.
the target, or we narrow the range of the candidate subsequence. Where we made
the recursive call binary search(data, target, low, mid −1) in the original version,
we simply replace high = mid − 1 in our new version and then continue to the
next iteration of the loop. Our original base case condition of low high has
simply been replaced by the opposite loop condition while low = high. In our
new implementation, we return False to designate a failed search if the while loop
ends (that is, without having ever returned True from within).
We can similarly develop a nonrecursive implementation (Code Fragment 4.16)
of the original recursive reverse method of Code Fragment 4.10.
1 def reverse iterative(S):
2 ”””Reverse elements in sequence S.”””
3 start, stop = 0, len(S)
4 while start stop − 1:
5 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last
6 start, stop = start + 1, stop − 1 # narrow the range
Code Fragment 4.16: Reversing the elements of a sequence using iteration.
In this new version, we update the values start and stop during each pass of the
loop, exiting once we reach the case of having one or less elements in that range.
Many other linear recursions can be expressed quite efficiently with iteration,
even if they were not formally tail recursions. For example, there are trivial non-
recursive implementations for computing factorials, summing elements of a se-
quence, or computing Fibonacci numbers efficiently. In fact, our implementation
of a Fibonacci generator, from Section 1.8, produces each subsequent value in O(1)
time, and thus takes O(n) time to generate the nth entry in the series.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-201-320.jpg)
![180 Chapter 4. Recursion
4.7 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-4.1 Describe a recursive algorithm for finding the maximum element in a se-
quence, S, of n elements. What is your running time and space usage?
R-4.2 Draw the recursion trace for the computation of power(2,5), using the
traditional function implemented in Code Fragment 4.11.
R-4.3 Draw the recursion trace for the computation of power(2,18), using the
repeated squaring algorithm, as implemented in Code Fragment 4.12.
R-4.4 Draw the recursion trace for the execution of function reverse(S, 0, 5)
(Code Fragment 4.10) on S = [4, 3, 6, 2, 6].
R-4.5 Draw the recursion trace for the execution of function PuzzleSolve(3,S,U)
(Code Fragment 4.14), where S is empty and U = {a,b,c,d}.
R-4.6 Describe a recursive function for computing the nth Harmonic number,
Hn = ∑n
i=1 1/i.
R-4.7 Describe a recursive function for converting a string of digits into the in-
teger it represents. For example, 13531 represents the integer 13,531.
R-4.8 Isabel has an interesting way of summing up the values in a sequence A of
n integers, where n is a power of two. She creates a new sequence B of half
the size of A and sets B[i] = A[2i]+A[2i+1], for i = 0,1,...,(n/2)−1. If
B has size 1, then she outputs B[0]. Otherwise, she replaces A with B, and
repeats the process. What is the running time of her algorithm?
Creativity
C-4.9 Write a short recursive Python function that finds the minimum and max-
imum values in a sequence without using any loops.
C-4.10 Describe a recursive algorithm to compute the integer part of the base-two
logarithm of n using only addition and integer division.
C-4.11 Describe an efficient recursive function for solving the element unique-
ness problem, which runs in time that is at most O(n2) in the worst case
without using sorting.
C-4.12 Give a recursive algorithm to compute the product of two positive integers,
m and n, using only addition and subtraction.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-202-320.jpg)
![182 Chapter 4. Recursion
C-4.21 Suppose you are given an n-element sequence, S, containing distinct in-
tegers that are listed in increasing order. Given a number k, describe a
recursive algorithm to find two integers in S that sum to k, if such a pair
exists. What is the running time of your algorithm?
C-4.22 Develop a nonrecursive implementation of the version of power from
Code Fragment 4.12 that uses repeated squaring.
Projects
P-4.23 Implement a recursive function with signature find(path, filename) that
reports all entries of the file system rooted at the given path having the
given file name.
P-4.24 Write a program for solving summation puzzles by enumerating and test-
ing all possible configurations. Using your program, solve the three puz-
zles given in Section 4.4.3.
P-4.25 Provide a nonrecursive implementation of the draw interval function for
the English ruler project of Section 4.1.2. There should be precisely 2c −1
lines of output if c represents the length of the center tick. If incrementing
a counter from 0 to 2c −2, the number of dashes for each tick line should
be exactly one more than the number of consecutive 1’s at the end of the
binary representation of the counter.
P-4.26 Write a program that can solve instances of the Tower of Hanoi problem
(from Exercise C-4.14).
P-4.27 Python’s os module provides a function with signature walk(path) that
is a generator yielding the tuple (dirpath, dirnames, filenames) for each
subdirectory of the directory identified by string path, such that string
dirpath is the full path to the subdirectory, dirnames is a list of the names
of the subdirectories within dirpath, and filenames is a list of the names
of non-directory entries of dirpath. For example, when visiting the cs016
subdirectory of the file system shown in Figure 4.6, the walk would yield
( /user/rt/courses/cs016 , [ homeworks , programs ], [ grades ]).
Give your own implementation of such a walk function.
Chapter Notes
The use of recursion in programs belongs to the folkore of computer science (for example,
see the article of Dijkstra [36]). It is also at the heart of functional programming languages
(for example, see the classic book by Abelson, Sussman, and Sussman [1]). Interestingly,
binary search was first published in 1946, but was not published in a fully correct form
until 1962. For further discussions on lessons learned, please see papers by Bentley [14]
and Lesuisse [68].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-204-320.jpg)
![184 Chapter 5. Array-Based Sequences
5.1 Python’s Sequence Types
In this chapter, we explore Python’s various “sequence” classes, namely the built-
in list, tuple, and str classes. There is significant commonality between these
classes, most notably: each supports indexing to access an individual element of a
sequence, using a syntax such as seq[k], and each uses a low-level concept known
as an array to represent the sequence. However, there are significant differences in
the abstractions that these classes represent, and in the way that instances of these
classes are represented internally by Python. Because these classes are used so
widely in Python programs, and because they will become building blocks upon
which we will develop more complex data structures, it is imperative that we estab-
lish a clear understanding of both the public behavior and inner workings of these
classes.
Public Behaviors
A proper understanding of the outward semantics for a class is a necessity for a
good programmer. While the basic usage of lists, strings, and tuples may seem
straightforward, there are several important subtleties regarding the behaviors as-
sociated with these classes (such as what it means to make a copy of a sequence, or
to take a slice of a sequence). Having a misunderstanding of a behavior can easily
lead to inadvertent bugs in a program. Therefore, we establish an accurate men-
tal model for each of these classes. These images will help when exploring more
advanced usage, such as representing a multidimensional data set as a list of lists.
Implementation Details
A focus on the internal implementations of these classes seems to go against our
stated principles of object-oriented programming. In Section 2.1.2, we emphasized
the principle of encapsulation, noting that the user of a class need not know about
the internal details of the implementation. While it is true that one only needs to
understand the syntax and semantics of a class’s public interface in order to be able
to write legal and correct code that uses instances of the class, the efficiency of a
program depends greatly on the efficiency of the components upon which it relies.
Asymptotic and Experimental Analyses
In describing the efficiency of various operations for Python’s sequence classes,
we will rely on the formal asymptotic analysis notations established in Chapter 3.
We will also perform experimental analyses of the primary operations to provide
empirical evidence that is consistent with the more theoretical asymptotic analyses.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-206-320.jpg)
![5.2. Low-Level Arrays 187
5.2.1 Referential Arrays
As another motivating example, assume that we want a medical information system
to keep track of the patients currently assigned to beds in a certain hospital. If we
assume that the hospital has 200 beds, and conveniently that those beds are num-
bered from 0 to 199, we might consider using an array-based structure to maintain
the names of the patients currently assigned to those beds. For example, in Python
we might use a list of names, such as:
[ Rene , Joseph , Janet , Jonas , Helen , Virginia , ... ]
To represent such a list with an array, Python must adhere to the requirement that
each cell of the array use the same number of bytes. Yet the elements are strings,
and strings naturally have different lengths. Python could attempt to reserve enough
space for each cell to hold the maximum length string (not just of currently stored
strings, but of any string we might ever want to store), but that would be wasteful.
Instead, Python represents a list or tuple instance using an internal storage
mechanism of an array of object references. At the lowest level, what is stored
is a consecutive sequence of memory addresses at which the elements of the se-
quence reside. A high-level diagram of such a list is shown in Figure 5.4.
0 3
1 2 5
4
Rene Virginia
Joseph Helen
Jonas
Janet
Figure 5.4: An array storing references to strings.
Although the relative size of the individual elements may vary, the number of
bits used to store the memory address of each element is fixed (e.g., 64-bits per
address). In this way, Python can support constant-time access to a list or tuple
element based on its index.
In Figure 5.4, we characterize a list of strings that are the names of the patients
in a hospital. It is more likely that a medical information system would manage
more comprehensive information on each patient, perhaps represented as an in-
stance of a Patient class. From the perspective of the list implementation, the same
principle applies: The list will simply keep a sequence of references to those ob-
jects. Note as well that a reference to the None object can be used as an element
of the list to represent an empty bed in the hospital.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-209-320.jpg)
![188 Chapter 5. Array-Based Sequences
The fact that lists and tuples are referential structures is significant to the se-
mantics of these classes. A single list instance may include multiple references
to the same object as elements of the list, and it is possible for a single object to
be an element of two or more lists, as those lists simply store references back to
that object. As an example, when you compute a slice of a list, the result is a new
list instance, but that new list has references to the same elements that are in the
original list, as portrayed in Figure 5.5.
3 4 5 6 7
0 1 2
0 1 2
primes:
temp:
3 11
5
2 19
17
13
7
Figure 5.5: The result of the command temp = primes[3:6].
When the elements of the list are immutable objects, as with the integer in-
stances in Figure 5.5, the fact that the two lists share elements is not that signifi-
cant, as neither of the lists can cause a change to the shared object. If, for example,
the command temp[2] = 15 were executed from this configuration, that does not
change the existing integer object; it changes the reference in cell 2 of the temp list
to reference a different object. The resulting configuration is shown in Figure 5.6.
3 4 5 6 7
0 1 2
0 1 2
primes:
temp:
13
11
5
2
15
3 19
17
7
Figure 5.6: The result of the command temp[2] = 15 upon the configuration por-
trayed in Figure 5.5.
The same semantics is demonstrated when making a new list as a copy of an
existing one, with a syntax such as backup = list(primes). This produces a new
list that is a shallow copy (see Section 2.6), in that it references the same elements
as in the first list. With immutable elements, this point is moot. If the contents of
the list were of a mutable type, a deep copy, meaning a new list with new elements,
can be produced by using the deepcopy function from the copy module.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-210-320.jpg)
![5.2. Low-Level Arrays 189
As a more striking example, it is a common practice in Python to initialize an
array of integers using a syntax such as counters = [0] 8. This syntax produces
a list of length eight, with all eight elements being the value zero. Technically, all
eight cells of the list reference the same object, as portrayed in Figure 5.7.
4 5 6 7
0 1 2 3
counters:
0
Figure 5.7: The result of the command data = [0] 8.
At first glance, the extreme level of aliasing in this configuration may seem
alarming. However, we rely on the fact that the referenced integer is immutable.
Even a command such as counters[2] += 1 does not technically change the value
of the existing integer instance. This computes a new integer, with value 0+1, and
sets cell 2 to reference the newly computed value. The resulting configuration is
shown in Figure 5.8.
4 5 6 7
0 1 2 3
counters:
0
1
Figure 5.8: The result of command data[2] += 1 upon the list from Figure 5.7.
As a final manifestation of the referential nature of lists, we note that the extend
command is used to add all elements from one list to the end of another list. The
extended list does not receive copies of those elements, it receives references to
those elements. Figure 5.9 portrays the effect of a call to extend.
3 4 5 6 7 8 10
9
2
1
0
0 1 2
primes:
extras:
29 31
7 19
3 17
13
11
2 5 23
Figure 5.9: The effect of command primes.extend(extras), shown in light gray.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-211-320.jpg)
![5.2. Low-Level Arrays 191
Primary support for compact arrays is in a module named array. That module
defines a class, also named array, providing compact storage for arrays of primitive
data types. A portrayal of such an array of integers is shown in Figure 5.10.
3 4 5 6 7
0 1 2
17
5
2 3 7 11 13 19
Figure 5.10: Integers stored compactly as elements of a Python array.
The public interface for the array class conforms mostly to that of a Python list.
However, the constructor for the array class requires a type code as a first parameter,
which is a character that designates the type of data that will be stored in the array.
As a tangible example, the type code, i , designates an array of (signed) integers,
typically represented using at least 16-bits each. We can declare the array shown in
Figure 5.10 as,
primes = array( i , [2, 3, 5, 7, 11, 13, 17, 19])
The type code allows the interpreter to determine precisely how many bits are
needed per element of the array. The type codes supported by the array module,
as shown in Table 5.1, are formally based upon the native data types used by the
C programming language (the language in which the the most widely used distri-
bution of Python is implemented). The precise number of bits for the C data types
is system-dependent, but typical ranges are shown in the table.
Code C Data Type Typical Number of Bytes
b signed char 1
B unsigned char 1
u Unicode char 2 or 4
h signed short int 2
H unsigned short int 2
i signed int 2 or 4
I unsigned int 2 or 4
l signed long int 4
L unsigned long int 4
f float 4
d float 8
Table 5.1: Type codes supported by the array module.
The array module does not provide support for making compact arrays of user-
defined data types. Compact arrays of such structures can be created with the lower-
level support of a module named ctypes. (See Section 5.3.1 for more discussion of
the ctypes module.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-213-320.jpg)
![5.3. Dynamic Arrays and Amortization 193
1 import sys # provides getsizeof function
2 data = [ ]
3 for k in range(n): # NOTE: must fix choice of n
4 a = len(data) # number of elements
5 b = sys.getsizeof(data) # actual size in bytes
6 print( Length: {0:3d}; Size in bytes: {1:4d} .format(a, b))
7 data.append(None) # increase length by one
Code Fragment 5.1: An experiment to explore the relationship between a list’s
length and its underlying size in Python.
Length: 0; Size in bytes: 72
Length: 1; Size in bytes: 104
Length: 2; Size in bytes: 104
Length: 3; Size in bytes: 104
Length: 4; Size in bytes: 104
Length: 5; Size in bytes: 136
Length: 6; Size in bytes: 136
Length: 7; Size in bytes: 136
Length: 8; Size in bytes: 136
Length: 9; Size in bytes: 200
Length: 10; Size in bytes: 200
Length: 11; Size in bytes: 200
Length: 12; Size in bytes: 200
Length: 13; Size in bytes: 200
Length: 14; Size in bytes: 200
Length: 15; Size in bytes: 200
Length: 16; Size in bytes: 200
Length: 17; Size in bytes: 272
Length: 18; Size in bytes: 272
Length: 19; Size in bytes: 272
Length: 20; Size in bytes: 272
Length: 21; Size in bytes: 272
Length: 22; Size in bytes: 272
Length: 23; Size in bytes: 272
Length: 24; Size in bytes: 272
Length: 25; Size in bytes: 272
Length: 26; Size in bytes: 352
Code Fragment 5.2: Sample output from the experiment of Code Fragment 5.1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-215-320.jpg)
![5.3. Dynamic Arrays and Amortization 195
5.3.1 Implementing a Dynamic Array
Although the Python list class provides a highly optimized implementation of dy-
namic arrays, upon which we rely for the remainder of this book, it is instructive to
see how such a class might be implemented.
The key is to provide means to grow the array A that stores the elements of a
list. Of course, we cannot actually grow that array, as its capacity is fixed. If an
element is appended to a list at a time when the underlying array is full, we perform
the following steps:
1. Allocate a new array B with larger capacity.
2. Set B[i] = A[i], for i = 0,...,n−1, where n denotes current number of items.
3. Set A = B, that is, we henceforth use B as the array supporting the list.
4. Insert the new element in the new array.
An illustration of this process is shown in Figure 5.12.
B
A A
B A
(a) (b) (c)
Figure 5.12: An illustration of the three steps for “growing” a dynamic array: (a)
create new array B; (b) store elements of A in B; (c) reassign reference A to the new
array. Not shown is the future garbage collection of the old array, or the insertion
of the new element.
The remaining issue to consider is how large of a new array to create. A com-
monly used rule is for the new array to have twice the capacity of the existing array
that has been filled. In Section 5.3.2, we will provide a mathematical analysis to
justify such a choice.
In Code Fragment 5.3, we offer a concrete implementation of dynamic arrays
in Python. Our DynamicArray class is designed using ideas described in this sec-
tion. While consistent with the interface of a Python list class, we provide only
limited functionality in the form of an append method, and accessors len and
getitem . Support for creating low-level arrays is provided by a module named
ctypes. Because we will not typically use such a low-level structure in the remain-
der of this book, we omit a detailed explanation of the ctypes module. Instead,
we wrap the necessary command for declaring the raw array within a private util-
ity method make array. The hallmark expansion procedure is performed in our
nonpublic resize method.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-217-320.jpg)
![196 Chapter 5. Array-Based Sequences
1 import ctypes # provides low-level arrays
2
3 class DynamicArray:
4 ”””A dynamic array class akin to a simplified Python list.”””
5
6 def init (self):
7 ”””Create an empty array.”””
8 self. n = 0 # count actual elements
9 self. capacity = 1 # default array capacity
10 self. A = self. make array(self. capacity) # low-level array
11
12 def len (self):
13 ”””Return number of elements stored in the array.”””
14 return self. n
15
16 def getitem (self, k):
17 ”””Return element at index k.”””
18 if not 0 = k self. n:
19 raise IndexError( invalid index )
20 return self. A[k] # retrieve from array
21
22 def append(self, obj):
23 ”””Add object to end of the array.”””
24 if self. n == self. capacity: # not enough room
25 self. resize(2 self. capacity) # so double capacity
26 self. A[self. n] = obj
27 self. n += 1
28
29 def resize(self, c): # nonpublic utitity
30 ”””Resize internal array to capacity c.”””
31 B = self. make array(c) # new (bigger) array
32 for k in range(self. n): # for each existing value
33 B[k] = self. A[k]
34 self. A = B # use the bigger array
35 self. capacity = c
36
37 def make array(self, c): # nonpublic utitity
38 ”””Return new array with capacity c.”””
39 return (c ctypes.py object)( ) # see ctypes documentation
Code Fragment 5.3: An implementation of a DynamicArray class, using a raw array
from the ctypes module as storage.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-218-320.jpg)
![5.3. Dynamic Arrays and Amortization 201
5.3.3 Python’s List Class
The experiments of Code Fragment 5.1 and 5.2, at the beginning of Section 5.3,
provide empirical evidence that Python’s list class is using a form of dynamic arrays
for its storage. Yet, a careful examination of the intermediate array capacities (see
Exercises R-5.2 and C-5.13) suggests that Python is not using a pure geometric
progression, nor is it using an arithmetic progression.
With that said, it is clear that Python’s implementation of the append method
exhibits amortized constant-time behavior. We can demonstrate this fact experi-
mentally. A single append operation typically executes so quickly that it would be
difficult for us to accurately measure the time elapsed at that granularity, although
we should notice some of the more expensive operations in which a resize is per-
formed. We can get a more accurate measure of the amortized cost per operation
by performing a series of n append operations on an initially empty list and deter-
mining the average cost of each. A function to perform that experiment is given in
Code Fragment 5.4.
1 from time import time # import time function from time module
2 def compute average(n):
3 ”””Perform n appends to an empty list and return average time elapsed.”””
4 data = [ ]
5 start = time( ) # record the start time (in seconds)
6 for k in range(n):
7 data.append(None)
8 end = time( ) # record the end time (in seconds)
9 return (end − start) / n # compute average per operation
Code Fragment 5.4: Measuring the amortized cost of append for Python’s list class.
Technically, the time elapsed between the start and end includes the time to
manage the iteration of the for loop, in addition to the append calls. The empirical
results of the experiment, for increasingly large values of n, are shown in Table 5.2.
We see higher average cost for the smaller data sets, perhaps in part due to the over-
head of the loop range. There is also natural variance in measuring the amortized
cost in this way, because of the impact of the final resize event relative to n. Taken
as a whole, there seems clear evidence that the amortized time for each append is
independent of n.
n 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000
μs 0.219 0.158 0.164 0.151 0.147 0.147 0.149
Table 5.2: Average running time of append, measured in microseconds, as observed
over a sequence of n calls, starting with an empty list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-223-320.jpg)
![202 Chapter 5. Array-Based Sequences
5.4 Efficiency of Python’s Sequence Types
In the previous section, we began to explore the underpinnings of Python’s list
class, in terms of implementation strategies and efficiency. We continue in this
section by examining the performance of all of Python’s sequence types.
5.4.1 Python’s List and Tuple Classes
The nonmutating behaviors of the list class are precisely those that are supported
by the tuple class. We note that tuples are typically more memory efficient than
lists because they are immutable; therefore, there is no need for an underlying
dynamic array with surplus capacity. We summarize the asymptotic efficiency of
the nonmutating behaviors of the list and tuple classes in Table 5.3. An explanation
of this analysis follows.
Operation Running Time
len(data) O(1)
data[j] O(1)
data.count(value) O(n)
data.index(value) O(k+1)
value in data O(k+1)
data1 == data2
O(k+1)
(similarly !=, , =, , =)
data[j:k] O(k− j +1)
data1 + data2 O(n1 +n2)
c data O(cn)
Table 5.3: Asymptotic performance of the nonmutating behaviors of the list and
tuple classes. Identifiers data, data1, and data2 designate instances of the list or
tuple class, and n, n1, and n2 their respective lengths. For the containment check
and index method, k represents the index of the leftmost occurrence (with k = n if
there is no occurrence). For comparisons between two sequences, we let k denote
the leftmost index at which they disagree or else k = min(n1,n2).
Constant-Time Operations
The length of an instance is returned in constant time because an instance explicitly
maintains such state information. The constant-time efficiency of syntax data[j] is
assured by the underlying access into an array.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-224-320.jpg)
![5.4. Efficiency of Python’s Sequence Types 203
Searching for Occurrences of a Value
Each of the count, index, and contains methods proceed through iteration
of the sequence from left to right. In fact, Code Fragment 2.14 of Section 2.4.3
demonstrates how those behaviors might be implemented. Notably, the loop for
computing the count must proceed through the entire sequence, while the loops
for checking containment of an element or determining the index of an element
immediately exit once they find the leftmost occurrence of the desired value, if
one exists. So while count always examines the n elements of the sequence,
index and contains examine n elements in the worst case, but may be faster.
Empirical evidence can be found by setting data = list(range(10000000)) and
then comparing the relative efficiency of the test, 5 in data, relative to the test,
9999995 in data, or even the failed test, −5 in data.
Lexicographic Comparisons
Comparisons between two sequences are defined lexicographically. In the worst
case, evaluating such a condition requires an iteration taking time proportional
to the length of the shorter of the two sequences (because when one sequence
ends, the lexicographic result can be determined). However, in some cases the
result of the test can be evaluated more efficiently. For example, if evaluating
[7, 3, ...] [7, 5, ...], it is clear that the result is True without examining the re-
mainders of those lists, because the second element of the left operand is strictly
less than the second element of the right operand.
Creating New Instances
The final three behaviors in Table 5.3 are those that construct a new instance based
on one or more existing instances. In all cases, the running time depends on the
construction and initialization of the new result, and therefore the asymptotic be-
havior is proportional to the length of the result. Therefore, we find that slice
data[6000000:6000008] can be constructed almost immediately because it has only
eight elements, while slice data[6000000:7000000] has one million elements, and
thus is more time-consuming to create.
Mutating Behaviors
The efficiency of the mutating behaviors of the list class are described in Table 5.3.
The simplest of those behaviors has syntax data[j] = val, and is supported by the
special setitem method. This operation has worst-case O(1) running time be-
cause it simply replaces one element of a list with a new value. No other elements
are affected and the size of the underlying array does not change. The more inter-
esting behaviors to analyze are those that add or remove elements from the list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-225-320.jpg)
![204 Chapter 5. Array-Based Sequences
Operation Running Time
data[j] = val O(1)
data.append(value) O(1)∗
data.insert(k, value) O(n−k+1)∗
data.pop() O(1)∗
data.pop(k)
O(n−k)∗
del data[k]
data.remove(value) O(n)∗
data1.extend(data2)
O(n2)∗
data1 += data2
data.reverse() O(n)
data.sort() O(nlogn)
∗amortized
Table 5.4: Asymptotic performance of the mutating behaviors of the list class. Iden-
tifiers data, data1, and data2 designate instances of the list class, and n, n1, and n2
their respective lengths.
Adding Elements to a List
In Section 5.3 we fully explored the append method. In the worst case, it requires
Ω(n) time because the underlying array is resized, but it uses O(1) time in the amor-
tized sense. Lists also support a method, with signature insert(k, value), that inserts
a given value into the list at index 0 ≤ k ≤ n while shifting all subsequent elements
back one slot to make room. For the purpose of illustration, Code Fragment 5.5 pro-
vides an implementation of that method, in the context of our DynamicArray class
introduced in Code Fragment 5.3. There are two complicating factors in analyzing
the efficiency of such an operation. First, we note that the addition of one element
may require a resizing of the dynamic array. That portion of the work requires Ω(n)
worst-case time but only O(1) amortized time, as per append. The other expense
for insert is the shifting of elements to make room for the new item. The time for
1 def insert(self, k, value):
2 ”””Insert value at index k, shifting subsequent values rightward.”””
3 # (for simplicity, we assume 0 = k = n in this verion)
4 if self. n == self. capacity: # not enough room
5 self. resize(2 self. capacity) # so double capacity
6 for j in range(self. n, k, −1): # shift rightmost first
7 self. A[j] = self. A[j−1]
8 self. A[k] = value # store newest element
9 self. n += 1
Code Fragment 5.5: Implementation of insert for our DynamicArray class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-226-320.jpg)
![206 Chapter 5. Array-Based Sequences
Removing Elements from a List
Python’s list class offers several ways to remove an element from a list. A call to
pop() removes the last element from a list. This is most efficient, because all other
elements remain in their original location. This is effectively an O(1) operation,
but the bound is amortized because Python will occasionally shrink the underlying
dynamic array to conserve memory.
The parameterized version, pop(k), removes the element that is at index k n
of a list, shifting all subsequent elements leftward to fill the gap that results from
the removal. The efficiency of this operation is O(n−k), as the amount of shifting
depends upon the choice of index k, as illustrated in Figure 5.17. Note well that this
implies that pop(0) is the most expensive call, using Ω(n) time. (see experiments
in Exercise R-5.8.)
k
2
1
0 n−1
Figure 5.17: Removing an element at index k of a dynamic array.
The list class offers another method, named remove, that allows the caller to
specify the value that should be removed (not the index at which it resides). For-
mally, it removes only the first occurrence of such a value from a list, or raises a
ValueError if no such value is found. An implementation of such behavior is given
in Code Fragment 5.6, again using our DynamicArray class for illustration.
Interestingly, there is no “efficient” case for remove; every call requires Ω(n)
time. One part of the process searches from the beginning until finding the value at
index k, while the rest iterates from k to the end in order to shift elements leftward.
This linear behavior can be observed experimentally (see Exercise C-5.24).
1 def remove(self, value):
2 ”””Remove first occurrence of value (or raise ValueError).”””
3 # note: we do not consider shrinking the dynamic array in this version
4 for k in range(self. n):
5 if self. A[k] == value: # found a match!
6 for j in range(k, self. n − 1): # shift others to fill gap
7 self. A[j] = self. A[j+1]
8 self. A[self. n − 1] = None # help garbage collection
9 self. n −= 1 # we have one less item
10 return # exit immediately
11 raise ValueError( value not found ) # only reached if no match
Code Fragment 5.6: Implementation of remove for our DynamicArray class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-228-320.jpg)
![5.4. Efficiency of Python’s Sequence Types 207
Extending a List
Python provides a method named extend that is used to add all elements of one list
to the end of a second list. In effect, a call to data.extend(other) produces the same
outcome as the code,
for element in other:
data.append(element)
In either case, the running time is proportional to the length of the other list, and
amortized because the underlying array for the first list may be resized to accom-
modate the additional elements.
In practice, the extend method is preferable to repeated calls to append because
the constant factors hidden in the asymptotic analysis are significantly smaller. The
greater efficiency of extend is threefold. First, there is always some advantage to
using an appropriate Python method, because those methods are often implemented
natively in a compiled language (rather than as interpreted Python code). Second,
there is less overhead to a single function call that accomplishes all the work, versus
many individual function calls. Finally, increased efficiency of extend comes from
the fact that the resulting size of the updated list can be calculated in advance. If the
second data set is quite large, there is some risk that the underlying dynamic array
might be resized multiple times when using repeated calls to append. With a single
call to extend, at most one resize operation will be performed. Exercise C-5.22
explores the relative efficiency of these two approaches experimentally.
Constructing New Lists
There are several syntaxes for constructing new lists. In almost all cases, the asymp-
totic efficiency of the behavior is linear in the length of the list that is created. How-
ever, as with the case in the preceding discussion of extend, there are significant
differences in the practical efficiency.
Section 1.9.2 introduces the topic of list comprehension, using an example
such as squares = [ k k for k in range(1, n+1) ] as a shorthand for
squares = [ ]
for k in range(1, n+1):
squares.append(k k)
Experiments should show that the list comprehension syntax is significantly faster
than building the list by repeatedly appending (see Exercise C-5.23).
Similarly, it is a common Python idiom to initialize a list of constant values
using the multiplication operator, as in [0] n to produce a list of length n with
all values equal to zero. Not only is this succinct for the programmer; it is more
efficient than building such a list incrementally.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-229-320.jpg)
![5.4. Efficiency of Python’s Sequence Types 209
While the preceding code fragment accomplishes the goal, it may be terribly
inefficient. Because strings are immutable, the command, letters += c, would
presumably compute the concatenation, letters + c, as a new string instance and
then reassign the identifier, letters, to that result. Constructing that new string
would require time proportional to its length. If the final result has n characters, the
series of concatenations would take time proportional to the familiar sum 1+ 2+
3+···+n, and therefore O(n2) time.
Inefficient code of this type is widespread in Python, perhaps because of the
somewhat natural appearance of the code, and mistaken presumptions about how
the += operator is evaluated with strings. Some later implementations of the
Python interpreter have developed an optimization to allow such code to complete
in linear time, but this is not guaranteed for all Python implementations. The op-
timization is as follows. The reason that a command, letters += c, causes a new
string instance to be created is that the original string must be left unchanged if
another variable in a program refers to that string. On the other hand, if Python
knew that there were no other references to the string in question, it could imple-
ment += more efficiently by directly mutating the string (as a dynamic array). As
it happens, the Python interpreter already maintains what are known as reference
counts for each object; this count is used in part to determine if an object can be
garbage collected. (See Section 15.1.2.) But in this context, it provides a means to
detect when no other references exist to a string, thereby allowing the optimization.
A more standard Python idiom to guarantee linear time composition of a string
is to use a temporary list to store individual pieces, and then to rely on the join
method of the str class to compose the final result. Using this technique with our
previous example would appear as follows:
temp = [ ] # start with empty list
for c in document:
if c.isalpha():
temp.append(c) # append alphabetic character
letters = .join(temp) # compose overall result
This approach is guaranteed to run in O(n) time. First, we note that the series of
up to n append calls will require a total of O(n) time, as per the definition of the
amortized cost of that operation. The final call to join also guarantees that it takes
time that is linear in the final length of the composed string.
As we discussed at the end of the previous section, we can further improve
the practical execution time by using a list comprehension syntax to build up the
temporary list, rather than by repeated calls to append. That solution appears as,
letters = .join([c for c in document if c.isalpha()])
Better yet, we can entirely avoid the temporary list with a generator comprehension:
letters = .join(c for c in document if c.isalpha())](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-231-320.jpg)
![5.5. Using Array-Based Sequences 211
A Class for High Scores
To maintain a sequence of high scores, we develop a class named Scoreboard. A
scoreboard is limited to a certain number of high scores that can be saved; once that
limit is reached, a new score only qualifies for the scoreboard if it is strictly higher
than the lowest “high score” on the board. The length of the desired scoreboard may
depend on the game, perhaps 10, 50, or 500. Since that limit may vary depending on
the game, we allow it to be specified as a parameter to our Scoreboard constructor.
Internally, we will use a Python list named board in order to manage the
GameEntry instances that represent the high scores. Since we expect the score-
board to eventually reach full capacity, we initialize the list to be large enough to
hold the maximum number of scores, but we initially set all entries to None. By
allocating the list with maximum capacity initially, it never needs to be resized. As
entries are added, we will maintain them from highest to lowest score, starting at
index 0 of the list. We illustrate a typical state of the data structure in Figure 5.18.
3
2
0 1 9
7 8
6
5
4
660
Mike 1105 Paul 720 Rose 590
Rob 750 Anna Jack 510
Figure 5.18: An illustration of an ordered list of length ten, storing references to six
GameEntry objects in the cells from index 0 to 5, with the rest being None.
A complete Python implementation of the Scoreboard class is given in Code
Fragment 5.8. The constructor is rather simple. The command
self. board = [None] capacity
creates a list with the desired length, yet all entries equal to None. We maintain
an additional instance variable, n, that represents the number of actual entries
currently in our table. For convenience, our class supports the getitem method
to retrieve an entry at a given index with a syntax board[i] (or None if no such entry
exists), and we support a simple str method that returns a string representation
of the entire scoreboard, with one entry per line.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-233-320.jpg)
![212 Chapter 5. Array-Based Sequences
1 class Scoreboard:
2 ”””Fixed-length sequence of high scores in nondecreasing order.”””
3
4 def init (self, capacity=10):
5 ”””Initialize scoreboard with given maximum capacity.
6
7 All entries are initially None.
8 ”””
9 self. board = [None] capacity # reserve space for future scores
10 self. n = 0 # number of actual entries
11
12 def getitem (self, k):
13 ”””Return entry at index k.”””
14 return self. board[k]
15
16 def str (self):
17 ”””Return string representation of the high score list.”””
18 return n .join(str(self. board[j]) for j in range(self. n))
19
20 def add(self, entry):
21 ”””Consider adding entry to high scores.”””
22 score = entry.get score()
23
24 # Does new entry qualify as a high score?
25 # answer is yes if board not full or score is higher than last entry
26 good = self. n len(self. board) or score self. board[−1].get score()
27
28 if good:
29 if self. n len(self. board): # no score drops from list
30 self. n += 1 # so overall number increases
31
32 # shift lower scores rightward to make room for new entry
33 j = self. n − 1
34 while j 0 and self. board[j−1].get score( ) score:
35 self. board[j] = self. board[j−1] # shift entry from j-1 to j
36 j −= 1 # and decrement j
37 self. board[j] = entry # when done, add new entry
Code Fragment 5.8: Python code for a Scoreboard class that maintains an ordered
series of scores as GameEntry objects.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-234-320.jpg)
![214 Chapter 5. Array-Based Sequences
5.5.2 Sorting a Sequence
In the previous subsection, we considered an application for which we added an ob-
ject to a sequence at a given position while shifting other elements so as to keep the
previous order intact. In this section, we use a similar technique to solve the sorting
problem, that is, starting with an unordered sequence of elements and rearranging
them into nondecreasing order.
The Insertion-Sort Algorithm
We study several sorting algorithms in this book, most of which are described in
Chapter 12. As a warm-up, in this section we describe a nice, simple sorting al-
gorithm known as insertion-sort. The algorithm proceeds as follows for an array-
based sequence. We start with the first element in the array. One element by itself
is already sorted. Then we consider the next element in the array. If it is smaller
than the first, we swap them. Next we consider the third element in the array. We
swap it leftward until it is in its proper order with the first two elements. We then
consider the fourth element, and swap it leftward until it is in the proper order with
the first three. We continue in this manner with the fifth element, the sixth, and so
on, until the whole array is sorted. We can express the insertion-sort algorithm in
pseudo-code, as shown in Code Fragment 5.9.
Algorithm InsertionSort(A):
Input: An array A of n comparable elements
Output: The array A with elements rearranged in nondecreasing order
for k from 1 to n − 1 do
Insert A[k] at its proper location within A[0], A[1], ..., A[k].
Code Fragment 5.9: High-level description of the insertion-sort algorithm.
This is a simple, high-level description of insertion-sort. If we look back to
Code Fragment 5.8 of Section 5.5.1, we see that the task of inserting a new en-
try into the list of high scores is almost identical to the task of inserting a newly
considered element in insertion-sort (except that game scores were ordered from
high to low). We provide a Python implementation of insertion-sort in Code Frag-
ment 5.10, using an outer loop to consider each element in turn, and an inner
loop that moves a newly considered element to its proper location relative to the
(sorted) subarray of elements that are to its left. We illustrate an example run of the
insertion-sort algorithm in Figure 5.20.
The nested loops of insertion-sort lead to an O(n2) running time in the worst
case. The most work is done if the array is initially in reverse order. On the other
hand, if the initial array is nearly sorted or perfectly sorted, insertion-sort runs in
O(n) time because there are few or no iterations of the inner loop.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-236-320.jpg)
![5.5. Using Array-Based Sequences 215
1 def insertion sort(A):
2 ”””Sort list of comparable elements into nondecreasing order.”””
3 for k in range(1, len(A)): # from 1 to n-1
4 cur = A[k] # current element to be inserted
5 j = k # find correct index j for current
6 while j 0 and A[j−1] cur: # element A[j-1] must be after current
7 A[j] = A[j−1]
8 j −= 1
9 A[j] = cur # cur is now in the right place
Code Fragment 5.10: Python code for performing insertion-sort on a list.
insert
insert
insert
0
0
0
0
0
0
0
0
0
0 0 0
Done!
0
C A E H G F
B C A E H G F
D
B E H G F
C D
A H G F
B C D E
A F
B C D E H
A G F
B C D E H
E H G F
D
C
B
B E H G F
D
C
A F
B C D E H
B C D E H
G
B
A
A B C D E G H A B
C
F
F
G
A
G
H
E
A
D
C D E H
H
G
F
E
D
G
C
A
B D
no move
2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
cur
1 2 3 4 5 6 7
move move
move
no move
no move
no move
no move
move no move
move move
1
Figure 5.20: Execution of the insertion-sort algorithm on an array of eight charac-
ters. Each row corresponds to an iteration of the outer loop, and each copy of the
sequence in a row corresponds to an iteration of the inner loop. The current element
that is being inserted is highlighted in the array, and shown as the cur value.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-237-320.jpg)
![216 Chapter 5. Array-Based Sequences
5.5.3 Simple Cryptography
An interesting application of strings and lists is cryptography, the science of secret
messages and their applications. This field studies ways of performing encryp-
tion, which takes a message, called the plaintext, and converts it into a scrambled
message, called the ciphertext. Likewise, cryptography also studies corresponding
ways of performing decryption, which takes a ciphertext and turns it back into its
original plaintext.
Arguably the earliest encryption scheme is the Caesar cipher, which is named
after Julius Caesar, who used this scheme to protect important military messages.
(All of Caesar’s messages were written in Latin, of course, which already makes
them unreadable for most of us!) The Caesar cipher is a simple way to obscure a
message written in a language that forms words with an alphabet.
The Caesar cipher involves replacing each letter in a message with the letter that
is a certain number of letters after it in the alphabet. So, in an English message, we
might replace each A with D, each B with E, each C with F, and so on, if shifting by
three characters. We continue this approach all the way up to W, which is replaced
with Z. Then, we let the substitution pattern wrap around, so that we replace X
with A, Y with B, and Z with C.
Converting Between Strings and Character Lists
Given that strings are immutable, we cannot directly edit an instance to encrypt it.
Instead, our goal will be to generate a new string. A convenient technique for per-
forming string transformations is to create an equivalent list of characters, edit the
list, and then reassemble a (new) string based on the list. The first step can be per-
formed by sending the string as a parameter to the constructor of the list class. For
example, the expression list( bird ) produces the result [ b , i , r , d ].
Conversely, we can use a list of characters to build a string by invoking the join
method on an empty string, with the list of characters as the parameter. For exam-
ple, the call .join([ b , i , r , d ]) returns the string bird .
Using Characters as Array Indices
If we were to number our letters like array indices, so that A is 0, B is 1, C is 2,
and so on, then we can write the Caesar cipher with a rotation of r as a simple
formula: Replace each letter i with the letter (i + r) mod 26, where mod is the
modulo operator, which returns the remainder after performing an integer division.
This operator is denoted with % in Python, and it is exactly the operator we need
to easily perform the wrap around at the end of the alphabet. For 26 mod 26 is
0, 27 mod 26 is 1, and 28 mod 26 is 2. The decryption algorithm for the Caesar
cipher is just the opposite—we replace each letter with the one r places before it,
with wrap around (that is, letter i is replaced by letter (i−r) mod 26).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-238-320.jpg)
![218 Chapter 5. Array-Based Sequences
1 class CaesarCipher:
2 ”””Class for doing encryption and decryption using a Caesar cipher.”””
3
4 def init (self, shift):
5 ”””Construct Caesar cipher using given integer shift for rotation.”””
6 encoder = [None] 26 # temp array for encryption
7 decoder = [None] 26 # temp array for decryption
8 for k in range(26):
9 encoder[k] = chr((k + shift) % 26 + ord( A ))
10 decoder[k] = chr((k − shift) % 26 + ord( A ))
11 self. forward = .join(encoder) # will store as string
12 self. backward = .join(decoder) # since fixed
13
14 def encrypt(self, message):
15 ”””Return string representing encripted message.”””
16 return self. transform(message, self. forward)
17
18 def decrypt(self, secret):
19 ”””Return decrypted message given encrypted secret.”””
20 return self. transform(secret, self. backward)
21
22 def transform(self, original, code):
23 ”””Utility to perform transformation based on given code string.”””
24 msg = list(original)
25 for k in range(len(msg)):
26 if msg[k].isupper():
27 j = ord(msg[k]) − ord( A ) # index from 0 to 25
28 msg[k] = code[j] # replace this character
29 return .join(msg)
30
31 if name == __main__ :
32 cipher = CaesarCipher(3)
33 message = THE EAGLE IS IN PLAY; MEET AT JOE S.
34 coded = cipher.encrypt(message)
35 print( Secret: , coded)
36 answer = cipher.decrypt(coded)
37 print( Message: , answer)
Code Fragment 5.11: A complete Python class for the Caesar cipher.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-240-320.jpg)
![5.6. Multidimensional Data Sets 219
5.6 Multidimensional Data Sets
Lists, tuples, and strings in Python are one-dimensional. We use a single index to
access each element of the sequence. Many computer applications involve mul-
tidimensional data sets. For example, computer graphics are often modeled in
either two or three dimensions. Geographic information may be naturally repre-
sented in two dimensions, medical imaging may provide three-dimensional scans
of a patient, and a company’s valuation is often based upon a high number of in-
dependent financial measures that can be modeled as multidimensional data. A
two-dimensional array is sometimes also called a matrix. We may use two indices,
say i and j, to refer to the cells in the matrix. The first index usually refers to a
row number and the second to a column number, and these are traditionally zero-
indexed in computer science. Figure 5.22 illustrates a two-dimensional data set
with integer values. This data might, for example, represent the number of stores
in various regions of Manhattan.
22 18 709 5 33 10 4 56 82 440
45 32 830 120 750 660 13 77 20 105
4 880 45 66 61 28 650 7 510 67
940 12 36 3 20 100 306 590 0 500
50 65 42 49 88 25 70 126 83 288
398 233 5 83 59 232 49 8 365 90
33 58 632 87 94 5 59 204 120 829
62 394 3 4 102 140 183 390 16 26
8
0
1
2
3
4
5
6
7
0 1 2 3 4 5 6 7 9
Figure 5.22: Illustration of a two-dimensional integer data set, which has 8 rows
and 10 columns. The rows and columns are zero-indexed. If this data set were
named stores, the value of stores[3][5] is 100 and the value of stores[6][2] is 632.
A common representation for a two-dimensional data set in Python is as a list
of lists. In particular, we can represent a two-dimensional array as a list of rows,
with each row itself being a list of values. For example, the two-dimensional data
22 18 709 5 33
45 32 830 120 750
4 880 45 66 61
might be stored in Python as follows.
data = [ [22, 18, 709, 5, 33], [45, 32, 830, 120, 750], [4, 880, 45, 66, 61] ]
An advantage of this representation is that we can naturally use a syntax such
as data[1][3] to represent the value that has row index 1 and column index 3, as
data[1], the second entry in the outer list, is itself a list, and thus indexable.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-241-320.jpg)
![220 Chapter 5. Array-Based Sequences
Constructing a Multidimensional List
To quickly initialize a one-dimensional list, we generally rely on a syntax such as
data = [0] n to create a list of n zeros. On page 189, we emphasized that from
a technical perspective, this creates a list of length n with all entries referencing
the same integer instance, but that there was no meaningful consequence of such
aliasing because of the immutability of the int class in Python.
We have to be considerably more careful when creating a list of lists. If our
goal were to create the equivalent of a two-dimensional list of integers, with r rows
and c columns, and to initialize all values to zero, a flawed approach might be to
try the command
data = ([0] c) r # Warning: this is a mistake
While([0] c) is indeed a list of c zeros, multiplying that list by r unfortunately cre-
ates a single list with length r ·c, just as [2,4,6] 2 results in list [2, 4, 6, 2, 4, 6].
A better, yet still flawed attempt is to make a list that contains the list of c zeros
as its only element, and then to multiply that list by r. That is, we could try the
command
data = [ [0] c ] r # Warning: still a mistake
This is much closer, as we actually do have a structure that is formally a list of lists.
The problem is that all r entries of the list known as data are references to the same
instance of a list of c zeros. Figure 5.23 provides a portrayal of such aliasing.
0
0
0
0 0 0 0 0
1 2 3 4 5
1 2
data:
Figure 5.23: A flawed representation of a 3×6 data set as a list of lists, created with
the command data = [ [0] 6 ] 3. (For simplicity, we overlook the fact that the
values in the secondary list are referential.)
This is truly a problem. Setting an entry such as data[2][0] = 100 would change
the first entry of the secondary list to reference a new value, 100. Yet that cell of
the secondary list also represents the value data[0][0], because “row” data[0] and
“row” data[2] refer to the same secondary list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-242-320.jpg)
![5.6. Multidimensional Data Sets 221
0 0 0
0
0
0 0 0 0 0 0
0
0 0 0
0 0
0 0
0 0 0
3
1 2 2 3 4 5 1 2 4 5
1 2
data:
3 4 5 1
Figure 5.24: A valid representation of a 3×6 data set as a list of lists. (For simplic-
ity, we overlook the fact that the values in the secondary lists are referential.)
To properly initialize a two-dimensional list, we must ensure that each cell of
the primary list refers to an independent instance of a secondary list. This can be
accomplished through the use of Python’s list comprehension syntax.
data = [ [0] c for j in range(r) ]
This command produces a valid configuration, similar to the one shown in Fig-
ure 5.24. By using list comprehension, the expression [0] c is reevaluated for
each pass of the embedded for loop. Therefore, we get r distinct secondary lists, as
desired. (We note that the variable j in that command is irrelevant; we simply need
a for loop that iterates r times.)
Two-Dimensional Arrays and Positional Games
Many computer games, be they strategy games, simulation games, or first-person
conflict games, involve objects that reside in a two-dimensional space. Software for
such positional games need a way of representing such a two-dimensional “board,”
and in Python the list of lists is a natural choice.
Tic-Tac-Toe
As most school children know, Tic-Tac-Toe is a game played in a three-by-three
board. Two players—X and O—alternate in placing their respective marks in the
cells of this board, starting with player X. If either player succeeds in getting three
of his or her marks in a row, column, or diagonal, then that player wins.
This is admittedly not a sophisticated positional game, and it’s not even that
much fun to play, since a good player O can always force a tie. Tic-Tac-Toe’s saving
grace is that it is a nice, simple example showing how two-dimensional arrays can
be used for positional games. Software for more sophisticated positional games,
such as checkers, chess, or the popular simulation games, are all based on the same
approach we illustrate here for using a two-dimensional array for Tic-Tac-Toe.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-243-320.jpg)
![222 Chapter 5. Array-Based Sequences
Our representation of a 3 × 3 board will be a list of lists of characters, with
X or O designating a player’s move, or designating an empty space. For
example, the board configuration
X
O O
O X
X
will be stored internally as
[ [ O , X , O ], [ , X , ], [ , O , X ] ]
We develop a complete Python class for maintaining a Tic-Tac-Toe board for
two players. That class will keep track of the moves and report a winner, but it
does not perform any strategy or allow someone to play Tic-Tac-Toe against the
computer. The details of such a program are beyond the scope of this chapter, but
it might nonetheless make a good course project (see Exercise P-8.68).
Before presenting the implementation of the class, we demonstrate its public
interface with a simple test in Code Fragment 5.12.
1 game = TicTacToe()
2 # X moves: # O moves:
3 game.mark(1, 1); game.mark(0, 2)
4 game.mark(2, 2); game.mark(0, 0)
5 game.mark(0, 1); game.mark(2, 1)
6 game.mark(1, 2); game.mark(1, 0)
7 game.mark(2, 0)
8
9 print(game)
10 winner = game.winner()
11 if winner is None:
12 print( Tie )
13 else:
14 print(winner, wins )
Code Fragment 5.12: A simple test for our Tic-Tac-Toe class.
The basic operations are that a new game instance represents an empty board,
that the mark(i,j) method adds a mark at the given position for the current player
(with the software managing the alternating of turns), and that the game board can
be printed and the winner determined. The complete source code for the TicTacToe
class is given in Code Fragment 5.13. Our mark method performs error checking
to make sure that valid indices are sent, that the position is not already occupied,
and that no further moves are made after someone wins the game.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-244-320.jpg)
![5.6. Multidimensional Data Sets 223
1 class TicTacToe:
2 ”””Management of a Tic-Tac-Toe game (does not do strategy).”””
3
4 def init (self):
5 ”””Start a new game.”””
6 self. board = [ [ ] 3 for j in range(3) ]
7 self. player = X
8
9 def mark(self, i, j):
10 ”””Put an X or O mark at position (i,j) for next player s turn.”””
11 if not (0 = i = 2 and 0 = j = 2):
12 raise ValueError( Invalid board position )
13 if self. board[i][j] != :
14 raise ValueError( Board position occupied )
15 if self.winner( ) is not None:
16 raise ValueError( Game is already complete )
17 self. board[i][j] = self. player
18 if self. player == X :
19 self. player = O
20 else:
21 self. player = X
22
23 def is win(self, mark):
24 ”””Check whether the board configuration is a win for the given player.”””
25 board = self. board # local variable for shorthand
26 return (mark == board[0][0] == board[0][1] == board[0][2] or # row 0
27 mark == board[1][0] == board[1][1] == board[1][2] or # row 1
28 mark == board[2][0] == board[2][1] == board[2][2] or # row 2
29 mark == board[0][0] == board[1][0] == board[2][0] or # column 0
30 mark == board[0][1] == board[1][1] == board[2][1] or # column 1
31 mark == board[0][2] == board[1][2] == board[2][2] or # column 2
32 mark == board[0][0] == board[1][1] == board[2][2] or # diagonal
33 mark == board[0][2] == board[1][1] == board[2][0]) # rev diag
34
35 def winner(self):
36 ”””Return mark of winning player, or None to indicate a tie.”””
37 for mark in XO :
38 if self. is win(mark):
39 return mark
40 return None
41
42 def str (self):
43 ”””Return string representation of current game board.”””
44 rows = [ | .join(self. board[r]) for r in range(3)]
45 return n-----n .join(rows)
Code Fragment 5.13: A complete Python class for managing a Tic-Tac-Toe game.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-245-320.jpg)
![Chapter Notes 227
C-5.29 A useful operation in databases is the natural join. If we view a database
as a list of ordered pairs of objects, then the natural join of databases A
and B is the list of all ordered triples (x,y,z) such that the pair (x,y) is in
A and the pair (y,z) is in B. Describe and analyze an efficient algorithm
for computing the natural join of a list A of n pairs and a list B of m pairs.
C-5.30 When Bob wants to send Alice a message M on the Internet, he breaks M
into n data packets, numbers the packets consecutively, and injects them
into the network. When the packets arrive at Alice’s computer, they may
be out of order, so Alice must assemble the sequence of n packets in order
before she can be sure she has the entire message. Describe an efficient
scheme for Alice to do this, assuming that she knows the value of n. What
is the running time of this algorithm?
C-5.31 Describe a way to use recursion to add all the numbers in an n × n data
set, represented as a list of lists.
Projects
P-5.32 Write a Python function that takes two three-dimensional numeric data
sets and adds them componentwise.
P-5.33 Write a Python program for a matrix class that can add and multiply two-
dimensional arrays of numbers, assuming the dimensions agree appropri-
ately for the operation.
P-5.34 Write a program that can perform the Caesar cipher for English messages
that include both upper- and lowercase characters.
P-5.35 Implement a class, SubstitutionCipher, with a constructor that takes a
string with the 26 uppercase letters in an arbitrary order and uses that for
the forward mapping for encryption (akin to the self. forward string in
our CaesarCipher class of Code Fragment 5.11). You should derive the
backward mapping from the forward version.
P-5.36 Redesign the CaesarCipher class as a subclass of the SubstitutionCipher
from the previous problem.
P-5.37 Design a RandomCipher class as a subclass of the SubstitutionCipher
from Exercise P-5.35, so that each instance of the class relies on a random
permutation of letters for its mapping.
Chapter Notes
The fundamental data structures of arrays belong to the folklore of computer science. They
were first chronicled in the computer science literature by Knuth in his seminal book on
Fundamental Algorithms [64].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-249-320.jpg)
![230 Chapter 6. Stacks, Queues, and Deques
6.1.1 The Stack Abstract Data Type
Stacks are the simplest of all data structures, yet they are also among the most
important. They are used in a host of different applications, and as a tool for many
more sophisticated data structures and algorithms. Formally, a stack is an abstract
data type (ADT) such that an instance S supports the following two methods:
S.push(e): Add element e to the top of stack S.
S.pop(): Remove and return the top element from the stack S;
an error occurs if the stack is empty.
Additionally, let us define the following accessor methods for convenience:
S.top(): Return a reference to the top element of stack S, without
removing it; an error occurs if the stack is empty.
S.is empty(): Return True if stack S does not contain any elements.
len(S): Return the number of elements in stack S; in Python, we
implement this with the special method len .
By convention, we assume that a newly created stack is empty, and that there is no
a priori bound on the capacity of the stack. Elements added to the stack can have
arbitrary type.
Example 6.3: The following table shows a series of stack operations and their
effects on an initially empty stack S of integers.
Operation Return Value Stack Contents
S.push(5) – [5]
S.push(3) – [5, 3]
len(S) 2 [5, 3]
S.pop() 3 [5]
S.is empty() False [5]
S.pop() 5 [ ]
S.is empty() True [ ]
S.pop() “error” [ ]
S.push(7) – [7]
S.push(9) – [7, 9]
S.top() 9 [7, 9]
S.push(4) – [7, 9, 4]
len(S) 3 [7, 9, 4]
S.pop() 4 [7, 9]
S.push(6) – [7, 9, 6]
S.push(8) – [7, 9, 6, 8]
S.pop() 8 [7, 9, 6]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-252-320.jpg)
![6.1. Stacks 231
6.1.2 Simple Array-Based Stack Implementation
We can implement a stack quite easily by storing its elements in a Python list. The
list class already supports adding an element to the end with the append method,
and removing the last element with the pop method, so it is natural to align the top
of the stack at the end of the list, as shown in Figure 6.2.
0
M
B C D E F G K L
A
1 2 top
Figure 6.2: Implementing a stack with a Python list, storing the top element in the
rightmost cell.
Although a programmer could directly use the list class in place of a formal
stack class, lists also include behaviors (e.g., adding or removing elements from
arbitrary positions) that would break the abstraction that the stack ADT represents.
Also, the terminology used by the list class does not precisely align with traditional
nomenclature for a stack ADT, in particular the distinction between append and
push. Instead, we demonstrate how to use a list for internal storage while providing
a public interface consistent with a stack.
The Adapter Pattern
The adapter design pattern applies to any context where we effectively want to
modify an existing class so that its methods match those of a related, but different,
class or interface. One general way to apply the adapter pattern is to define a new
class in such a way that it contains an instance of the existing class as a hidden
field, and then to implement each method of the new class using methods of this
hidden instance variable. By applying the adapter pattern in this way, we have
created a new class that performs some of the same functions as an existing class,
but repackaged in a more convenient way. In the context of the stack ADT, we can
adapt Python’s list class using the correspondences shown in Table 6.1.
Stack Method Realization with Python list
S.push(e) L.append(e)
S.pop() L.pop()
S.top() L[−1]
S.is empty() len(L) == 0
len(S) len(L)
Table 6.1: Realization of a stack S as an adaptation of a Python list L.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-253-320.jpg)
![232 Chapter 6. Stacks, Queues, and Deques
Implementing a Stack Using a Python List
We use the adapter design pattern to define an ArrayStack class that uses an un-
derlying Python list for storage. (We choose the name ArrayStack to emphasize
that the underlying storage is inherently array based.) One question that remains is
what our code should do if a user calls pop or top when the stack is empty. Our
ADT suggests that an error occurs, but we must decide what type of error. When
pop is called on an empty Python list, it formally raises an IndexError, as lists are
index-based sequences. That choice does not seem appropriate for a stack, since
there is no assumption of indices. Instead, we can define a new exception class that
is more appropriate. Code Fragment 6.1 defines such an Empty class as a trivial
subclass of the Python Exception class.
class Empty(Exception):
”””Error attempting to access an element from an empty container.”””
pass
Code Fragment 6.1: Definition for an Empty exception class.
The formal definition for our ArrayStack class is given in Code Fragment 6.2.
The constructor establishes the member self. data as an initially empty Python list,
for internal storage. The rest of the public stack behaviors are implemented, using
the corresponding adaptation that was outlined in Table 6.1.
Example Usage
Below, we present an example of the use of our ArrayStack class, mirroring the
operations at the beginning of Example 6.3 on page 230.
S = ArrayStack( ) # contents: [ ]
S.push(5) # contents: [5]
S.push(3) # contents: [5, 3]
print(len(S)) # contents: [5, 3]; outputs 2
print(S.pop()) # contents: [5]; outputs 3
print(S.is empty()) # contents: [5]; outputs False
print(S.pop()) # contents: [ ]; outputs 5
print(S.is empty()) # contents: [ ]; outputs True
S.push(7) # contents: [7]
S.push(9) # contents: [7, 9]
print(S.top()) # contents: [7, 9]; outputs 9
S.push(4) # contents: [7, 9, 4]
print(len(S)) # contents: [7, 9, 4]; outputs 3
print(S.pop()) # contents: [7, 9]; outputs 4
S.push(6) # contents: [7, 9, 6]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-254-320.jpg)
![6.1. Stacks 233
1 class ArrayStack:
2 ”””LIFO Stack implementation using a Python list as underlying storage.”””
3
4 def init (self):
5 ”””Create an empty stack.”””
6 self. data = [ ] # nonpublic list instance
7
8 def len (self):
9 ”””Return the number of elements in the stack.”””
10 return len(self. data)
11
12 def is empty(self):
13 ”””Return True if the stack is empty.”””
14 return len(self. data) == 0
15
16 def push(self, e):
17 ”””Add element e to the top of the stack.”””
18 self. data.append(e) # new item stored at end of list
19
20 def top(self):
21 ”””Return (but do not remove) the element at the top of the stack.
22
23 Raise Empty exception if the stack is empty.
24 ”””
25 if self.is empty():
26 raise Empty( Stack is empty )
27 return self. data[−1] # the last item in the list
28
29 def pop(self):
30 ”””Remove and return the element from the top of the stack (i.e., LIFO).
31
32 Raise Empty exception if the stack is empty.
33 ”””
34 if self.is empty():
35 raise Empty( Stack is empty )
36 return self. data.pop( ) # remove last item from list
Code Fragment 6.2: Implementing a stack using a Python list as storage.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-255-320.jpg)
![236 Chapter 6. Stacks, Queues, and Deques
6.1.4 Matching Parentheses and HTML Tags
In this subsection, we explore two related applications of stacks, both of which
involve testing for pairs of matching delimiters. In our first application, we consider
arithmetic expressions that may contain various pairs of grouping symbols, such as
• Parentheses: “(” and “)”
• Braces: “{” and “}”
• Brackets: “[” and “]”
Each opening symbol must match its corresponding closing symbol. For example, a
left bracket, “[,” must match a corresponding right bracket, “],” as in the expression
[(5+x)-(y+z)]. The following examples further illustrate this concept:
• Correct: ()(()){([()])}
• Correct: ((()(()){([()])}))
• Incorrect: )(()){([()])}
• Incorrect: ({[])}
• Incorrect: (
We leave the precise definition of a matching group of symbols to Exercise R-6.6.
An Algorithm for Matching Delimiters
An important task when processing arithmetic expressions is to make sure their
delimiting symbols match up correctly. Code Fragment 6.4 presents a Python im-
plementation of such an algorithm. A discussion of the code follows.
1 def is matched(expr):
2 ”””Return True if all delimiters are properly match; False otherwise.”””
3 lefty = ({[ # opening delimiters
4 righty = )}] # respective closing delims
5 S = ArrayStack()
6 for c in expr:
7 if c in lefty:
8 S.push(c) # push left delimiter on stack
9 elif c in righty:
10 if S.is empty():
11 return False # nothing to match with
12 if righty.index(c) != lefty.index(S.pop()):
13 return False # mismatched
14 return S.is empty( ) # were all symbols matched?
Code Fragment 6.4: Function for matching delimiters in an arithmetic expression.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-258-320.jpg)
![6.1. Stacks 237
We assume the input is a sequence of characters, such as [(5+x)-(y+z)] .
We perform a left-to-right scan of the original sequence, using a stack S to facilitate
the matching of grouping symbols. Each time we encounter an opening symbol,
we push that symbol onto S, and each time we encounter a closing symbol, we pop
a symbol from the stack S (assuming S is not empty), and check that these two
symbols form a valid pair. If we reach the end of the expression and the stack is
empty, then the original expression was properly matched. Otherwise, there must
be an opening delimiter on the stack without a matching symbol.
If the length of the original expression is n, the algorithm will make at most
n calls to push and n calls to pop. Those calls run in a total of O(n) time, even con-
sidering the amortized nature of the O(1) time bound for those methods. Given that
our selection of possible delimiters, ({[, has constant size, auxiliary tests such as
c in lefty and righty.index(c) each run in O(1) time. Combining these operations,
the matching algorithm on a sequence of length n runs in O(n) time.
Matching Tags in a Markup Language
Another application of matching delimiters is in the validation of markup languages
such as HTML or XML. HTML is the standard format for hyperlinked documents
on the Internet and XML is an extensible markup language used for a variety of
structured data sets. We show a sample HTML document and a possible rendering
in Figure 6.3.
body
center
h1 The Little Boat /h1
/center
p The storm tossed the little
boat like a cheap sneaker in an
old washing machine. The three
drunken fishermen were used to
such treatment, of course, but
not the tree salesman, who even as
a stowaway now felt that he
had overpaid for the voyage. /p
ol
li Will the salesman die? /li
li What color is the boat? /li
li And what about Naomi? /li
/ol
/body
The Little Boat
The storm tossed the little boat
like a cheap sneaker in an
old washing machine. The three
drunken fishermen were used to
such treatment, of course, but not
the tree salesman, who even as
a stowaway now felt that he had
overpaid for the voyage.
1. Will the salesman die?
2. What color is the boat?
3. And what about Naomi?
(a) (b)
Figure 6.3: Illustrating HTML tags. (a) An HTML document; (b) its rendering.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-259-320.jpg)
![238 Chapter 6. Stacks, Queues, and Deques
In an HTML document, portions of text are delimited by HTML tags. A simple
opening HTML tag has the form “name” and the corresponding closing tag has
the form “/name”. For example, we see the body tag on the first line of
Figure 6.3(a), and the matching /body tag at the close of that document. Other
commonly used HTML tags that are used in this example include:
• body: document body
• h1: section header
• center: center justify
• p: paragraph
• ol: numbered (ordered) list
• li: list item
Ideally, an HTML document should have matching tags, although most browsers
tolerate a certain number of mismatching tags. In Code Fragment 6.5, we give a
Python function that matches tags in a string representing an HTML document. We
make a left-to-right pass through the raw string, using index j to track our progress
and the find method of the str class to locate the and characters that define
the tags. Opening tags are pushed onto the stack, and matched against closing tags
as they are popped from the stack, just as we did when matching delimiters in Code
Fragment 6.4. By similar analysis, this algorithm runs in O(n) time, where n is the
number of characters in the raw HTML source.
1 def is matched html(raw):
2 ”””Return True if all HTML tags are properly match; False otherwise.”””
3 S = ArrayStack()
4 j = raw.find( ) # find first ’’ character (if any)
5 while j != −1:
6 k = raw.find( , j+1) # find next ’’ character
7 if k == −1:
8 return False # invalid tag
9 tag = raw[j+1:k] # strip away
10 if not tag.startswith( / ): # this is opening tag
11 S.push(tag)
12 else: # this is closing tag
13 if S.is empty():
14 return False # nothing to match with
15 if tag[1:] != S.pop():
16 return False # mismatched delimiter
17 j = raw.find( , k+1) # find next ’’ character (if any)
18 return S.is empty( ) # were all opening tags matched?
Code Fragment 6.5: Function for testing if an HTML document has matching tags.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-260-320.jpg)
![240 Chapter 6. Stacks, Queues, and Deques
6.2.1 The Queue Abstract Data Type
Formally, the queue abstract data type defines a collection that keeps objects in a
sequence, where element access and deletion are restricted to the first element in
the queue, and element insertion is restricted to the back of the sequence. This
restriction enforces the rule that items are inserted and deleted in a queue accord-
ing to the first-in, first-out (FIFO) principle. The queue abstract data type (ADT)
supports the following two fundamental methods for a queue Q:
Q.enqueue(e): Add element e to the back of queue Q.
Q.dequeue(): Remove and return the first element from queue Q;
an error occurs if the queue is empty.
The queue ADT also includes the following supporting methods (with first being
analogous to the stack’s top method):
Q.first(): Return a reference to the element at the front of queue Q,
without removing it; an error occurs if the queue is empty.
Q.is empty(): Return True if queue Q does not contain any elements.
len(Q): Return the number of elements in queue Q; in Python,
we implement this with the special method len .
By convention, we assume that a newly created queue is empty, and that there
is no a priori bound on the capacity of the queue. Elements added to the queue can
have arbitrary type.
Example 6.4: The following table shows a series of queue operations and their
effects on an initially empty queue Q of integers.
Operation Return Value first ← Q ← last
Q.enqueue(5) – [5]
Q.enqueue(3) – [5, 3]
len(Q) 2 [5, 3]
Q.dequeue() 5 [3]
Q.is empty() False [3]
Q.dequeue() 3 [ ]
Q.is empty() True [ ]
Q.dequeue() “error” [ ]
Q.enqueue(7) – [7]
Q.enqueue(9) – [7, 9]
Q.first() 7 [7, 9]
Q.enqueue(4) – [7, 9, 4]
len(Q) 3 [7, 9, 4]
Q.dequeue() 7 [9, 4]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-262-320.jpg)
![6.2. Queues 243
1 class ArrayQueue:
2 ”””FIFO queue implementation using a Python list as underlying storage.”””
3 DEFAULT CAPACITY = 10 # moderate capacity for all new queues
4
5 def init (self):
6 ”””Create an empty queue.”””
7 self. data = [None] ArrayQueue.DEFAULT CAPACITY
8 self. size = 0
9 self. front = 0
10
11 def len (self):
12 ”””Return the number of elements in the queue.”””
13 return self. size
14
15 def is empty(self):
16 ”””Return True if the queue is empty.”””
17 return self. size == 0
18
19 def first(self):
20 ”””Return (but do not remove) the element at the front of the queue.
21
22 Raise Empty exception if the queue is empty.
23 ”””
24 if self.is empty():
25 raise Empty( Queue is empty )
26 return self. data[self. front]
27
28 def dequeue(self):
29 ”””Remove and return the first element of the queue (i.e., FIFO).
30
31 Raise Empty exception if the queue is empty.
32 ”””
33 if self.is empty():
34 raise Empty( Queue is empty )
35 answer = self. data[self. front]
36 self. data[self. front] = None # help garbage collection
37 self. front = (self. front + 1) % len(self. data)
38 self. size −= 1
39 return answer
Code Fragment 6.6: Array-based implementation of a queue (continued in Code
Fragment 6.7).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-265-320.jpg)
![244 Chapter 6. Stacks, Queues, and Deques
40 def enqueue(self, e):
41 ”””Add an element to the back of queue.”””
42 if self. size == len(self. data):
43 self. resize(2 len(self.data)) # double the array size
44 avail = (self. front + self. size) % len(self. data)
45 self. data[avail] = e
46 self. size += 1
47
48 def resize(self, cap): # we assume cap = len(self)
49 ”””Resize to a new list of capacity = len(self).”””
50 old = self. data # keep track of existing list
51 self. data = [None] cap # allocate list with new capacity
52 walk = self. front
53 for k in range(self. size): # only consider existing elements
54 self. data[k] = old[walk] # intentionally shift indices
55 walk = (1 + walk) % len(old) # use old size as modulus
56 self. front = 0 # front has been realigned
Code Fragment 6.7: Array-based implementation of a queue (continued from Code
Fragment 6.6).
The implementation of len and is empty are trivial, given knowledge of
the size. The implementation of the front method is also simple, as the front
index tells us precisely where the desired element is located within the data list,
assuming that list is not empty.
Adding and Removing Elements
The goal of the enqueue method is to add a new element to the back of the queue.
We need to determine the proper index at which to place the new element. Although
we do not explicitly maintain an instance variable for the back of the queue, we
compute the location of the next opening based on the formula:
avail = (self. front + self. size) % len(self. data)
Note that we are using the size of the queue as it exists prior to the addition of the
new element. For example, consider a queue with capacity 10, current size 3, and
first element at index 5. The three elements of such a queue are stored at indices 5,
6, and 7. The new element should be placed at index (front + size) = 8. In a case
with wrap-around, the use of the modular arithmetic achieves the desired circular
semantics. For example, if our hypothetical queue had 3 elements with the first at
index 8, our computation of (8+3) % 10 evaluates to 1, which is perfect since the
three existing elements occupy indices 8, 9, and 0.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-266-320.jpg)
![6.2. Queues 245
When the dequeue method is called, the current value of self. front designates
the index of the value that is to be removed and returned. We keep a local refer-
ence to the element that will be returned, setting answer = self. data[self. front]
just prior to removing the reference to that object from the list, with the assignment
self. data[self. front] = None. Our reason for the assignment to None relates to
Python’s mechanism for reclaiming unused space. Internally, Python maintains a
count of the number of references that exist to each object. If that count reaches
zero, the object is effectively inaccessible, thus the system may reclaim that mem-
ory for future use. (For more details, see Section 15.1.2.) Since we are no longer
responsible for storing a dequeued element, we remove the reference to it from our
list so as to reduce that element’s reference count.
The second significant responsibility of the dequeue method is to update the
value of front to reflect the removal of the element, and the presumed promotion
of the second element to become the new first. In most cases, we simply want
to increment the index by one, but because of the possibility of a wrap-around
configuration, we rely on modular arithmetic as originally described on page 242.
Resizing the Queue
When enqueue is called at a time when the size of the queue equals the size of the
underlying list, we rely on a standard technique of doubling the storage capacity of
the underlying list. In this way, our approach is similar to the one used when we
implemented a DynamicArray in Section 5.3.1.
However, more care is needed in the queue’s resize utility than was needed in
the corresponding method of the DynamicArray class. After creating a temporary
reference to the old list of values, we allocate a new list that is twice the size and
copy references from the old list to the new list. While transferring the contents, we
intentionally realign the front of the queue with index 0 in the new array, as shown
in Figure 6.7. This realignment is not purely cosmetic. Since the modular arith-
metic depends on the size of the array, our state would be flawed had we transferred
each element to its same index in the new array.
E G H
I J K
E F G H I J K
F
f
1 2
f = 0
Figure 6.7: Resizing the queue, while realigning the front element with index 0.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-267-320.jpg)
![248 Chapter 6. Stacks, Queues, and Deques
Example 6.5: The following table shows a series of operations and their effects
on an initially empty deque D of integers.
Operation Return Value Deque
D.add last(5) – [5]
D.add first(3) – [3, 5]
D.add first(7) – [7, 3, 5]
D.first() 7 [7, 3, 5]
D.delete last() 5 [7, 3]
len(D) 2 [7, 3]
D.delete last() 3 [7]
D.delete last() 7 [ ]
D.add first(6) – [6]
D.last() 6 [6]
D.add first(8) – [8, 6]
D.is empty() False [8, 6]
D.last() 6 [8, 6]
6.3.2 Implementing a Deque with a Circular Array
We can implement the deque ADT in much the same way as the ArrayQueue class
provided in Code Fragments 6.6 and 6.7 of Section 6.2.2 (so much so that we leave
the details of an ArrayDeque implementation to Exercise P-6.32). We recommend
maintaining the same three instance variables: data, size, and front. Whenever
we need to know the index of the back of the deque, or the first available slot
beyond the back of the deque, we use modular arithmetic for the computation. For
example, our implementation of the last() method uses the index
back = (self. front + self. size − 1) % len(self. data)
Our implementation of the ArrayDeque.add last method is essentially the same
as that for ArrayQueue.enqueue, including the reliance on a resize utility. Like-
wise, the implementation of the ArrayDeque.delete first method is the same as
ArrayQueue.dequeue. Implementations of add first and delete last use similar
techniques. One subtlety is that a call to add first may need to wrap around the
beginning of the array, so we rely on modular arithmetic to circularly decrement
the index, as
self. front = (self. front − 1) % len(self. data) # cyclic shift
The efficiency of an ArrayDeque is similar to that of an ArrayQueue, with all
operations having O(1) running time, but with that bound being amortized for op-
erations that may change the size of the underlying list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-270-320.jpg)
![6.3. Double-Ended Queues 249
6.3.3 Deques in the Python Collections Module
An implementation of a deque class is available in Python’s standard collections
module. A summary of the most commonly used behaviors of the collections.deque
class is given in Table 6.4. It uses more asymmetric nomenclature than our ADT.
Our Deque ADT collections.deque Description
len(D) len(D) number of elements
D.add first() D.appendleft() add to beginning
D.add last() D.append() add to end
D.delete first() D.popleft() remove from beginning
D.delete last() D.pop() remove from end
D.first() D[0] access first element
D.last() D[−1] access last element
D[j] access arbitrary entry by index
D[j] = val modify arbitrary entry by index
D.clear() clear all contents
D.rotate(k) circularly shift rightward k steps
D.remove(e) remove first matching element
D.count(e) count number of matches for e
Table 6.4: Comparison of our deque ADT and the collections.deque class.
The collections.deque interface was chosen to be consistent with established
naming conventions of Python’s list class, for which append and pop are presumed
to act at the end of the list. Therefore, appendleft and popleft designate an opera-
tion at the beginning of the list. The library deque also mimics a list in that it is an
indexed sequence, allowing arbitrary access or modification using the D[j] syntax.
The library deque constructor also supports an optional maxlen parameter to
force a fixed-length deque. However, if a call to append at either end is invoked
when the deque is full, it does not throw an error; instead, it causes one element to
be dropped from the opposite side. That is, calling appendleft when the deque is
full causes an implicit pop from the right side to make room for the new element.
The current Python distribution implements collections.deque with a hybrid ap-
proach that uses aspects of circular arrays, but organized into blocks that are them-
selves organized in a doubly linked list (a data structure that we will introduce in
the next chapter). The deque class is formally documented to guarantee O(1)-time
operations at either end, but O(n)-time worst-case operations when using index
notation near the middle of the deque.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-271-320.jpg)
![250 Chapter 6. Stacks, Queues, and Deques
6.4 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-6.1 What values are returned during the following series of stack operations, if
executed upon an initially empty stack? push(5), push(3), pop(), push(2),
push(8), pop(), pop(), push(9), push(1), pop(), push(7), push(6), pop(),
pop(), push(4), pop(), pop().
R-6.2 Suppose an initially empty stack S has executed a total of 25 push opera-
tions, 12 top operations, and 10 pop operations, 3 of which raised Empty
errors that were caught and ignored. What is the current size of S?
R-6.3 Implement a function with signature transfer(S, T) that transfers all ele-
ments from stack S onto stack T, so that the element that starts at the top
of S is the first to be inserted onto T, and the element at the bottom of S
ends up at the top of T.
R-6.4 Give a recursive method for removing all the elements from a stack.
R-6.5 Implement a function that reverses a list of elements by pushing them onto
a stack in one order, and writing them back to the list in reversed order.
R-6.6 Give a precise and complete definition of the concept of matching for
grouping symbols in an arithmetic expression. Your definition may be
recursive.
R-6.7 What values are returned during the following sequence of queue opera-
tions, if executed on an initially empty queue? enqueue(5), enqueue(3),
dequeue(), enqueue(2), enqueue(8), dequeue(), dequeue(), enqueue(9),
enqueue(1), dequeue(), enqueue(7), enqueue(6), dequeue(), dequeue(),
enqueue(4), dequeue(), dequeue().
R-6.8 Suppose an initially empty queue Q has executed a total of 32 enqueue
operations, 10 first operations, and 15 dequeue operations, 5 of which
raised Empty errors that were caught and ignored. What is the current
size of Q?
R-6.9 Had the queue of the previous problem been an instance of ArrayQueue
that used an initial array of capacity 30, and had its size never been greater
than 30, what would be the final value of the front instance variable?
R-6.10 Consider what happens if the loop in the ArrayQueue. resize method at
lines 53–55 of Code Fragment 6.7 had been implemented as:
for k in range(self. size):
self. data[k] = old[k] # rather than old[walk]
Give a clear explanation of what could go wrong.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-272-320.jpg)
![252 Chapter 6. Stacks, Queues, and Deques
C-6.21 Show how to use a stack S and a queue Q to generate all possible subsets
of an n-element set T nonrecursively.
C-6.22 Postfix notation is an unambiguous way of writing an arithmetic expres-
sion without parentheses. It is defined so that if “(exp1)op(exp2)” is a
normal, fully parenthesized expression whose operation is op, the postfix
version of this is “pexp1 pexp2 op”, where pexp1 is the postfix version of
exp1 and pexp2 is the postfix version of exp2. The postfix version of a sin-
gle number or variable is just that number or variable. For example, the
postfix version of “((5+2)∗(8−3))/4” is “5 2 + 8 3 − ∗ 4 /”. Describe
a nonrecursive way of evaluating an expression in postfix notation.
C-6.23 Suppose you have three nonempty stacks R, S, and T. Describe a sequence
of operations that results in S storing all elements originally in T below all
of S’s original elements, with both sets of those elements in their original
order. The final configuration for R should be the same as its original
configuration. For example, if R = [1,2,3], S = [4,5], and T = [6,7,8,9],
the final configuration should have R = [1,2,3] and S = [6,7,8,9,4,5].
C-6.24 Describe how to implement the stack ADT using a single queue as an
instance variable, and only constant additional local memory within the
method bodies. What is the running time of the push(), pop(), and top()
methods for your design?
C-6.25 Describe how to implement the queue ADT using two stacks as instance
variables, such that all queue operations execute in amortized O(1) time.
Give a formal proof of the amortized bound.
C-6.26 Describe how to implement the double-ended queue ADT using two stacks
as instance variables. What are the running times of the methods?
C-6.27 Suppose you have a stack S containing n elements and a queue Q that is
initially empty. Describe how you can use Q to scan S to see if it contains a
certain element x, with the additional constraint that your algorithm must
return the elements back to S in their original order. You may only use S,
Q, and a constant number of other variables.
C-6.28 Modify the ArrayQueue implementation so that the queue’s capacity is
limited to maxlen elements, where maxlen is an optional parameter to the
constructor (that defaults to None). If enqueue is called when the queue
is at full capacity, throw a Full exception (defined similarly to Empty).
C-6.29 In certain applications of the queue ADT, it is common to repeatedly
dequeue an element, process it in some way, and then immediately en-
queue the same element. Modify the ArrayQueue implementation to in-
clude a rotate() method that has semantics identical to the combina-
tion, Q.enqueue(Q.dequeue()). However, your implementation should
be more efficient than making two separate calls (for example, because
there is no need to modify size).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-274-320.jpg)
![254 Chapter 6. Stacks, Queues, and Deques
P-6.36 When a share of common stock of some company is sold, the capital
gain (or, sometimes, loss) is the difference between the share’s selling
price and the price originally paid to buy it. This rule is easy to under-
stand for a single share, but if we sell multiple shares of stock bought
over a long period of time, then we must identify the shares actually be-
ing sold. A standard accounting principle for identifying which shares of
a stock were sold in such a case is to use a FIFO protocol—the shares
sold are the ones that have been held the longest (indeed, this is the de-
fault method built into several personal finance software packages). For
example, suppose we buy 100 shares at $20 each on day 1, 20 shares at
$24 on day 2, 200 shares at $36 on day 3, and then sell 150 shares on day
4 at $30 each. Then applying the FIFO protocol means that of the 150
shares sold, 100 were bought on day 1, 20 were bought on day 2, and 30
were bought on day 3. The capital gain in this case would therefore be
100·10+20·6+30·(−6), or $940. Write a program that takes as input
a sequence of transactions of the form “buy x share(s) at y each”
or “sell x share(s) at y each,” assuming that the transactions oc-
cur on consecutive days and the values x and y are integers. Given this
input sequence, the output should be the total capital gain (or loss) for the
entire sequence, using the FIFO protocol to identify shares.
P-6.37 Design an ADT for a two-color, double-stack ADT that consists of two
stacks—one “red” and one “blue”—and has as its operations color-coded
versions of the regular stack ADT operations. For example, this ADT
should support both a red push operation and a blue push operation. Give
an efficient implementation of this ADT using a single array whose ca-
pacity is set at some value N that is assumed to always be larger than the
sizes of the red and blue stacks combined.
Chapter Notes
We were introduced to the approach of defining data structures first in terms of their ADTs
and then in terms of concrete implementations by the classic books by Aho, Hopcroft, and
Ullman [5, 6]. Exercises C-6.30, and C-6.31 are similar to interview questions said to be
from a well-known software company. For further study of abstract data types, see Liskov
and Guttag [71], Cardelli and Wegner [23], or Demurjian [33].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-276-320.jpg)
![298 Chapter 7. Linked Lists
P-7.45 An array A is sparse if most of its entries are empty (i.e., None). A list
L can be used to implement such an array efficiently. In particular, for
each nonempty cell A[i], we can store an entry (i,e) in L, where e is the
element stored at A[i]. This approach allows us to represent A using O(m)
storage, where m is the number of nonempty entries in A. Provide such
a SparseArray class that minimally supports methods getitem (j) and
setitem (j, e) to provide standard indexing operations. Analyze the
efficiency of these methods.
P-7.46 Although we have used a doubly linked list to implement the positional
list ADT, it is possible to support the ADT with an array-based imple-
mentation. The key is to use the composition pattern and store a sequence
of position items, where each item stores an element as well as that ele-
ment’s current index in the array. Whenever an element’s place in the array
is changed, the recorded index in the position must be updated to match.
Given a complete class providing such an array-based implementation of
the positional list ADT. What is the efficiency of the various operations?
P-7.47 Implement a CardHand class that supports a person arranging a group of
cards in his or her hand. The simulator should represent the sequence of
cards using a single positional list ADT so that cards of the same suit are
kept together. Implement this strategy by means of four “fingers” into the
hand, one for each of the suits of hearts, clubs, spades, and diamonds,
so that adding a new card to the person’s hand or playing a correct card
from the hand can be done in constant time. The class should support the
following methods:
• add card(r, s): Add a new card with rank r and suit s to the hand.
• play(s): Remove and return a card of suit s from the player’s hand;
if there is no card of suit s, then remove and return an arbitrary card
from the hand.
• iter (): Iterate through all cards currently in the hand.
• all of suit(s): Iterate through all cards of suit s that are currently in
the hand.
Chapter Notes
A view of data structures as collections (and other principles of object-oriented design)
can be found in object-oriented design books by Booch [17], Budd [20], Goldberg and
Robson [42], and Liskov and Guttag [71]. Our positional list ADT is derived from the
“position” abstraction introduced by Aho, Hopcroft, and Ullman [6], and the list ADT of
Wood [104]. Implementations of linked lists are discussed by Knuth [64].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-320-320.jpg)
![326 Chapter 8. Trees
The level numbering function f suggests a representation of a binary tree T
by means of an array-based structure A (such as a Python list), with the element
at position p of T stored at index f(p) of the array. We show an example of an
array-based representation of a binary tree in Figure 8.13.
/
4 2
0
2
1
3 4 5 6
12
11
8
7
3 1
+
×
9 5
−
+
0 6 12
1 2 3 4 5 7 8 9 10 11 13 14
5
× + + 4 − 2 3 1 9
/
Figure 8.13: Representation of a binary tree by means of an array.
One advantage of an array-based representation of a binary tree is that a posi-
tion p can be represented by the single integer f(p), and that position-based meth-
ods such as root, parent, left, and right can be implemented using simple arithmetic
operations on the number f(p). Based on our formula for the level numbering, the
left child of p has index 2f(p) + 1, the right child of p has index 2f(p) + 2, and
the parent of p has index (f(p)− 1)/2. We leave the details of a complete im-
plementation as an exercise (R-8.18).
The space usage of an array-based representation depends greatly on the shape
of the tree. Let n be the number of nodes of T, and let fM be the maximum value
of f(p) over all the nodes of T. The array A requires length N = 1 + fM, since
elements range from A[0] to A[ fM]. Note that A may have a number of empty cells
that do not refer to existing nodes of T. In fact, in the worst case, N = 2n − 1,
the justification of which is left as an exercise (R-8.16). In Section 9.3, we will
see a class of binary trees, called “heaps” for which N = n. Thus, in spite of the
worst-case space usage, there are applications for which the array representation
of a binary tree is space efficient. Still, for general binary trees, the exponential
worst-case space requirement of this representation is prohibitive.
Another drawback of an array representation is that some update operations for
trees cannot be efficiently supported. For example, deleting a node and promoting
its child takes O(n) time because it is not just the child that moves locations within
the array, but all descendants of that child.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-348-320.jpg)
![8.4. Tree Traversal Algorithms 339
1 def preorder label(T, p, d, path):
2 ”””Print labeled representation of subtree of T rooted at p at depth d.”””
3 label = . .join(str(j+1) for j in path) # displayed labels are one-indexed
4 print(2 d + label, p.element())
5 path.append(0) # path entries are zero-indexed
6 for c in T.children(p):
7 preorder label(T, c, d+1, path) # child depth is d+1
8 path[−1] += 1
9 path.pop()
Code Fragment 8.24: Efficient recursion for printing an indented and labeled pre-
sentation of a preorder traversal.
Parenthetic Representations of a Tree
It is not possible to reconstruct a general tree, given only the preorder sequence
of elements, as in Figure 8.20(a). Some additional context is necessary for the
structure of the tree to be well defined. The use of indentation or numbered labels
provides such context, with a very human-friendly presentation. However, there
are more concise string representations of trees that are computer-friendly.
In this section, we explore one such representation. The parenthetic string
representation P(T) of tree T is recursively defined as follows. If T consists of a
single position p, then
P(T) = str(p.element()).
Otherwise, it is defined recursively as,
P(T) = str(p.element())+ ( +P(T1)+ , + ··· + , +P(Tk)+ )
where p is the root of T and T1,T2,...,Tk are the subtrees rooted at the children
of p, which are given in order if T is an ordered tree. We are using “+” here to
denote string concatenation. As an example, the parenthetic representation of the
tree of Figure 8.2 would appear as follows (line breaks are cosmetic):
Electronics R’Us (RD, Sales (Domestic, International (Canada,
S. America, Overseas (Africa, Europe, Asia, Australia))),
Purchasing, Manufacturing (TV, CD, Tuner))
Although the parenthetic representation is essentially a preorder traversal, we
cannot easily produce the additional punctuation using the formal implementation
of preorder, as given in Code Fragment 8.17. The opening parenthesis must be
produced just before the loop over a position’s children and the closing parenthesis
must be produced just after that loop. Furthermore, the separating commas must
be produced. The Python function parenthesize, shown in Code Fragment 8.25, is
a custom traversal that prints such a parenthetic string representation of a tree T.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-361-320.jpg)
![8.4. Tree Traversal Algorithms 343
1 class EulerTour:
2 ”””Abstract base class for performing Euler tour of a tree.
3
4 hook previsit and hook postvisit may be overridden by subclasses.
5 ”””
6 def init (self, tree):
7 ”””Prepare an Euler tour template for given tree.”””
8 self. tree = tree
9
10 def tree(self):
11 ”””Return reference to the tree being traversed.”””
12 return self. tree
13
14 def execute(self):
15 ”””Perform the tour and return any result from post visit of root.”””
16 if len(self. tree) 0:
17 return self. tour(self. tree.root(), 0, [ ]) # start the recursion
18
19 def tour(self, p, d, path):
20 ”””Perform tour of subtree rooted at Position p.
21
22 p Position of current node being visited
23 d depth of p in the tree
24 path list of indices of children on path from root to p
25 ”””
26 self. hook previsit(p, d, path) # ”pre visit” p
27 results = [ ]
28 path.append(0) # add new index to end of path before recursion
29 for c in self. tree.children(p):
30 results.append(self. tour(c, d+1, path)) # recur on child s subtree
31 path[−1] += 1 # increment index
32 path.pop( ) # remove extraneous index from end of path
33 answer = self. hook postvisit(p, d, path, results) # ”post visit” p
34 return answer
35
36 def hook previsit(self, p, d, path): # can be overridden
37 pass
38
39 def hook postvisit(self, p, d, path, results): # can be overridden
40 pass
Code Fragment 8.28: An EulerTour base class providing a framework for perform-
ing Euler tour traversals of a tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-365-320.jpg)
![8.4. Tree Traversal Algorithms 345
A labeled version of an indented, preorder presentation, akin to Code Frag-
ment 8.24, could be generated by the new subclass of EulerTour shown in Code
Fragment 8.30.
1 class PreorderPrintIndentedLabeledTour(EulerTour):
2 def hook previsit(self, p, d, path):
3 label = . .join(str(j+1) for j in path) # labels are one-indexed
4 print(2 d + label, p.element())
Code Fragment 8.30: A subclass of EulerTour that produces a labeled and indented,
preorder list of a tree’s elements.
To produce the parenthetic string representation, originally achieved with Code
Fragment 8.25, we define a subclass that overrides both the previsit and postvisit
hooks. Our new implementation is given in Code Fragment 8.31.
1 class ParenthesizeTour(EulerTour):
2 def hook previsit(self, p, d, path):
3 if path and path[−1] 0: # p follows a sibling
4 print( , , end= ) # so preface with comma
5 print(p.element(), end= ) # then print element
6 if not self.tree().is leaf(p): # if p has children
7 print( ( , end= ) # print opening parenthesis
8
9 def hook postvisit(self, p, d, path, results):
10 if not self.tree().is leaf(p): # if p has children
11 print( ) , end= ) # print closing parenthesis
Code Fragment 8.31: A subclass of EulerTour that prints a parenthetic string repre-
sentation of a tree.
Notice that in this implementation, we need to invoke a method on the tree instance
that is being traversed from within the hooks. The public tree() method of the
EulerTour class serves as an accessor for that tree.
Finally, the task of computing disk space, as originally implemented in Code
Fragment 8.26, can be performed quite easily with the EulerTour subclass shown
in Code Fragment 8.32. The postvisit result of the root will be returned by the call
to execute().
1 class DiskSpaceTour(EulerTour):
2 def hook postvisit(self, p, d, path, results):
3 # we simply add space associated with p to that of its subtrees
4 return p.element().space( ) + sum(results)
Code Fragment 8.32: A subclass of EulerTour that computes disk space for a tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-367-320.jpg)
![346 Chapter 8. Trees
The Euler Tour Traversal of a Binary Tree
In Section 8.4.6, we introduced the concept of an Euler tour traversal of a general
graph, using the template method pattern in designing the EulerTour class. That
class provided methods hook previsit and hook postvisit that could be overrid-
den to customize a tour. In Code Fragment 8.33 we provide a BinaryEulerTour
specialization that includes an additional hook invisit that is called once for each
position—after its left subtree is traversed, but before its right subtree is traversed.
Our implementation of BinaryEulerTour replaces the original tour utility to
specialize to the case in which a node has at most two children. If a node has only
one child, a tour differentiates between whether that is a left child or a right child,
with the “in visit” taking place after the visit of a sole left child, but before the visit
of a sole right child. In the case of a leaf, the three hooks are called in succession.
1 class BinaryEulerTour(EulerTour):
2 ”””Abstract base class for performing Euler tour of a binary tree.
3
4 This version includes an additional hook invisit that is called after the tour
5 of the left subtree (if any), yet before the tour of the right subtree (if any).
6
7 Note: Right child is always assigned index 1 in path, even if no left sibling.
8 ”””
9 def tour(self, p, d, path):
10 results = [None, None] # will update with results of recursions
11 self. hook previsit(p, d, path) # ”pre visit” for p
12 if self. tree.left(p) is not None: # consider left child
13 path.append(0)
14 results[0] = self. tour(self. tree.left(p), d+1, path)
15 path.pop()
16 self. hook invisit(p, d, path) # ”in visit” for p
17 if self. tree.right(p) is not None: # consider right child
18 path.append(1)
19 results[1] = self. tour(self. tree.right(p), d+1, path)
20 path.pop()
21 answer = self. hook postvisit(p, d, path, results) # ”post visit” p
22 return answer
23
24 def hook invisit(self, p, d, path): pass # can be overridden
Code Fragment 8.33: A BinaryEulerTour base class providing a specialized tour for
binary trees. The original EulerTour base class was given in Code Fragment 8.28.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-368-320.jpg)
![8.5. Case Study: An Expression Tree 349
1 class ExpressionTree(LinkedBinaryTree):
2 ”””An arithmetic expression tree.”””
3
4 def init (self, token, left=None, right=None):
5 ”””Create an expression tree.
6
7 In a single parameter form, token should be a leaf value (e.g., 42 ),
8 and the expression tree will have that value at an isolated node.
9
10 In a three-parameter version, token should be an operator,
11 and left and right should be existing ExpressionTree instances
12 that become the operands for the binary operator.
13 ”””
14 super(). init ( ) # LinkedBinaryTree initialization
15 if not isinstance(token, str):
16 raise TypeError( Token must be a string )
17 self. add root(token) # use inherited, nonpublic method
18 if left is not None: # presumably three-parameter form
19 if token not in +-*x/ :
20 raise ValueError( token must be valid operator )
21 self. attach(self.root(), left, right) # use inherited, nonpublic method
22
23 def str (self):
24 ”””Return string representation of the expression.”””
25 pieces = [ ] # sequence of piecewise strings to compose
26 self. parenthesize recur(self.root(), pieces)
27 return .join(pieces)
28
29 def parenthesize recur(self, p, result):
30 ”””Append piecewise representation of p s subtree to resulting list.”””
31 if self.is leaf(p):
32 result.append(str(p.element())) # leaf value as a string
33 else:
34 result.append( ( ) # opening parenthesis
35 self. parenthesize recur(self.left(p), result) # left subtree
36 result.append(p.element()) # operator
37 self. parenthesize recur(self.right(p), result) # right subtree
38 result.append( ) ) # closing parenthesis
Code Fragment 8.35: The beginning of an ExpressionTree class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-371-320.jpg)
![8.5. Case Study: An Expression Tree 351
Building an Expression Tree
The constructor for the ExpressionTree class, from Code Fragment 8.35, provides
basic functionality for combining existing trees to build larger expression trees.
However, the question still remains how to construct a tree that represents an ex-
pression for a given string, such as (((3+1)x4)/((9-5)+2)) .
To automate this process, we rely on a bottom-up construction algorithm, as-
suming that a string can first be tokenized so that multidigit numbers are treated
atomically (see Exercise R-8.30), and that the expression is fully parenthesized.
The algorithm uses a stack S while scanning tokens of the input expression E to
find values, operators, and right parentheses. (Left parentheses are ignored.)
• When we see an operator ◦, we push that string on the stack.
• When we see a literal value v, we create a single-node expression tree T
storing v, and push T on the stack.
• When we see a right parenthesis, ) , we pop the top three items from the
stack S, which represent a subexpression (E1 ◦ E2). We then construct a
tree T using trees for E1 and E2 as subtrees of the root storing ◦, and push
the resulting tree T back on the stack.
We repeat this until the expression E has been processed, at which time the top
element on the stack is the expression tree for E. The total running time is O(n).
An implementation of this algorithm is given in Code Fragment 8.38 in the form
of a stand-alone function named build expression tree, which produces and returns
an appropriate ExpressionTree instance, assuming the input has been tokenized.
1 def build expression tree(tokens):
2 ”””Returns an ExpressionTree based upon by a tokenized expression.”””
3 S = [ ] # we use Python list as stack
4 for t in tokens:
5 if t in +-x*/ : # t is an operator symbol
6 S.append(t) # push the operator symbol
7 elif t not in () : # consider t to be a literal
8 S.append(ExpressionTree(t)) # push trivial tree storing value
9 elif t == ) : # compose a new tree from three constituent parts
10 right = S.pop( ) # right subtree as per LIFO
11 op = S.pop( ) # operator symbol
12 left = S.pop( ) # left subtree
13 S.append(ExpressionTree(op, left, right)) # repush tree
14 # we ignore a left parenthesis
15 return S.pop()
Code Fragment 8.38: Implementation of a build expression tree that produces an
ExpressionTree from a sequence of tokens representing an arithmetic expression.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-373-320.jpg)
![354 Chapter 8. Trees
R-8.22 In what order are positions visited during a postorder traversal of the tree
of Figure 8.8?
R-8.23 Let T be an ordered tree with more than one node. Is it possible that the
preorder traversal of T visits the nodes in the same order as the postorder
traversal of T? If so, give an example; otherwise, explain why this cannot
occur. Likewise, is it possible that the preorder traversal of T visits the
nodes in the reverse order of the postorder traversal of T? If so, give an
example; otherwise, explain why this cannot occur.
R-8.24 Answer the previous question for the case when T is a proper binary tree
with more than one node.
R-8.25 Consider the example of a breadth-first traversal given in Figure 8.17.
Using the annotated numbers from that figure, describe the contents of
the queue before each pass of the while loop in Code Fragment 8.14. To
get started, the queue has contents {1} before the first pass, and contents
{2,3,4} before the second pass.
R-8.26 The collections.deque class supports an extend method that adds a col-
lection of elements to the end of the queue at once. Reimplement the
breadthfirst method of the Tree class to take advantage of this feature.
R-8.27 Give the output of the function parenthesize(T, T.root()), as described
in Code Fragment 8.25, when T is the tree of Figure 8.8.
R-8.28 What is the running time of parenthesize(T, T.root()), as given in Code
Fragment 8.25, for a tree T with n nodes?
R-8.29 Describe, in pseudo-code, an algorithm for computing the number of de-
scendants of each node of a binary tree. The algorithm should be based
on the Euler tour traversal.
R-8.30 The build expression tree method of the ExpressionTree class requires
input that is an iterable of string tokens. We used a convenient exam-
ple, (((3+1)x4)/((9-5)+2)) , in which each character is its own to-
ken, so that the string itself sufficed as input to build expression tree.
In general, a string, such as (35 + 14) , must be explicitly tokenized
into list [ ( , 35 , + , 14 , ) ] so as to ignore whitespace and to
recognize multidigit numbers as a single token. Write a utility method,
tokenize(raw), that returns such a list of tokens for a raw string.
Creativity
C-8.31 Define the internal path length, I(T), of a tree T to be the sum of the
depths of all the internal positions in T. Likewise, define the external path
length, E(T), of a tree T to be the sum of the depths of all the external
positions in T. Show that if T is a proper binary tree with n positions, then
E(T) = I(T)+n−1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-376-320.jpg)
![Chapter Notes 361
P-8.68 Write a program that can play Tic-Tac-Toe effectively. (See Section 5.6.)
To do this, you will need to create a game tree T, which is a tree where
each position corresponds to a game configuration, which, in this case,
is a representation of the Tic-Tac-Toe board. (See Section 8.4.2.) The
root corresponds to the initial configuration. For each internal position p
in T, the children of p correspond to the game states we can reach from
p’s game state in a single legal move for the appropriate player, A (the
first player) or B (the second player). Positions at even depths correspond
to moves for A and positions at odd depths correspond to moves for B.
Leaves are either final game states or are at a depth beyond which we do
not want to explore. We score each leaf with a value that indicates how
good this state is for player A. In large games, like chess, we have to use a
heuristic scoring function, but for small games, like Tic-Tac-Toe, we can
construct the entire game tree and score leaves as +1, 0, −1, indicating
whether player A has a win, draw, or lose in that configuration. A good
algorithm for choosing moves is minimax. In this algorithm, we assign a
score to each internal position p in T, such that if p represents A’s turn, we
compute p’s score as the maximum of the scores of p’s children (which
corresponds to A’s optimal play from p). If an internal node p represents
B’s turn, then we compute p’s score as the minimum of the scores of p’s
children (which corresponds to B’s optimal play from p).
P-8.69 Implement the tree ADT using the binary tree representation described in
Exercise C-8.43. You may adapt the LinkedBinaryTree implementation.
P-8.70 Write a program that takes as input a general tree T and a position p of T
and converts T to another tree with the same set of position adjacencies,
but now with p as its root.
Chapter Notes
Discussions of the classic preorder, inorder, and postorder tree traversal methods can be
found in Knuth’s Fundamental Algorithms book [64]. The Euler tour traversal technique
comes from the parallel algorithms community; it is introduced by Tarjan and Vishkin [93]
and is discussed by JáJá [54] and by Karp and Ramachandran [58]. The algorithm for
drawing a tree is generally considered to be a part of the “folklore” of graph-drawing al-
gorithms. The reader interested in graph drawing is referred to the book by Di Battista,
Eades, Tamassia, and Tollis [34] and the survey by Tamassia and Liotta [92]. The puzzle
in Exercise R-8.12 was communicated by Micha Sharir.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-383-320.jpg)
![9.3. Heaps 371
(14,E)
(5,A) (6,Z)
(20,B)
(7,Q)
(9,F)
(15,K)
(11,S)
(16,X) (25,J) (13,W)
(12,H)
(4,C)
Figure 9.1: Example of a heap storing 13 entries with integer keys. The last position
is the one storing entry (13,W).
The tree in Figure 9.1 is complete because levels 0, 1, and 2 are full, and the six
nodes in level 3 are in the six leftmost possible positions at that level. In formalizing
what we mean by the leftmost possible positions, we refer to the discussion of level
numbering from Section 8.3.2, in the context of an array-based representation of a
binary tree. (In fact, in Section 9.3.3 we will discuss the use of an array to represent
a heap.) A complete binary tree with n elements is one that has positions with level
numbering 0 through n − 1. For example, in an array-based representation of the
above tree, its 13 entries would be stored consecutively from A[0] to A[12].
The Height of a Heap
Let h denote the height of T. Insisting that T be complete also has an important
consequence, as shown in Proposition 9.2.
Proposition 9.2: A heap T storing n entries has height h = logn.
Justification: From the fact that T is complete, we know that the number of
nodes in levels 0 through h−1 of T is precisely 1+2+4+···+2h−1 = 2h −1, and
that the number of nodes in level h is at least 1 and at most 2h. Therefore
n ≥ 2h
−1+1 = 2h
and n ≤ 2h
−1+2h
= 2h+1
−1.
By taking the logarithm of both sides of inequality 2h ≤ n, we see that height
h ≤ logn. By rearranging terms and taking the logarithm of both sides of inequality
n ≤ 2h+1 − 1, we see that log(n + 1) − 1 ≤ h. Since h is an integer, these two
inequalities imply that h = logn.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-393-320.jpg)
![376 Chapter 9. Priority Queues
9.3.3 Array-Based Representation of a Complete Binary Tree
The array-based representation of a binary tree (Section 8.3.2) is especially suitable
for a complete binary tree T. We recall that in this implementation, the elements
of T are stored in an array-based list A such that the element at position p in T is
stored in A with index equal to the level number f(p) of p, defined as follows:
• If p is the root of T, then f(p) = 0.
• If p is the left child of position q, then f(p) = 2f(q)+1.
• If p is the right child of position q, then f(p) = 2f(q)+2.
With this implementation, the elements of T have contiguous indices in the range
[0,n−1] and the last position of T is always at index n−1, where n is the number
of positions of T. For example, Figure 9.4 illustrates the array-based representation
of the heap structure originally portrayed in Figure 9.1.
0 1 2 3 4 5 6 7 8 9 10 11 12
(4,C) (6,Z) (15,K) (9,F) (7,Q) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (8,W)
(5,A)
Figure 9.4: An array-based representation of the heap from Figure 9.1.
Implementing a priority queue using an array-based heap representation allows
us to avoid some complexities of a node-based tree structure. In particular, the add
and remove min operations of a priority queue both depend on locating the last
index of a heap of size n. With the array-based representation, the last position
is at index n− 1 of the array. Locating the last position of a complete binary tree
implemented with a linked structure requires more effort. (See Exercise C-9.34.)
If the size of a priority queue is not known in advance, use of an array-based
representation does introduce the need to dynamically resize the array on occasion,
as is done with a Python list. The space usage of such an array-based representation
of a complete binary tree with n nodes is O(n), and the time bounds of methods for
adding or removing elements become amortized. (See Section 5.3.1.)
9.3.4 Python Heap Implementation
We provide a Python implementation of a heap-based priority queue in Code Frag-
ments 9.4 and 9.5. We use an array-based representation, maintaining a Python list
of item composites. Although we do not formally use the binary tree ADT, Code
Fragment 9.4 includes nonpublic utility functions that compute the level numbering
of a parent or child of another. This allows us to describe the rest of our algorithms
using tree-like terminology of parent, left, and right. However, the relevant vari-
ables are integer indexes (not “position” objects). We use recursion to implement
the repetition in the upheap and downheap utilities.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-398-320.jpg)
![9.3. Heaps 377
1 class HeapPriorityQueue(PriorityQueueBase): # base class defines Item
2 ”””A min-oriented priority queue implemented with a binary heap.”””
3 #------------------------------ nonpublic behaviors ------------------------------
4 def parent(self, j):
5 return (j−1) // 2
6
7 def left(self, j):
8 return 2 j + 1
9
10 def right(self, j):
11 return 2 j + 2
12
13 def has left(self, j):
14 return self. left(j) len(self. data) # index beyond end of list?
15
16 def has right(self, j):
17 return self. right(j) len(self. data) # index beyond end of list?
18
19 def swap(self, i, j):
20 ”””Swap the elements at indices i and j of array.”””
21 self. data[i], self. data[j] = self. data[j], self. data[i]
22
23 def upheap(self, j):
24 parent = self. parent(j)
25 if j 0 and self. data[j] self. data[parent]:
26 self. swap(j, parent)
27 self. upheap(parent) # recur at position of parent
28
29 def downheap(self, j):
30 if self. has left(j):
31 left = self. left(j)
32 small child = left # although right may be smaller
33 if self. has right(j):
34 right = self. right(j)
35 if self. data[right] self. data[left]:
36 small child = right
37 if self. data[small child] self. data[j]:
38 self. swap(j, small child)
39 self. downheap(small child) # recur at position of small child
Code Fragment 9.4: An implementation of a priority queue using an array-based
heap (continued in Code Fragment 9.5). The extends the PriorityQueueBase class
from Code Fragment 9.1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-399-320.jpg)
![378 Chapter 9. Priority Queues
40 #------------------------------ public behaviors ------------------------------
41 def init (self):
42 ”””Create a new empty Priority Queue.”””
43 self. data = [ ]
44
45 def len (self):
46 ”””Return the number of items in the priority queue.”””
47 return len(self. data)
48
49 def add(self, key, value):
50 ”””Add a key-value pair to the priority queue.”””
51 self. data.append(self. Item(key, value))
52 self. upheap(len(self. data) − 1) # upheap newly added position
53
54 def min(self):
55 ”””Return but do not remove (k,v) tuple with minimum key.
56
57 Raise Empty exception if empty.
58 ”””
59 if self.is empty():
60 raise Empty( Priority queue is empty. )
61 item = self. data[0]
62 return (item. key, item. value)
63
64 def remove min(self):
65 ”””Remove and return (k,v) tuple with minimum key.
66
67 Raise Empty exception if empty.
68 ”””
69 if self.is empty():
70 raise Empty( Priority queue is empty. )
71 self. swap(0, len(self. data) − 1) # put minimum item at the end
72 item = self. data.pop( ) # and remove it from the list;
73 self. downheap(0) # then fix new root
74 return (item. key, item. value)
Code Fragment 9.5: An implementation of a priority queue using an array-based
heap (continued from Code Fragment 9.4).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-400-320.jpg)
![382 Chapter 9. Priority Queues
Python Implementation of a Bottom-Up Heap Construction
Implementing a bottom-up heap construction is quite easy, given the existence of
a “down-heap” utility function. The “merging” of two equally sized heaps that
are subtrees of a common position p, as described in the opening of this section,
can be accomplished simply by down-heaping p’s entry. For example, that is what
happened to the key 14 in going from Figure 9.5(f) to (g).
With our array-based representation of a heap, if we initially store all n items in
arbitrary order within the array, we can implement the bottom-up heap construction
process with a single loop that makes a call to downheap from each position of
the tree, as long as those calls are ordered starting with the deepest level and ending
with the root of the tree. In fact, that loop can start with the deepest nonleaf, since
there is no effect when down-heap is called at a leaf position.
In Code Fragment 9.6, we augment the original HeapPriorityQueue class from
Section 9.3.4 to provide support for the bottom-up construction of an initial col-
lection. We introduce a nonpublic utility method, heapify, that calls downheap
on each nonleaf position, beginning with the deepest and concluding with a call at
the root of the tree. We have redesigned the constructor of the class to accept an
optional parameter that can be any sequence of (k,v) tuples. Rather than initializing
self. data to an empty list, we use a list comprehension syntax (see Section 1.9.2)
to create an initial list of item composites based on the given contents. We de-
clare an empty sequence as the default parameter value so that the default syntax
HeapPriorityQueue() continues to result in an empty priority queue.
def init (self, contents=()):
”””Create a new priority queue.
By default, queue will be empty. If contents is given, it should be as an
iterable sequence of (k,v) tuples specifying the initial contents.
”””
self. data = [ self. Item(k,v) for k,v in contents ] # empty by default
if len(self. data) 1:
self. heapify()
def heapify(self):
start = self. parent(len(self) − 1) # start at PARENT of last leaf
for j in range(start, −1, −1): # going to and including the root
self. downheap(j)
Code Fragment 9.6: Revision to the HeapPriorityQueue class of Code Frag-
ments 9.4 and 9.5 to support a linear-time construction given an initial sequence
of entries.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-404-320.jpg)
![384 Chapter 9. Priority Queues
9.3.7 Python’s heapq Module
Python’s standard distribution includes a heapq module that provides support for
heap-based priority queues. That module does not provide any priority queue class;
instead it provides functions that allow a standard Python list to be managed as a
heap. Its model is essentially the same as our own, with n elements stored in list
cells L[0] through L[n − 1], based on the level-numbering indices with the small-
est element at the root in L[0]. We note that heapq does not separately manage
associated values; elements serve as their own key.
The heapq module supports the following functions, all of which presume that
existing list L satisfies the heap-order property prior to the call:
heappush(L, e): Push element e onto list L and restore the heap-order
property. The function executes in O(logn) time.
heappop(L): Pop and return the element with smallest value from list L,
and reestablish the heap-order property. The operation
executes in O(logn) time.
heappushpop(L, e): Push element e on list L and then pop and return the
smallest item. The time is O(logn), but it is slightly more
efficient than separate calls to push and pop because the
size of the list never changes. If the newly pushed el-
ement becomes the smallest, it is immediately returned.
Otherwise, the new element takes the place of the popped
element at the root and a down-heap is performed.
heapreplace(L, e): Similar to heappushpop, but equivalent to the pop be-
ing performed before the push (in other words, the new
element cannot be returned as the smallest). Again, the
time is O(logn), but it is more efficient that two separate
operations.
The module supports additional functions that operate on sequences that do not
previously satisfy the heap-order property.
heapify(L): Transform unordered list to satisfy the heap-order prop-
erty. This executes in O(n) time by using the bottom-up
construction algorithm.
nlargest(k, iterable): Produce a list of the k largest values from a given iterable.
This can be implemented to run in O(n + klogn) time,
where we use n to denote the length of the iterable (see
Exercise C-9.42).
nsmallest(k, iterable): Produce a list of the k smallest values from a given it-
erable. This can be implemented to run in O(n+klogn)
time, using similar technique as with nlargest.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-406-320.jpg)
![9.5. Adaptable Priority Queues 393
ride the remove min method because the only change in behavior for the adaptable
priority queue is again provided by the overridden swap method.
The update and remove methods provide the core new functionality for the
adaptable priority queue. We perform robust checking of the validity of a locator
that is sent by a caller (although in the interest of space, our displayed code does
not do preliminary type-checking to ensure that the parameter is indeed a Locator
instance). To ensure that a locator is associated with a current element of the given
priority queue, we examine the index that is encapsulated within the locator object,
and then verify that the entry of the list at that index is the very same locator.
In conclusion, the adaptable priority queue provides the same asymptotic effi-
ciency and space usage as the nonadaptive version, and provides logarithmic per-
formance for the new locator-based update and remove methods. A summary of
the performance is given in Table 9.4.
1 class AdaptableHeapPriorityQueue(HeapPriorityQueue):
2 ”””A locator-based priority queue implemented with a binary heap.”””
3
4 #------------------------------ nested Locator class ------------------------------
5 class Locator(HeapPriorityQueue. Item):
6 ”””Token for locating an entry of the priority queue.”””
7 slots = _index # add index as additional field
8
9 def init (self, k, v, j):
10 super(). init (k,v)
11 self. index = j
12
13 #------------------------------ nonpublic behaviors ------------------------------
14 # override swap to record new indices
15 def swap(self, i, j):
16 super(). swap(i,j) # perform the swap
17 self. data[i]. index = i # reset locator index (post-swap)
18 self. data[j]. index = j # reset locator index (post-swap)
19
20 def bubble(self, j):
21 if j 0 and self. data[j] self. data[self. parent(j)]:
22 self. upheap(j)
23 else:
24 self. downheap(j)
Code Fragment 9.8: An implementation of an adaptable priority queue (continued
in Code Fragment 9.9). This extends the HeapPriorityQueue class of Code Frag-
ments 9.4 and 9.5](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-415-320.jpg)
![394 Chapter 9. Priority Queues
25 def add(self, key, value):
26 ”””Add a key-value pair.”””
27 token = self.Locator(key, value, len(self. data)) # initiaize locator index
28 self. data.append(token)
29 self. upheap(len(self. data) − 1)
30 return token
31
32 def update(self, loc, newkey, newval):
33 ”””Update the key and value for the entry identified by Locator loc.”””
34 j = loc. index
35 if not (0 = j len(self) and self. data[j] is loc):
36 raise ValueError( Invalid locator )
37 loc. key = newkey
38 loc. value = newval
39 self. bubble(j)
40
41 def remove(self, loc):
42 ”””Remove and return the (k,v) pair identified by Locator loc.”””
43 j = loc. index
44 if not (0 = j len(self) and self. data[j] is loc):
45 raise ValueError( Invalid locator )
46 if j == len(self) − 1: # item at last position
47 self. data.pop( ) # just remove it
48 else:
49 self. swap(j, len(self)−1) # swap item to the last position
50 self. data.pop( ) # remove it from the list
51 self. bubble(j) # fix item displaced by the swap
52 return (loc. key, loc. value)
Code Fragment 9.9: An implementation of an adaptable priority queue (continued
from Code Fragment 9.8).
Operation Running Time
len(P), P.is empty(), P.min() O(1)
P.add(k,v) O(logn)∗
P.update(loc, k, v) O(logn)
P.remove(loc) O(logn)∗
P.remove min() O(logn)∗
∗amortized with dynamic array
Table 9.4: Running times of the methods of an adaptable priority queue, P, of size n,
realized by means of our array-based heap representation. The space requirement
is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-416-320.jpg)
![400 Chapter 9. Priority Queues
P-9.55 Write a program that can process a sequence of stock buy and sell orders
as described in Exercise C-9.50.
P-9.56 Let S be a set of n points in the plane with distinct integer x- and y-
coordinates. Let T be a complete binary tree storing the points from S
at its external nodes, such that the points are ordered left to right by in-
creasing x-coordinates. For each node v in T, let S(v) denote the subset of
S consisting of points stored in the subtree rooted at v. For the root r of
T, define top(r) to be the point in S = S(r) with maximum y-coordinate.
For every other node v, define top(r) to be the point in S with highest y-
coordinate in S(v) that is not also the highest y-coordinate in S(u), where
u is the parent of v in T (if such a point exists). Such labeling turns T into
a priority search tree. Describe a linear-time algorithm for turning T into
a priority search tree. Implement this approach.
P-9.57 One of the main applications of priority queues is in operating systems—
for scheduling jobs on a CPU. In this project you are to build a program
that schedules simulated CPU jobs. Your program should run in a loop,
each iteration of which corresponds to a time slice for the CPU. Each job
is assigned a priority, which is an integer between −20 (highest priority)
and 19 (lowest priority), inclusive. From among all jobs waiting to be pro-
cessed in a time slice, the CPU must work on a job with highest priority.
In this simulation, each job will also come with a length value, which is an
integer between 1 and 100, inclusive, indicating the number of time slices
that are needed to process this job. For simplicity, you may assume jobs
cannot be interrupted—once it is scheduled on the CPU, a job runs for a
number of time slices equal to its length. Your simulator must output the
name of the job running on the CPU in each time slice and must process
a sequence of commands, one per time slice, each of which is of the form
“add job name with length n and priority p” or “no new job this slice”.
P-9.58 Develop a Python implementation of an adaptable priority queue that is
based on an unsorted list and supports location-aware entries.
Chapter Notes
Knuth’s book on sorting and searching [65] describes the motivation and history for the
selection-sort, insertion-sort, and heap-sort algorithms. The heap-sort algorithm is due
to Williams [103], and the linear-time heap construction algorithm is due to Floyd [39].
Additional algorithms and analyses for heaps and heap-sort variations can be found in
papers by Bentley [15], Carlsson [24], Gonnet and Munro [45], McDiarmid and Reed [74],
and Schaffer and Sedgewick [88].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-422-320.jpg)
![402 Chapter 10. Maps, Hash Tables, and Skip Lists
10.1 Maps and Dictionaries
Python’s dict class is arguably the most significant data structure in the language. It
represents an abstraction known as a dictionary in which unique keys are mapped
to associated values. Because of the relationship they express between keys and
values, dictionaries are commonly known as associative arrays or maps. In this
book, we use the term dictionary when specifically discussing Python’s dict class,
and the term map when discussing the more general notion of the abstract data type.
As a simple example, Figure 10.1 illustrates a map from the names of countries
to their associated units of currency.
Rupee
Turkey Spain China United States India
Greece
Lira Euro Yuan Dollar
Figure 10.1: A map from countries (the keys) to their units of currency (the values).
We note that the keys (the country names) are assumed to be unique, but the values
(the currency units) are not necessarily unique. For example, we note that Spain
and Greece both use the euro for currency. Maps use an array-like syntax for in-
dexing, such as currency[ Greece ] to access a value associated with a given key
or currency[ Greece ] = Drachma to remap it to a new value. Unlike a stan-
dard array, indices for a map need not be consecutive nor even numeric. Common
applications of maps include the following.
• A university’s information system relies on some form of a student ID as a
key that is mapped to that student’s associated record (such as the student’s
name, address, and course grades) serving as the value.
• The domain-name system (DNS) maps a host name, such as www.wiley.com,
to an Internet-Protocol (IP) address, such as 208.215.179.146.
• A social media site typically relies on a (nonnumeric) username as a key that
can be efficiently mapped to a particular user’s associated information.
• A computer graphics system may map a color name, such as turquoise ,
to the triple of numbers that describes the color’s RGB (red-green-blue) rep-
resentation, such as (64,224,208).
• Python uses a dictionary to represent each namespace, mapping an identifying
string, such as pi , to an associated object, such as 3.14159.
In this chapter and the next we demonstrate that a map may be implemented so
that a search for a key, and its associated value, can be performed very efficiently,
thereby supporting fast lookup in such applications.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-424-320.jpg)
![10.1. Maps and Dictionaries 403
10.1.1 The Map ADT
In this section, we introduce the map ADT, and define its behaviors to be consistent
with those of Python’s built-in dict class. We begin by listing what we consider the
most significant five behaviors of a map M as follows:
M[k]: Return the value v associated with key k in map M, if
one exists; otherwise raise a KeyError. In Python, this is
implemented with the special method getitem .
M[k] = v: Associate value v with key k in map M, replacing the ex-
isting value if the map already contains an item with key
equal to k. In Python, this is implemented with the special
method setitem .
del M[k]: Remove from map M the item with key equal to k; if M
has no such item, then raise a KeyError. In Python, this is
implemented with the special method delitem .
len(M): Return the number of items in map M. In Python, this is
implemented with the special method len .
iter(M): The default iteration for a map generates a sequence of
keys in the map. In Python, this is implemented with the
special method iter , and it allows loops of the form,
for k in M.
We have highlighted the above five behaviors because they demonstrate the core
functionality of a map—namely, the ability to query, add, modify, or delete a key-
value pair, and the ability to report all such pairs. For additional convenience,
map M should also support the following behaviors:
k in M: Return True if the map contains an item with key k. In
Python, this is implemented with the special contains
method.
M.get(k, d=None): Return M[k] if key k exists in the map; otherwise return
default value d. This provides a form to query M[k] with-
out risk of a KeyError.
M.setdefault(k, d): If key k exists in the map, simply return M[k]; if key k
does not exist, set M[k] = d and return that value.
M.pop(k, d=None): Remove the item associated with key k from the map and
return its associated value v. If key k is not in the map,
return default value d (or raise KeyError if parameter d is
None).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-425-320.jpg)
![404 Chapter 10. Maps, Hash Tables, and Skip Lists
M.popitem(): Remove an arbitrary key-value pair from the map, and re-
turn a (k,v) tuple representing the removed pair. If map is
empty, raise a KeyError.
M.clear(): Remove all key-value pairs from the map.
M.keys(): Return a set-like view of all keys of M.
M.values(): Return a set-like view of all values of M.
M.items(): Return a set-like view of (k,v) tuples for all entries of M.
M.update(M2): Assign M[k] = v for every (k,v) pair in map M2.
M == M2: Return True if maps M and M2 have identical key-value
associations.
M != M2: Return True if maps M and M2 do not have identical key-
value associations.
Example 10.1: In the following, we show the effect of a series of operations on
an initially empty map storing items with integer keys and single-character values.
We use the literal syntax for Python’s dict class to describe the map contents.
Operation Return Value Map
len(M) 0 { }
M[ K ] = 2 – { K : 2}
M[ B ] = 4 – { K : 2, B : 4}
M[ U ] = 2 – { K : 2, B : 4, U : 2}
M[ V ] = 8 – { K : 2, B : 4, U : 2, V : 8}
M[ K ] = 9 – { K : 9, B : 4, U : 2, V : 8}
M[ B ] 4 { K : 9, B : 4, U : 2, V : 8}
M[ X ] KeyError { K : 9, B : 4, U : 2, V : 8}
M.get( F ) None { K : 9, B : 4, U : 2, V : 8}
M.get( F , 5) 5 { K : 9, B : 4, U : 2, V : 8}
M.get( K , 5) 9 { K : 9, B : 4, U : 2, V : 8}
len(M) 4 { K : 9, B : 4, U : 2, V : 8}
del M[ V ] – { K : 9, B : 4, U : 2}
M.pop( K ) 9 { B : 4, U : 2}
M.keys() B , U { B : 4, U : 2}
M.values() 4, 2 { B : 4, U : 2}
M.items() ( B , 4), ( U , 2) { B : 4, U : 2}
M.setdefault( B , 1) 4 { B : 4, U : 2}
M.setdefault( A , 1) 1 { A : 1, B : 4, U : 2}
M.popitem() ( B , 4) { A : 1, U : 2}](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-426-320.jpg)
![10.1. Maps and Dictionaries 405
10.1.2 Application: Counting Word Frequencies
As a case study for using a map, consider the problem of counting the number
of occurrences of words in a document. This is a standard task when performing a
statistical analysis of a document, for example, when categorizing an email or news
article. A map is an ideal data structure to use here, for we can use words as keys
and word counts as values. We show such an application in Code Fragment 10.1.
We break apart the original document using a combination of file and string
methods that results in a loop over a lowercased version of all whitespace separated
pieces of the document. We omit all nonalphabetic characters so that parentheses,
apostrophes, and other such punctuation are not considered part of a word.
In terms of map operations, we begin with an empty Python dictionary named
freq. During the first phase of the algorithm, we execute the command
freq[word] = 1 + freq.get(word, 0)
for each word occurrence. We use the get method on the right-hand side because the
current word might not exist in the dictionary; the default value of 0 is appropriate
in that case.
During the second phase of the algorithm, after the full document has been pro-
cessed, we examine the contents of the frequency map, looping over freq.items()
to determine which word has the most occurrences.
1 freq = { }
2 for piece in open(filename).read().lower().split():
3 # only consider alphabetic characters within this piece
4 word = .join(c for c in piece if c.isalpha())
5 if word: # require at least one alphabetic character
6 freq[word] = 1 + freq.get(word, 0)
7
8 max word =
9 max count = 0
10 for (w,c) in freq.items(): # (key, value) tuples represent (word, count)
11 if c max count:
12 max word = w
13 max count = c
14 print( The most frequent word is , max word)
15 print( Its number of occurrences is , max count)
Code Fragment 10.1: A program for counting word frequencies in a document, and
reporting the most frequent word. We use Python’s dict class for the map. We
convert the input to lowercase and ignore any nonalphabetic characters.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-427-320.jpg)
![406 Chapter 10. Maps, Hash Tables, and Skip Lists
10.1.3 Python’s MutableMapping Abstract Base Class
Section 2.4.3 provides an introduction to the concept of an abstract base class
and the role of such classes in Python’s collections module. Methods that are de-
clared to be abstract in such a base class must be implemented by concrete sub-
classes. However, an abstract base class may provide concrete implementation of
other methods that depend upon use of the presumed abstract methods. (This is an
example of the template method design pattern.)
The collections module provides two abstract base classes that are relevant to
our current discussion: the Mapping and MutableMapping classes. The Mapping
class includes all nonmutating methods supported by Python’s dict class, while the
MutableMapping class extends that to include the mutating methods. What we
define as the map ADT in Section 10.1.1 is akin to the MutableMapping abstract
base class in Python’s collections module.
The significance of these abstract base classes is that they provide a framework
to assist in creating a user-defined map class. In particular, the MutableMapping
class provides concrete implementations for all behaviors other than the first five
outlined in Section 10.1.1: getitem , setitem , delitem , len , and
iter . As we implement the map abstraction with various data structures, as
long as we provide the five core behaviors, we can inherit all other derived behav-
iors by simply declaring MutableMapping as a parent class.
To better understand the MutableMapping class, we provide a few examples of
how concrete behaviors can be derived from the five core abstractions. For example,
the contains method, supporting the syntax k in M, could be implemented by
making a guarded attempt to retrieve self[k] to determine if the key exists.
def contains (self, k):
try:
self[k] # access via getitem (ignore result)
return True
except KeyError:
return False # attempt failed
A similar approach might be used to provide the logic of the setdefault method.
def setdefault(self, k, d):
try:
return self[k] # if getitem succeeds, return value
except KeyError: # otherwise:
self[k] = d # set default value with setitem
return d # and return that newly assigned value
We leave as exercises the implementations of the remaining concrete methods of
the MutableMapping class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-428-320.jpg)
![10.1. Maps and Dictionaries 409
1 class UnsortedTableMap(MapBase):
2 ”””Map implementation using an unordered list.”””
3
4 def init (self):
5 ”””Create an empty map.”””
6 self. table = [ ] # list of Item’s
7
8 def getitem (self, k):
9 ”””Return value associated with key k (raise KeyError if not found).”””
10 for item in self. table:
11 if k == item. key:
12 return item. value
13 raise KeyError( Key Error: + repr(k))
14
15 def setitem (self, k, v):
16 ”””Assign value v to key k, overwriting existing value if present.”””
17 for item in self. table:
18 if k == item. key: # Found a match:
19 item. value = v # reassign value
20 return # and quit
21 # did not find match for key
22 self. table.append(self. Item(k,v))
23
24 def delitem (self, k):
25 ”””Remove item associated with key k (raise KeyError if not found).”””
26 for j in range(len(self. table)):
27 if k == self. table[j]. key: # Found a match:
28 self. table.pop(j) # remove item
29 return # and quit
30 raise KeyError( Key Error: + repr(k))
31
32 def len (self):
33 ”””Return number of items in the map.”””
34 return len(self. table)
35
36 def iter (self):
37 ”””Generate iteration of the map s keys.”””
38 for item in self. table:
39 yield item. key # yield the KEY
Code Fragment 10.3: An implementation of a map using a Python list as an unsorted
table. Parent class MapBase is given in Code Fragment 10.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-431-320.jpg)
![410 Chapter 10. Maps, Hash Tables, and Skip Lists
10.2 Hash Tables
In this section, we introduce one of the most practical data structures for imple-
menting a map, and the one that is used by Python’s own implementation of the
dict class. This structure is known as a hash table.
Intuitively, a map M supports the abstraction of using keys as indices with a
syntax such as M[k]. As a mental warm-up, consider a restricted setting in which
a map with n items uses keys that are known to be integers in a range from 0 to
N −1 for some N ≥ n. In this case, we can represent the map using a lookup table
of length N, as diagrammed in Figure 10.3.
0 1 2 3 4 5 6 7 8 9 10
D Z C Q
Figure 10.3: A lookup table with length 11 for a map containing items (1,D), (3,Z),
(6,C), and (7,Q).
In this representation, we store the value associated with key k at index k of the
table (presuming that we have a distinct way to represent an empty slot). Basic map
operations of getitem , setitem , and delitem can be implemented in
O(1) worst-case time.
There are two challenges in extending this framework to the more general set-
ting of a map. First, we may not wish to devote an array of length N if it is the case
that N n. Second, we do not in general require that a map’s keys be integers.
The novel concept for a hash table is the use of a hash function to map general
keys to corresponding indices in a table. Ideally, keys will be well distributed in the
range from 0 to N −1 by a hash function, but in practice there may be two or more
distinct keys that get mapped to the same index. As a result, we will conceptualize
our table as a bucket array, as shown in Figure 10.4, in which each bucket may
manage a collection of items that are sent to a specific index by the hash function.
(To save space, an empty bucket may be replaced by None.)
0 1 2 3 4 5 6 7 8 9 10
(1,D) (25,C)
(3,F)
(14,Z)
(39,C)
(6,A) (7,Q)
Figure 10.4: A bucket array of capacity 11 with items (1,D), (25,C), (3,F), (14,Z),
(6,A), (39,C), and (7,Q), using a simple hash function.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-432-320.jpg)
![10.2. Hash Tables 411
10.2.1 Hash Functions
The goal of a hash function, h, is to map each key k to an integer in the range
[0,N − 1], where N is the capacity of the bucket array for a hash table. Equipped
with such a hash function, h, the main idea of this approach is to use the hash
function value, h(k), as an index into our bucket array, A, instead of the key k
(which may not be appropriate for direct use as an index). That is, we store the
item (k,v) in the bucket A[h(k)].
If there are two or more keys with the same hash value, then two different items
will be mapped to the same bucket in A. In this case, we say that a collision has
occurred. To be sure, there are ways of dealing with collisions, which we will
discuss later, but the best strategy is to try to avoid them in the first place. We say
that a hash function is “good” if it maps the keys in our map so as to sufficiently
minimize collisions. For practical reasons, we also would like a hash function to
be fast and easy to compute.
It is common to view the evaluation of a hash function, h(k), as consisting of
two portions—a hash code that maps a key k to an integer, and a compression
function that maps the hash code to an integer within a range of indices, [0,N −1],
for a bucket array. (See Figure 10.5.)
-1
hash code
1 2
0
-2
... ...
compression function
1 2
0 N-1
...
Arbitrary Objects
Figure 10.5: Two parts of a hash function: a hash code and a compression function.
The advantage of separating the hash function into two such components is that
the hash code portion of that computation is independent of a specific hash table
size. This allows the development of a general hash code for each object that can
be used for a hash table of any size; only the compression function depends upon
the table size. This is particularly convenient, because the underlying bucket array
for a hash table may be dynamically resized, depending on the number of items
currently stored in the map. (See Section 10.2.3.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-433-320.jpg)
![412 Chapter 10. Maps, Hash Tables, and Skip Lists
Hash Codes
The first action that a hash function performs is to take an arbitrary key k in our
map and compute an integer that is called the hash code for k; this integer need not
be in the range [0,N −1], and may even be negative. We desire that the set of hash
codes assigned to our keys should avoid collisions as much as possible. For if the
hash codes of our keys cause collisions, then there is no hope for our compression
function to avoid them. In this subsection, we begin by discussing the theory of
hash codes. Following that, we discuss practical implementations of hash codes in
Python.
Treating the Bit Representation as an Integer
To begin, we note that, for any data type X that is represented using at most as many
bits as our integer hash codes, we can simply take as a hash code for X an integer
interpretation of its bits. For example, the hash code for key 314 could simply be
314. The hash code for a floating-point number such as 3.14 could be based upon
an interpretation of the bits of the floating-point representation as an integer.
For a type whose bit representation is longer than a desired hash code, the above
scheme is not immediately applicable. For example, Python relies on 32-bit hash
codes. If a floating-point number uses a 64-bit representation, its bits cannot be
viewed directly as a hash code. One possibility is to use only the high-order 32 bits
(or the low-order 32 bits). This hash code, of course, ignores half of the information
present in the original key, and if many of the keys in our map only differ in these
bits, then they will collide using this simple hash code.
A better approach is to combine in some way the high-order and low-order por-
tions of a 64-bit key to form a 32-bit hash code, which takes all the original bits
into consideration. A simple implementation is to add the two components as 32-
bit numbers (ignoring overflow), or to take the exclusive-or of the two components.
These approaches of combining components can be extended to any object x whose
binary representation can be viewed as an n-tuple (x0,x1,...,xn−1) of 32-bit inte-
gers, for example, by forming a hash code for x as ∑n−1
i=0 xi, or as x0 ⊕x1 ⊕···⊕xn−1,
where the ⊕ symbol represents the bitwise exclusive-or operation (which is ˆ in
Python).
Polynomial Hash Codes
The summation and exclusive-or hash codes, described above, are not good choices
for character strings or other variable-length objects that can be viewed as tuples of
the form (x0,x1,...,xn−1), where the order of the xi’s is significant. For example,
consider a 16-bit hash code for a character string s that sums the Unicode values
of the characters in s. This hash code unfortunately produces lots of unwanted](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-434-320.jpg)
![416 Chapter 10. Maps, Hash Tables, and Skip Lists
Compression Functions
The hash code for a key k will typically not be suitable for immediate use with a
bucket array, because the integer hash code may be negative or may exceed the ca-
pacity of the bucket array. Thus, once we have determined an integer hash code for
a key object k, there is still the issue of mapping that integer into the range [0,N −1].
This computation, known as a compression function, is the second action per-
formed as part of an overall hash function. A good compression function is one
that minimizes the number of collisions for a given set of distinct hash codes.
The Division Method
A simple compression function is the division method, which maps an integer i to
i mod N,
where N, the size of the bucket array, is a fixed positive integer. Additionally, if we
take N to be a prime number, then this compression function helps “spread out” the
distribution of hashed values. Indeed, if N is not prime, then there is greater risk
that patterns in the distribution of hash codes will be repeated in the distribution of
hash values, thereby causing collisions. For example, if we insert keys with hash
codes {200,205,210,215,220,... ,600} into a bucket array of size 100, then each
hash code will collide with three others. But if we use a bucket array of size 101,
then there will be no collisions. If a hash function is chosen well, it should ensure
that the probability of two different keys getting hashed to the same bucket is 1/N.
Choosing N to be a prime number is not always enough, however, for if there is
a repeated pattern of hash codes of the form pN + q for several different p’s, then
there will still be collisions.
The MAD Method
A more sophisticated compression function, which helps eliminate repeated pat-
terns in a set of integer keys, is the Multiply-Add-and-Divide (or “MAD”) method.
This method maps an integer i to
[(ai+b) mod p] mod N,
where N is the size of the bucket array, p is a prime number larger than N, and a
and b are integers chosen at random from the interval [0, p− 1], with a 0. This
compression function is chosen in order to eliminate repeated patterns in the set of
hash codes and get us closer to having a “good” hash function, that is, one such that
the probability any two different keys collide is 1/N. This good behavior would be
the same as we would have if these keys were “thrown” into A uniformly at random.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-438-320.jpg)
![10.2. Hash Tables 417
10.2.2 Collision-Handling Schemes
The main idea of a hash table is to take a bucket array, A, and a hash function, h, and
use them to implement a map by storing each item (k,v) in the “bucket” A[h(k)].
This simple idea is challenged, however, when we have two distinct keys, k1 and k2,
such that h(k1) = h(k2). The existence of such collisions prevents us from simply
inserting a new item (k,v) directly into the bucket A[h(k)]. It also complicates our
procedure for performing insertion, search, and deletion operations.
Separate Chaining
A simple and efficient way for dealing with collisions is to have each bucket A[ j]
store its own secondary container, holding items (k,v) such that h(k) = j. A natural
choice for the secondary container is a small map instance implemented using a list,
as described in Section 10.1.5. This collision resolution rule is known as separate
chaining, and is illustrated in Figure 10.6.
A
1 2 3 4 5 6 7 8 9 10
0 11 12
12
38
25
90
54
28
41
36
18 10
Figure 10.6: A hash table of size 13, storing 10 items with integer keys, with colli-
sions resolved by separate chaining. The compression function is h(k) = k mod 13.
For simplicity, we do not show the values associated with the keys.
In the worst case, operations on an individual bucket take time proportional to
the size of the bucket. Assuming we use a good hash function to index the n items
of our map in a bucket array of capacity N, the expected size of a bucket is n/N.
Therefore, if given a good hash function, the core map operations run in O( n/N).
The ratio λ = n/N, called the load factor of the hash table, should be bounded by
a small constant, preferably below 1. As long as λ is O(1), the core operations on
the hash table run in O(1) expected time.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-439-320.jpg)
![418 Chapter 10. Maps, Hash Tables, and Skip Lists
Open Addressing
The separate chaining rule has many nice properties, such as affording simple im-
plementations of map operations, but it nevertheless has one slight disadvantage:
It requires the use of an auxiliary data structure—a list—to hold items with collid-
ing keys. If space is at a premium (for example, if we are writing a program for a
small handheld device), then we can use the alternative approach of always storing
each item directly in a table slot. This approach saves space because no auxiliary
structures are employed, but it requires a bit more complexity to deal with colli-
sions. There are several variants of this approach, collectively referred to as open
addressing schemes, which we discuss next. Open addressing requires that the load
factor is always at most 1 and that items are stored directly in the cells of the bucket
array itself.
Linear Probing and Its Variants
A simple method for collision handling with open addressing is linear probing.
With this approach, if we try to insert an item (k,v) into a bucket A[ j] that is already
occupied, where j = h(k), then we next try A[(j +1) mod N]. If A[(j +1) mod N]
is also occupied, then we try A[(j + 2) mod N], and so on, until we find an empty
bucket that can accept the new item. Once this bucket is located, we simply in-
sert the item there. Of course, this collision resolution strategy requires that we
change the implementation when searching for an existing key—the first step of all
getitem , setitem , or delitem operations. In particular, to attempt
to locate an item with key equal to k, we must examine consecutive slots, starting
from A[h(k)], until we either find an item with that key or we find an empty bucket.
(See Figure 10.7.) The name “linear probing” comes from the fact that accessing a
cell of the bucket array can be viewed as a “probe.”
26
1 2 3 4 5 6 7 8 9 10
0
New element with
key = 15 to be inserted
Must probe 4 times
before finding empty slot
5 37 16 21
13
Figure 10.7: Insertion into a hash table with integer keys using linear probing. The
hash function is h(k) = k mod 11. Values associated with keys are not shown.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-440-320.jpg)
![10.2. Hash Tables 419
To implement a deletion, we cannot simply remove a found item from its slot
in the array. For example, after the insertion of key 15 portrayed in Figure 10.7,
if the item with key 37 were trivially deleted, a subsequent search for 15 would
fail because that search would start by probing at index 4, then index 5, and then
index 6, at which an empty cell is found. A typical way to get around this diffi-
culty is to replace a deleted item with a special “available” marker object. With
this special marker possibly occupying spaces in our hash table, we modify our
search algorithm so that the search for a key k will skip over cells containing the
available marker and continue probing until reaching the desired item or an empty
bucket (or returning back to where we started from). Additionally, our algorithm
for setitem should remember an available cell encountered during the search
for k, since this is a valid place to put a new item (k,v), if no existing item is found.
Although use of an open addressing scheme can save space, linear probing
suffers from an additional disadvantage. It tends to cluster the items of a map into
contiguous runs, which may even overlap (particularly if more than half of the cells
in the hash table are occupied). Such contiguous runs of occupied hash cells cause
searches to slow down considerably.
Another open addressing strategy, known as quadratic probing, iteratively tries
the buckets A[(h(k)+ f(i)) mod N], for i = 0,1,2,..., where f(i) = i2, until finding
an empty bucket. As with linear probing, the quadratic probing strategy compli-
cates the removal operation, but it does avoid the kinds of clustering patterns that
occur with linear probing. Nevertheless, it creates its own kind of clustering, called
secondary clustering, where the set of filled array cells still has a non-uniform
pattern, even if we assume that the original hash codes are distributed uniformly.
When N is prime and the bucket array is less than half full, the quadratic probing
strategy is guaranteed to find an empty slot. However, this guarantee is not valid
once the table becomes at least half full, or if N is not chosen as a prime number;
we explore the cause of this type of clustering in an exercise (C-10.36).
An open addressing strategy that does not cause clustering of the kind produced
by linear probing or the kind produced by quadratic probing is the double hashing
strategy. In this approach, we choose a secondary hash function, h, and if h maps
some key k to a bucket A[h(k)] that is already occupied, then we iteratively try
the buckets A[(h(k) + f(i)) mod N] next, for i = 1,2,3,..., where f(i) = i · h(k).
In this scheme, the secondary hash function is not allowed to evaluate to zero; a
common choice is h(k) = q−(k mod q), for some prime number q N. Also, N
should be a prime.
Another approach to avoid clustering with open addressing is to iteratively try
buckets A[(h(k) + f(i)) mod N] where f(i) is based on a pseudo-random number
generator, providing a repeatable, but somewhat arbitrary, sequence of subsequent
probes that depends upon bits of the original hash code. This is the approach cur-
rently used by Python’s dictionary class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-441-320.jpg)
![10.2. Hash Tables 421
Operation List Hash Table
expected worst case
getitem O(n) O(1) O(n)
setitem O(n) O(1) O(n)
delitem O(n) O(1) O(n)
len O(1) O(1) O(1)
iter O(n) O(n) O(n)
Table 10.2: Comparison of the running times of the methods of a map realized by
means of an unsorted list (as in Section 10.1.5) or a hash table. We let n denote
the number of items in the map, and we assume that the bucket array supporting
the hash table is maintained such that its capacity is proportional to the number of
items in the map.
In practice, hash tables are among the most efficient means for implementing
a map, and it is essentially taken for granted by programmers that their core oper-
ations run in constant time. Python’s dict class is implemented with hashing, and
the Python interpreter relies on dictionaries to retrieve an object that is referenced
by an identifier in a given namespace. (See Sections 1.10 and 2.5.) The basic com-
mand c = a + b involves two calls to getitem in the dictionary for the local
namespace to retrieve the values identified as a and b, and a call to setitem
to store the result associated with name c in that namespace. In our own algorithm
analysis, we simply presume that such dictionary operations run in constant time,
independent of the number of entries in the namespace. (Admittedly, the number
of entries in a typical namespace can almost surely be bounded by a constant.)
In a 2003 academic paper [31], researchers discuss the possibility of exploiting
a hash table’s worst-case performance to cause a denial-of-service (DoS) attack
of Internet technologies. For many published algorithms that compute hash codes,
they note that an attacker could precompute a very large number of moderate-length
strings that all hash to the identical 32-bit hash code. (Recall that by any of the
hashing schemes we describe, other than double hashing, if two keys are mapped
to the same hash code, they will be inseparable in the collision resolution.)
In late 2011, another team of researchers demonstrated an implementation of
just such an attack [61]. Web servers allow a series of key-value parameters to be
embedded in a URL using a syntax such as ?key1=val1key2=val2key3=val3.
Typically, those key-value pairs are immediately stored in a map by the server,
and a limit is placed on the length and number of such parameters presuming that
storage time in the map will be linear in the number of entries. If all keys were
to collide, that storage requires quadratic time (causing the server to perform an
inordinate amount of work). In spring of 2012, Python developers distributed a
security patch that introduces randomization into the computation of hash codes
for strings, making it less tractable to reverse engineer a set of colliding strings.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-443-320.jpg)
![10.2. Hash Tables 423
1 class HashMapBase(MapBase):
2 ”””Abstract base class for map using hash-table with MAD compression.”””
3
4 def init (self, cap=11, p=109345121):
5 ”””Create an empty hash-table map.”””
6 self. table = cap [ None ]
7 self. n = 0 # number of entries in the map
8 self. prime = p # prime for MAD compression
9 self. scale = 1 + randrange(p−1) # scale from 1 to p-1 for MAD
10 self. shift = randrange(p) # shift from 0 to p-1 for MAD
11
12 def hash function(self, k):
13 return (hash(k) self. scale + self. shift) % self. prime % len(self. table)
14
15 def len (self):
16 return self. n
17
18 def getitem (self, k):
19 j = self. hash function(k)
20 return self. bucket getitem(j, k) # may raise KeyError
21
22 def setitem (self, k, v):
23 j = self. hash function(k)
24 self. bucket setitem(j, k, v) # subroutine maintains self. n
25 if self. n len(self. table) // 2: # keep load factor = 0.5
26 self. resize(2 len(self. table) − 1) # number 2ˆx - 1 is often prime
27
28 def delitem (self, k):
29 j = self. hash function(k)
30 self. bucket delitem(j, k) # may raise KeyError
31 self. n −= 1
32
33 def resize(self, c): # resize bucket array to capacity c
34 old = list(self.items()) # use iteration to record existing items
35 self. table = c [None] # then reset table to desired capacity
36 self. n = 0 # n recomputed during subsequent adds
37 for (k,v) in old:
38 self[k] = v # reinsert old key-value pair
Code Fragment 10.4: A base class for our hash table implementations, extending
our MapBase class from Code Fragment 10.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-445-320.jpg)
![424 Chapter 10. Maps, Hash Tables, and Skip Lists
Separate Chaining
Code Fragment 10.5 provides a concrete implementation of a hash table with sepa-
rate chaining, in the form of the ChainHashMap class. To represent a single bucket,
it relies on an instance of the UnsortedTableMap class from Code Fragment 10.3.
The first three methods in the class use index j to access the potential bucket in
the bucket array, and a check for the special case in which that table entry is None.
The only time we need a new bucket structure is when bucket setitem is called on
an otherwise empty slot. The remaining functionality relies on map behaviors that
are already supported by the individual UnsortedTableMap instances. We need a
bit of forethought to determine whether the application of setitem on the chain
causes a net increase in the size of the map (that is, whether the given key is new).
1 class ChainHashMap(HashMapBase):
2 ”””Hash map implemented with separate chaining for collision resolution.”””
3
4 def bucket getitem(self, j, k):
5 bucket = self. table[j]
6 if bucket is None:
7 raise KeyError( Key Error: + repr(k)) # no match found
8 return bucket[k] # may raise KeyError
9
10 def bucket setitem(self, j, k, v):
11 if self. table[j] is None:
12 self. table[j] = UnsortedTableMap( ) # bucket is new to the table
13 oldsize = len(self. table[j])
14 self. table[j][k] = v
15 if len(self. table[j]) oldsize: # key was new to the table
16 self. n += 1 # increase overall map size
17
18 def bucket delitem(self, j, k):
19 bucket = self. table[j]
20 if bucket is None:
21 raise KeyError( Key Error: + repr(k)) # no match found
22 del bucket[k] # may raise KeyError
23
24 def iter (self):
25 for bucket in self. table:
26 if bucket is not None: # a nonempty slot
27 for key in bucket:
28 yield key
Code Fragment 10.5: Concrete hash map class with separate chaining.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-446-320.jpg)
![10.2. Hash Tables 425
Linear Probing
Our implementation of a ProbeHashMap class, using open addressing with linear
probing, is given in Code Fragments 10.6 and 10.7. In order to support deletions,
we use a technique described in Section 10.2.2 in which we place a special marker
in a table location at which an item has been deleted, so that we can distinguish
between it and a location that has always been empty. In our implementation, we
declare a class-level attribute, AVAIL, as a sentinel. (We use an instance of the
built-in object class because we do not care about any behaviors of the sentinel,
just our ability to differentiate it from other objects.)
The most challenging aspect of open addressing is to properly trace the series
of probes when collisions occur during an insertion or search for an item. To this
end, we define a nonpublic utility, find slot, that searches for an item with key k
in “bucket” j (that is, where j is the index returned by the hash function for key k).
1 class ProbeHashMap(HashMapBase):
2 ”””Hash map implemented with linear probing for collision resolution.”””
3 AVAIL = object( ) # sentinal marks locations of previous deletions
4
5 def is available(self, j):
6 ”””Return True if index j is available in table.”””
7 return self. table[j] is None or self. table[j] is ProbeHashMap. AVAIL
8
9 def find slot(self, j, k):
10 ”””Search for key k in bucket at index j.
11
12 Return (success, index) tuple, described as follows:
13 If match was found, success is True and index denotes its location.
14 If no match found, success is False and index denotes first available slot.
15 ”””
16 firstAvail = None
17 while True:
18 if self. is available(j):
19 if firstAvail is None:
20 firstAvail = j # mark this as first avail
21 if self. table[j] is None:
22 return (False, firstAvail) # search has failed
23 elif k == self. table[j]. key:
24 return (True, j) # found a match
25 j = (j + 1) % len(self. table) # keep looking (cyclically)
Code Fragment 10.6: Concrete ProbeHashMap class that uses linear probing for
collision resolution (continued in Code Fragment 10.7).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-447-320.jpg)
![426 Chapter 10. Maps, Hash Tables, and Skip Lists
26 def bucket getitem(self, j, k):
27 found, s = self. find slot(j, k)
28 if not found:
29 raise KeyError( Key Error: + repr(k)) # no match found
30 return self. table[s]. value
31
32 def bucket setitem(self, j, k, v):
33 found, s = self. find slot(j, k)
34 if not found:
35 self. table[s] = self. Item(k,v) # insert new item
36 self. n += 1 # size has increased
37 else:
38 self. table[s]. value = v # overwrite existing
39
40 def bucket delitem(self, j, k):
41 found, s = self. find slot(j, k)
42 if not found:
43 raise KeyError( Key Error: + repr(k)) # no match found
44 self. table[s] = ProbeHashMap. AVAIL # mark as vacated
45
46 def iter (self):
47 for j in range(len(self. table)): # scan entire table
48 if not self. is available(j):
49 yield self. table[j]. key
Code Fragment 10.7: Concrete ProbeHashMap class that uses linear probing for
collision resolution (continued from Code Fragment 10.6).
The three primary map operations each rely on the find slot utility. When at-
tempting to retrieve the value associated with a given key, we must continue probing
until we find the key, or until we reach a table slot with the None value. We cannot
stop the search upon reaching an AVAIL sentinel, because it represents a location
that may have been filled when the desired item was once inserted.
When a key-value pair is being assigned in the map, we must attempt to find
an existing item with the given key, so that we might overwrite its value, before
adding a new item to the map. Therefore, we must search beyond any occurrences
of the AVAIL sentinel when inserting. However, if no match is found, we prefer to
repurpose the first slot marked with AVAIL, if any, when placing the new element
in the table. The find slot method enacts this logic, continuing the search until a
truly empty slot, but returning the index of the first available slot for an insertion.
When deleting an existing item within bucket delitem, we intentionally set
the table entry to the AVAIL sentinel in accordance with our strategy.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-448-320.jpg)
![430 Chapter 10. Maps, Hash Tables, and Skip Lists
1 class SortedTableMap(MapBase):
2 ”””Map implementation using a sorted table.”””
3
4 #----------------------------- nonpublic behaviors -----------------------------
5 def find index(self, k, low, high):
6 ”””Return index of the leftmost item with key greater than or equal to k.
7
8 Return high + 1 if no such item qualifies.
9
10 That is, j will be returned such that:
11 all items of slice table[low:j] have key k
12 all items of slice table[j:high+1] have key = k
13 ”””
14 if high low:
15 return high + 1 # no element qualifies
16 else:
17 mid = (low + high) // 2
18 if k == self. table[mid]. key:
19 return mid # found exact match
20 elif k self. table[mid]. key:
21 return self. find index(k, low, mid − 1) # Note: may return mid
22 else:
23 return self. find index(k, mid + 1, high) # answer is right of mid
24
25 #----------------------------- public behaviors -----------------------------
26 def init (self):
27 ”””Create an empty map.”””
28 self. table = [ ]
29
30 def len (self):
31 ”””Return number of items in the map.”””
32 return len(self. table)
33
34 def getitem (self, k):
35 ”””Return value associated with key k (raise KeyError if not found).”””
36 j = self. find index(k, 0, len(self. table) − 1)
37 if j == len(self. table) or self. table[j]. key != k:
38 raise KeyError( Key Error: + repr(k))
39 return self. table[j]. value
Code Fragment 10.8: An implementation of a SortedTableMap class (continued in
Code Fragments 10.9 and 10.10).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-452-320.jpg)
![10.3. Sorted Maps 431
40 def setitem (self, k, v):
41 ”””Assign value v to key k, overwriting existing value if present.”””
42 j = self. find index(k, 0, len(self. table) − 1)
43 if j len(self. table) and self. table[j]. key == k:
44 self. table[j]. value = v # reassign value
45 else:
46 self. table.insert(j, self. Item(k,v)) # adds new item
47
48 def delitem (self, k):
49 ”””Remove item associated with key k (raise KeyError if not found).”””
50 j = self. find index(k, 0, len(self. table) − 1)
51 if j == len(self. table) or self. table[j]. key != k:
52 raise KeyError( Key Error: + repr(k))
53 self. table.pop(j) # delete item
54
55 def iter (self):
56 ”””Generate keys of the map ordered from minimum to maximum.”””
57 for item in self. table:
58 yield item. key
59
60 def reversed (self):
61 ”””Generate keys of the map ordered from maximum to minimum.”””
62 for item in reversed(self. table):
63 yield item. key
64
65 def find min(self):
66 ”””Return (key,value) pair with minimum key (or None if empty).”””
67 if len(self. table) 0:
68 return (self. table[0]. key, self. table[0]. value)
69 else:
70 return None
71
72 def find max(self):
73 ”””Return (key,value) pair with maximum key (or None if empty).”””
74 if len(self. table) 0:
75 return (self. table[−1]. key, self. table[−1]. value)
76 else:
77 return None
Code Fragment 10.9: An implementation of a SortedTableMap class (together with
Code Fragments 10.8 and 10.10).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-453-320.jpg)
![432 Chapter 10. Maps, Hash Tables, and Skip Lists
78 def find ge(self, k):
79 ”””Return (key,value) pair with least key greater than or equal to k.”””
80 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
81 if j len(self. table):
82 return (self. table[j]. key, self. table[j]. value)
83 else:
84 return None
85
86 def find lt(self, k):
87 ”””Return (key,value) pair with greatest key strictly less than k.”””
88 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
89 if j 0:
90 return (self. table[j−1]. key, self. table[j−1]. value) # Note use of j-1
91 else:
92 return None
93
94 def find gt(self, k):
95 ”””Return (key,value) pair with least key strictly greater than k.”””
96 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
97 if j len(self. table) and self. table[j]. key == k:
98 j += 1 # advanced past match
99 if j len(self. table):
100 return (self. table[j]. key, self. table[j]. value)
101 else:
102 return None
103
104 def find range(self, start, stop):
105 ”””Iterate all (key,value) pairs such that start = key stop.
106
107 If start is None, iteration begins with minimum key of map.
108 If stop is None, iteration continues through the maximum key of map.
109 ”””
110 if start is None:
111 j = 0
112 else:
113 j = self. find index(start, 0, len(self. table)−1) # find first result
114 while j len(self. table) and (stop is None or self. table[j]. key stop):
115 yield (self. table[j]. key, self. table[j]. value)
116 j += 1
Code Fragment 10.10: An implementation of a SortedTableMap class (continued
from Code Fragments 10.9 and 10.10). We omit the find le method due to space.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-454-320.jpg)
![10.3. Sorted Maps 433
Analysis
We conclude by analyzing the performance of our SortedTableMap implementa-
tion. A summary of the running times for all methods of the sorted map ADT
(including the traditional map operations) is given in Table 10.3. It should be clear
that the len , find min, and find max methods run in O(1) time, and that iter-
ating the keys of the table in either direction can be peformed in O(n) time.
The analysis for the various forms of search all depend on the fact that a binary
search on a table with n entries runs in O(logn) time. This claim was originally
shown as Proposition 4.2 in Section 4.2, and that analysis clearly applies to our
find index method as well. We therefore claim an O(logn) worst-case running
time for methods getitem , find lt, find gt, find le, and find ge. Each of these
makes a single call to find index, followed by a constant number of additional
steps to determine the appropriate answer based on the index. The analysis of
find range is a bit more interesting. It begins with a binary search to find the first
item within the range (if any). After that, it executes a loop that takes O(1) time per
iteration to report subsequent values until reaching the end of the range. If there are
s items reported in the range, the total running time is O(s+logn).
In contrast to the efficient search operations, update operations for a sorted table
may take considerable time. Although binary search can help identify the index at
which an update occurs, both insertions and deletions require, in the worst case, that
linearly many existing elements be shifted in order to maintain the sorted order of
the table. Specifically, the potential call to table.insert from within setitem
and table.pop from within delitem lead to O(n) worst-case time. (See the
discussion of corresponding operations of the list class in Section 5.4.1.)
In conclusion, sorted tables are primarily used in situations where we expect
many searches but relatively few updates.
Operation Running Time
len(M) O(1)
k in M O(logn)
M[k] = v O(n) worst case; O(logn) if existing k
del M[k] O(n) worst case
M.find min(), M.find max() O(1)
M.find lt(k), M.find gt(k)
O(logn)
M.find le(k), M.find ge(k)
M.find range(start, stop) O(s+logn) where s items are reported
iter(M), reversed(M) O(n)
Table 10.3: Performance of a sorted map, as implemented with SortedTableMap.
We use n to denote the number of items in the map at the time the operation is
performed. The space requirement is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-455-320.jpg)
![436 Chapter 10. Maps, Hash Tables, and Skip Lists
Maintaining a Maxima Set with a Sorted Map
We can store the set of maxima pairs in a sorted map, M, so that the cost is the
key field and performance (speed) is the value field. We can then implement opera-
tions add(c, p), which adds a new cost-performance pair (c, p), and best(c), which
returns the best pair with cost at most c, as shown in Code Fragment 10.11.
1 class CostPerformanceDatabase:
2 ”””Maintain a database of maximal (cost,performance) pairs.”””
3
4 def init (self):
5 ”””Create an empty database.”””
6 self. M = SortedTableMap( ) # or a more efficient sorted map
7
8 def best(self, c):
9 ”””Return (cost,performance) pair with largest cost not exceeding c.
10
11 Return None if there is no such pair.
12 ”””
13 return self. M.find le(c)
14
15 def add(self, c, p):
16 ”””Add new entry with cost c and performance p.”””
17 # determine if (c,p) is dominated by an existing pair
18 other = self. M.find le(c) # other is at least as cheap as c
19 if other is not None and other[1] = p: # if its performance is as good,
20 return # (c,p) is dominated, so ignore
21 self. M[c] = p # else, add (c,p) to database
22 # and now remove any pairs that are dominated by (c,p)
23 other = self. M.find gt(c) # other more expensive than c
24 while other is not None and other[1] = p:
25 del self. M[other[0]]
26 other = self. M.find gt(c)
Code Fragment 10.11: An implementation of a class maintaining a set of maxima
cost-performance pairs using a sorted map.
Unfortunately, if we implement M using the SortedTableMap, the add behavior
has O(n) worst-case running time. If, on the other hand, we implement M using
a skip list, which we next describe, we can perform best(c) queries in O(logn)
expected time and add(c, p) updates in O((1+ r)logn) expected time, where r is
the number of points removed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-458-320.jpg)
![10.4. Skip Lists 439
10.4.1 Search and Update Operations in a Skip List
The skip-list structure affords simple map search and update algorithms. In fact,
all of the skip-list search and update algorithms are based on an elegant SkipSearch
method that takes a key k and finds the position p of the item in list S0 that has the
largest key less than or equal to k (which is possibly −∞).
Searching in a Skip List
Suppose we are given a search key k. We begin the SkipSearch method by setting
a position variable p to the topmost, left position in the skip list S, called the start
position of S. That is, the start position is the position of Sh storing the special
entry with key −∞. We then perform the following steps (see Figure 10.11), where
key(p) denotes the key of the item at position p:
1. If S.below(p) is None, then the search terminates—we are at the bottom and
have located the item in S with the largest key less than or equal to the search
key k. Otherwise, we drop down to the next lower level in the present tower
by setting p = S.below(p).
2. Starting at position p, we move p forward until it is at the rightmost position
on the present level such that key(p) ≤ k. We call this the scan forward step.
Note that such a position always exists, since each level contains the keys
+∞ and −∞. It may be that p remains where it started after we perform
such a forward scan for this level.
3. Return to step 1.
55
S1
S2
S3
S4
S5
+∞
+∞
+∞
+∞
+∞
+∞
-∞
-∞
-∞ 12
12
-∞
17
17 25
25
20
17 31 38 39
-∞
-∞ 17
17 25
25 31
31 38 44
44 50
55
55
55
S0
Figure 10.11: Example of a search in a skip list. The positions examined when
searching for key 50 are highlighted.
We give a pseudo-code description of the skip-list search algorithm, SkipSearch,
in Code Fragment 10.12. Given this method, the map operation M[k] is performed
by computing p = SkipSearch(k) and testing whether or not key(p) = k. If these
two keys are equal, we return the associated value; otherwise, we raise a KeyError.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-461-320.jpg)
![440 Chapter 10. Maps, Hash Tables, and Skip Lists
Algorithm SkipSearch(k):
Input: A search key k
Output: Position p in the bottom list S0 with the largest key such that key(p) ≤ k
p = start {begin at start position}
while below(p) = None do
p = below(p) {drop down}
while k ≥ key(next(p)) do
p = next(p) {scan forward}
return p.
Code Fragment 10.12: Algorthm to search a skip list S for key k.
As it turns out, the expected running time of algorithm SkipSearch on a skip list
with n entries is O(logn). We postpone the justification of this fact, however, until
after we discuss the implementation of the update methods for skip lists. Navigation
starting at the position identified by SkipSearch(k) can be easily used to provide the
additional forms of searches in the sorted map ADT (e.g., find gt, find range).
Insertion in a Skip List
The execution of the map operation M[k] = v begins with a call to SkipSearch(k).
This gives us the position p of the bottom-level item with the largest key less than or
equal to k (note that p may hold the special item with key −∞). If key(p) = k, the
associated value is overwritten with v. Otherwise, we need to create a new tower for
item (k,v). We insert (k,v) immediately after position p within S0. After inserting
the new item at the bottom level, we use randomization to decide the height of the
tower for the new item. We “flip” a coin, and if the flip comes up tails, then we stop
here. Else (the flip comes up heads), we backtrack to the previous (next higher)
level and insert (k,v) in this level at the appropriate position. We again flip a coin;
if it comes up heads, we go to the next higher level and repeat. Thus, we continue
to insert the new item (k,v) in lists until we finally get a flip that comes up tails.
We link together all the references to the new item (k,v) created in this process to
create its tower. A coin flip can be simulated with Python’s built-in pseudo-random
number generator from the random module by calling randrange(2), which returns
0 or 1, each with probability 1/2.
We give the insertion algorithm for a skip list S in Code Fragment 10.13 and
we illustrate it in Figure 10.12. The algorithm uses an insertAfterAbove(p,q,(k,v))
method that inserts a position storing the item (k,v) after position p (on the same
level as p) and above position q, returning the new position r (and setting internal
references so that next, prev, above, and below methods will work correctly for p,
q, and r). The expected running time of the insertion algorithm on a skip list with
n entries is O(logn), which we show in Section 10.4.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-462-320.jpg)
![442 Chapter 10. Maps, Hash Tables, and Skip Lists
Removal in a Skip List
Like the search and insertion algorithms, the removal algorithm for a skip list is
quite simple. In fact, it is even easier than the insertion algorithm. That is, to per-
form the map operation del M[k] we begin by executing method SkipSearch(k).
If the position p stores an entry with key different from k, we raise a KeyError.
Otherwise, we remove p and all the positions above p, which are easily accessed
by using above operations to climb up the tower of this entry in S starting at posi-
tion p. While removing levels of the tower, we reestablish links between the hor-
izontal neighbors of each removed position. The removal algorithm is illustrated
in Figure 10.13 and a detailed description of it is left as an exercise (R-10.24). As
we show in the next subsection, deletion operation in a skip list with n entries has
O(logn) expected running time.
Before we give this analysis, however, there are some minor improvements to
the skip-list data structure we would like to discuss. First, we do not actually need
to store references to values at the levels of the skip list above the bottom level,
because all that is needed at these levels are references to keys. In fact, we can
more efficiently represent a tower as a single object, storing the key-value pair,
and maintaining j previous references and j next references if the tower reaches
level Sj. Second, for the horizontal axes, it is possible to keep the list singly linked,
storing only the next references. We can perform insertions and removals in strictly
a top-down, scan-forward fashion. We explore the details of this optimization in
Exercise C-10.44. Neither of these optimizations improve the asymptotic perfor-
mance of skip lists by more than a constant factor, but these improvements can,
nevertheless, be meaningful in practice. In fact, experimental evidence suggests
that optimized skip lists are faster in practice than AVL trees and other balanced
search trees, which are discussed in Chapter 11.
31
S5
S4
S3
S2
S1
-∞
-∞
-∞ 12
12
-∞
17
17 25
25 31
31
42
55
50
55
+∞
+∞
+∞
+∞
+∞
-∞
-∞
17
38
38 39 42
42
42
44
44
55
55
+∞
17
17
20 25
25
S0
Figure 10.13: Removal of the entry with key 25 from the skip list of Figure 10.12.
The positions visited after the search for the position of S0 holding the entry are
highlighted. The positions removed are drawn with dashed lines.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-464-320.jpg)
![10.4. Skip Lists 445
So we have yet to bound the number of scan-forward steps we make. Let ni be
the number of keys examined while scanning forward at level i. Observe that, after
the key at the starting position, each additional key examined in a scan-forward at
level i cannot also belong to level i + 1. If any of these keys were on the previous
level, we would have encountered them in the previous scan-forward step. Thus,
the probability that any key is counted in ni is 1/2. Therefore, the expected value of
ni is exactly equal to the expected number of times we must flip a fair coin before
it comes up heads. This expected value is 2. Hence, the expected amount of time
spent scanning forward at any level i is O(1). Since S has O(logn) levels with high
probability, a search in S takes expected time O(logn). By a similar analysis, we
can show that the expected running time of an insertion or a removal is O(logn).
Space Usage in a Skip List
Finally, let us turn to the space requirement of a skip list S with n entries. As we
observed above, the expected number of positions at level i is n/2i, which means
that the expected total number of positions in S is
h
∑
i=0
n
2i
= n
h
∑
i=0
1
2i
.
Using Proposition 3.5 on geometric summations, we have
h
∑
i=0
1
2i
=
1
2
h+1
−1
1
2 −1
= 2·
1−
1
2h+1
2 for all h ≥ 0.
Hence, the expected space requirement of S is O(n).
Table 10.4 summarizes the performance of a sorted map realized by a skip list.
Operation Running Time
len(M) O(1)
k in M O(logn) expected
M[k] = v O(logn) expected
del M[k] O(logn) expected
M.find min(), M.find max() O(1)
M.find lt(k), M.find gt(k)
O(logn) expected
M.find le(k), M.find ge(k)
M.find range(start, stop) O(s+logn) expected, with s items reported
iter(M), reversed(M) O(n)
Table 10.4: Performance of a sorted map implemented with a skip list. We use n to
denote the number of entries in the dictionary at the time the operation is performed.
The expected space requirement is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-467-320.jpg)
![10.5. Sets, Multisets, and Multimaps 451
1 class MultiMap:
2 ”””A multimap class built upon use of an underlying map for storage.”””
3 MapType = dict # Map type; can be redefined by subclass
4
5 def init (self):
6 ”””Create a new empty multimap instance.”””
7 self. map = self. MapType( ) # create map instance for storage
8 self. n = 0
9
10 def iter (self):
11 ”””Iterate through all (k,v) pairs in multimap.”””
12 for k,secondary in self. map.items():
13 for v in secondary:
14 yield (k,v)
15
16 def add(self, k, v):
17 ”””Add pair (k,v) to multimap.”””
18 container = self. map.setdefault(k, [ ]) # create empty list, if needed
19 container.append(v)
20 self. n += 1
21
22 def pop(self, k):
23 ”””Remove and return arbitrary (k,v) with key k (or raise KeyError).”””
24 secondary = self. map[k] # may raise KeyError
25 v = secondary.pop()
26 if len(secondary) == 0:
27 del self. map[k] # no pairs left
28 self. n −= 1
29 return (k, v)
30
31 def find(self, k):
32 ”””Return arbitrary (k,v) pair with given key (or raise KeyError).”””
33 secondary = self. map[k] # may raise KeyError
34 return (k, secondary[0])
35
36 def find all(self, k):
37 ”””Generate iteration of all (k,v) pairs with given key.”””
38 secondary = self. map.get(k, [ ]) # empty list, by default
39 for v in secondary:
40 yield (k,v)
Code Fragment 10.17: An implementation of a MultiMap using a dict for storage.
The len method, which returns self. n, is omitted from this listing.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-473-320.jpg)
![10.6. Exercises 453
R-10.12 What is the result of Exercise R-10.9 when collisions are handled by dou-
ble hashing using the secondary hash function h(k) = 7−(k mod 7)?
R-10.13 What is the worst-case time for putting n entries in an initially empty hash
table, with collisions resolved by chaining? What is the best case?
R-10.14 Show the result of rehashing the hash table shown in Figure 10.6 into a
table of size 19 using the new hash function h(k) = 3k mod 17.
R-10.15 Our HashMapBase class maintains a load factor λ ≤ 0.5. Reimplement
that class to allow the user to specify the maximum load, and adjust the
concrete subclasses accordingly.
R-10.16 Give a pseudo-code description of an insertion into a hash table that uses
quadratic probing to resolve collisions, assuming we also use the trick of
replacing deleted entries with a special “deactivated entry” object.
R-10.17 Modify our ProbeHashMap to use quadratic probing.
R-10.18 Explain why a hash table is not suited to implement a sorted map.
R-10.19 Describe how a sorted list implemented as a doubly linked list could be
used to implement the sorted map ADT.
R-10.20 What is the worst-case asymptotic running time for performing n deletions
from a SortedTableMap instance that initially contains 2n entries?
R-10.21 Consider the following variant of the find index method from Code Frag-
ment 10.8, in the context of the SortedTableMap class:
def find index(self, k, low, high):
if high low:
return high + 1
else:
mid = (low + high) // 2
if self. table[mid]. key k:
return self. find index(k, mid + 1, high)
else:
return self. find index(k, low, mid − 1)
Does this always produce the same result as the original version? Justify
your answer.
R-10.22 What is the expected running time of the methods for maintaining a max-
ima set if we insert n pairs such that each pair has lower cost and perfor-
mance than one before it? What is contained in the sorted map at the end
of this series of operations? What if each pair had a lower cost and higher
performance than the one before it?
R-10.23 Draw an example skip list S that results from performing the following
series of operations on the skip list shown in Figure 10.13: del S[38],
S[48] = x , S[24] = y , del S[55]. Record your coin flips, as well.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-475-320.jpg)
![454 Chapter 10. Maps, Hash Tables, and Skip Lists
R-10.24 Give a pseudo-code description of the delitem map operation when
using a skip list.
R-10.25 Give a concrete implementation of the pop method, in the context of a
MutableSet abstract base class, that relies only on the five core set behav-
iors described in Section 10.5.2.
R-10.26 Give a concrete implementation of the isdisjoint method in the context
of the MutableSet abstract base class, relying only on the five primary
abstract methods of that class. Your algorithm should run in O(min(n,m))
where n and m denote the respective cardinalities of the two sets.
R-10.27 What abstraction would you use to manage a database of friends’ birth-
days in order to support efficient queries such as “find all friends whose
birthday is today” and “find the friend who will be the next to celebrate a
birthday”?
Creativity
C-10.28 On page 406 of Section 10.1.3, we give an implementation of the method
setdefault as it might appear in the MutableMapping abstract base class.
While that method accomplishes the goal in a general fashion, its effi-
ciency is less than ideal. In particular, when the key is new, there will be
a failed search due to the initial use of getitem , and then a subse-
quent insertion via setitem . For a concrete implementation, such as
the UnsortedTableMap, this is twice the work because a complete scan
of the table will take place during the failed getitem , and then an-
other complete scan of the table takes place due to the implementation of
setitem . A better solution is for the UnsortedTableMap class to over-
ride setdefault to provide a direct solution that performs a single search.
Give such an implementation of UnsortedTableMap.setdefault.
C-10.29 Repeat Exercise C-10.28 for the ProbeHashMap class.
C-10.30 Repeat Exercise C-10.28 for the ChainHashMap class.
C-10.31 For an ideal compression function, the capacity of the bucket array for a
hash table should be a prime number. Therefore, we consider the problem
of locating a prime number in a range [M,2M]. Implement a method for
finding such a prime by using the sieve algorithm. In this algorithm, we
allocate a 2M cell Boolean array A, such that cell i is associated with the
integer i. We then initialize the array cells to all be “true” and we “mark
off” all the cells that are multiples of 2, 3, 5, 7, and so on. This process
can stop after it reaches a number larger than
√
2M. (Hint: Consider a
bootstrapping method for finding the primes up to
√
2M.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-476-320.jpg)
![10.6. Exercises 455
C-10.32 Perform experiments on our ChainHashMap and ProbeHashMap classes
to measure its efficiency using random key sets and varying limits on the
load factor (see Exercise R-10.15).
C-10.33 Our implementation of separate chaining in ChainHashMap conserves
memory by representing empty buckets in the table as None, rather than
as empty instances of a secondary structure. Because many of these buck-
ets will hold a single item, a better optimization is to have those slots of
the table directly reference the Item instance, and to reserve use of sec-
ondary containers for buckets that have two or more items. Modify our
implementation to provide this additional optimization.
C-10.34 Computing a hash code can be expensive, especially for lengthy keys. In
our hash table implementations, we compute the hash code when first in-
serting an item, and recompute each item’s hash code each time we resize
our table. Python’s dict class makes an interesting trade-off. The hash
code is computed once, when an item is inserted, and the hash code is
stored as an extra field of the item composite, so that it need not be recom-
puted. Reimplement our HashTableBase class to use such an approach.
C-10.35 Describe how to perform a removal from a hash table that uses linear
probing to resolve collisions where we do not use a special marker to
represent deleted elements. That is, we must rearrange the contents so that
it appears that the removed entry was never inserted in the first place.
C-10.36 The quadratic probing strategy has a clustering problem related to the way
it looks for open slots. Namely, when a collision occurs at bucket h(k), it
checks buckets A[(h(k)+i2) mod N], for i = 1,2,...,N −1.
a. Show that i2 mod N will assume at most (N + 1)/2 distinct values,
for N prime, as i ranges from 1 to N − 1. As a part of this justifica-
tion, note that i2 mod N = (N −i)2 mod N for all i.
b. A better strategy is to choose a prime N such that N mod 4 = 3 and
then to check the buckets A[(h(k) ± i2) mod N] as i ranges from 1
to (N − 1)/2, alternating between plus and minus. Show that this
alternate version is guaranteed to check every bucket in A.
C-10.37 Refactor our ProbeHashMap design so that the sequence of secondary
probes for collision resolution can be more easily customized. Demon-
strate your new framework by providing separate concrete subclasses for
linear probing and quadratic probing.
C-10.38 Design a variation of binary search for performing the multimap opera-
tion find all(k) implemented with a sorted search table that includes du-
plicates, and show that it runs in time O(s+logn), where n is the number
of elements in the dictionary and s is the number of items with given key k.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-477-320.jpg)
![458 Chapter 10. Maps, Hash Tables, and Skip Lists
Chapter Notes
Hashing is a well-studied technique. The reader interested in further study is encouraged
to explore the book by Knuth [65], as well as the book by Vitter and Chen [100]. Skip
lists were introduced by Pugh [86]. Our analysis of skip lists is a simplification of a pre-
sentation given by Motwani and Raghavan [80]. For a more in-depth analysis of skip lists,
please see the various research papers on skip lists that have appeared in the data structures
literature [59, 81, 84]. Exercise C-10.36 was contributed by James Lee.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-480-320.jpg)
![460 Chapter 11. Search Trees
11.1 Binary Search Trees
In Chapter 8 we introduced the tree data structure and demonstrated a variety of
applications. One important use is as a search tree (as described on page 332). In
this chapter, we use a search tree structure to efficiently implement a sorted map.
The three most fundamental methods of a map M (see Section 10.1.1) are:
M[k]: Return the value v associated with key k in map M, if one exists;
otherwise raise a KeyError; implemented with getitem method.
M[k] = v: Associate value v with key k in map M, replacing the existing value
if the map already contains an item with key equal to k; implemented
with setitem method.
del M[k]: Remove from map M the item with key equal to k; if M has no such
item, then raise a KeyError; implemented with delitem method.
The sorted map ADT includes additional functionality (see Section 10.3), guar-
anteeing that an iteration reports keys in sorted order, and supporting additional
searches such as find gt(k) and find range(start, stop).
Binary trees are an excellent data structure for storing items of a map, assuming
we have an order relation defined on the keys. In this context, a binary search tree
is a binary tree T with each position p storing a key-value pair (k,v) such that:
• Keys stored in the left subtree of p are less than k.
• Keys stored in the right subtree of p are greater than k.
An example of such a binary search tree is given in Figure 11.1. As a matter of
convenience, we will not diagram the values associated with keys in this chapter,
since those values do not affect the placement of items within a search tree.
80
82
54
28 93
65
32 97
44
88
17
8
76
29
Figure 11.1: A binary search tree with integer keys. We omit the display of associ-
ated values in this chapter, since they are not relevant to the order of items within a
search tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-482-320.jpg)
![11.1. Binary Search Trees 465
11.1.3 Insertions and Deletions
Algorithms for inserting or deleting entries of a binary search tree are fairly straight-
forward, although not trivial.
Insertion
The map command M[k] = v, as supported by the setitem method, begins
with a search for key k (assuming the map is nonempty). If found, that item’s
existing value is reassigned. Otherwise, a node for the new item can be inserted
into the underlying tree T in place of the empty subtree that was reached at the end
of the failed search. The binary search tree property is sustained by that placement
(note that it is placed exactly where a search would expect it). Pseudo-code for
such a TreeInsert algorithm is given in in Code Fragment 11.3.
Algorithm TreeInsert(T, k, v):
Input: A search key k to be associated with value v
p = TreeSearch(T,T.root(),k)
if k == p.key() then
Set p’s value to v
else if k p.key() then
add node with item (k,v) as left child of p
else
add node with item (k,v) as right child of p
Code Fragment 11.3: Algorithm for inserting a key-value pair into a map that is
represented as a binary search tree.
An example of insertion into a binary search tree is shown in Figure 11.4.
82
54
28 93
65
32 97
44
88
17
8
80
76
29
68
54
28 93
65
32 97
44
88
17
8
80
76
82
29
(a) (b)
Figure 11.4: Insertion of an item with key 68 into the search tree of Figure 11.2.
Finding the position to insert is shown in (a), and the resulting tree is shown in (b).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/85/Data-structures-and-Algorithms-in-Python-pdf-487-320.jpg)
![1.1. Python Overview 3
1.1.2 Preview of a Python Program
As a simple introduction, Code Fragment 1.1 presents a Python program that com-
putes the grade-point average (GPA) for a student based on letter grades that are
entered by a user. Many of the techniques demonstrated in this example will be
discussed in the remainder of this chapter. At this point, we draw attention to a few
high-level issues, for readers who are new to Python as a programming language.
Python’s syntax relies heavily on the use of whitespace. Individual statements
are typically concluded with a newline character, although a command can extend
to another line, either with a concluding backslash character (), or if an opening
delimiter has not yet been closed, such as the { character in defining value map.
Whitespace is also key in delimiting the bodies of control structures in Python.
Specifically, a block of code is indented to designate it as the body of a control
structure, and nested control structures use increasing amounts of indentation. In
Code Fragment 1.1, the body of the while loop consists of the subsequent 8 lines,
including a nested conditional structure.
Comments are annotations provided for human readers, yet ignored by the
Python interpreter. The primary syntax for comments in Python is based on use
of the # character, which designates the remainder of the line as a comment.
print( Welcome to the GPA calculator. )
print( Please enter all your letter grades, one per line. )
print( Enter a blank line to designate the end. )
# map from letter grade to point value
points = { A+ :4.0, A :4.0, A- :3.67, B+ :3.33, B :3.0, B- :2.67,
C+ :2.33, C :2.0, C :1.67, D+ :1.33, D :1.0, F :0.0}
num courses = 0
total points = 0
done = False
while not done:
grade = input( ) # read line from user
if grade == : # empty line was entered
done = True
elif grade not in points: # unrecognized grade entered
print(Unknown grade {0} being ignored.format(grade))
else:
num courses += 1
total points += points[grade]
if num courses 0: # avoid division by zero
print( Your GPA is {0:.3} .format(total points / num courses))
Code Fragment 1.1: A Python program that computes a grade-point average (GPA).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-25-2048.jpg)
![1.2. Objects in Python 9
Sequence Types: The list, tuple, and str Classes
The list, tuple, and str classes are sequence types in Python, representing a col-
lection of values in which the order is significant. The list class is the most general,
representing a sequence of arbitrary objects (akin to an “array” in other languages).
The tuple class is an immutable version of the list class, benefiting from a stream-
lined internal representation. The str class is specially designed for representing
an immutable sequence of text characters. We note that Python does not have a
separate class for characters; they are just strings with length one.
The list Class
A list instance stores a sequence of objects. A list is a referential structure, as it
technically stores a sequence of references to its elements (see Figure 1.4). El-
ements of a list may be arbitrary objects (including the None object). Lists are
array-based sequences and are zero-indexed, thus a list of length n has elements
indexed from 0 to n−1 inclusive. Lists are perhaps the most used container type in
Python and they will be extremely central to our study of data structures and algo-
rithms. They have many valuable behaviors, including the ability to dynamically
expand and contract their capacities as needed. In this chapter, we will discuss only
the most basic properties of lists. We revisit the inner working of all of Python’s
sequence types as the focus of Chapter 5.
Python uses the characters [ ] as delimiters for a list literal, with [ ] itself being
an empty list. As another example, [ red , green , blue ] is a list containing
three string instances. The contents of a list literal need not be expressed as literals;
if identifiers a and b have been established, then syntax [a, b] is legitimate.
The list() constructor produces an empty list by default. However, the construc-
tor will accept any parameter that is of an iterable type. We will discuss iteration
further in Section 1.8, but examples of iterable types include all of the standard con-
tainer types (e.g., strings, list, tuples, sets, dictionaries). For example, the syntax
list( hello ) produces a list of individual characters, [ h , e , l , l , o ].
Because an existing list is itself iterable, the syntax backup = list(data) can be
used to construct a new list instance referencing the same contents as the original.
3 4 5 6 7
0 1 2 10
9
8
primes:
13 19 23 29 31
7
5
3
2 11 17
Figure 1.4: Python’s internal representation of a list of integers, instantiated as
prime = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31]. The implicit indices of the ele-
ments are shown below each entry.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-31-2048.jpg)
![10 Chapter 1. Python Primer
The tuple Class
The tuple class provides an immutable version of a sequence, and therefore its
instances have an internal representation that may be more streamlined than that of
a list. While Python uses the [ ] characters to delimit a list, parentheses delimit a
tuple, with () being an empty tuple. There is one important subtlety. To express
a tuple of length one as a literal, a comma must be placed after the element, but
within the parentheses. For example, (17,) is a one-element tuple. The reason for
this requirement is that, without the trailing comma, the expression (17) is viewed
as a simple parenthesized numeric expression.
The str Class
Python’s str class is specifically designed to efficiently represent an immutable
sequence of characters, based upon the Unicode international character set. Strings
have a more compact internal representation than the referential lists and tuples, as
portrayed in Figure 1.5.
0
A
S M P L E
3 4 5
1 2
Figure 1.5: A Python string, which is an indexed sequence of characters.
String literals can be enclosed in single quotes, as in hello , or double
quotes, as in hello. This choice is convenient, especially when using an-
other of the quotation characters as an actual character in the sequence, as in
Don t worry. Alternatively, the quote delimiter can be designated using a
backslash as a so-called escape character, as in Don t worry . Because the
backslash has this purpose, the backslash must itself be escaped to occur as a natu-
ral character of the string literal, as in C:Python , for a string that would be
displayed as C:Python. Other commonly escaped characters are n for newline
and t for tab. Unicode characters can be included, such as 20u20AC for the
string 20 .
Python also supports using the delimiter or to begin and end a string
literal. The advantage of such triple-quoted strings is that newline characters can
be embedded naturally (rather than escaped as n). This can greatly improve the
readability of long, multiline strings in source code. For example, at the beginning
of Code Fragment 1.1, rather than use separate print statements for each line of
introductory output, we can use a single print statement, as follows:
print(”””Welcome to the GPA calculator.
Please enter all your letter grades, one per line.
Enter a blank line to designate the end.”””)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-32-2048.jpg)
![1.2. Objects in Python 11
The set and frozenset Classes
Python’s set class represents the mathematical notion of a set, namely a collection
of elements, without duplicates, and without an inherent order to those elements.
The major advantage of using a set, as opposed to a list, is that it has a highly
optimized method for checking whether a specific element is contained in the set.
This is based on a data structure known as a hash table (which will be the primary
topic of Chapter 10). However, there are two important restrictions due to the
algorithmic underpinnings. The first is that the set does not maintain the elements
in any particular order. The second is that only instances of immutable types can be
added to a Python set. Therefore, objects such as integers, floating-point numbers,
and character strings are eligible to be elements of a set. It is possible to maintain a
set of tuples, but not a set of lists or a set of sets, as lists and sets are mutable. The
frozenset class is an immutable form of the set type, so it is legal to have a set of
frozensets.
Python uses curly braces { and } as delimiters for a set, for example, as {17}
or { red , green , blue }. The exception to this rule is that { } does not
represent an empty set; for historical reasons, it represents an empty dictionary
(see next paragraph). Instead, the constructor syntax set() produces an empty set.
If an iterable parameter is sent to the constructor, then the set of distinct elements
is produced. For example, set( hello ) produces { h , e , l , o }.
The dict Class
Python’s dict class represents a dictionary, or mapping, from a set of distinct keys
to associated values. For example, a dictionary might map from unique student ID
numbers, to larger student records (such as the student’s name, address, and course
grades). Python implements a dict using an almost identical approach to that of a
set, but with storage of the associated values.
A dictionary literal also uses curly braces, and because dictionaries were intro-
duced in Python prior to sets, the literal form { } produces an empty dictionary.
A nonempty dictionary is expressed using a comma-separated series of key:value
pairs. For example, the dictionary { ga : Irish , de : German } maps
ga to Irish and de to German .
The constructor for the dict class accepts an existing mapping as a parameter,
in which case it creates a new dictionary with identical associations as the existing
one. Alternatively, the constructor accepts a sequence of key-value pairs as a pa-
rameter, as in dict(pairs) with pairs = [( ga , Irish ), ( de , German )].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-33-2048.jpg)
![14 Chapter 1. Python Primer
Python carefully extends the semantics of // and % to cases where one or both
operands are negative. For the sake of notation, let us assume that variables n
and m represent respectively the dividend and divisor of a quotient n
m, and that
q = n // m and r = n % m. Python guarantees that q m + r will equal n. We
already saw an example of this identity with positive operands, as 6∗4+ 3 = 27.
When the divisor m is positive, Python further guarantees that 0 ≤ r m. As
a consequence, we find that −27 // 4 evaluates to −7 and −27 % 4 evaluates
to 1, as (−7)∗4+ 1 = −27. When the divisor is negative, Python guarantees that
m r ≤ 0. As an example, 27 // −4 is −7 and 27 % −4 is −1, satisfying the
identity 27 = (−7)∗(−4)+(−1).
The conventions for the // and % operators are even extended to floating-
point operands, with the expression q = n // m being the integral floor of the
quotient, and r = n % m being the “remainder” to ensure that q m + r equals
n. For example, 8.2 // 3.14 evaluates to 2.0 and 8.2 % 3.14 evaluates to 1.92, as
2.0∗3.14+1.92 = 8.2.
Bitwise Operators
Python provides the following bitwise operators for integers:
∼ bitwise complement (prefix unary operator)
bitwise and
| bitwise or
ˆ bitwise exclusive-or
shift bits left, filling in with zeros
shift bits right, filling in with sign bit
Sequence Operators
Each of Python’s built-in sequence types (str, tuple, and list) support the following
operator syntaxes:
s[j] element at index j
s[start:stop] slice including indices [start,stop)
s[start:stop:step] slice including indices start, start + step,
start + 2 step, . . . , up to but not equalling or stop
s + t concatenation of sequences
k s shorthand for s + s + s + ... (k times)
val in s containment check
val not in s non-containment check
Python relies on zero-indexing of sequences, thus a sequence of length n has ele-
ments indexed from 0 to n− 1 inclusive. Python also supports the use of negative
indices, which denote a distance from the end of the sequence; index −1 denotes
the last element, index −2 the second to last, and so on. Python uses a slicing](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-36-2048.jpg)
![1.3. Expressions, Operators, and Precedence 15
notation to describe subsequences of a sequence. Slices are described as half-open
intervals, with a start index that is included, and a stop index that is excluded. For
example, the syntax data[3:8] denotes a subsequence including the five indices:
3,4,5,6,7. An optional “step” value, possibly negative, can be indicated as a third
parameter of the slice. If a start index or stop index is omitted in the slicing nota-
tion, it is presumed to designate the respective extreme of the original sequence.
Because lists are mutable, the syntax s[j] = val can be used to replace an ele-
ment at a given index. Lists also support a syntax, del s[j], that removes the desig-
nated element from the list. Slice notation can also be used to replace or delete a
sublist.
The notation val in s can be used for any of the sequences to see if there is an
element equivalent to val in the sequence. For strings, this syntax can be used to
check for a single character or for a larger substring, as with amp in example .
All sequences define comparison operations based on lexicographic order, per-
forming an element by element comparison until the first difference is found. For
example, [5, 6, 9] [5, 7] because of the entries at index 1. Therefore, the follow-
ing operations are supported by sequence types:
s == t equivalent (element by element)
s != t not equivalent
s t lexicographically less than
s = t lexicographically less than or equal to
s t lexicographically greater than
s = t lexicographically greater than or equal to
Operators for Sets and Dictionaries
Sets and frozensets support the following operators:
key in s containment check
key not in s non-containment check
s1 == s2 s1 is equivalent to s2
s1 != s2 s1 is not equivalent to s2
s1 = s2 s1 is subset of s2
s1 s2 s1 is proper subset of s2
s1 = s2 s1 is superset of s2
s1 s2 s1 is proper superset of s2
s1 | s2 the union of s1 and s2
s1 s2 the intersection of s1 and s2
s1 − s2 the set of elements in s1 but not s2
s1 ˆ s2 the set of elements in precisely one of s1 or s2
Note well that sets do not guarantee a particular order of their elements, so the
comparison operators, such as , are not lexicographic; rather, they are based on
the mathematical notion of a subset. As a result, the comparison operators define](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-37-2048.jpg)
![16 Chapter 1. Python Primer
a partial order, but not a total order, as disjoint sets are neither “less than,” “equal
to,” or “greater than” each other. Sets also support many fundamental behaviors
through named methods (e.g., add, remove); we will explore their functionality
more fully in Chapter 10.
Dictionaries, like sets, do not maintain a well-defined order on their elements.
Furthermore, the concept of a subset is not typically meaningful for dictionaries, so
the dict class does not support operators such as . Dictionaries support the notion
of equivalence, with d1 == d2 if the two dictionaries contain the same set of key-
value pairs. The most widely used behavior of dictionaries is accessing a value
associated with a particular key k with the indexing syntax, d[k]. The supported
operators are as follows:
d[key] value associated with given key
d[key] = value set (or reset) the value associated with given key
del d[key] remove key and its associated value from dictionary
key in d containment check
key not in d non-containment check
d1 == d2 d1 is equivalent to d2
d1 != d2 d1 is not equivalent to d2
Dictionaries also support many useful behaviors through named methods, which
we explore more fully in Chapter 10.
Extended Assignment Operators
Python supports an extended assignment operator for most binary operators, for
example, allowing a syntax such as count += 5. By default, this is a shorthand for
the more verbose count = count + 5. For an immutable type, such as a number or
a string, one should not presume that this syntax changes the value of the existing
object, but instead that it will reassign the identifier to a newly constructed value.
(See discussion of Figure 1.3.) However, it is possible for a type to redefine such
semantics to mutate the object, as the list class does for the += operator.
alpha = [1, 2, 3]
beta = alpha # an alias for alpha
beta += [4, 5] # extends the original list with two more elements
beta = beta + [6, 7] # reassigns beta to a new list [1, 2, 3, 4, 5, 6, 7]
print(alpha) # will be [1, 2, 3, 4, 5]
This example demonstrates the subtle difference between the list semantics for the
syntax beta += foo versus beta = beta + foo.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-38-2048.jpg)
![1.3. Expressions, Operators, and Precedence 17
1.3.1 Compound Expressions and Operator Precedence
Programming languages must have clear rules for the order in which compound
expressions, such as 5 + 2 3, are evaluated. The formal order of precedence
for operators in Python is given in Table 1.3. Operators in a category with higher
precedence will be evaluated before those with lower precedence, unless the expres-
sion is otherwise parenthesized. Therefore, we see that Python gives precedence to
multiplication over addition, and therefore evaluates the expression 5 + 2 3 as
5 + (2 3), with value 11, but the parenthesized expression (5 + 2) 3 evalu-
ates to value 21. Operators within a category are typically evaluated from left to
right, thus 5 − 2 + 3 has value 6. Exceptions to this rule include that unary oper-
ators and exponentiation are evaluated from right to left.
Python allows a chained assignment, such as x = y = 0, to assign multiple
identifiers to the rightmost value. Python also allows the chaining of comparison
operators. For example, the expression 1 = x + y = 10 is evaluated as the
compound (1 = x + y) and (x + y = 10), but without computing the inter-
mediate value x + y twice.
Operator Precedence
Type Symbols
1 member access expr.member
2
function/method calls expr(...)
container subscripts/slices expr[...]
3 exponentiation
4 unary operators +expr, −expr, ˜expr
5 multiplication, division , /, //, %
6 addition, subtraction +, −
7 bitwise shifting ,
8 bitwise-and
9 bitwise-xor ˆ
10 bitwise-or |
11
comparisons is, is not, ==, !=, , =, , =
containment in, not in
12 logical-not not expr
13 logical-and and
14 logical-or or
15 conditional val1 if cond else val2
16 assignments =, +=, −=, =, etc.
Table 1.3: Operator precedence in Python, with categories ordered from highest
precedence to lowest precedence. When stated, we use expr to denote a literal,
identifier, or result of a previously evaluated expression. All operators without
explicit mention of expr are binary operators, with syntax expr1 operator expr2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-39-2048.jpg)
![20 Chapter 1. Python Primer
1.4.2 Loops
Python offers two distinct looping constructs. A while loop allows general repeti-
tion based upon the repeated testing of a Boolean condition. A for loop provides
convenient iteration of values from a defined series (such as characters of a string,
elements of a list, or numbers within a given range). We discuss both forms in this
section.
While Loops
The syntax for a while loop in Python is as follows:
while condition:
body
As with an if statement, condition can be an arbitrary Boolean expression, and
body can be an arbitrary block of code (including nested control structures). The
execution of a while loop begins with a test of the Boolean condition. If that condi-
tion evaluates to True, the body of the loop is performed. After each execution of
the body, the loop condition is retested, and if it evaluates to True, another iteration
of the body is performed. When the conditional test evaluates to False (assuming
it ever does), the loop is exited and the flow of control continues just beyond the
body of the loop.
As an example, here is a loop that advances an index through a sequence of
characters until finding an entry with value X or reaching the end of the sequence.
j = 0
while j len(data) and data[j] != X :
j += 1
The len function, which we will introduce in Section 1.5.2, returns the length of a
sequence such as a list or string. The correctness of this loop relies on the short-
circuiting behavior of the and operator, as described on page 12. We intention-
ally test j len(data) to ensure that j is a valid index, prior to accessing element
data[j]. Had we written that compound condition with the opposite order, the eval-
uation of data[j] would eventually raise an IndexError when X is not found. (See
Section 1.7 for discussion of exceptions.)
As written, when this loop terminates, variable j’s value will be the index of
the leftmost occurrence of X , if found, or otherwise the length of the sequence
(which is recognizable as an invalid index to indicate failure of the search). It is
worth noting that this code behaves correctly, even in the special case when the list
is empty, as the condition j len(data) will initially fail and the body of the loop
will never be executed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-42-2048.jpg)
![1.4. Control Flow 21
For Loops
Python’s for-loop syntax is a more convenient alternative to a while loop when
iterating through a series of elements. The for-loop syntax can be used on any
type of iterable structure, such as a list, tuple str, set, dict, or file (we will discuss
iterators more formally in Section 1.8). Its general syntax appears as follows.
for element in iterable:
body # body may refer to element as an identifier
For readers familiar with Java, the semantics of Python’s for loop is similar to the
“for each” loop style introduced in Java 1.5.
As an instructive example of such a loop, we consider the task of computing
the sum of a list of numbers. (Admittedly, Python has a built-in function, sum, for
this purpose.) We perform the calculation with a for loop as follows, assuming that
data identifies the list:
total = 0
for val in data:
total += val # note use of the loop variable, val
The loop body executes once for each element of the data sequence, with the iden-
tifier, val, from the for-loop syntax assigned at the beginning of each pass to a
respective element. It is worth noting that val is treated as a standard identifier. If
the element of the original data happens to be mutable, the val identifier can be
used to invoke its methods. But a reassignment of identifier val to a new value has
no affect on the original data, nor on the next iteration of the loop.
As a second classic example, we consider the task of finding the maximum
value in a list of elements (again, admitting that Python’s built-in max function
already provides this support). If we can assume that the list, data, has at least one
element, we could implement this task as follows:
biggest = data[0] # as we assume nonempty list
for val in data:
if val biggest:
biggest = val
Although we could accomplish both of the above tasks with a while loop, the
for-loop syntax had an advantage of simplicity, as there is no need to manage an
explicit index into the list nor to author a Boolean loop condition. Furthermore, we
can use a for loop in cases for which a while loop does not apply, such as when
iterating through a collection, such as a set, that does not support any direct form
of indexing.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-43-2048.jpg)
![22 Chapter 1. Python Primer
Index-Based For Loops
The simplicity of a standard for loop over the elements of a list is wonderful; how-
ever, one limitation of that form is that we do not know where an element resides
within the sequence. In some applications, we need knowledge of the index of an
element within the sequence. For example, suppose that we want to know where
the maximum element in a list resides.
Rather than directly looping over the elements of the list in that case, we prefer
to loop over all possible indices of the list. For this purpose, Python provides
a built-in class named range that generates integer sequences. (We will discuss
generators in Section 1.8.) In simplest form, the syntax range(n) generates the
series of n values from 0 to n − 1. Conveniently, these are precisely the series of
valid indices into a sequence of length n. Therefore, a standard Python idiom for
looping through the series of indices of a data sequence uses a syntax,
for j in range(len(data)):
In this case, identifier j is not an element of the data—it is an integer. But the
expression data[j] can be used to retrieve the respective element. For example, we
can find the index of the maximum element of a list as follows:
big index = 0
for j in range(len(data)):
if data[j] data[big index]:
big index = j
Break and Continue Statements
Python supports a break statement that immediately terminate a while or for loop
when executed within its body. More formally, if applied within nested control
structures, it causes the termination of the most immediately enclosing loop. As
a typical example, here is code that determines whether a target value occurs in a
data set:
found = False
for item in data:
if item == target:
found = True
break
Python also supports a continue statement that causes the current iteration of a
loop body to stop, but with subsequent passes of the loop proceeding as expected.
We recommend that the break and continue statements be used sparingly. Yet,
there are situations in which these commands can be effectively used to avoid in-
troducing overly complex logical conditions.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-44-2048.jpg)
![1.5. Functions 25
These assignment statements establish identifier data as an alias for grades and
target as a name for the string literal A . (See Figure 1.7.)
...
str
A
data target
grades
list
Figure 1.7: A portrayal of parameter passing in Python, for the function call
count(grades, A ). Identifiers data and target are formal parameters defined
within the local scope of the count function.
The communication of a return value from the function back to the caller is
similarly implemented as an assignment. Therefore, with our sample invocation of
prizes = count(grades, A ), the identifier prizes in the caller’s scope is assigned
to the object that is identified as n in the return statement within our function body.
An advantage to Python’s mechanism for passing information to and from a
function is that objects are not copied. This ensures that the invocation of a function
is efficient, even in a case where a parameter or return value is a complex object.
Mutable Parameters
Python’s parameter passing model has additional implications when a parameter is
a mutable object. Because the formal parameter is an alias for the actual parameter,
the body of the function may interact with the object in ways that change its state.
Considering again our sample invocation of the count function, if the body of the
function executes the command data.append( F ), the new entry is added to the
end of the list identified as data within the function, which is one and the same as
the list known to the caller as grades. As an aside, we note that reassigning a new
value to a formal parameter with a function body, such as by setting data = [ ],
does not alter the actual parameter; such a reassignment simply breaks the alias.
Our hypothetical example of a count method that appends a new element to a
list lacks common sense. There is no reason to expect such a behavior, and it would
be quite a poor design to have such an unexpected effect on the parameter. There
are, however, many legitimate cases in which a function may be designed (and
clearly documented) to modify the state of a parameter. As a concrete example,
we present the following implementation of a method named scale that’s primary
purpose is to multiply all entries of a numeric data set by a given factor.
def scale(data, factor):
for j in range(len(data)):
data[j] = factor](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-47-2048.jpg)
![26 Chapter 1. Python Primer
Default Parameter Values
Python provides means for functions to support more than one possible calling
signature. Such a function is said to be polymorphic (which is Greek for “many
forms”). Most notably, functions can declare one or more default values for pa-
rameters, thereby allowing the caller to invoke a function with varying numbers of
actual parameters. As an artificial example, if a function is declared with signature
def foo(a, b=15, c=27):
there are three parameters, the last two of which offer default values. A caller is
welcome to send three actual parameters, as in foo(4, 12, 8), in which case the de-
fault values are not used. If, on the other hand, the caller only sends one parameter,
foo(4), the function will execute with parameters values a=4, b=15, c=27. If a
caller sends two parameters, they are assumed to be the first two, with the third be-
ing the default. Thus, foo(8, 20) executes with a=8, b=20, c=27. However, it is
illegal to define a function with a signature such as bar(a, b=15, c) with b having
a default value, yet not the subsequent c; if a default parameter value is present for
one parameter, it must be present for all further parameters.
As a more motivating example for the use of a default parameter, we revisit
the task of computing a student’s GPA (see Code Fragment 1.1). Rather than as-
sume direct input and output with the console, we prefer to design a function that
computes and returns a GPA. Our original implementation uses a fixed mapping
from each letter grade (such as a B−) to a corresponding point value (such as
2.67). While that point system is somewhat common, it may not agree with the
system used by all schools. (For example, some may assign an A+ grade a value
higher than 4.0.) Therefore, we design a compute gpa function, given in Code
Fragment 1.2, which allows the caller to specify a custom mapping from grades to
values, while offering the standard point system as a default.
def compute gpa(grades, points={ A+ :4.0, A :4.0, A- :3.67, B+ :3.33,
B :3.0, B- :2.67, C+ :2.33, C :2.0,
C :1.67, D+ :1.33, D :1.0, F :0.0}):
num courses = 0
total points = 0
for g in grades:
if g in points: # a recognizable grade
num courses += 1
total points += points[g]
return total points / num courses
Code Fragment 1.2: A function that computes a student’s GPA with a point value
system that can be customized as an optional parameter.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-48-2048.jpg)
![1.6. Simple Input and Output 31
When reading a numeric value from the user, a programmer must use the input
function to get the string of characters, and then use the int or float syntax to
construct the numeric value that character string represents. That is, if a call to
response = input() reports that the user entered the characters, 2013 , the syntax
int(response) could be used to produce the integer value 2013. It is quite common
to combine these operations with a syntax such as
year = int(input( In what year were you born? ))
if we assume that the user will enter an appropriate response. (In Section 1.7 we
discuss error handling in such a situation.)
Because input returns a string as its result, use of that function can be combined
with the existing functionality of the string class, as described in Appendix A. For
example, if the user enters multiple pieces of information on the same line, it is
common to call the split method on the result, as in
reply = input( Enter x and y, separated by spaces: )
pieces = reply.split( ) # returns a list of strings, as separated by spaces
x = float(pieces[0])
y = float(pieces[1])
A Sample Program
Here is a simple, but complete, program that demonstrates the use of the input
and print functions. The tools for formatting the final output is discussed in Ap-
pendix A.
age = int(input( Enter your age in years: ))
max heart rate = 206.9 − (0.67 age) # as per Med Sci Sports Exerc.
target = 0.65 max heart rate
print( Your target fat-burning heart rate is , target)
1.6.2 Files
Files are typically accessed in Python beginning with a call to a built-in function,
named open, that returns a proxy for interactions with the underlying file. For
example, the command, fp = open( sample.txt ), attempts to open a file named
sample.txt, returning a proxy that allows read-only access to the text file.
The open function accepts an optional second parameter that determines the
access mode. The default mode is r for reading. Other common modes are w
for writing to the file (causing any existing file with that name to be overwritten),
or a for appending to the end of an existing file. Although we focus on use of
text files, it is possible to work with binary files, using access modes such as rb
or wb .](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-53-2048.jpg)
![34 Chapter 1. Python Primer
Sending the wrong number, type, or value of parameters to a function is another
common cause for an exception. For example, a call to abs( hello ) will raise a
TypeError because the parameter is not numeric, and a call to abs(3, 5) will raise
a TypeError because one parameter is expected. A ValueError is typically raised
when the correct number and type of parameters are sent, but a value is illegitimate
for the context of the function. For example, the int constructor accepts a string,
as with int( 137 ), but a ValueError is raised if that string does not represent an
integer, as with int( 3.14 ) or int( hello ).
Python’s sequence types (e.g., list, tuple, and str) raise an IndexError when
syntax such as data[k] is used with an integer k that is not a valid index for the given
sequence (as described in Section 1.2.3). Sets and dictionaries raise a KeyError
when an attempt is made to access a nonexistent element.
1.7.1 Raising an Exception
An exception is thrown by executing the raise statement, with an appropriate in-
stance of an exception class as an argument that designates the problem. For exam-
ple, if a function for computing a square root is sent a negative value as a parameter,
it can raise an exception with the command:
raise ValueError( x cannot be negative )
This syntax raises a newly created instance of the ValueError class, with the error
message serving as a parameter to the constructor. If this exception is not caught
within the body of the function, the execution of the function immediately ceases
and the exception is propagated to the calling context (and possibly beyond).
When checking the validity of parameters sent to a function, it is customary
to first verify that a parameter is of an appropriate type, and then to verify that it
has an appropriate value. For example, the sqrt function in Python’s math library
performs error-checking that might be implemented as follows:
def sqrt(x):
if not isinstance(x, (int, float)):
raise TypeError( x must be numeric )
elif x 0:
raise ValueError( x cannot be negative )
# do the real work here...
Checking the type of an object can be performed at run-time using the built-in
function, isinstance. In simplest form, isinstance(obj, cls) returns True if object,
obj, is an instance of class, cls, or any subclass of that type. In the above example, a
more general form is used with a tuple of allowable types indicated with the second
parameter. After confirming that the parameter is numeric, the function enforces
an expectation that the number be nonnegative, raising a ValueError otherwise.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-56-2048.jpg)
![1.7. Exception Handling 35
How much error-checking to perform within a function is a matter of debate.
Checking the type and value of each parameter demands additional execution time
and, if taken to an extreme, seems counter to the nature of Python. Consider the
built-in sum function, which computes a sum of a collection of numbers. An im-
plementation with rigorous error-checking might be written as follows:
def sum(values):
if not isinstance(values, collections.Iterable):
raise TypeError( parameter must be an iterable type )
total = 0
for v in values:
if not isinstance(v, (int, float)):
raise TypeError( elements must be numeric )
total = total+ v
return total
The abstract base class, collections.Iterable, includes all of Python’s iterable con-
tainers types that guarantee support for the for-loop syntax (e.g., list, tuple, set);
we discuss iterables in Section 1.8, and the use of modules, such as collections, in
Section 1.11. Within the body of the for loop, each element is verified as numeric
before being added to the total. A far more direct and clear implementation of this
function can be written as follows:
def sum(values):
total = 0
for v in values:
total = total + v
return total
Interestingly, this simple implementation performs exactly like Python’s built-in
version of the function. Even without the explicit checks, appropriate exceptions
are raised naturally by the code. In particular, if values is not an iterable type, the
attempt to use the for-loop syntax raises a TypeError reporting that the object is not
iterable. In the case when a user sends an iterable type that includes a nonnumer-
ical element, such as sum([3.14, oops ]), a TypeError is naturally raised by the
evaluation of expression total + v. The error message
unsupported operand type(s) for +: ’float’ and ’str’
should be sufficiently informative to the caller. Perhaps slightly less obvious is the
error that results from sum([ alpha , beta ]). It will technically report a failed
attempt to add an int and str, due to the initial evaluation of total + alpha ,
when total has been initialized to 0.
In the remainder of this book, we tend to favor the simpler implementations
in the interest of clean presentation, performing minimal error-checking in most
situations.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-57-2048.jpg)
![1.8. Iterators and Generators 39
1.8 Iterators and Generators
In Section 1.4.2, we introduced the for-loop syntax beginning as:
for element in iterable:
and we noted that there are many types of objects in Python that qualify as being
iterable. Basic container types, such as list, tuple, and set, qualify as iterable types.
Furthermore, a string can produce an iteration of its characters, a dictionary can
produce an iteration of its keys, and a file can produce an iteration of its lines. User-
defined types may also support iteration. In Python, the mechanism for iteration is
based upon the following conventions:
• An iterator is an object that manages an iteration through a series of values. If
variable, i, identifies an iterator object, then each call to the built-in function,
next(i), produces a subsequent element from the underlying series, with a
StopIteration exception raised to indicate that there are no further elements.
• An iterable is an object, obj, that produces an iterator via the syntax iter(obj).
By these definitions, an instance of a list is an iterable, but not itself an iterator.
With data = [1, 2, 4, 8], it is not legal to call next(data). However, an iterator
object can be produced with syntax, i = iter(data), and then each subsequent call
to next(i) will return an element of that list. The for-loop syntax in Python simply
automates this process, creating an iterator for the give iterable, and then repeatedly
calling for the next element until catching the StopIteration exception.
More generally, it is possible to create multiple iterators based upon the same
iterable object, with each iterator maintaining its own state of progress. However,
iterators typically maintain their state with indirect reference back to the original
collection of elements. For example, calling iter(data) on a list instance produces
an instance of the list iterator class. That iterator does not store its own copy of the
list of elements. Instead, it maintains a current index into the original list, represent-
ing the next element to be reported. Therefore, if the contents of the original list
are modified after the iterator is constructed, but before the iteration is complete,
the iterator will be reporting the updated contents of the list.
Python also supports functions and classes that produce an implicit iterable se-
ries of values, that is, without constructing a data structure to store all of its values
at once. For example, the call range(1000000) does not return a list of numbers; it
returns a range object that is iterable. This object generates the million values one
at a time, and only as needed. Such a lazy evaluation technique has great advan-
tage. In the case of range, it allows a loop of the form, for j in range(1000000):,
to execute without setting aside memory for storing one million values. Also, if
such a loop were to be interrupted in some fashion, no time will have been spent
computing unused values of the range.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-61-2048.jpg)
![40 Chapter 1. Python Primer
We see lazy evaluation used in many of Python’s libraries. For example, the
dictionary class supports methods keys(), values(), and items(), which respec-
tively produce a “view” of all keys, values, or (key,value) pairs within a dictionary.
None of these methods produces an explicit list of results. Instead, the views that
are produced are iterable objects based upon the actual contents of the dictionary.
An explicit list of values from such an iteration can be immediately constructed by
calling the list class constructor with the iteration as a parameter. For example, the
syntax list(range(1000)) produces a list instance with values from 0 to 999, while
the syntax list(d.values()) produces a list that has elements based upon the current
values of dictionary d. We can similarly construct a tuple or set instance based
upon a given iterable.
Generators
In Section 2.3.4, we will explain how to define a class whose instances serve as
iterators. However, the most convenient technique for creating iterators in Python
is through the use of generators. A generator is implemented with a syntax that
is very similar to a function, but instead of returning values, a yield statement is
executed to indicate each element of the series. As an example, consider the goal
of determining all factors of a positive integer. For example, the number 100 has
factors 1, 2, 4, 5, 10, 20, 25, 50, 100. A traditional function might produce and
return a list containing all factors, implemented as:
def factors(n): # traditional function that computes factors
results = [ ] # store factors in a new list
for k in range(1,n+1):
if n % k == 0: # divides evenly, thus k is a factor
results.append(k) # add k to the list of factors
return results # return the entire list
In contrast, an implementation of a generator for computing those factors could be
implemented as follows:
def factors(n): # generator that computes factors
for k in range(1,n+1):
if n % k == 0: # divides evenly, thus k is a factor
yield k # yield this factor as next result
Notice use of the keyword yield rather than return to indicate a result. This indi-
cates to Python that we are defining a generator, rather than a traditional function. It
is illegal to combine yield and return statements in the same implementation, other
than a zero-argument return statement to cause a generator to end its execution. If
a programmer writes a loop such as for factor in factors(100):, an instance of our
generator is created. For each iteration of the loop, Python executes our procedure](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-62-2048.jpg)
![1.9. Additional Python Conveniences 43
1.9.2 Comprehension Syntax
A very common programming task is to produce one series of values based upon
the processing of another series. Often, this task can be accomplished quite simply
in Python using what is known as a comprehension syntax. We begin by demon-
strating list comprehension, as this was the first form to be supported by Python.
Its general form is as follows:
[ expression for value in iterable if condition ]
We note that both expression and condition may depend on value, and that the
if-clause is optional. The evaluation of the comprehension is logically equivalent
to the following traditional control structure for computing a resulting list:
result = [ ]
for value in iterable:
if condition:
result.append(expression)
As a concrete example, a list of the squares of the numbers from 1 to n, that is
[1,4,9,16,25,... ,n2], can be created by traditional means as follows:
squares = [ ]
for k in range(1, n+1):
squares.append(k k)
With list comprehension, this logic is expressed as follows:
squares = [k k for k in range(1, n+1)]
As a second example, Section 1.8 introduced the goal of producing a list of factors
for an integer n. That task is accomplished with the following list comprehension:
factors = [k for k in range(1,n+1) if n % k == 0]
Python supports similar comprehension syntaxes that respectively produce a
set, generator, or dictionary. We compare those syntaxes using our example for
producing the squares of numbers.
[ k k for k in range(1, n+1) ] list comprehension
{ k k for k in range(1, n+1) } set comprehension
( k k for k in range(1, n+1) ) generator comprehension
{ k : k k for k in range(1, n+1) } dictionary comprehension
The generator syntax is particularly attractive when results do not need to be stored
in memory. For example, to compute the sum of the first n squares, the genera-
tor syntax, total = sum(k k for k in range(1, n+1)), is preferred to the use of an
explicitly instantiated list comprehension as the parameter.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-65-2048.jpg)
![44 Chapter 1. Python Primer
1.9.3 Packing and Unpacking of Sequences
Python provides two additional conveniences involving the treatment of tuples and
other sequence types. The first is rather cosmetic. If a series of comma-separated
expressions are given in a larger context, they will be treated as a single tuple, even
if no enclosing parentheses are provided. For example, the assignment
data = 2, 4, 6, 8
results in identifier, data, being assigned to the tuple (2, 4, 6, 8). This behavior
is called automatic packing of a tuple. One common use of packing in Python is
when returning multiple values from a function. If the body of a function executes
the command,
return x, y
it will be formally returning a single object that is the tuple (x, y).
As a dual to the packing behavior, Python can automatically unpack a se-
quence, allowing one to assign a series of individual identifiers to the elements
of sequence. As an example, we can write
a, b, c, d = range(7, 11)
which has the effect of assigning a=7, b=8, c=9, and d=10, as those are the four
values in the sequence returned by the call to range. For this syntax, the right-hand
side expression can be any iterable type, as long as the number of variables on the
left-hand side is the same as the number of elements in the iteration.
This technique can be used to unpack tuples returned by a function. For exam-
ple, the built-in function, divmod(a, b), returns the pair of values (a // b, a % b)
associated with an integer division. Although the caller can consider the return
value to be a single tuple, it is possible to write
quotient, remainder = divmod(a, b)
to separately identify the two entries of the returned tuple. This syntax can also be
used in the context of a for loop, when iterating over a sequence of iterables, as in
for x, y in [ (7, 2), (5, 8), (6, 4) ]:
In this example, there will be three iterations of the loop. During the first pass, x=7
and y=2, and so on. This style of loop is quite commonly used to iterate through
key-value pairs that are returned by the items() method of the dict class, as in:
for k, v in mapping.items():](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-66-2048.jpg)
![50 Chapter 1. Python Primer
next number in a sequence based upon one or more past numbers that it has gen-
erated. Indeed, a simple yet popular pseudo-random number generator chooses its
next number based solely on the most recently chosen number and some additional
parameters using the following formula.
next = (a*current + b) % n;
where a, b, and n are appropriately chosen integers. Python uses a more advanced
technique known as a Mersenne twister. It turns out that the sequences generated
by these techniques can be proven to be statistically uniform, which is usually
good enough for most applications requiring random numbers, such as games. For
applications, such as computer security settings, where one needs unpredictable
random sequences, this kind of formula should not be used. Instead, one should
ideally sample from a source that is actually random, such as radio static coming
from outer space.
Since the next number in a pseudo-random generator is determined by the pre-
vious number(s), such a generator always needs a place to start, which is called its
seed. The sequence of numbers generated for a given seed will always be the same.
One common trick to get a different sequence each time a program is run is to use
a seed that will be different for each run. For example, we could use some timed
input from a user or the current system time in milliseconds.
Python’s random module provides support for pseudo-random number gener-
ation by defining a Random class; instances of that class serve as generators with
independent state. This allows different aspects of a program to rely on their own
pseudo-random number generator, so that calls to one generator do not affect the
sequence of numbers produced by another. For convenience, all of the methods
supported by the Random class are also supported as stand-alone functions of the
random module (essentially using a single generator instance for all top-level calls).
Syntax Description
seed(hashable)
Initializes the pseudo-random number generator
based upon the hash value of the parameter
random()
Returns a pseudo-random floating-point
value in the interval [0.0,1.0).
randint(a,b)
Returns a pseudo-random integer
in the closed interval [a,b].
randrange(start, stop, step)
Returns a pseudo-random integer in the standard
Python range indicated by the parameters.
choice(seq)
Returns an element of the given sequence
chosen pseudo-randomly.
shuffle(seq)
Reorders the elements of the given
sequence pseudo-randomly.
Table 1.8: Methods supported by instances of the Random class, and as top-level
functions of the random module.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-72-2048.jpg)
![1.12. Exercises 51
1.12 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-1.1 Write a short Python function, is multiple(n, m), that takes two integer
values and returns True if n is a multiple of m, that is, n = mi for some
integer i, and False otherwise.
R-1.2 Write a short Python function, is even(k), that takes an integer value and
returns True if k is even, and False otherwise. However, your function
cannot use the multiplication, modulo, or division operators.
R-1.3 Write a short Python function, minmax(data), that takes a sequence of
one or more numbers, and returns the smallest and largest numbers, in the
form of a tuple of length two. Do not use the built-in functions min or
max in implementing your solution.
R-1.4 Write a short Python function that takes a positive integer n and returns
the sum of the squares of all the positive integers smaller than n.
R-1.5 Give a single command that computes the sum from Exercise R-1.4, rely-
ing on Python’s comprehension syntax and the built-in sum function.
R-1.6 Write a short Python function that takes a positive integer n and returns
the sum of the squares of all the odd positive integers smaller than n.
R-1.7 Give a single command that computes the sum from Exercise R-1.6, rely-
ing on Python’s comprehension syntax and the built-in sum function.
R-1.8 Python allows negative integers to be used as indices into a sequence,
such as a string. If string s has length n, and expression s[k] is used for in-
dex −n ≤ k 0, what is the equivalent index j ≥ 0 such that s[j] references
the same element?
R-1.9 What parameters should be sent to the range constructor, to produce a
range with values 50, 60, 70, 80?
R-1.10 What parameters should be sent to the range constructor, to produce a
range with values 8, 6, 4, 2, 0, −2, −4, −6, −8?
R-1.11 Demonstrate how to use Python’s list comprehension syntax to produce
the list [1, 2, 4, 8, 16, 32, 64, 128, 256].
R-1.12 Python’s random module includes a function choice(data) that returns a
random element from a non-empty sequence. The random module in-
cludes a more basic function randrange, with parameterization similar to
the built-in range function, that return a random choice from the given
range. Using only the randrange function, implement your own version
of the choice function.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-73-2048.jpg)
![52 Chapter 1. Python Primer
Creativity
C-1.13 Write a pseudo-code description of a function that reverses a list of n
integers, so that the numbers are listed in the opposite order than they
were before, and compare this method to an equivalent Python function
for doing the same thing.
C-1.14 Write a short Python function that takes a sequence of integer values and
determines if there is a distinct pair of numbers in the sequence whose
product is odd.
C-1.15 Write a Python function that takes a sequence of numbers and determines
if all the numbers are different from each other (that is, they are distinct).
C-1.16 In our implementation of the scale function (page 25), the body of the loop
executes the command data[j] = factor. We have discussed that numeric
types are immutable, and that use of the = operator in this context causes
the creation of a new instance (not the mutation of an existing instance).
How is it still possible, then, that our implementation of scale changes the
actual parameter sent by the caller?
C-1.17 Had we implemented the scale function (page 25) as follows, does it work
properly?
def scale(data, factor):
for val in data:
val = factor
Explain why or why not.
C-1.18 Demonstrate how to use Python’s list comprehension syntax to produce
the list [0, 2, 6, 12, 20, 30, 42, 56, 72, 90].
C-1.19 Demonstrate how to use Python’s list comprehension syntax to produce
the list [ a , b , c , ..., z ], but without having to type all 26 such
characters literally.
C-1.20 Python’s random module includes a function shuffle(data) that accepts a
list of elements and randomly reorders the elements so that each possi-
ble order occurs with equal probability. The random module includes a
more basic function randint(a, b) that returns a uniformly random integer
from a to b (including both endpoints). Using only the randint function,
implement your own version of the shuffle function.
C-1.21 Write a Python program that repeatedly reads lines from standard input
until an EOFError is raised, and then outputs those lines in reverse order
(a user can indicate end of input by typing ctrl-D).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-74-2048.jpg)
![1.12. Exercises 53
C-1.22 Write a short Python program that takes two arrays a and b of length n
storing int values, and returns the dot product of a and b. That is, it returns
an array c of length n such that c[i] = a[i]·b[i], for i = 0,...,n−1.
C-1.23 Give an example of a Python code fragment that attempts to write an ele-
ment to a list based on an index that may be out of bounds. If that index
is out of bounds, the program should catch the exception that results, and
print the following error message:
“Don’t try buffer overflow attacks in Python!”
C-1.24 Write a short Python function that counts the number of vowels in a given
character string.
C-1.25 Write a short Python function that takes a string s, representing a sentence,
and returns a copy of the string with all punctuation removed. For exam-
ple, if given the string Let s try, Mike., this function would return
Lets try Mike.
C-1.26 Write a short program that takes as input three integers, a, b, and c, from
the console and determines if they can be used in a correct arithmetic
formula (in the given order), like “a+b = c,” “a = b−c,” or “a∗b = c.”
C-1.27 In Section 1.8, we provided three different implementations of a generator
that computes factors of a given integer. The third of those implementa-
tions, from page 41, was the most efficient, but we noted that it did not
yield the factors in increasing order. Modify the generator so that it reports
factors in increasing order, while maintaining its general performance ad-
vantages.
C-1.28 The p-norm of a vector v = (v1,v2,...,vn) in n-dimensional space is de-
fined as
v =
p
vp
1 +vp
2 +···+vp
n.
For the special case of p = 2, this results in the traditional Euclidean
norm, which represents the length of the vector. For example, the Eu-
clidean norm of a two-dimensional vector with coordinates (4,3) has a
Euclidean norm of
√
42 +32 =
√
16+9 =
√
25 = 5. Give an implemen-
tation of a function named norm such that norm(v, p) returns the p-norm
value of v and norm(v) returns the Euclidean norm of v. You may assume
that v is a list of numbers.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-75-2048.jpg)
![Chapter Notes 55
Chapter Notes
The official Python Web site (http://www.python.org) has a wealth of information, in-
cluding a tutorial and full documentation of the built-in functions, classes, and standard
modules. The Python interpreter is itself a useful reference, as the interactive command
help(foo) provides documentation for any function, class, or module that foo identifies.
Books providing an introduction to programming in Python include titles authored by
Campbell et al. [22], Cedar [25], Dawson [32], Goldwasser and Letscher [43], Lutz [72],
Perkovic [82], and Zelle [105]. More complete reference books on Python include titles by
Beazley [12], and Summerfield [91].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-77-2048.jpg)
![64 Chapter 2. Object-Oriented Programming
2.2.2 Pseudo-Code
As an intermediate step before the implementation of a design, programmers are
often asked to describe algorithms in a way that is intended for human eyes only.
Such descriptions are called pseudo-code. Pseudo-code is not a computer program,
but is more structured than usual prose. It is a mixture of natural language and
high-level programming constructs that describe the main ideas behind a generic
implementation of a data structure or algorithm. Because pseudo-code is designed
for a human reader, not a computer, we can communicate high-level ideas, without
being burdened with low-level implementation details. At the same time, we should
not gloss over important steps. Like many forms of human communication, finding
the right balance is an important skill that is refined through practice.
In this book, we rely on a pseudo-code style that we hope will be evident to
Python programmers, yet with a mix of mathematical notations and English prose.
For example, we might use the phrase “indicate an error” rather than a formal raise
statement. Following conventions of Python, we rely on indentation to indicate
the extent of control structures and on an indexing notation in which entries of a
sequence A with length n are indexed from A[0] to A[n− 1]. However, we choose
to enclose comments within curly braces { like these } in our pseudo-code, rather
than using Python’s # character.
2.2.3 Coding Style and Documentation
Programs should be made easy to read and understand. Good programmers should
therefore be mindful of their coding style, and develop a style that communicates
the important aspects of a program’s design for both humans and computers. Con-
ventions for coding style tend to vary between different programming communities.
The official Style Guide for Python Code is available online at
http://www.python.org/dev/peps/pep-0008/
The main principles that we adopt are as follows:
• Python code blocks are typically indented by 4 spaces. However, to avoid
having our code fragments overrun the book’s margins, we use 2 spaces for
each level of indentation. It is strongly recommended that tabs be avoided, as
tabs are displayed with differing widths across systems, and tabs and spaces
are not viewed as identical by the Python interpreter. Many Python-aware
editors will automatically replace tabs with an appropriate number of spaces.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-86-2048.jpg)
![66 Chapter 2. Object-Oriented Programming
Documentation
Python provides integrated support for embedding formal documentation directly
in source code using a mechanism known as a docstring. Formally, any string literal
that appears as the first statement within the body of a module, class, or function
(including a member function of a class) will be considered to be a docstring. By
convention, those string literals should be delimited within triple quotes (”””). As
an example, our version of the scale function from page 25 could be documented
as follows:
def scale(data, factor):
”””Multiply all entries of numeric data list by the given factor.”””
for j in range(len(data)):
data[j] = factor
It is common to use the triple-quoted string delimiter for a docstring, even when
the string fits on a single line, as in the above example. More detailed docstrings
should begin with a single line that summarizes the purpose, followed by a blank
line, and then further details. For example, we might more clearly document the
scale function as follows:
def scale(data, factor):
”””Multiply all entries of numeric data list by the given factor.
data an instance of any mutable sequence type (such as a list)
containing numeric elements
factor a number that serves as the multiplicative factor for scaling
”””
for j in range(len(data)):
data[j] = factor
A docstring is stored as a field of the module, function, or class in which it
is declared. It serves as documentation and can be retrieved in a variety of ways.
For example, the command help(x), within the Python interpreter, produces the
documentation associated with the identified object x. An external tool named
pydoc is distributed with Python and can be used to generate formal documentation
as text or as a Web page. Guidelines for authoring useful docstrings are available
at:
http://www.python.org/dev/peps/pep-0257/
In this book, we will try to present docstrings when space allows. Omitted
docstrings can be found in the online version of our source code.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-88-2048.jpg)
![2.3. Class Definitions 73
Testing the Class
In Code Fragment 2.3, we demonstrate some basic usage of the CreditCard class,
inserting three cards into a list named wallet. We use loops to make some charges
and payments, and use various accessors to print results to the console.
These tests are enclosed within a conditional, if name == __main__ :,
so that they can be embedded in the source code with the class definition. Using
the terminology of Section 2.2.4, these tests provide method coverage, as each of
the methods is called at least once, but it does not provide statement coverage, as
there is never a case in which a charge is rejected due to the credit limit. This
is not a particular advanced from of testing as the output of the given tests must
be manually audited in order to determine whether the class behaved as expected.
Python has tools for more formal testing (see discussion of the unittest module
in Section 2.2.4), so that resulting values can be automatically compared to the
predicted outcomes, with output generated only when an error is detected.
53 if name == __main__ :
54 wallet = [ ]
55 wallet.append(CreditCard( John Bowman , California Savings ,
56 5391 0375 9387 5309 , 2500) )
57 wallet.append(CreditCard( John Bowman , California Federal ,
58 3485 0399 3395 1954 , 3500) )
59 wallet.append(CreditCard( John Bowman , California Finance ,
60 5391 0375 9387 5309 , 5000) )
61
62 for val in range(1, 17):
63 wallet[0].charge(val)
64 wallet[1].charge(2 val)
65 wallet[2].charge(3 val)
66
67 for c in range(3):
68 print( Customer = , wallet[c].get customer())
69 print( Bank = , wallet[c].get bank())
70 print( Account = , wallet[c].get account())
71 print( Limit = , wallet[c].get limit())
72 print( Balance = , wallet[c].get balance())
73 while wallet[c].get balance( ) 100:
74 wallet[c].make payment(100)
75 print( New balance = , wallet[c].get balance())
76 print()
Code Fragment 2.3: Testing the CreditCard class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-95-2048.jpg)
![2.3. Class Definitions 75
Common Syntax Special Method Form
a + b a. add (b); alternatively b. radd (a)
a − b a. sub (b); alternatively b. rsub (a)
a b a. mul (b); alternatively b. rmul (a)
a / b a. truediv (b); alternatively b. rtruediv (a)
a // b a. floordiv (b); alternatively b. rfloordiv (a)
a % b a. mod (b); alternatively b. rmod (a)
a b a. pow (b); alternatively b. rpow (a)
a b a. lshift (b); alternatively b. rlshift (a)
a b a. rshift (b); alternatively b. rrshift (a)
a b a. and (b); alternatively b. rand (a)
a ˆ b a. xor (b); alternatively b. rxor (a)
a | b a. or (b); alternatively b. ror (a)
a += b a. iadd (b)
a −= b a. isub (b)
a = b a. imul (b)
... ...
+a a. pos ()
−a a. neg ()
˜a a. invert ()
abs(a) a. abs ()
a b a. lt (b)
a = b a. le (b)
a b a. gt (b)
a = b a. ge (b)
a == b a. eq (b)
a != b a. ne (b)
v in a a. contains (v)
a[k] a. getitem (k)
a[k] = v a. setitem (k,v)
del a[k] a. delitem (k)
a(arg1, arg2, ...) a. call (arg1, arg2, ...)
len(a) a. len ()
hash(a) a. hash ()
iter(a) a. iter ()
next(a) a. next ()
bool(a) a. bool ()
float(a) a. float ()
int(a) a. int ()
repr(a) a. repr ()
reversed(a) a. reversed ()
str(a) a. str ()
Table 2.1: Overloaded operations, implemented with Python’s special methods.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-97-2048.jpg)
![2.3. Class Definitions 77
2.3.3 Example: Multidimensional Vector Class
To demonstrate the use of operator overloading via special methods, we provide
an implementation of a Vector class, representing the coordinates of a vector in a
multidimensional space. For example, in a three-dimensional space, we might wish
to represent a vector with coordinates 5,−2, 3 . Although it might be tempting to
directly use a Python list to represent those coordinates, a list does not provide an
appropriate abstraction for a geometric vector. In particular, if using lists, the ex-
pression [5, −2, 3] + [1, 4, 2] results in the list [5, −2, 3, 1, 4, 2]. When working
with vectors, if u = 5,−2, 3 and v = 1, 4, 2 , one would expect the expression,
u + v, to return a three-dimensional vector with coordinates 6, 2, 5 .
We therefore define a Vector class, in Code Fragment 2.4, that provides a better
abstraction for the notion of a geometric vector. Internally, our vector relies upon
an instance of a list, named coords, as its storage mechanism. By keeping the
internal list encapsulated, we can enforce the desired public interface for instances
of our class. A demonstration of supported behaviors includes the following:
v = Vector(5) # construct five-dimensional 0, 0, 0, 0, 0
v[1] = 23 # 0, 23, 0, 0, 0 (based on use of setitem )
v[−1] = 45 # 0, 23, 0, 0, 45 (also via setitem )
print(v[4]) # print 45 (via getitem )
u = v + v # 0, 46, 0, 0, 90 (via add )
print(u) # print 0, 46, 0, 0, 90
total = 0
for entry in v: # implicit iteration via len and getitem
total += entry
We implement many of the behaviors by trivially invoking a similar behavior
on the underlying list of coordinates. However, our implementation of add
is customized. Assuming the two operands are vectors with the same length, this
method creates a new vector and sets the coordinates of the new vector to be equal
to the respective sum of the operands’ elements.
It is interesting to note that the class definition, as given in Code Fragment 2.4,
automatically supports the syntax u = v + [5, 3, 10, −2, 1], resulting in a new
vector that is the element-by-element “sum” of the first vector and the list in-
stance. This is a result of Python’s polymorphism. Literally, “polymorphism”
means “many forms.” Although it is tempting to think of the other parameter of
our add method as another Vector instance, we never declared it as such.
Within the body, the only behaviors we rely on for parameter other is that it sup-
ports len(other) and access to other[j]. Therefore, our code executes when the
right-hand operand is a list of numbers (with matching length).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-99-2048.jpg)
![78 Chapter 2. Object-Oriented Programming
1 class Vector:
2 ”””Represent a vector in a multidimensional space.”””
3
4 def init (self, d):
5 ”””Create d-dimensional vector of zeros.”””
6 self. coords = [0] d
7
8 def len (self):
9 ”””Return the dimension of the vector.”””
10 return len(self. coords)
11
12 def getitem (self, j):
13 ”””Return jth coordinate of vector.”””
14 return self. coords[j]
15
16 def setitem (self, j, val):
17 ”””Set jth coordinate of vector to given value.”””
18 self. coords[j] = val
19
20 def add (self, other):
21 ”””Return sum of two vectors.”””
22 if len(self) != len(other): # relies on len method
23 raise ValueError( dimensions must agree )
24 result = Vector(len(self)) # start with vector of zeros
25 for j in range(len(self)):
26 result[j] = self[j] + other[j]
27 return result
28
29 def eq (self, other):
30 ”””Return True if vector has same coordinates as other.”””
31 return self. coords == other. coords
32
33 def ne (self, other):
34 ”””Return True if vector differs from other.”””
35 return not self == other # rely on existing eq definition
36
37 def str (self):
38 ”””Produce string representation of vector.”””
39 return + str(self. coords)[1:−1] + # adapt list representation
Code Fragment 2.4: Definition of a simple Vector class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-100-2048.jpg)
![2.3. Class Definitions 79
2.3.4 Iterators
Iteration is an important concept in the design of data structures. We introduced
Python’s mechanism for iteration in Section 1.8. In short, an iterator for a collec-
tion provides one key behavior: It supports a special method named next that
returns the next element of the collection, if any, or raises a StopIteration exception
to indicate that there are no further elements.
Fortunately, it is rare to have to directly implement an iterator class. Our pre-
ferred approach is the use of the generator syntax (also described in Section 1.8),
which automatically produces an iterator of yielded values.
Python also helps by providing an automatic iterator implementation for any
class that defines both len and getitem . To provide an instructive exam-
ple of a low-level iterator, Code Fragment 2.5 demonstrates just such an iterator
class that works on any collection that supports both len and getitem .
This class can be instantiated as SequenceIterator(data). It operates by keeping an
internal reference to the data sequence, as well as a current index into the sequence.
Each time next is called, the index is incremented, until reaching the end of
the sequence.
1 class SequenceIterator:
2 ”””An iterator for any of Python s sequence types.”””
3
4 def init (self, sequence):
5 ”””Create an iterator for the given sequence.”””
6 self. seq = sequence # keep a reference to the underlying data
7 self. k = −1 # will increment to 0 on first call to next
8
9 def next (self):
10 ”””Return the next element, or else raise StopIteration error.”””
11 self. k += 1 # advance to next index
12 if self. k len(self. seq):
13 return(self. seq[self. k]) # return the data element
14 else:
15 raise StopIteration( ) # there are no more elements
16
17 def iter (self):
18 ”””By convention, an iterator must return itself as an iterator.”””
19 return self
Code Fragment 2.5: An iterator class for any sequence type.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-101-2048.jpg)
![80 Chapter 2. Object-Oriented Programming
2.3.5 Example: Range Class
As the final example for this section, we develop our own implementation of a
class that mimics Python’s built-in range class. Before introducing our class, we
discuss the history of the built-in version. Prior to Python 3 being released, range
was implemented as a function, and it returned a list instance with elements in
the specified range. For example, range(2, 10, 2) returned the list [2, 4, 6, 8].
However, a typical use of the function was to support a for-loop syntax, such as
for k in range(10000000). Unfortunately, this caused the instantiation and initial-
ization of a list with the range of numbers. That was an unnecessarily expensive
step, in terms of both time and memory usage.
The mechanism used to support ranges in Python 3 is entirely different (to be
fair, the “new” behavior existed in Python 2 under the name xrange). It uses a
strategy known as lazy evaluation. Rather than creating a new list instance, range
is a class that can effectively represent the desired range of elements without ever
storing them explicitly in memory. To better explore the built-in range class, we
recommend that you create an instance as r = range(8, 140, 5). The result is a
relatively lightweight object, an instance of the range class, that has only a few
behaviors. The syntax len(r) will report the number of elements that are in the
given range (27, in our example). A range also supports the getitem method,
so that syntax r[15] reports the sixteenth element in the range (as r[0] is the first
element). Because the class supports both len and getitem , it inherits
automatic support for iteration (see Section 2.3.4), which is why it is possible to
execute a for loop over a range.
At this point, we are ready to demonstrate our own version of such a class. Code
Fragment 2.6 provides a class we name Range (so as to clearly differentiate it from
built-in range). The biggest challenge in the implementation is properly computing
the number of elements that belong in the range, given the parameters sent by the
caller when constructing a range. By computing that value in the constructor, and
storing it as self. length, it becomes trivial to return it from the len method. To
properly implement a call to getitem (k), we simply take the starting value of
the range plus k times the step size (i.e., for k=0, we return the start value). There
are a few subtleties worth examining in the code:
• To properly support optional parameters, we rely on the technique described
on page 27, when discussing a functional version of range.
• We compute the number of elements in the range as
max(0, (stop − start + step − 1) // step)
It is worth testing this formula for both positive and negative step sizes.
• The getitem method properly supports negative indices by converting
an index −k to len(self)−k before computing the result.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-102-2048.jpg)
![94 Chapter 2. Object-Oriented Programming
1 from abc import ABCMeta, abstractmethod # need these definitions
2
3 class Sequence(metaclass=ABCMeta):
4 ”””Our own version of collections.Sequence abstract base class.”””
5
6 @abstractmethod
7 def len (self):
8 ”””Return the length of the sequence.”””
9
10 @abstractmethod
11 def getitem (self, j):
12 ”””Return the element at index j of the sequence.”””
13
14 def contains (self, val):
15 ”””Return True if val found in the sequence; False otherwise.”””
16 for j in range(len(self)):
17 if self[j] == val: # found match
18 return True
19 return False
20
21 def index(self, val):
22 ”””Return leftmost index at which val is found (or raise ValueError).”””
23 for j in range(len(self)):
24 if self[j] == val: # leftmost match
25 return j
26 raise ValueError( value not in sequence ) # never found a match
27
28 def count(self, val):
29 ”””Return the number of elements equal to given value.”””
30 k = 0
31 for j in range(len(self)):
32 if self[j] == val: # found a match
33 k += 1
34 return k
Code Fragment 2.14: An abstract base class akin to collections.Sequence.
This implementation relies on two advanced Python techniques. The first is that
we declare the ABCMeta class of the abc module as a metaclass of our Sequence
class. A metaclass is different from a superclass, in that it provides a template for
the class definition itself. Specifically, the ABCMeta declaration assures that the
constructor for the class raises an error.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-116-2048.jpg)
![2.4. Inheritance 95
The second advanced technique is the use of the @abstractmethod decorator
immediately before the len and getitem methods are declared. That de-
clares these two particular methods to be abstract, meaning that we do not provide
an implementation within our Sequence base class, but that we expect any concrete
subclasses to support those two methods. Python enforces this expectation, by dis-
allowing instantiation for any subclass that does not override the abstract methods
with concrete implementations.
The rest of the Sequence class definition provides tangible implementations for
other behaviors, under the assumption that the abstract len and getitem
methods will exist in a concrete subclass. If you carefully examine the source code,
the implementations of methods contains , index, and count do not rely on any
assumption about the self instances, other than that syntax len(self) and self[j] are
supported (by special methods len and getitem , respectively). Support
for iteration is automatic as well, as described in Section 2.3.4.
In the remainder of this book, we omit the formality of using the abc module.
If we need an “abstract” base class, we simply document the expectation that sub-
classes provide assumed functionality, without technical declaration of the methods
as abstract. But we will make use of the wonderful abstract base classes that are
defined within the collections module (such as Sequence). To use such a class, we
need only rely on standard inheritance techniques.
For example, our Range class, from Code Fragment 2.6 of Section 2.3.5, is an
example of a class that supports the len and getitem methods. But that
class does not support methods count or index. Had we originally declared it with
Sequence as a superclass, then it would also inherit the count and index methods.
The syntax for such a declaration would begin as:
class Range(collections.Sequence):
Finally, we emphasize that if a subclass provides its own implementation of
an inherited behaviors from a base class, the new definition overrides the inherited
one. This technique can be used when we have the ability to provide a more effi-
cient implementation for a behavior than is achieved by the generic approach. As
an example, the general implementation of contains for a sequence is based
on a loop used to search for the desired value. For our Range class, there is an
opportunity for a more efficient determination of containment. For example, it
is evident that the expression, 100000 in Range(0, 2000000, 100), should evalu-
ate to True, even without examining the individual elements of the range, because
the range starts with zero, has an increment of 100, and goes until 2 million; it
must include 100000, as that is a multiple of 100 that is between the start and
stop values. Exercise C-2.27 explores the goal of providing an implementation of
Range. contains that avoids the use of a (time-consuming) loop.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-117-2048.jpg)
![102 Chapter 2. Object-Oriented Programming
red
green
blue
color
52
163
169
list list
warmtones palette
red
green
blue
color
43
124
249
Figure 2.10: A shallow copy of a list of colors.
This is a better situation than our first attempt, as we can legitimately add
or remove elements from palette without affecting warmtones. However, if we
edit a color instance from the palette list, we effectively change the contents of
warmtones. Although palette and warmtones are distinct lists, there remains indi-
rect aliasing, for example, with palette[0] and warmtones[0] as aliases for the same
color instance.
We prefer that palette be what is known as a deep copy of warmtones. In a
deep copy, the new copy references its own copies of those objects referenced by
the original version. (See Figure 2.11.)
blue
color
52
163
169
list
red
green
blue
color
43
124
249 red
green
blue
color
52
163
169
list
red
green
blue
color
43
124
249
warmtones palette
red
green
Figure 2.11: A deep copy of a list of colors.
Python’s copy Module
To create a deep copy, we could populate our list by explicitly making copies of
the original color instances, but this requires that we know how to make copies of
colors (rather than aliasing). Python provides a very convenient module, named
copy, that can produce both shallow copies and deep copies of arbitrary objects.
This module supports two functions: the copy function creates a shallow copy
of its argument, and the deepcopy function creates a deep copy of its argument.
After importing the module, we may create a deep copy for our example, as shown
in Figure 2.11, using the command:
palette = copy.deepcopy(warmtones)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-124-2048.jpg)
![104 Chapter 2. Object-Oriented Programming
R-2.11 In Section 2.3.3, we note that our Vector class supports a syntax such as
v = u + [5, 3, 10, −2, 1], in which the sum of a vector and list returns
a new vector. However, the syntax v = [5, 3, 10, −2, 1] + u is illegal.
Explain how the Vector class definition can be revised so that this syntax
generates a new vector.
R-2.12 Implement the mul method for the Vector class of Section 2.3.3, so
that the expression v 3 returns a new vector with coordinates that are 3
times the respective coordinates of v.
R-2.13 Exercise R-2.12 asks for an implementation of mul , for the Vector
class of Section 2.3.3, to provide support for the syntax v 3. Implement
the rmul method, to provide additional support for syntax 3 v.
R-2.14 Implement the mul method for the Vector class of Section 2.3.3, so
that the expression u v returns a scalar that represents the dot product of
the vectors, that is, ∑d
i=1 ui ·vi.
R-2.15 The Vector class of Section 2.3.3 provides a constructor that takes an in-
teger d, and produces a d-dimensional vector with all coordinates equal to
0. Another convenient form for creating a new vector would be to send the
constructor a parameter that is some iterable type representing a sequence
of numbers, and to create a vector with dimension equal to the length of
that sequence and coordinates equal to the sequence values. For example,
Vector([4, 7, 5]) would produce a three-dimensional vector with coordi-
nates 4, 7, 5. Modify the constructor so that either of these forms is
acceptable; that is, if a single integer is sent, it produces a vector of that
dimension with all zeros, but if a sequence of numbers is provided, it pro-
duces a vector with coordinates based on that sequence.
R-2.16 Our Range class, from Section 2.3.5, relies on the formula
max(0, (stop − start + step − 1) // step)
to compute the number of elements in the range. It is not immediately ev-
ident why this formula provides the correct calculation, even if assuming
a positive step size. Justify this formula, in your own words.
R-2.17 Draw a class inheritance diagram for the following set of classes:
• Class Goat extends object and adds an instance variable tail and
methods milk() and jump().
• Class Pig extends object and adds an instance variable nose and
methods eat(food) and wallow().
• Class Horse extends object and adds instance variables height and
color, and methods run() and jump().
• Class Racer extends Horse and adds a method race().
• Class Equestrian extends Horse, adding an instance variable weight
and methods trot() and is trained().](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-126-2048.jpg)
![108 Chapter 2. Object-Oriented Programming
Chapter Notes
For a broad overview of developments in computer science and engineering, we refer the
reader to The Computer Science and Engineering Handbook [96]. For more information
about the Therac-25 incident, please see the paper by Leveson and Turner [69].
The reader interested in studying object-oriented programming further, is referred to
the books by Booch [17], Budd [20], and Liskov and Guttag [71]. Liskov and Guttag
also provide a nice discussion of abstract data types, as does the survey paper by Cardelli
and Wegner [23] and the book chapter by Demurjian [33] in the The Computer Science
and Engineering Handbook [96]. Design patterns are described in the book by Gamma et
al. [41].
Books with specific focus on object-oriented programming in Python include those
by Goldwasser and Letscher [43] at the introductory level, and by Phillips [83] at a more
advanced level,](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-130-2048.jpg)
![3.3. Asymptotic Analysis 123
3.3 Asymptotic Analysis
In algorithm analysis, we focus on the growth rate of the running time as a function
of the input size n, taking a “big-picture” approach. For example, it is often enough
just to know that the running time of an algorithm grows proportionally to n.
We analyze algorithms using a mathematical notation for functions that disre-
gards constant factors. Namely, we characterize the running times of algorithms
by using functions that map the size of the input, n, to values that correspond to
the main factor that determines the growth rate in terms of n. This approach re-
flects that each basic step in a pseudo-code description or a high-level language
implementation may correspond to a small number of primitive operations. Thus,
we can perform an analysis of an algorithm by estimating the number of primitive
operations executed up to a constant factor, rather than getting bogged down in
language-specific or hardware-specific analysis of the exact number of operations
that execute on the computer.
As a tangible example, we revisit the goal of finding the largest element of a
Python list; we first used this example when introducing for loops on page 21 of
Section 1.4.2. Code Fragment 3.1 presents a function named find max for this task.
1 def find max(data):
2 ”””Return the maximum element from a nonempty Python list.”””
3 biggest = data[0] # The initial value to beat
4 for val in data: # For each value:
5 if val biggest # if it is greater than the best so far,
6 biggest = val # we have found a new best (so far)
7 return biggest # When loop ends, biggest is the max
Code Fragment 3.1: A function that returns the maximum value of a Python list.
This is a classic example of an algorithm with a running time that grows pro-
portional to n, as the loop executes once for each data element, with some fixed
number of primitive operations executing for each pass. In the remainder of this
section, we provide a framework to formalize this claim.
3.3.1 The “Big-Oh” Notation
Let f(n) and g(n) be functions mapping positive integers to positive real numbers.
We say that f(n) is O(g(n)) if there is a real constant c 0 and an integer constant
n0 ≥ 1 such that
f(n) ≤ cg(n), for n ≥ n0.
This definition is often referred to as the “big-Oh” notation, for it is sometimes pro-
nounced as “f(n) is big-Oh of g(n).” Figure 3.5 illustrates the general definition.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-145-2048.jpg)
![130 Chapter 3. Algorithm Analysis
If we must draw a line between efficient and inefficient algorithms, therefore,
it is natural to make this distinction be that between those algorithms running in
polynomial time and those running in exponential time. That is, make the distinc-
tion between algorithms with a running time that is O(nc), for some constant c 1,
and those with a running time that is O(bn), for some constant b 1. Like so many
notions we have discussed in this section, this too should be taken with a “grain of
salt,” for an algorithm running in O(n100) time should probably not be considered
“efficient.” Even so, the distinction between polynomial-time and exponential-time
algorithms is considered a robust measure of tractability.
3.3.3 Examples of Algorithm Analysis
Now that we have the big-Oh notation for doing algorithm analysis, let us give some
examples by characterizing the running time of some simple algorithms using this
notation. Moreover, in keeping with our earlier promise, we illustrate below how
each of the seven functions given earlier in this chapter can be used to characterize
the running time of an example algorithm.
Rather than use pseudo-code in this section, we give complete Python imple-
mentations for our examples. We use Python’s list class as the natural representa-
tion for an “array” of values. In Chapter 5, we will fully explore the underpinnings
of Python’s list class, and the efficiency of the various behaviors that it supports. In
this section, we rely on just a few of its behaviors, discussing their efficiencies as
introduced.
Constant-Time Operations
Given an instance, named data, of the Python list class, a call to the function,
len(data), is evaluated in constant time. This is a very simple algorithm because
the list class maintains, for each list, an instance variable that records the current
length of the list. This allows it to immediately report that length, rather than take
time to iteratively count each of the elements in the list. Using asymptotic notation,
we say that this function runs in O(1) time; that is, the running time of this function
is independent of the length, n, of the list.
Another central behavior of Python’s list class is that it allows access to an arbi-
trary element of the list using syntax, data[j], for integer index j. Because Python’s
lists are implemented as array-based sequences, references to a list’s elements are
stored in a consecutive block of memory. The jth element of the list can be found,
not by iterating through the list one element at a time, but by validating the index,
and using it as an offset into the underlying array. In turn, computer hardware sup-
ports constant-time access to an element based on its memory address. Therefore,
we say that the expression data[j] is evaluated in O(1) time for a Python list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-152-2048.jpg)
![3.3. Asymptotic Analysis 131
Revisiting the Problem of Finding the Maximum of a Sequence
For our next example, we revisit the find max algorithm, given in Code Frag-
ment 3.1 on page 123, for finding the largest value in a sequence. Proposition 3.7
on page 125 claimed an O(n) run-time for the find max algorithm. Consistent with
our earlier analysis of syntax data[0], the initialization uses O(1) time. The loop
executes n times, and within each iteration, it performs one comparison and possi-
bly one assignment statement (as well as maintenance of the loop variable). Finally,
we note that the mechanism for enacting a return statement in Python uses O(1)
time. Combining these steps, we have that the find max function runs in O(n) time.
Further Analysis of the Maximum-Finding Algorithm
A more interesting question about find max is how many times we might update
the current “biggest” value. In the worst case, if the data is given to us in increasing
order, the biggest value is reassigned n − 1 times. But what if the input is given
to us in random order, with all orders equally likely; what would be the expected
number of times we update the biggest value in this case? To answer this question,
note that we update the current biggest in an iteration of the loop only if the current
element is bigger than all the elements that precede it. If the sequence is given to
us in random order, the probability that the jth element is the largest of the first j
elements is 1/j (assuming uniqueness). Hence, the expected number of times we
update the biggest (including initialization) is Hn = ∑n
j=1 1/j, which is known as
the nth Harmonic number. It turns out (see Proposition B.16) that Hn is O(logn).
Therefore, the expected number of times the biggest value is updated by find max
on a randomly ordered sequence is O(logn).
Prefix Averages
The next problem we consider is computing what are known as prefix averages
of a sequence of numbers. Namely, given a sequence S consisting of n num-
bers, we want to compute a sequence A such that A[ j] is the average of elements
S[0],...,S[ j], for j = 0,...,n−1, that is,
A[ j] =
∑
j
i=0 S[i]
j +1
.
Computing prefix averages has many applications in economics and statistics. For
example, given the year-by-year returns of a mutual fund, ordered from recent to
past, an investor will typically want to see the fund’s average annual returns for the
most recent year, the most recent three years, the most recent five years, and so on.
Likewise, given a stream of daily Web usage logs, a Web site manager may wish
to track average usage trends over various time periods. We analyze three different
implementations that solve this problem but with rather different running times.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-153-2048.jpg)
![132 Chapter 3. Algorithm Analysis
A Quadratic-Time Algorithm
Our first algorithm for computing prefix averages, named prefix average1, is shown
in Code Fragment 3.2. It computes every element of A separately, using an inner
loop to compute the partial sum.
1 def prefix average1(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 for j in range(n):
6 total = 0 # begin computing S[0] + ... + S[j]
7 for i in range(j + 1):
8 total += S[i]
9 A[j] = total / (j+1) # record the average
10 return A
Code Fragment 3.2: Algorithm prefix average1.
In order to analyze the prefix average1 algorithm, we consider the various steps
that are executed.
• The statement, n = len(S), executes in constant time, as described at the
beginning of Section 3.3.3.
• The statement, A = [0] n, causes the creation and initialization of a Python
list with length n, and with all entries equal to zero. This uses a constant
number of primitive operations per element, and thus runs in O(n) time.
• There are two nested for loops, which are controlled, respectively, by coun-
ters j and i. The body of the outer loop, controlled by counter j, is ex-
ecuted n times, for j = 0,...,n − 1. Therefore, statements total = 0 and
A[j] = total / (j+1) are executed n times each. This implies that these two
statements, plus the management of counter j in the range, contribute a num-
ber of primitive operations proportional to n, that is, O(n) time.
• The body of the inner loop, which is controlled by counter i, is executed j+1
times, depending on the current value of the outer loop counter j. Thus, state-
ment total += S[i], in the inner loop, is executed 1+ 2+ 3+ ··· + n times.
By recalling Proposition 3.3, we know that 1+2+3+···+n = n(n+1)/2,
which implies that the statement in the inner loop contributes O(n2) time.
A similar argument can be done for the primitive operations associated with
maintaining counter i, which also take O(n2) time.
The running time of implementation prefix average1 is given by the sum of three
terms. The first and the second terms are O(n), and the third term is O(n2). By a
simple application of Proposition 3.9, the running time of prefix average1 is O(n2).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-154-2048.jpg)
![3.3. Asymptotic Analysis 133
Our second implementation for computing prefix averages, prefix average2, is
presented in Code Fragment 3.3.
1 def prefix average2(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 for j in range(n):
6 A[j] = sum(S[0:j+1]) / (j+1) # record the average
7 return A
Code Fragment 3.3: Algorithm prefix average2.
This approach is essentially the same high-level algorithm as in prefix average1,
but we have replaced the inner loop by using the single expression sum(S[0:j+1])
to compute the partial sum, S[0]+··· +S[ j]. While the use of that function greatly
simplifies the presentation of the algorithm, it is worth asking how it affects the
efficiency. Asymptotically, this implementation is no better. Even though the ex-
pression, sum(S[0:j+1]), seems like a single command, it is a function call and
an evaluation of that function takes O(j + 1) time in this context. Technically, the
computation of the slice, S[0:j+1], also uses O(j + 1) time, as it constructs a new
list instance for storage. So the running time of prefix average2 is still dominated
by a series of steps that take time proportional to 1+2+3+···+n, and thus O(n2).
A Linear-Time Algorithm
Our final algorithm, prefix averages3, is given in Code Fragment 3.4. Just as with
our first two algorithms, we are interested in computing, for each j, the prefix sum
S[0] + S[1] + ··· + S[ j], denoted as total in our code, so that we can then compute
the prefix average A[j] =total / (j + 1). However, there is a key difference that
results in much greater efficiency.
1 def prefix average3(S):
2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].”””
3 n = len(S)
4 A = [0] n # create new list of n zeros
5 total = 0 # compute prefix sum as S[0] + S[1] + ...
6 for j in range(n):
7 total += S[j] # update prefix sum to include S[j]
8 A[j] = total / (j+1) # compute average based on current sum
9 return A
Code Fragment 3.4: Algorithm prefix average3.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-155-2048.jpg)
![134 Chapter 3. Algorithm Analysis
In our first two algorithms, the prefix sum is computed anew for each value of j.
That contributed O(j) time for each j, leading to the quadratic behavior. In algo-
rithm prefix average3, we maintain the current prefix sum dynamically, effectively
computing S[0]+S[1]+··· +S[ j] as total + S[j], where value total is equal to the
sum S[0] + S[1] + ··· + S[ j − 1] computed by the previous pass of the loop over j.
The analysis of the running time of algorithm prefix average3 follows:
• Initializing variables n and total uses O(1) time.
• Initializing the list A uses O(n) time.
• There is a single for loop, which is controlled by counter j. The maintenance
of that counter by the range iterator contributes a total of O(n) time.
• The body of the loop is executed n times, for j = 0,...,n − 1. Thus, state-
ments total += S[j] and A[j] = total / (j+1) are executed n times each.
Since each of these statements uses O(1) time per iteration, their overall
contribution is O(n) time.
The running time of algorithm prefix average3 is given by the sum of the four
terms. The first is O(1) and the remaining three are O(n). By a simple application
of Proposition 3.9, the running time of prefix average3 is O(n), which is much
better than the quadratic time of algorithms prefix average1 and prefix average2.
Three-Way Set Disjointness
Suppose we are given three sequences of numbers, A, B, and C. We will assume
that no individual sequence contains duplicate values, but that there may be some
numbers that are in two or three of the sequences. The three-way set disjointness
problem is to determine if the intersection of the three sequences is empty, namely,
that there is no element x such that x ∈ A, x ∈ B, and x ∈ C. A simple Python
function to determine this property is given in Code Fragment 3.5.
1 def disjoint1(A, B, C):
2 ”””Return True if there is no element common to all three lists.”””
3 for a in A:
4 for b in B:
5 for c in C:
6 if a == b == c:
7 return False # we found a common value
8 return True # if we reach this, sets are disjoint
Code Fragment 3.5: Algorithm disjoint1 for testing three-way set disjointness.
This simple algorithm loops through each possible triple of values from the
three sets to see if those values are equivalent. If each of the original sets has size
n, then the worst-case running time of this function is O(n3).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-156-2048.jpg)
![136 Chapter 3. Algorithm Analysis
1 def unique1(S):
2 ”””Return True if there are no duplicate elements in sequence S.”””
3 for j in range(len(S)):
4 for k in range(j+1, len(S)):
5 if S[j] == S[k]:
6 return False # found duplicate pair
7 return True # if we reach this, elements were unique
Code Fragment 3.7: Algorithm unique1 for testing element uniqueness.
those pairs refer to elements that are equivalent to each other. It does this using two
nested for loops, such that the first iteration of the outer loop causes n−1 iterations
of the inner loop, the second iteration of the outer loop causes n − 2 iterations of
the inner loop, and so on. Thus, the worst-case running time of this function is
proportional to
(n−1)+(n−2)+···+2+1,
which we recognize as the familiar O(n2) summation from Proposition 3.3.
Using Sorting as a Problem-Solving Tool
An even better algorithm for the element uniqueness problem is based on using
sorting as a problem-solving tool. In this case, by sorting the sequence of elements,
we are guaranteed that any duplicate elements will be placed next to each other.
Thus, to determine if there are any duplicates, all we need to do is perform a sin-
gle pass over the sorted sequence, looking for consecutive duplicates. A Python
implementation of this algorithm is as follows:
1 def unique2(S):
2 ”””Return True if there are no duplicate elements in sequence S.”””
3 temp = sorted(S) # create a sorted copy of S
4 for j in range(1, len(temp)):
5 if S[j−1] == S[j]:
6 return False # found duplicate pair
7 return True # if we reach this, elements were unique
Code Fragment 3.8: Algorithm unique2 for testing element uniqueness.
The built-in function, sorted, as described in Section 1.5.2, produces a copy of
the original list with elements in sorted order. It guarantees a worst-case running
time of O(nlogn); see Chapter 12 for a discussion of common sorting algorithms.
Once the data is sorted, the subsequent loop runs in O(n) time, and so the entire
unique2 algorithm runs in O(nlogn) time.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-158-2048.jpg)
![140 Chapter 3. Algorithm Analysis
Loop Invariants
The final justification technique we discuss in this section is the loop invariant. To
prove some statement L about a loop is correct, define L in terms of a series of
smaller statements L0,L1,...,Lk, where:
1. The initial claim, L0, is true before the loop begins.
2. If Lj−1 is true before iteration j, then Lj will be true after iteration j.
3. The final statement, Lk, implies the desired statement L to be true.
Let us give a simple example of using a loop-invariant argument to justify the
correctness of an algorithm. In particular, we use a loop invariant to justify that
the function, find (see Code Fragment 3.9), finds the smallest index at which ele-
ment val occurs in sequence S.
1 def find(S, val):
2 ”””Return index j such that S[j] == val, or -1 if no such element.”””
3 n = len(S)
4 j = 0
5 while j n:
6 if S[j] == val:
7 return j # a match was found at index j
8 j += 1
9 return −1
Code Fragment 3.9: Algorithm for finding the first index at which a given element
occurs in a Python list.
To show that find is correct, we inductively define a series of statements, Lj,
that lead to the correctness of our algorithm. Specifically, we claim the following
is true at the beginning of iteration j of the while loop:
Lj: val is not equal to any of the first j elements of S.
This claim is true at the beginning of the first iteration of the loop, because j is 0
and there are no elements among the first 0 in S (this kind of a trivially true claim
is said to hold vacuously). In iteration j, we compare element val to element S[ j]
and return the index j if these two elements are equivalent, which is clearly correct
and completes the algorithm in this case. If the two elements val and S[ j] are not
equal, then we have found one more element not equal to val and we increment
the index j. Thus, the claim Lj will be true for this new value of j; hence, it is
true at the beginning of the next iteration. If the while loop terminates without
ever returning an index in S, then we have j = n. That is, Ln is true—there are no
elements of S equal to val. Therefore, the algorithm correctly returns −1 to indicate
that val is not in S.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-162-2048.jpg)
![3.5. Exercises 143
1 def example1(S):
2 ”””Return the sum of the elements in sequence S.”””
3 n = len(S)
4 total = 0
5 for j in range(n): # loop from 0 to n-1
6 total += S[j]
7 return total
8
9 def example2(S):
10 ”””Return the sum of the elements with even index in sequence S.”””
11 n = len(S)
12 total = 0
13 for j in range(0, n, 2): # note the increment of 2
14 total += S[j]
15 return total
16
17 def example3(S):
18 ”””Return the sum of the prefix sums of sequence S.”””
19 n = len(S)
20 total = 0
21 for j in range(n): # loop from 0 to n-1
22 for k in range(1+j): # loop from 0 to j
23 total += S[k]
24 return total
25
26 def example4(S):
27 ”””Return the sum of the prefix sums of sequence S.”””
28 n = len(S)
29 prefix = 0
30 total = 0
31 for j in range(n):
32 prefix += S[j]
33 total += prefix
34 return total
35
36 def example5(A, B): # assume that A and B have equal length
37 ”””Return the number of elements in B equal to the sum of prefix sums in A.”””
38 n = len(A)
39 count = 0
40 for i in range(n): # loop from 0 to n-1
41 total = 0
42 for j in range(n): # loop from 0 to n-1
43 for k in range(1+j): # loop from 0 to j
44 total += A[k]
45 if B[i] == total:
46 count += 1
47 return count
Code Fragment 3.10: Some sample algorithms for analysis.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-165-2048.jpg)
![144 Chapter 3. Algorithm Analysis
R-3.32 Given an n-element sequence S, Algorithm D calls Algorithm E on each
element S[i]. Algorithm E runs in O(i) time when it is called on element
S[i]. What is the worst-case running time of Algorithm D?
R-3.33 Al and Bob are arguing about their algorithms. Al claims his O(nlogn)-
time method is always faster than Bob’s O(n2)-time method. To settle the
issue, they perform a set of experiments. To Al’s dismay, they find that if
n 100, the O(n2)-time algorithm runs faster, and only when n ≥ 100 is
the O(nlogn)-time one better. Explain how this is possible.
R-3.34 There is a well-known city (which will go nameless here) whose inhabi-
tants have the reputation of enjoying a meal only if that meal is the best
they have ever experienced in their life. Otherwise, they hate it. Assum-
ing meal quality is distributed uniformly across a person’s life, describe
the expected number of times inhabitants of this city are happy with their
meals?
Creativity
C-3.35 Assuming it is possible to sort n numbers in O(nlogn) time, show that it
is possible to solve the three-way set disjointness problem in O(nlogn)
time.
C-3.36 Describe an efficient algorithm for finding the ten largest elements in a
sequence of size n. What is the running time of your algorithm?
C-3.37 Give an example of a positive function f(n) such that f(n) is neither O(n)
nor Ω(n).
C-3.38 Show that ∑n
i=1 i2 is O(n3).
C-3.39 Show that ∑n
i=1 i/2i 2. (Hint: Try to bound this sum term by term with
a geometric progression.)
C-3.40 Show that logb f(n) is Θ(log f(n)) if b 1 is a constant.
C-3.41 Describe an algorithm for finding both the minimum and maximum of n
numbers using fewer than 3n/2 comparisons. (Hint: First, construct a
group of candidate minimums and a group of candidate maximums.)
C-3.42 Bob built a Web site and gave the URL only to his n friends, which he
numbered from 1 to n. He told friend number i that he/she can visit the
Web site at most i times. Now Bob has a counter, C, keeping track of the
total number of visits to the site (but not the identities of who visits). What
is the minimum value for C such that Bob can know that one of his friends
has visited his/her maximum allowed number of times?
C-3.43 Draw a visual justification of Proposition 3.3 analogous to that of Fig-
ure 3.3(b) for the case when n is odd.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-166-2048.jpg)
![3.5. Exercises 145
C-3.44 Communication security is extremely important in computer networks,
and one way many network protocols achieve security is to encrypt mes-
sages. Typical cryptographic schemes for the secure transmission of mes-
sages over such networks are based on the fact that no efficient algorithms
are known for factoring large integers. Hence, if we can represent a secret
message by a large prime number p, we can transmit, over the network,
the number r = p·q, where q p is another large prime number that acts
as the encryption key. An eavesdropper who obtains the transmitted num-
ber r on the network would have to factor r in order to figure out the secret
message p.
Using factoring to figure out a message is very difficult without knowing
the encryption key q. To understand why, consider the following naive
factoring algorithm:
for p in range(2,r):
if r % p == 0: # if p divides r
return The secret message is p!
a. Suppose that the eavesdropper uses the above algorithm and has a
computer that can carry out in 1 microsecond (1 millionth of a sec-
ond) a division between two integers of up to 100 bits each. Give an
estimate of the time that it will take in the worst case to decipher the
secret message p if the transmitted message r has 100 bits.
b. What is the worst-case time complexity of the above algorithm?
Since the input to the algorithm is just one large number r, assume
that the input size n is the number of bytes needed to store r, that is,
n = (log2 r)/8+1, and that each division takes time O(n).
C-3.45 A sequence S contains n − 1 unique integers in the range [0,n − 1], that
is, there is one number from this range that is not in S. Design an O(n)-
time algorithm for finding that number. You are only allowed to use O(1)
additional space besides the sequence S itself.
C-3.46 Al says he can prove that all sheep in a flock are the same color:
Base case: One sheep. It is clearly the same color as itself.
Induction step: A flock of n sheep. Take a sheep, a, out. The remaining
n − 1 are all the same color by induction. Now put sheep a back in and
take out a different sheep, b. By induction, the n− 1 sheep (now with a)
are all the same color. Therefore, all the sheep in the flock are the same
color. What is wrong with Al’s “justification”?
C-3.47 Let S be a set of n lines in the plane such that no two are parallel and
no three meet in the same point. Show, by induction, that the lines in S
determine Θ(n2) intersection points.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-167-2048.jpg)
![146 Chapter 3. Algorithm Analysis
C-3.48 Consider the following “justification” that the Fibonacci function, F(n)
(see Proposition 3.20) is O(n):
Base case (n ≤ 2): F(1) = 1 and F(2) = 2.
Induction step (n 2): Assume claim true for n n. Consider n. F(n) =
F(n−2)+F(n−1). By induction, F(n−2) is O(n−2) and F(n−1) is
O(n−1). Then, F(n) is O((n−2)+(n−1)), by the identity presented in
Exercise R-3.11. Therefore, F(n) is O(n).
What is wrong with this “justification”?
C-3.49 Consider the Fibonacci function, F(n) (see Proposition 3.20). Show by
induction that F(n) is Ω((3/2)n).
C-3.50 Let p(x) be a polynomial of degree n, that is, p(x) = ∑n
i=0 aixi.
(a) Describe a simple O(n2)-time algorithm for computing p(x).
(b) Describe an O(nlogn)-time algorithm for computing p(x), based upon
a more efficient calculation of xi.
(c) Now consider a rewriting of p(x) as
p(x) = a0 +x(a1 +x(a2 +x(a3 +···+x(an−1 +xan)···))),
which is known as Horner’s method. Using the big-Oh notation, charac-
terize the number of arithmetic operations this method executes.
C-3.51 Show that the summation ∑n
i=1 logi is O(nlogn).
C-3.52 Show that the summation ∑n
i=1 logi is Ω(nlogn).
C-3.53 An evil king has n bottles of wine, and a spy has just poisoned one of
them. Unfortunately, they do not know which one it is. The poison is very
deadly; just one drop diluted even a billion to one will still kill. Even so,
it takes a full month for the poison to take effect. Design a scheme for
determining exactly which one of the wine bottles was poisoned in just
one month’s time while expending O(logn) taste testers.
C-3.54 A sequence S contains n integers taken from the interval [0,4n], with repe-
titions allowed. Describe an efficient algorithm for determining an integer
value k that occurs the most often in S. What is the running time of your
algorithm?
Projects
P-3.55 Perform an experimental analysis of the three algorithms prefix average1,
prefix average2, and prefix average3, from Section 3.3.3. Visualize their
running times as a function of the input size with a log-log chart.
P-3.56 Perform an experimental analysis that compares the relative running times
of the functions shown in Code Fragment 3.10.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-168-2048.jpg)
![Chapter Notes 147
P-3.57 Perform experimental analysis to test the hypothesis that Python’s sorted
method runs in O(nlogn) time on average.
P-3.58 For each of the three algorithms, unique1, unique2, and unique3, which
solve the element uniqueness problem, perform an experimental analysis
to determine the largest value of n such that the given algorithm runs in
one minute or less.
Chapter Notes
The big-Oh notation has prompted several comments about its proper use [19, 49, 63].
Knuth [64, 63] defines it using the notation f(n) = O(g(n)), but says this “equality” is only
“one way.” We have chosen to take a more standard view of equality and view the big-Oh
notation as a set, following Brassard [19]. The reader interested in studying average-case
analysis is referred to the book chapter by Vitter and Flajolet [101]. For some additional
mathematical tools, please refer to Appendix B.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-169-2048.jpg)
![4.1. Illustrative Examples 155
4.1.3 Binary Search
In this section, we describe a classic recursive algorithm, binary search, that is used
to efficiently locate a target value within a sorted sequence of n elements. This is
among the most important of computer algorithms, and it is the reason that we so
often store data in sorted order (as in Figure 4.4).
37
5
0 1 2 3 4 6 7 8 9 10 11 12 13 14 15
9
2 4 5 7 8 12 14 17 19 22 25 27 28 33
Figure 4.4: Values stored in sorted order within an indexable sequence, such as a
Python list. The numbers at top are the indices.
When the sequence is unsorted, the standard approach to search for a target
value is to use a loop to examine every element, until either finding the target or
exhausting the data set. This is known as the sequential search algorithm. This
algorithm runs in O(n) time (i.e., linear time) since every element is inspected in
the worst case.
When the sequence is sorted and indexable, there is a much more efficient
algorithm. (For intuition, think about how you would accomplish this task by
hand!) For any index j, we know that all the values stored at indices 0,..., j − 1
are less than or equal to the value at index j, and all the values stored at indices
j +1,...,n−1 are greater than or equal to that at index j. This observation allows
us to quickly “home in” on a search target using a variant of the children’s game
“high-low.” We call an element of the sequence a candidate if, at the current stage
of the search, we cannot rule out that this item matches the target. The algorithm
maintains two parameters, low and high, such that all the candidate entries have
index at least low and at most high. Initially, low = 0 and high = n− 1. We then
compare the target value to the median candidate, that is, the item data[mid] with
index
mid = (low +high)/2.
We consider three cases:
• If the target equals data[mid], then we have found the item we are looking
for, and the search terminates successfully.
• If target data[mid], then we recur on the first half of the sequence, that is,
on the interval of indices from low to mid−1.
• If target data[mid], then we recur on the second half of the sequence, that
is, on the interval of indices from mid+1 to high.
An unsuccessful search occurs if low high, as the interval [low,high] is empty.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-177-2048.jpg)
![156 Chapter 4. Recursion
This algorithm is known as binary search. We give a Python implementation
in Code Fragment 4.3, and an illustration of the execution of the algorithm in Fig-
ure 4.5. Whereas sequential search runs in O(n) time, the more efficient binary
search runs in O(logn) time. This is a significant improvement, given that if n
is one billion, logn is only 30. (We defer our formal analysis of binary search’s
running time to Proposition 4.2 in Section 4.2.)
1 def binary search(data, target, low, high):
2 ”””Return True if target is found in indicated portion of a Python list.
3
4 The search only considers the portion from data[low] to data[high] inclusive.
5 ”””
6 if low high:
7 return False # interval is empty; no match
8 else:
9 mid = (low + high) // 2
10 if target == data[mid]: # found a match
11 return True
12 elif target data[mid]:
13 # recur on the portion left of the middle
14 return binary search(data, target, low, mid − 1)
15 else:
16 # recur on the portion right of the middle
17 return binary search(data, target, mid + 1, high)
Code Fragment 4.3: An implementation of the binary search algorithm.
mid
high
high
low
low mid
low mid
low=mid=high
high
14 19 22 25 27 28 33 37
6 7 8 9 10 11 12 13 14 15
7
5
4
2 9
8
9
2 4 5 7 8 12 14 17
37
33
28
27
25
22
19
9
2 4 5 7 8 12 14 17 19 22 25 27 28 33 37
19 22 25 27 28 33 37
5
0 1 2 3 4
17
14
12
9
2 4 5 7 8 12 17
Figure 4.5: Example of a binary search for target value 22.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-178-2048.jpg)
![4.1. Illustrative Examples 159
Python’s os Module
To provide a Python implementation of a recursive algorithm for computing disk
usage, we rely on Python’s os module, which provides robust tools for interacting
with the operating system during the execution of a program. This is an extensive
library, but we will only need the following four functions:
• os.path.getsize(path)
Return the immediate disk usage (measured in bytes) for the file or directory
that is identified by the string path (e.g., /user/rt/courses).
• os.path.isdir(path)
Return True if entry designated by string path is a directory; False otherwise.
• os.listdir(path)
Return a list of strings that are the names of all entries within a directory
designated by string path. In our sample file system, if the parameter is
/user/rt/courses, this returns the list [ cs016 , cs252 ].
• os.path.join(path, filename)
Compose the path string and filename string using an appropriate operating
system separator between the two (e.g., the / character for a Unix/Linux
system, and the character for Windows). Return the string that represents
the full path to the file.
Python Implementation
With use of the os module, we now convert the algorithm from Code Fragment 4.4
into the Python implementation of Code Fragment 4.5.
1 import os
2
3 def disk usage(path):
4 ”””Return the number of bytes used by a file/folder and any descendents.”””
5 total = os.path.getsize(path) # account for direct usage
6 if os.path.isdir(path): # if this is a directory,
7 for filename in os.listdir(path): # then for each child:
8 childpath = os.path.join(path, filename) # compose full path to child
9 total += disk usage(childpath) # add child’s usage to total
10
11 print ( {0:7} .format(total), path) # descriptive output (optional)
12 return total # return the grand total
Code Fragment 4.5: A recursive function for reporting disk usage of a file system.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-181-2048.jpg)
![4.3. Recursion Run Amok 165
4.3 Recursion Run Amok
Although recursion is a very powerful tool, it can easily be misused in various ways.
In this section, we examine several problems in which a poorly implemented recur-
sion causes drastic inefficiency, and we discuss some strategies for recognizing and
avoid such pitfalls.
We begin by revisiting the element uniqueness problem, defined on page 135
of Section 3.3.3. We can use the following recursive formulation to determine if
all n elements of a sequence are unique. As a base case, when n = 1, the elements
are trivially unique. For n ≥ 2, the elements are unique if and only if the first n−1
elements are unique, the last n−1 items are unique, and the first and last elements
are different (as that is the only pair that was not already checked as a subcase). A
recursive implementation based on this idea is given in Code Fragment 4.6, named
unique3 (to differentiate it from unique1 and unique2 from Chapter 3).
1 def unique3(S, start, stop):
2 ”””Return True if there are no duplicate elements in slice S[start:stop].”””
3 if stop − start = 1: return True # at most one item
4 elif not unique(S, start, stop−1): return False # first part has duplicate
5 elif not unique(S, start+1, stop): return False # second part has duplicate
6 else: return S[start] != S[stop−1] # do first and last differ?
Code Fragment 4.6: Recursive unique3 for testing element uniqueness.
Unfortunately, this is a terribly inefficient use of recursion. The nonrecursive
part of each call uses O(1) time, so the overall running time will be proportional to
the total number of recursive invocations. To analyze the problem, we let n denote
the number of entries under consideration, that is, let n= stop − start.
If n = 1, then the running time of unique3 is O(1), since there are no recursive
calls for this case. In the general case, the important observation is that a single call
to unique3 for a problem of size n may result in two recursive calls on problems of
size n − 1. Those two calls with size n − 1 could in turn result in four calls (two
each) with a range of size n − 2, and thus eight calls with size n − 3 and so on.
Thus, in the worst case, the total number of function calls is given by the geometric
summation
1+2+4+···+2n−1
,
which is equal to 2n − 1 by Proposition 3.5. Thus, the running time of function
unique3 is O(2n). This is an incredibly inefficient function for solving the ele-
ment uniqueness problem. Its inefficiency comes not from the fact that it uses
recursion—it comes from the fact that it uses recursion poorly, which is something
we address in Exercise C-4.11.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-187-2048.jpg)
![170 Chapter 4. Recursion
A recursive algorithm for computing the sum of a sequence of numbers based
on this intuition is implemented in Code Fragment 4.9.
1 def linear sum(S, n):
2 ”””Return the sum of the first n numbers of sequence S.”””
3 if n == 0:
4 return 0
5 else:
6 return linear sum(S, n−1) + S[n−1]
Code Fragment 4.9: Summing the elements of a sequence using linear recursion.
A recursion trace of the linear sum function for a small example is given in
Figure 4.10. For an input of size n, the linear sum algorithm makes n+1 function
calls. Hence, it will take O(n) time, because it spends a constant amount of time
performing the nonrecursive part of each call. Moreover, we can also see that the
memory space used by the algorithm (in addition to the sequence S) is also O(n), as
we use a constant amount of memory space for each of the n+1 activation records
in the trace at the time we make the final recursive call (with n = 0).
return 15 + S[4] = 15 + 8 = 23
linear sum(S, 5)
linear sum(S, 4)
linear sum(S, 3)
linear sum(S, 2)
linear sum(S, 1)
linear sum(S, 0)
return 0
return 0 + S[0] = 0 + 4 = 4
return 4 + S[1] = 4 + 3 = 7
return 7 + S[2] = 7 + 6 = 13
return 13 + S[3] = 13 + 2 = 15
Figure 4.10: Recursion trace for an execution of linear sum(S, 5) with input pa-
rameter S = [4, 3, 6, 2, 8].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-192-2048.jpg)
![4.4. Further Examples of Recursion 171
Reversing a Sequence with Recursion
Next, let us consider the problem of reversing the n elements of a sequence, S, so
that the first element becomes the last, the second element becomes second to the
last, and so on. We can solve this problem using linear recursion, by observing that
the reversal of a sequence can be achieved by swapping the first and last elements
and then recursively reversing the remaining elements. We present an implemen-
tation of this algorithm in Code Fragment 4.10, using the convention that the first
time we call this algorithm we do so as reverse(S, 0, len(S)).
1 def reverse(S, start, stop):
2 ”””Reverse elements in implicit slice S[start:stop].”””
3 if start stop − 1: # if at least 2 elements:
4 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last
5 reverse(S, start+1, stop−1) # recur on rest
Code Fragment 4.10: Reversing the elements of a sequence using linear recursion.
Note that there are two implicit base case scenarios: When start == stop, the
implicit range is empty, and when start == stop−1, the implicit range has only
one element. In either of these cases, there is no need for action, as a sequence
with zero elements or one element is trivially equal to its reversal. When otherwise
invoking recursion, we are guaranteed to make progress towards a base case, as
the difference, stop−start, decreases by two with each call (see Figure 4.11). If n
is even, we will eventually reach the start == stop case, and if n is odd, we will
eventually reach the start == stop − 1 case.
The above argument implies that the recursive algorithm of Code Fragment 4.10
is guaranteed to terminate after a total of 1+
n
2
recursive calls. Since each call
involves a constant amount of work, the entire process runs in O(n) time.
6
3 6 2 8 9 5
5 9 8 2 6 3 4
5 3 6 2 8 9 4
5 9 8 2 6 3 4
5 9 6 2 8 3 4
5
0 1 2 3 4
4
Figure 4.11: A trace of the recursion for reversing a sequence. The shaded portion
has yet to be reversed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-193-2048.jpg)
![174 Chapter 4. Recursion
4.4.2 Binary Recursion
When a function makes two recursive calls, we say that it uses binary recursion.
We have already seen several examples of binary recursion, most notably when
drawing the English ruler (Section 4.1.2), or in the bad fibonacci function of Sec-
tion 4.3. As another application of binary recursion, let us revisit the problem of
summing the n elements of a sequence, S, of numbers. Computing the sum of one
or zero elements is trivial. With two or more elements, we can recursively com-
pute the sum of the first half, and the sum of the second half, and add these sums
together. Our implementation of such an algorithm, in Code Fragment 4.13, is
initially invoked as binary sum(A, 0, len(A)).
1 def binary sum(S, start, stop):
2 ”””Return the sum of the numbers in implicit slice S[start:stop].”””
3 if start = stop: # zero elements in slice
4 return 0
5 elif start == stop−1: # one element in slice
6 return S[start]
7 else: # two or more elements in slice
8 mid = (start + stop) // 2
9 return binary sum(S, start, mid) + binary sum(S, mid, stop)
Code Fragment 4.13: Summing the elements of a sequence using binary recursion.
To analyze algorithm binary sum, we consider, for simplicity, the case where
n is a power of two. Figure 4.13 shows the recursion trace of an execution of
binary sum(0, 8). We label each box with the values of parameters start:stop
for that call. The size of the range is divided in half at each recursive call, and
so the depth of the recursion is 1 + log2 n. Therefore, binary sum uses O(logn)
amount of additional space, which is a big improvement over the O(n) space used
by the linear sum function of Code Fragment 4.9. However, the running time of
binary sum is O(n), as there are 2n−1 function calls, each requiring constant time.
0:1 1:2 2:3 4:5 6:7 7:8
3:4 5:6
0:2 4:6 6:8
2:4
0:4 4:8
0:8
Figure 4.13: Recursion trace for the execution of binary sum(0, 8).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-196-2048.jpg)
![4.6. Eliminating Tail Recursion 179
1 def binary search iterative(data, target):
2 ”””Return True if target is found in the given Python list.”””
3 low = 0
4 high = len(data)−1
5 while low = high:
6 mid = (low + high) // 2
7 if target == data[mid]: # found a match
8 return True
9 elif target data[mid]:
10 high = mid − 1 # only consider values left of mid
11 else:
12 low = mid + 1 # only consider values right of mid
13 return False # loop ended without success
Code Fragment 4.15: A nonrecursive implementation of binary search.
the target, or we narrow the range of the candidate subsequence. Where we made
the recursive call binary search(data, target, low, mid −1) in the original version,
we simply replace high = mid − 1 in our new version and then continue to the
next iteration of the loop. Our original base case condition of low high has
simply been replaced by the opposite loop condition while low = high. In our
new implementation, we return False to designate a failed search if the while loop
ends (that is, without having ever returned True from within).
We can similarly develop a nonrecursive implementation (Code Fragment 4.16)
of the original recursive reverse method of Code Fragment 4.10.
1 def reverse iterative(S):
2 ”””Reverse elements in sequence S.”””
3 start, stop = 0, len(S)
4 while start stop − 1:
5 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last
6 start, stop = start + 1, stop − 1 # narrow the range
Code Fragment 4.16: Reversing the elements of a sequence using iteration.
In this new version, we update the values start and stop during each pass of the
loop, exiting once we reach the case of having one or less elements in that range.
Many other linear recursions can be expressed quite efficiently with iteration,
even if they were not formally tail recursions. For example, there are trivial non-
recursive implementations for computing factorials, summing elements of a se-
quence, or computing Fibonacci numbers efficiently. In fact, our implementation
of a Fibonacci generator, from Section 1.8, produces each subsequent value in O(1)
time, and thus takes O(n) time to generate the nth entry in the series.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-201-2048.jpg)
![180 Chapter 4. Recursion
4.7 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-4.1 Describe a recursive algorithm for finding the maximum element in a se-
quence, S, of n elements. What is your running time and space usage?
R-4.2 Draw the recursion trace for the computation of power(2,5), using the
traditional function implemented in Code Fragment 4.11.
R-4.3 Draw the recursion trace for the computation of power(2,18), using the
repeated squaring algorithm, as implemented in Code Fragment 4.12.
R-4.4 Draw the recursion trace for the execution of function reverse(S, 0, 5)
(Code Fragment 4.10) on S = [4, 3, 6, 2, 6].
R-4.5 Draw the recursion trace for the execution of function PuzzleSolve(3,S,U)
(Code Fragment 4.14), where S is empty and U = {a,b,c,d}.
R-4.6 Describe a recursive function for computing the nth Harmonic number,
Hn = ∑n
i=1 1/i.
R-4.7 Describe a recursive function for converting a string of digits into the in-
teger it represents. For example, 13531 represents the integer 13,531.
R-4.8 Isabel has an interesting way of summing up the values in a sequence A of
n integers, where n is a power of two. She creates a new sequence B of half
the size of A and sets B[i] = A[2i]+A[2i+1], for i = 0,1,...,(n/2)−1. If
B has size 1, then she outputs B[0]. Otherwise, she replaces A with B, and
repeats the process. What is the running time of her algorithm?
Creativity
C-4.9 Write a short recursive Python function that finds the minimum and max-
imum values in a sequence without using any loops.
C-4.10 Describe a recursive algorithm to compute the integer part of the base-two
logarithm of n using only addition and integer division.
C-4.11 Describe an efficient recursive function for solving the element unique-
ness problem, which runs in time that is at most O(n2) in the worst case
without using sorting.
C-4.12 Give a recursive algorithm to compute the product of two positive integers,
m and n, using only addition and subtraction.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-202-2048.jpg)
![182 Chapter 4. Recursion
C-4.21 Suppose you are given an n-element sequence, S, containing distinct in-
tegers that are listed in increasing order. Given a number k, describe a
recursive algorithm to find two integers in S that sum to k, if such a pair
exists. What is the running time of your algorithm?
C-4.22 Develop a nonrecursive implementation of the version of power from
Code Fragment 4.12 that uses repeated squaring.
Projects
P-4.23 Implement a recursive function with signature find(path, filename) that
reports all entries of the file system rooted at the given path having the
given file name.
P-4.24 Write a program for solving summation puzzles by enumerating and test-
ing all possible configurations. Using your program, solve the three puz-
zles given in Section 4.4.3.
P-4.25 Provide a nonrecursive implementation of the draw interval function for
the English ruler project of Section 4.1.2. There should be precisely 2c −1
lines of output if c represents the length of the center tick. If incrementing
a counter from 0 to 2c −2, the number of dashes for each tick line should
be exactly one more than the number of consecutive 1’s at the end of the
binary representation of the counter.
P-4.26 Write a program that can solve instances of the Tower of Hanoi problem
(from Exercise C-4.14).
P-4.27 Python’s os module provides a function with signature walk(path) that
is a generator yielding the tuple (dirpath, dirnames, filenames) for each
subdirectory of the directory identified by string path, such that string
dirpath is the full path to the subdirectory, dirnames is a list of the names
of the subdirectories within dirpath, and filenames is a list of the names
of non-directory entries of dirpath. For example, when visiting the cs016
subdirectory of the file system shown in Figure 4.6, the walk would yield
( /user/rt/courses/cs016 , [ homeworks , programs ], [ grades ]).
Give your own implementation of such a walk function.
Chapter Notes
The use of recursion in programs belongs to the folkore of computer science (for example,
see the article of Dijkstra [36]). It is also at the heart of functional programming languages
(for example, see the classic book by Abelson, Sussman, and Sussman [1]). Interestingly,
binary search was first published in 1946, but was not published in a fully correct form
until 1962. For further discussions on lessons learned, please see papers by Bentley [14]
and Lesuisse [68].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-204-2048.jpg)
![184 Chapter 5. Array-Based Sequences
5.1 Python’s Sequence Types
In this chapter, we explore Python’s various “sequence” classes, namely the built-
in list, tuple, and str classes. There is significant commonality between these
classes, most notably: each supports indexing to access an individual element of a
sequence, using a syntax such as seq[k], and each uses a low-level concept known
as an array to represent the sequence. However, there are significant differences in
the abstractions that these classes represent, and in the way that instances of these
classes are represented internally by Python. Because these classes are used so
widely in Python programs, and because they will become building blocks upon
which we will develop more complex data structures, it is imperative that we estab-
lish a clear understanding of both the public behavior and inner workings of these
classes.
Public Behaviors
A proper understanding of the outward semantics for a class is a necessity for a
good programmer. While the basic usage of lists, strings, and tuples may seem
straightforward, there are several important subtleties regarding the behaviors as-
sociated with these classes (such as what it means to make a copy of a sequence, or
to take a slice of a sequence). Having a misunderstanding of a behavior can easily
lead to inadvertent bugs in a program. Therefore, we establish an accurate men-
tal model for each of these classes. These images will help when exploring more
advanced usage, such as representing a multidimensional data set as a list of lists.
Implementation Details
A focus on the internal implementations of these classes seems to go against our
stated principles of object-oriented programming. In Section 2.1.2, we emphasized
the principle of encapsulation, noting that the user of a class need not know about
the internal details of the implementation. While it is true that one only needs to
understand the syntax and semantics of a class’s public interface in order to be able
to write legal and correct code that uses instances of the class, the efficiency of a
program depends greatly on the efficiency of the components upon which it relies.
Asymptotic and Experimental Analyses
In describing the efficiency of various operations for Python’s sequence classes,
we will rely on the formal asymptotic analysis notations established in Chapter 3.
We will also perform experimental analyses of the primary operations to provide
empirical evidence that is consistent with the more theoretical asymptotic analyses.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-206-2048.jpg)
![5.2. Low-Level Arrays 187
5.2.1 Referential Arrays
As another motivating example, assume that we want a medical information system
to keep track of the patients currently assigned to beds in a certain hospital. If we
assume that the hospital has 200 beds, and conveniently that those beds are num-
bered from 0 to 199, we might consider using an array-based structure to maintain
the names of the patients currently assigned to those beds. For example, in Python
we might use a list of names, such as:
[ Rene , Joseph , Janet , Jonas , Helen , Virginia , ... ]
To represent such a list with an array, Python must adhere to the requirement that
each cell of the array use the same number of bytes. Yet the elements are strings,
and strings naturally have different lengths. Python could attempt to reserve enough
space for each cell to hold the maximum length string (not just of currently stored
strings, but of any string we might ever want to store), but that would be wasteful.
Instead, Python represents a list or tuple instance using an internal storage
mechanism of an array of object references. At the lowest level, what is stored
is a consecutive sequence of memory addresses at which the elements of the se-
quence reside. A high-level diagram of such a list is shown in Figure 5.4.
0 3
1 2 5
4
Rene Virginia
Joseph Helen
Jonas
Janet
Figure 5.4: An array storing references to strings.
Although the relative size of the individual elements may vary, the number of
bits used to store the memory address of each element is fixed (e.g., 64-bits per
address). In this way, Python can support constant-time access to a list or tuple
element based on its index.
In Figure 5.4, we characterize a list of strings that are the names of the patients
in a hospital. It is more likely that a medical information system would manage
more comprehensive information on each patient, perhaps represented as an in-
stance of a Patient class. From the perspective of the list implementation, the same
principle applies: The list will simply keep a sequence of references to those ob-
jects. Note as well that a reference to the None object can be used as an element
of the list to represent an empty bed in the hospital.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-209-2048.jpg)
![188 Chapter 5. Array-Based Sequences
The fact that lists and tuples are referential structures is significant to the se-
mantics of these classes. A single list instance may include multiple references
to the same object as elements of the list, and it is possible for a single object to
be an element of two or more lists, as those lists simply store references back to
that object. As an example, when you compute a slice of a list, the result is a new
list instance, but that new list has references to the same elements that are in the
original list, as portrayed in Figure 5.5.
3 4 5 6 7
0 1 2
0 1 2
primes:
temp:
3 11
5
2 19
17
13
7
Figure 5.5: The result of the command temp = primes[3:6].
When the elements of the list are immutable objects, as with the integer in-
stances in Figure 5.5, the fact that the two lists share elements is not that signifi-
cant, as neither of the lists can cause a change to the shared object. If, for example,
the command temp[2] = 15 were executed from this configuration, that does not
change the existing integer object; it changes the reference in cell 2 of the temp list
to reference a different object. The resulting configuration is shown in Figure 5.6.
3 4 5 6 7
0 1 2
0 1 2
primes:
temp:
13
11
5
2
15
3 19
17
7
Figure 5.6: The result of the command temp[2] = 15 upon the configuration por-
trayed in Figure 5.5.
The same semantics is demonstrated when making a new list as a copy of an
existing one, with a syntax such as backup = list(primes). This produces a new
list that is a shallow copy (see Section 2.6), in that it references the same elements
as in the first list. With immutable elements, this point is moot. If the contents of
the list were of a mutable type, a deep copy, meaning a new list with new elements,
can be produced by using the deepcopy function from the copy module.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-210-2048.jpg)
![5.2. Low-Level Arrays 189
As a more striking example, it is a common practice in Python to initialize an
array of integers using a syntax such as counters = [0] 8. This syntax produces
a list of length eight, with all eight elements being the value zero. Technically, all
eight cells of the list reference the same object, as portrayed in Figure 5.7.
4 5 6 7
0 1 2 3
counters:
0
Figure 5.7: The result of the command data = [0] 8.
At first glance, the extreme level of aliasing in this configuration may seem
alarming. However, we rely on the fact that the referenced integer is immutable.
Even a command such as counters[2] += 1 does not technically change the value
of the existing integer instance. This computes a new integer, with value 0+1, and
sets cell 2 to reference the newly computed value. The resulting configuration is
shown in Figure 5.8.
4 5 6 7
0 1 2 3
counters:
0
1
Figure 5.8: The result of command data[2] += 1 upon the list from Figure 5.7.
As a final manifestation of the referential nature of lists, we note that the extend
command is used to add all elements from one list to the end of another list. The
extended list does not receive copies of those elements, it receives references to
those elements. Figure 5.9 portrays the effect of a call to extend.
3 4 5 6 7 8 10
9
2
1
0
0 1 2
primes:
extras:
29 31
7 19
3 17
13
11
2 5 23
Figure 5.9: The effect of command primes.extend(extras), shown in light gray.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-211-2048.jpg)
![5.2. Low-Level Arrays 191
Primary support for compact arrays is in a module named array. That module
defines a class, also named array, providing compact storage for arrays of primitive
data types. A portrayal of such an array of integers is shown in Figure 5.10.
3 4 5 6 7
0 1 2
17
5
2 3 7 11 13 19
Figure 5.10: Integers stored compactly as elements of a Python array.
The public interface for the array class conforms mostly to that of a Python list.
However, the constructor for the array class requires a type code as a first parameter,
which is a character that designates the type of data that will be stored in the array.
As a tangible example, the type code, i , designates an array of (signed) integers,
typically represented using at least 16-bits each. We can declare the array shown in
Figure 5.10 as,
primes = array( i , [2, 3, 5, 7, 11, 13, 17, 19])
The type code allows the interpreter to determine precisely how many bits are
needed per element of the array. The type codes supported by the array module,
as shown in Table 5.1, are formally based upon the native data types used by the
C programming language (the language in which the the most widely used distri-
bution of Python is implemented). The precise number of bits for the C data types
is system-dependent, but typical ranges are shown in the table.
Code C Data Type Typical Number of Bytes
b signed char 1
B unsigned char 1
u Unicode char 2 or 4
h signed short int 2
H unsigned short int 2
i signed int 2 or 4
I unsigned int 2 or 4
l signed long int 4
L unsigned long int 4
f float 4
d float 8
Table 5.1: Type codes supported by the array module.
The array module does not provide support for making compact arrays of user-
defined data types. Compact arrays of such structures can be created with the lower-
level support of a module named ctypes. (See Section 5.3.1 for more discussion of
the ctypes module.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-213-2048.jpg)
![5.3. Dynamic Arrays and Amortization 193
1 import sys # provides getsizeof function
2 data = [ ]
3 for k in range(n): # NOTE: must fix choice of n
4 a = len(data) # number of elements
5 b = sys.getsizeof(data) # actual size in bytes
6 print( Length: {0:3d}; Size in bytes: {1:4d} .format(a, b))
7 data.append(None) # increase length by one
Code Fragment 5.1: An experiment to explore the relationship between a list’s
length and its underlying size in Python.
Length: 0; Size in bytes: 72
Length: 1; Size in bytes: 104
Length: 2; Size in bytes: 104
Length: 3; Size in bytes: 104
Length: 4; Size in bytes: 104
Length: 5; Size in bytes: 136
Length: 6; Size in bytes: 136
Length: 7; Size in bytes: 136
Length: 8; Size in bytes: 136
Length: 9; Size in bytes: 200
Length: 10; Size in bytes: 200
Length: 11; Size in bytes: 200
Length: 12; Size in bytes: 200
Length: 13; Size in bytes: 200
Length: 14; Size in bytes: 200
Length: 15; Size in bytes: 200
Length: 16; Size in bytes: 200
Length: 17; Size in bytes: 272
Length: 18; Size in bytes: 272
Length: 19; Size in bytes: 272
Length: 20; Size in bytes: 272
Length: 21; Size in bytes: 272
Length: 22; Size in bytes: 272
Length: 23; Size in bytes: 272
Length: 24; Size in bytes: 272
Length: 25; Size in bytes: 272
Length: 26; Size in bytes: 352
Code Fragment 5.2: Sample output from the experiment of Code Fragment 5.1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-215-2048.jpg)
![5.3. Dynamic Arrays and Amortization 195
5.3.1 Implementing a Dynamic Array
Although the Python list class provides a highly optimized implementation of dy-
namic arrays, upon which we rely for the remainder of this book, it is instructive to
see how such a class might be implemented.
The key is to provide means to grow the array A that stores the elements of a
list. Of course, we cannot actually grow that array, as its capacity is fixed. If an
element is appended to a list at a time when the underlying array is full, we perform
the following steps:
1. Allocate a new array B with larger capacity.
2. Set B[i] = A[i], for i = 0,...,n−1, where n denotes current number of items.
3. Set A = B, that is, we henceforth use B as the array supporting the list.
4. Insert the new element in the new array.
An illustration of this process is shown in Figure 5.12.
B
A A
B A
(a) (b) (c)
Figure 5.12: An illustration of the three steps for “growing” a dynamic array: (a)
create new array B; (b) store elements of A in B; (c) reassign reference A to the new
array. Not shown is the future garbage collection of the old array, or the insertion
of the new element.
The remaining issue to consider is how large of a new array to create. A com-
monly used rule is for the new array to have twice the capacity of the existing array
that has been filled. In Section 5.3.2, we will provide a mathematical analysis to
justify such a choice.
In Code Fragment 5.3, we offer a concrete implementation of dynamic arrays
in Python. Our DynamicArray class is designed using ideas described in this sec-
tion. While consistent with the interface of a Python list class, we provide only
limited functionality in the form of an append method, and accessors len and
getitem . Support for creating low-level arrays is provided by a module named
ctypes. Because we will not typically use such a low-level structure in the remain-
der of this book, we omit a detailed explanation of the ctypes module. Instead,
we wrap the necessary command for declaring the raw array within a private util-
ity method make array. The hallmark expansion procedure is performed in our
nonpublic resize method.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-217-2048.jpg)
![196 Chapter 5. Array-Based Sequences
1 import ctypes # provides low-level arrays
2
3 class DynamicArray:
4 ”””A dynamic array class akin to a simplified Python list.”””
5
6 def init (self):
7 ”””Create an empty array.”””
8 self. n = 0 # count actual elements
9 self. capacity = 1 # default array capacity
10 self. A = self. make array(self. capacity) # low-level array
11
12 def len (self):
13 ”””Return number of elements stored in the array.”””
14 return self. n
15
16 def getitem (self, k):
17 ”””Return element at index k.”””
18 if not 0 = k self. n:
19 raise IndexError( invalid index )
20 return self. A[k] # retrieve from array
21
22 def append(self, obj):
23 ”””Add object to end of the array.”””
24 if self. n == self. capacity: # not enough room
25 self. resize(2 self. capacity) # so double capacity
26 self. A[self. n] = obj
27 self. n += 1
28
29 def resize(self, c): # nonpublic utitity
30 ”””Resize internal array to capacity c.”””
31 B = self. make array(c) # new (bigger) array
32 for k in range(self. n): # for each existing value
33 B[k] = self. A[k]
34 self. A = B # use the bigger array
35 self. capacity = c
36
37 def make array(self, c): # nonpublic utitity
38 ”””Return new array with capacity c.”””
39 return (c ctypes.py object)( ) # see ctypes documentation
Code Fragment 5.3: An implementation of a DynamicArray class, using a raw array
from the ctypes module as storage.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-218-2048.jpg)
![5.3. Dynamic Arrays and Amortization 201
5.3.3 Python’s List Class
The experiments of Code Fragment 5.1 and 5.2, at the beginning of Section 5.3,
provide empirical evidence that Python’s list class is using a form of dynamic arrays
for its storage. Yet, a careful examination of the intermediate array capacities (see
Exercises R-5.2 and C-5.13) suggests that Python is not using a pure geometric
progression, nor is it using an arithmetic progression.
With that said, it is clear that Python’s implementation of the append method
exhibits amortized constant-time behavior. We can demonstrate this fact experi-
mentally. A single append operation typically executes so quickly that it would be
difficult for us to accurately measure the time elapsed at that granularity, although
we should notice some of the more expensive operations in which a resize is per-
formed. We can get a more accurate measure of the amortized cost per operation
by performing a series of n append operations on an initially empty list and deter-
mining the average cost of each. A function to perform that experiment is given in
Code Fragment 5.4.
1 from time import time # import time function from time module
2 def compute average(n):
3 ”””Perform n appends to an empty list and return average time elapsed.”””
4 data = [ ]
5 start = time( ) # record the start time (in seconds)
6 for k in range(n):
7 data.append(None)
8 end = time( ) # record the end time (in seconds)
9 return (end − start) / n # compute average per operation
Code Fragment 5.4: Measuring the amortized cost of append for Python’s list class.
Technically, the time elapsed between the start and end includes the time to
manage the iteration of the for loop, in addition to the append calls. The empirical
results of the experiment, for increasingly large values of n, are shown in Table 5.2.
We see higher average cost for the smaller data sets, perhaps in part due to the over-
head of the loop range. There is also natural variance in measuring the amortized
cost in this way, because of the impact of the final resize event relative to n. Taken
as a whole, there seems clear evidence that the amortized time for each append is
independent of n.
n 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000
μs 0.219 0.158 0.164 0.151 0.147 0.147 0.149
Table 5.2: Average running time of append, measured in microseconds, as observed
over a sequence of n calls, starting with an empty list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-223-2048.jpg)
![202 Chapter 5. Array-Based Sequences
5.4 Efficiency of Python’s Sequence Types
In the previous section, we began to explore the underpinnings of Python’s list
class, in terms of implementation strategies and efficiency. We continue in this
section by examining the performance of all of Python’s sequence types.
5.4.1 Python’s List and Tuple Classes
The nonmutating behaviors of the list class are precisely those that are supported
by the tuple class. We note that tuples are typically more memory efficient than
lists because they are immutable; therefore, there is no need for an underlying
dynamic array with surplus capacity. We summarize the asymptotic efficiency of
the nonmutating behaviors of the list and tuple classes in Table 5.3. An explanation
of this analysis follows.
Operation Running Time
len(data) O(1)
data[j] O(1)
data.count(value) O(n)
data.index(value) O(k+1)
value in data O(k+1)
data1 == data2
O(k+1)
(similarly !=, , =, , =)
data[j:k] O(k− j +1)
data1 + data2 O(n1 +n2)
c data O(cn)
Table 5.3: Asymptotic performance of the nonmutating behaviors of the list and
tuple classes. Identifiers data, data1, and data2 designate instances of the list or
tuple class, and n, n1, and n2 their respective lengths. For the containment check
and index method, k represents the index of the leftmost occurrence (with k = n if
there is no occurrence). For comparisons between two sequences, we let k denote
the leftmost index at which they disagree or else k = min(n1,n2).
Constant-Time Operations
The length of an instance is returned in constant time because an instance explicitly
maintains such state information. The constant-time efficiency of syntax data[j] is
assured by the underlying access into an array.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-224-2048.jpg)
![5.4. Efficiency of Python’s Sequence Types 203
Searching for Occurrences of a Value
Each of the count, index, and contains methods proceed through iteration
of the sequence from left to right. In fact, Code Fragment 2.14 of Section 2.4.3
demonstrates how those behaviors might be implemented. Notably, the loop for
computing the count must proceed through the entire sequence, while the loops
for checking containment of an element or determining the index of an element
immediately exit once they find the leftmost occurrence of the desired value, if
one exists. So while count always examines the n elements of the sequence,
index and contains examine n elements in the worst case, but may be faster.
Empirical evidence can be found by setting data = list(range(10000000)) and
then comparing the relative efficiency of the test, 5 in data, relative to the test,
9999995 in data, or even the failed test, −5 in data.
Lexicographic Comparisons
Comparisons between two sequences are defined lexicographically. In the worst
case, evaluating such a condition requires an iteration taking time proportional
to the length of the shorter of the two sequences (because when one sequence
ends, the lexicographic result can be determined). However, in some cases the
result of the test can be evaluated more efficiently. For example, if evaluating
[7, 3, ...] [7, 5, ...], it is clear that the result is True without examining the re-
mainders of those lists, because the second element of the left operand is strictly
less than the second element of the right operand.
Creating New Instances
The final three behaviors in Table 5.3 are those that construct a new instance based
on one or more existing instances. In all cases, the running time depends on the
construction and initialization of the new result, and therefore the asymptotic be-
havior is proportional to the length of the result. Therefore, we find that slice
data[6000000:6000008] can be constructed almost immediately because it has only
eight elements, while slice data[6000000:7000000] has one million elements, and
thus is more time-consuming to create.
Mutating Behaviors
The efficiency of the mutating behaviors of the list class are described in Table 5.3.
The simplest of those behaviors has syntax data[j] = val, and is supported by the
special setitem method. This operation has worst-case O(1) running time be-
cause it simply replaces one element of a list with a new value. No other elements
are affected and the size of the underlying array does not change. The more inter-
esting behaviors to analyze are those that add or remove elements from the list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-225-2048.jpg)
![204 Chapter 5. Array-Based Sequences
Operation Running Time
data[j] = val O(1)
data.append(value) O(1)∗
data.insert(k, value) O(n−k+1)∗
data.pop() O(1)∗
data.pop(k)
O(n−k)∗
del data[k]
data.remove(value) O(n)∗
data1.extend(data2)
O(n2)∗
data1 += data2
data.reverse() O(n)
data.sort() O(nlogn)
∗amortized
Table 5.4: Asymptotic performance of the mutating behaviors of the list class. Iden-
tifiers data, data1, and data2 designate instances of the list class, and n, n1, and n2
their respective lengths.
Adding Elements to a List
In Section 5.3 we fully explored the append method. In the worst case, it requires
Ω(n) time because the underlying array is resized, but it uses O(1) time in the amor-
tized sense. Lists also support a method, with signature insert(k, value), that inserts
a given value into the list at index 0 ≤ k ≤ n while shifting all subsequent elements
back one slot to make room. For the purpose of illustration, Code Fragment 5.5 pro-
vides an implementation of that method, in the context of our DynamicArray class
introduced in Code Fragment 5.3. There are two complicating factors in analyzing
the efficiency of such an operation. First, we note that the addition of one element
may require a resizing of the dynamic array. That portion of the work requires Ω(n)
worst-case time but only O(1) amortized time, as per append. The other expense
for insert is the shifting of elements to make room for the new item. The time for
1 def insert(self, k, value):
2 ”””Insert value at index k, shifting subsequent values rightward.”””
3 # (for simplicity, we assume 0 = k = n in this verion)
4 if self. n == self. capacity: # not enough room
5 self. resize(2 self. capacity) # so double capacity
6 for j in range(self. n, k, −1): # shift rightmost first
7 self. A[j] = self. A[j−1]
8 self. A[k] = value # store newest element
9 self. n += 1
Code Fragment 5.5: Implementation of insert for our DynamicArray class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-226-2048.jpg)
![206 Chapter 5. Array-Based Sequences
Removing Elements from a List
Python’s list class offers several ways to remove an element from a list. A call to
pop() removes the last element from a list. This is most efficient, because all other
elements remain in their original location. This is effectively an O(1) operation,
but the bound is amortized because Python will occasionally shrink the underlying
dynamic array to conserve memory.
The parameterized version, pop(k), removes the element that is at index k n
of a list, shifting all subsequent elements leftward to fill the gap that results from
the removal. The efficiency of this operation is O(n−k), as the amount of shifting
depends upon the choice of index k, as illustrated in Figure 5.17. Note well that this
implies that pop(0) is the most expensive call, using Ω(n) time. (see experiments
in Exercise R-5.8.)
k
2
1
0 n−1
Figure 5.17: Removing an element at index k of a dynamic array.
The list class offers another method, named remove, that allows the caller to
specify the value that should be removed (not the index at which it resides). For-
mally, it removes only the first occurrence of such a value from a list, or raises a
ValueError if no such value is found. An implementation of such behavior is given
in Code Fragment 5.6, again using our DynamicArray class for illustration.
Interestingly, there is no “efficient” case for remove; every call requires Ω(n)
time. One part of the process searches from the beginning until finding the value at
index k, while the rest iterates from k to the end in order to shift elements leftward.
This linear behavior can be observed experimentally (see Exercise C-5.24).
1 def remove(self, value):
2 ”””Remove first occurrence of value (or raise ValueError).”””
3 # note: we do not consider shrinking the dynamic array in this version
4 for k in range(self. n):
5 if self. A[k] == value: # found a match!
6 for j in range(k, self. n − 1): # shift others to fill gap
7 self. A[j] = self. A[j+1]
8 self. A[self. n − 1] = None # help garbage collection
9 self. n −= 1 # we have one less item
10 return # exit immediately
11 raise ValueError( value not found ) # only reached if no match
Code Fragment 5.6: Implementation of remove for our DynamicArray class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-228-2048.jpg)
![5.4. Efficiency of Python’s Sequence Types 207
Extending a List
Python provides a method named extend that is used to add all elements of one list
to the end of a second list. In effect, a call to data.extend(other) produces the same
outcome as the code,
for element in other:
data.append(element)
In either case, the running time is proportional to the length of the other list, and
amortized because the underlying array for the first list may be resized to accom-
modate the additional elements.
In practice, the extend method is preferable to repeated calls to append because
the constant factors hidden in the asymptotic analysis are significantly smaller. The
greater efficiency of extend is threefold. First, there is always some advantage to
using an appropriate Python method, because those methods are often implemented
natively in a compiled language (rather than as interpreted Python code). Second,
there is less overhead to a single function call that accomplishes all the work, versus
many individual function calls. Finally, increased efficiency of extend comes from
the fact that the resulting size of the updated list can be calculated in advance. If the
second data set is quite large, there is some risk that the underlying dynamic array
might be resized multiple times when using repeated calls to append. With a single
call to extend, at most one resize operation will be performed. Exercise C-5.22
explores the relative efficiency of these two approaches experimentally.
Constructing New Lists
There are several syntaxes for constructing new lists. In almost all cases, the asymp-
totic efficiency of the behavior is linear in the length of the list that is created. How-
ever, as with the case in the preceding discussion of extend, there are significant
differences in the practical efficiency.
Section 1.9.2 introduces the topic of list comprehension, using an example
such as squares = [ k k for k in range(1, n+1) ] as a shorthand for
squares = [ ]
for k in range(1, n+1):
squares.append(k k)
Experiments should show that the list comprehension syntax is significantly faster
than building the list by repeatedly appending (see Exercise C-5.23).
Similarly, it is a common Python idiom to initialize a list of constant values
using the multiplication operator, as in [0] n to produce a list of length n with
all values equal to zero. Not only is this succinct for the programmer; it is more
efficient than building such a list incrementally.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-229-2048.jpg)
![5.4. Efficiency of Python’s Sequence Types 209
While the preceding code fragment accomplishes the goal, it may be terribly
inefficient. Because strings are immutable, the command, letters += c, would
presumably compute the concatenation, letters + c, as a new string instance and
then reassign the identifier, letters, to that result. Constructing that new string
would require time proportional to its length. If the final result has n characters, the
series of concatenations would take time proportional to the familiar sum 1+ 2+
3+···+n, and therefore O(n2) time.
Inefficient code of this type is widespread in Python, perhaps because of the
somewhat natural appearance of the code, and mistaken presumptions about how
the += operator is evaluated with strings. Some later implementations of the
Python interpreter have developed an optimization to allow such code to complete
in linear time, but this is not guaranteed for all Python implementations. The op-
timization is as follows. The reason that a command, letters += c, causes a new
string instance to be created is that the original string must be left unchanged if
another variable in a program refers to that string. On the other hand, if Python
knew that there were no other references to the string in question, it could imple-
ment += more efficiently by directly mutating the string (as a dynamic array). As
it happens, the Python interpreter already maintains what are known as reference
counts for each object; this count is used in part to determine if an object can be
garbage collected. (See Section 15.1.2.) But in this context, it provides a means to
detect when no other references exist to a string, thereby allowing the optimization.
A more standard Python idiom to guarantee linear time composition of a string
is to use a temporary list to store individual pieces, and then to rely on the join
method of the str class to compose the final result. Using this technique with our
previous example would appear as follows:
temp = [ ] # start with empty list
for c in document:
if c.isalpha():
temp.append(c) # append alphabetic character
letters = .join(temp) # compose overall result
This approach is guaranteed to run in O(n) time. First, we note that the series of
up to n append calls will require a total of O(n) time, as per the definition of the
amortized cost of that operation. The final call to join also guarantees that it takes
time that is linear in the final length of the composed string.
As we discussed at the end of the previous section, we can further improve
the practical execution time by using a list comprehension syntax to build up the
temporary list, rather than by repeated calls to append. That solution appears as,
letters = .join([c for c in document if c.isalpha()])
Better yet, we can entirely avoid the temporary list with a generator comprehension:
letters = .join(c for c in document if c.isalpha())](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-231-2048.jpg)
![5.5. Using Array-Based Sequences 211
A Class for High Scores
To maintain a sequence of high scores, we develop a class named Scoreboard. A
scoreboard is limited to a certain number of high scores that can be saved; once that
limit is reached, a new score only qualifies for the scoreboard if it is strictly higher
than the lowest “high score” on the board. The length of the desired scoreboard may
depend on the game, perhaps 10, 50, or 500. Since that limit may vary depending on
the game, we allow it to be specified as a parameter to our Scoreboard constructor.
Internally, we will use a Python list named board in order to manage the
GameEntry instances that represent the high scores. Since we expect the score-
board to eventually reach full capacity, we initialize the list to be large enough to
hold the maximum number of scores, but we initially set all entries to None. By
allocating the list with maximum capacity initially, it never needs to be resized. As
entries are added, we will maintain them from highest to lowest score, starting at
index 0 of the list. We illustrate a typical state of the data structure in Figure 5.18.
3
2
0 1 9
7 8
6
5
4
660
Mike 1105 Paul 720 Rose 590
Rob 750 Anna Jack 510
Figure 5.18: An illustration of an ordered list of length ten, storing references to six
GameEntry objects in the cells from index 0 to 5, with the rest being None.
A complete Python implementation of the Scoreboard class is given in Code
Fragment 5.8. The constructor is rather simple. The command
self. board = [None] capacity
creates a list with the desired length, yet all entries equal to None. We maintain
an additional instance variable, n, that represents the number of actual entries
currently in our table. For convenience, our class supports the getitem method
to retrieve an entry at a given index with a syntax board[i] (or None if no such entry
exists), and we support a simple str method that returns a string representation
of the entire scoreboard, with one entry per line.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-233-2048.jpg)
![212 Chapter 5. Array-Based Sequences
1 class Scoreboard:
2 ”””Fixed-length sequence of high scores in nondecreasing order.”””
3
4 def init (self, capacity=10):
5 ”””Initialize scoreboard with given maximum capacity.
6
7 All entries are initially None.
8 ”””
9 self. board = [None] capacity # reserve space for future scores
10 self. n = 0 # number of actual entries
11
12 def getitem (self, k):
13 ”””Return entry at index k.”””
14 return self. board[k]
15
16 def str (self):
17 ”””Return string representation of the high score list.”””
18 return n .join(str(self. board[j]) for j in range(self. n))
19
20 def add(self, entry):
21 ”””Consider adding entry to high scores.”””
22 score = entry.get score()
23
24 # Does new entry qualify as a high score?
25 # answer is yes if board not full or score is higher than last entry
26 good = self. n len(self. board) or score self. board[−1].get score()
27
28 if good:
29 if self. n len(self. board): # no score drops from list
30 self. n += 1 # so overall number increases
31
32 # shift lower scores rightward to make room for new entry
33 j = self. n − 1
34 while j 0 and self. board[j−1].get score( ) score:
35 self. board[j] = self. board[j−1] # shift entry from j-1 to j
36 j −= 1 # and decrement j
37 self. board[j] = entry # when done, add new entry
Code Fragment 5.8: Python code for a Scoreboard class that maintains an ordered
series of scores as GameEntry objects.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-234-2048.jpg)
![214 Chapter 5. Array-Based Sequences
5.5.2 Sorting a Sequence
In the previous subsection, we considered an application for which we added an ob-
ject to a sequence at a given position while shifting other elements so as to keep the
previous order intact. In this section, we use a similar technique to solve the sorting
problem, that is, starting with an unordered sequence of elements and rearranging
them into nondecreasing order.
The Insertion-Sort Algorithm
We study several sorting algorithms in this book, most of which are described in
Chapter 12. As a warm-up, in this section we describe a nice, simple sorting al-
gorithm known as insertion-sort. The algorithm proceeds as follows for an array-
based sequence. We start with the first element in the array. One element by itself
is already sorted. Then we consider the next element in the array. If it is smaller
than the first, we swap them. Next we consider the third element in the array. We
swap it leftward until it is in its proper order with the first two elements. We then
consider the fourth element, and swap it leftward until it is in the proper order with
the first three. We continue in this manner with the fifth element, the sixth, and so
on, until the whole array is sorted. We can express the insertion-sort algorithm in
pseudo-code, as shown in Code Fragment 5.9.
Algorithm InsertionSort(A):
Input: An array A of n comparable elements
Output: The array A with elements rearranged in nondecreasing order
for k from 1 to n − 1 do
Insert A[k] at its proper location within A[0], A[1], ..., A[k].
Code Fragment 5.9: High-level description of the insertion-sort algorithm.
This is a simple, high-level description of insertion-sort. If we look back to
Code Fragment 5.8 of Section 5.5.1, we see that the task of inserting a new en-
try into the list of high scores is almost identical to the task of inserting a newly
considered element in insertion-sort (except that game scores were ordered from
high to low). We provide a Python implementation of insertion-sort in Code Frag-
ment 5.10, using an outer loop to consider each element in turn, and an inner
loop that moves a newly considered element to its proper location relative to the
(sorted) subarray of elements that are to its left. We illustrate an example run of the
insertion-sort algorithm in Figure 5.20.
The nested loops of insertion-sort lead to an O(n2) running time in the worst
case. The most work is done if the array is initially in reverse order. On the other
hand, if the initial array is nearly sorted or perfectly sorted, insertion-sort runs in
O(n) time because there are few or no iterations of the inner loop.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-236-2048.jpg)
![5.5. Using Array-Based Sequences 215
1 def insertion sort(A):
2 ”””Sort list of comparable elements into nondecreasing order.”””
3 for k in range(1, len(A)): # from 1 to n-1
4 cur = A[k] # current element to be inserted
5 j = k # find correct index j for current
6 while j 0 and A[j−1] cur: # element A[j-1] must be after current
7 A[j] = A[j−1]
8 j −= 1
9 A[j] = cur # cur is now in the right place
Code Fragment 5.10: Python code for performing insertion-sort on a list.
insert
insert
insert
0
0
0
0
0
0
0
0
0
0 0 0
Done!
0
C A E H G F
B C A E H G F
D
B E H G F
C D
A H G F
B C D E
A F
B C D E H
A G F
B C D E H
E H G F
D
C
B
B E H G F
D
C
A F
B C D E H
B C D E H
G
B
A
A B C D E G H A B
C
F
F
G
A
G
H
E
A
D
C D E H
H
G
F
E
D
G
C
A
B D
no move
2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7
1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7
cur
1 2 3 4 5 6 7
move move
move
no move
no move
no move
no move
move no move
move move
1
Figure 5.20: Execution of the insertion-sort algorithm on an array of eight charac-
ters. Each row corresponds to an iteration of the outer loop, and each copy of the
sequence in a row corresponds to an iteration of the inner loop. The current element
that is being inserted is highlighted in the array, and shown as the cur value.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-237-2048.jpg)
![216 Chapter 5. Array-Based Sequences
5.5.3 Simple Cryptography
An interesting application of strings and lists is cryptography, the science of secret
messages and their applications. This field studies ways of performing encryp-
tion, which takes a message, called the plaintext, and converts it into a scrambled
message, called the ciphertext. Likewise, cryptography also studies corresponding
ways of performing decryption, which takes a ciphertext and turns it back into its
original plaintext.
Arguably the earliest encryption scheme is the Caesar cipher, which is named
after Julius Caesar, who used this scheme to protect important military messages.
(All of Caesar’s messages were written in Latin, of course, which already makes
them unreadable for most of us!) The Caesar cipher is a simple way to obscure a
message written in a language that forms words with an alphabet.
The Caesar cipher involves replacing each letter in a message with the letter that
is a certain number of letters after it in the alphabet. So, in an English message, we
might replace each A with D, each B with E, each C with F, and so on, if shifting by
three characters. We continue this approach all the way up to W, which is replaced
with Z. Then, we let the substitution pattern wrap around, so that we replace X
with A, Y with B, and Z with C.
Converting Between Strings and Character Lists
Given that strings are immutable, we cannot directly edit an instance to encrypt it.
Instead, our goal will be to generate a new string. A convenient technique for per-
forming string transformations is to create an equivalent list of characters, edit the
list, and then reassemble a (new) string based on the list. The first step can be per-
formed by sending the string as a parameter to the constructor of the list class. For
example, the expression list( bird ) produces the result [ b , i , r , d ].
Conversely, we can use a list of characters to build a string by invoking the join
method on an empty string, with the list of characters as the parameter. For exam-
ple, the call .join([ b , i , r , d ]) returns the string bird .
Using Characters as Array Indices
If we were to number our letters like array indices, so that A is 0, B is 1, C is 2,
and so on, then we can write the Caesar cipher with a rotation of r as a simple
formula: Replace each letter i with the letter (i + r) mod 26, where mod is the
modulo operator, which returns the remainder after performing an integer division.
This operator is denoted with % in Python, and it is exactly the operator we need
to easily perform the wrap around at the end of the alphabet. For 26 mod 26 is
0, 27 mod 26 is 1, and 28 mod 26 is 2. The decryption algorithm for the Caesar
cipher is just the opposite—we replace each letter with the one r places before it,
with wrap around (that is, letter i is replaced by letter (i−r) mod 26).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-238-2048.jpg)
![218 Chapter 5. Array-Based Sequences
1 class CaesarCipher:
2 ”””Class for doing encryption and decryption using a Caesar cipher.”””
3
4 def init (self, shift):
5 ”””Construct Caesar cipher using given integer shift for rotation.”””
6 encoder = [None] 26 # temp array for encryption
7 decoder = [None] 26 # temp array for decryption
8 for k in range(26):
9 encoder[k] = chr((k + shift) % 26 + ord( A ))
10 decoder[k] = chr((k − shift) % 26 + ord( A ))
11 self. forward = .join(encoder) # will store as string
12 self. backward = .join(decoder) # since fixed
13
14 def encrypt(self, message):
15 ”””Return string representing encripted message.”””
16 return self. transform(message, self. forward)
17
18 def decrypt(self, secret):
19 ”””Return decrypted message given encrypted secret.”””
20 return self. transform(secret, self. backward)
21
22 def transform(self, original, code):
23 ”””Utility to perform transformation based on given code string.”””
24 msg = list(original)
25 for k in range(len(msg)):
26 if msg[k].isupper():
27 j = ord(msg[k]) − ord( A ) # index from 0 to 25
28 msg[k] = code[j] # replace this character
29 return .join(msg)
30
31 if name == __main__ :
32 cipher = CaesarCipher(3)
33 message = THE EAGLE IS IN PLAY; MEET AT JOE S.
34 coded = cipher.encrypt(message)
35 print( Secret: , coded)
36 answer = cipher.decrypt(coded)
37 print( Message: , answer)
Code Fragment 5.11: A complete Python class for the Caesar cipher.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-240-2048.jpg)
![5.6. Multidimensional Data Sets 219
5.6 Multidimensional Data Sets
Lists, tuples, and strings in Python are one-dimensional. We use a single index to
access each element of the sequence. Many computer applications involve mul-
tidimensional data sets. For example, computer graphics are often modeled in
either two or three dimensions. Geographic information may be naturally repre-
sented in two dimensions, medical imaging may provide three-dimensional scans
of a patient, and a company’s valuation is often based upon a high number of in-
dependent financial measures that can be modeled as multidimensional data. A
two-dimensional array is sometimes also called a matrix. We may use two indices,
say i and j, to refer to the cells in the matrix. The first index usually refers to a
row number and the second to a column number, and these are traditionally zero-
indexed in computer science. Figure 5.22 illustrates a two-dimensional data set
with integer values. This data might, for example, represent the number of stores
in various regions of Manhattan.
22 18 709 5 33 10 4 56 82 440
45 32 830 120 750 660 13 77 20 105
4 880 45 66 61 28 650 7 510 67
940 12 36 3 20 100 306 590 0 500
50 65 42 49 88 25 70 126 83 288
398 233 5 83 59 232 49 8 365 90
33 58 632 87 94 5 59 204 120 829
62 394 3 4 102 140 183 390 16 26
8
0
1
2
3
4
5
6
7
0 1 2 3 4 5 6 7 9
Figure 5.22: Illustration of a two-dimensional integer data set, which has 8 rows
and 10 columns. The rows and columns are zero-indexed. If this data set were
named stores, the value of stores[3][5] is 100 and the value of stores[6][2] is 632.
A common representation for a two-dimensional data set in Python is as a list
of lists. In particular, we can represent a two-dimensional array as a list of rows,
with each row itself being a list of values. For example, the two-dimensional data
22 18 709 5 33
45 32 830 120 750
4 880 45 66 61
might be stored in Python as follows.
data = [ [22, 18, 709, 5, 33], [45, 32, 830, 120, 750], [4, 880, 45, 66, 61] ]
An advantage of this representation is that we can naturally use a syntax such
as data[1][3] to represent the value that has row index 1 and column index 3, as
data[1], the second entry in the outer list, is itself a list, and thus indexable.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-241-2048.jpg)
![220 Chapter 5. Array-Based Sequences
Constructing a Multidimensional List
To quickly initialize a one-dimensional list, we generally rely on a syntax such as
data = [0] n to create a list of n zeros. On page 189, we emphasized that from
a technical perspective, this creates a list of length n with all entries referencing
the same integer instance, but that there was no meaningful consequence of such
aliasing because of the immutability of the int class in Python.
We have to be considerably more careful when creating a list of lists. If our
goal were to create the equivalent of a two-dimensional list of integers, with r rows
and c columns, and to initialize all values to zero, a flawed approach might be to
try the command
data = ([0] c) r # Warning: this is a mistake
While([0] c) is indeed a list of c zeros, multiplying that list by r unfortunately cre-
ates a single list with length r ·c, just as [2,4,6] 2 results in list [2, 4, 6, 2, 4, 6].
A better, yet still flawed attempt is to make a list that contains the list of c zeros
as its only element, and then to multiply that list by r. That is, we could try the
command
data = [ [0] c ] r # Warning: still a mistake
This is much closer, as we actually do have a structure that is formally a list of lists.
The problem is that all r entries of the list known as data are references to the same
instance of a list of c zeros. Figure 5.23 provides a portrayal of such aliasing.
0
0
0
0 0 0 0 0
1 2 3 4 5
1 2
data:
Figure 5.23: A flawed representation of a 3×6 data set as a list of lists, created with
the command data = [ [0] 6 ] 3. (For simplicity, we overlook the fact that the
values in the secondary list are referential.)
This is truly a problem. Setting an entry such as data[2][0] = 100 would change
the first entry of the secondary list to reference a new value, 100. Yet that cell of
the secondary list also represents the value data[0][0], because “row” data[0] and
“row” data[2] refer to the same secondary list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-242-2048.jpg)
![5.6. Multidimensional Data Sets 221
0 0 0
0
0
0 0 0 0 0 0
0
0 0 0
0 0
0 0
0 0 0
3
1 2 2 3 4 5 1 2 4 5
1 2
data:
3 4 5 1
Figure 5.24: A valid representation of a 3×6 data set as a list of lists. (For simplic-
ity, we overlook the fact that the values in the secondary lists are referential.)
To properly initialize a two-dimensional list, we must ensure that each cell of
the primary list refers to an independent instance of a secondary list. This can be
accomplished through the use of Python’s list comprehension syntax.
data = [ [0] c for j in range(r) ]
This command produces a valid configuration, similar to the one shown in Fig-
ure 5.24. By using list comprehension, the expression [0] c is reevaluated for
each pass of the embedded for loop. Therefore, we get r distinct secondary lists, as
desired. (We note that the variable j in that command is irrelevant; we simply need
a for loop that iterates r times.)
Two-Dimensional Arrays and Positional Games
Many computer games, be they strategy games, simulation games, or first-person
conflict games, involve objects that reside in a two-dimensional space. Software for
such positional games need a way of representing such a two-dimensional “board,”
and in Python the list of lists is a natural choice.
Tic-Tac-Toe
As most school children know, Tic-Tac-Toe is a game played in a three-by-three
board. Two players—X and O—alternate in placing their respective marks in the
cells of this board, starting with player X. If either player succeeds in getting three
of his or her marks in a row, column, or diagonal, then that player wins.
This is admittedly not a sophisticated positional game, and it’s not even that
much fun to play, since a good player O can always force a tie. Tic-Tac-Toe’s saving
grace is that it is a nice, simple example showing how two-dimensional arrays can
be used for positional games. Software for more sophisticated positional games,
such as checkers, chess, or the popular simulation games, are all based on the same
approach we illustrate here for using a two-dimensional array for Tic-Tac-Toe.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-243-2048.jpg)
![222 Chapter 5. Array-Based Sequences
Our representation of a 3 × 3 board will be a list of lists of characters, with
X or O designating a player’s move, or designating an empty space. For
example, the board configuration
X
O O
O X
X
will be stored internally as
[ [ O , X , O ], [ , X , ], [ , O , X ] ]
We develop a complete Python class for maintaining a Tic-Tac-Toe board for
two players. That class will keep track of the moves and report a winner, but it
does not perform any strategy or allow someone to play Tic-Tac-Toe against the
computer. The details of such a program are beyond the scope of this chapter, but
it might nonetheless make a good course project (see Exercise P-8.68).
Before presenting the implementation of the class, we demonstrate its public
interface with a simple test in Code Fragment 5.12.
1 game = TicTacToe()
2 # X moves: # O moves:
3 game.mark(1, 1); game.mark(0, 2)
4 game.mark(2, 2); game.mark(0, 0)
5 game.mark(0, 1); game.mark(2, 1)
6 game.mark(1, 2); game.mark(1, 0)
7 game.mark(2, 0)
8
9 print(game)
10 winner = game.winner()
11 if winner is None:
12 print( Tie )
13 else:
14 print(winner, wins )
Code Fragment 5.12: A simple test for our Tic-Tac-Toe class.
The basic operations are that a new game instance represents an empty board,
that the mark(i,j) method adds a mark at the given position for the current player
(with the software managing the alternating of turns), and that the game board can
be printed and the winner determined. The complete source code for the TicTacToe
class is given in Code Fragment 5.13. Our mark method performs error checking
to make sure that valid indices are sent, that the position is not already occupied,
and that no further moves are made after someone wins the game.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-244-2048.jpg)
![5.6. Multidimensional Data Sets 223
1 class TicTacToe:
2 ”””Management of a Tic-Tac-Toe game (does not do strategy).”””
3
4 def init (self):
5 ”””Start a new game.”””
6 self. board = [ [ ] 3 for j in range(3) ]
7 self. player = X
8
9 def mark(self, i, j):
10 ”””Put an X or O mark at position (i,j) for next player s turn.”””
11 if not (0 = i = 2 and 0 = j = 2):
12 raise ValueError( Invalid board position )
13 if self. board[i][j] != :
14 raise ValueError( Board position occupied )
15 if self.winner( ) is not None:
16 raise ValueError( Game is already complete )
17 self. board[i][j] = self. player
18 if self. player == X :
19 self. player = O
20 else:
21 self. player = X
22
23 def is win(self, mark):
24 ”””Check whether the board configuration is a win for the given player.”””
25 board = self. board # local variable for shorthand
26 return (mark == board[0][0] == board[0][1] == board[0][2] or # row 0
27 mark == board[1][0] == board[1][1] == board[1][2] or # row 1
28 mark == board[2][0] == board[2][1] == board[2][2] or # row 2
29 mark == board[0][0] == board[1][0] == board[2][0] or # column 0
30 mark == board[0][1] == board[1][1] == board[2][1] or # column 1
31 mark == board[0][2] == board[1][2] == board[2][2] or # column 2
32 mark == board[0][0] == board[1][1] == board[2][2] or # diagonal
33 mark == board[0][2] == board[1][1] == board[2][0]) # rev diag
34
35 def winner(self):
36 ”””Return mark of winning player, or None to indicate a tie.”””
37 for mark in XO :
38 if self. is win(mark):
39 return mark
40 return None
41
42 def str (self):
43 ”””Return string representation of current game board.”””
44 rows = [ | .join(self. board[r]) for r in range(3)]
45 return n-----n .join(rows)
Code Fragment 5.13: A complete Python class for managing a Tic-Tac-Toe game.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-245-2048.jpg)
![Chapter Notes 227
C-5.29 A useful operation in databases is the natural join. If we view a database
as a list of ordered pairs of objects, then the natural join of databases A
and B is the list of all ordered triples (x,y,z) such that the pair (x,y) is in
A and the pair (y,z) is in B. Describe and analyze an efficient algorithm
for computing the natural join of a list A of n pairs and a list B of m pairs.
C-5.30 When Bob wants to send Alice a message M on the Internet, he breaks M
into n data packets, numbers the packets consecutively, and injects them
into the network. When the packets arrive at Alice’s computer, they may
be out of order, so Alice must assemble the sequence of n packets in order
before she can be sure she has the entire message. Describe an efficient
scheme for Alice to do this, assuming that she knows the value of n. What
is the running time of this algorithm?
C-5.31 Describe a way to use recursion to add all the numbers in an n × n data
set, represented as a list of lists.
Projects
P-5.32 Write a Python function that takes two three-dimensional numeric data
sets and adds them componentwise.
P-5.33 Write a Python program for a matrix class that can add and multiply two-
dimensional arrays of numbers, assuming the dimensions agree appropri-
ately for the operation.
P-5.34 Write a program that can perform the Caesar cipher for English messages
that include both upper- and lowercase characters.
P-5.35 Implement a class, SubstitutionCipher, with a constructor that takes a
string with the 26 uppercase letters in an arbitrary order and uses that for
the forward mapping for encryption (akin to the self. forward string in
our CaesarCipher class of Code Fragment 5.11). You should derive the
backward mapping from the forward version.
P-5.36 Redesign the CaesarCipher class as a subclass of the SubstitutionCipher
from the previous problem.
P-5.37 Design a RandomCipher class as a subclass of the SubstitutionCipher
from Exercise P-5.35, so that each instance of the class relies on a random
permutation of letters for its mapping.
Chapter Notes
The fundamental data structures of arrays belong to the folklore of computer science. They
were first chronicled in the computer science literature by Knuth in his seminal book on
Fundamental Algorithms [64].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-249-2048.jpg)
![230 Chapter 6. Stacks, Queues, and Deques
6.1.1 The Stack Abstract Data Type
Stacks are the simplest of all data structures, yet they are also among the most
important. They are used in a host of different applications, and as a tool for many
more sophisticated data structures and algorithms. Formally, a stack is an abstract
data type (ADT) such that an instance S supports the following two methods:
S.push(e): Add element e to the top of stack S.
S.pop(): Remove and return the top element from the stack S;
an error occurs if the stack is empty.
Additionally, let us define the following accessor methods for convenience:
S.top(): Return a reference to the top element of stack S, without
removing it; an error occurs if the stack is empty.
S.is empty(): Return True if stack S does not contain any elements.
len(S): Return the number of elements in stack S; in Python, we
implement this with the special method len .
By convention, we assume that a newly created stack is empty, and that there is no
a priori bound on the capacity of the stack. Elements added to the stack can have
arbitrary type.
Example 6.3: The following table shows a series of stack operations and their
effects on an initially empty stack S of integers.
Operation Return Value Stack Contents
S.push(5) – [5]
S.push(3) – [5, 3]
len(S) 2 [5, 3]
S.pop() 3 [5]
S.is empty() False [5]
S.pop() 5 [ ]
S.is empty() True [ ]
S.pop() “error” [ ]
S.push(7) – [7]
S.push(9) – [7, 9]
S.top() 9 [7, 9]
S.push(4) – [7, 9, 4]
len(S) 3 [7, 9, 4]
S.pop() 4 [7, 9]
S.push(6) – [7, 9, 6]
S.push(8) – [7, 9, 6, 8]
S.pop() 8 [7, 9, 6]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-252-2048.jpg)
![6.1. Stacks 231
6.1.2 Simple Array-Based Stack Implementation
We can implement a stack quite easily by storing its elements in a Python list. The
list class already supports adding an element to the end with the append method,
and removing the last element with the pop method, so it is natural to align the top
of the stack at the end of the list, as shown in Figure 6.2.
0
M
B C D E F G K L
A
1 2 top
Figure 6.2: Implementing a stack with a Python list, storing the top element in the
rightmost cell.
Although a programmer could directly use the list class in place of a formal
stack class, lists also include behaviors (e.g., adding or removing elements from
arbitrary positions) that would break the abstraction that the stack ADT represents.
Also, the terminology used by the list class does not precisely align with traditional
nomenclature for a stack ADT, in particular the distinction between append and
push. Instead, we demonstrate how to use a list for internal storage while providing
a public interface consistent with a stack.
The Adapter Pattern
The adapter design pattern applies to any context where we effectively want to
modify an existing class so that its methods match those of a related, but different,
class or interface. One general way to apply the adapter pattern is to define a new
class in such a way that it contains an instance of the existing class as a hidden
field, and then to implement each method of the new class using methods of this
hidden instance variable. By applying the adapter pattern in this way, we have
created a new class that performs some of the same functions as an existing class,
but repackaged in a more convenient way. In the context of the stack ADT, we can
adapt Python’s list class using the correspondences shown in Table 6.1.
Stack Method Realization with Python list
S.push(e) L.append(e)
S.pop() L.pop()
S.top() L[−1]
S.is empty() len(L) == 0
len(S) len(L)
Table 6.1: Realization of a stack S as an adaptation of a Python list L.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-253-2048.jpg)
![232 Chapter 6. Stacks, Queues, and Deques
Implementing a Stack Using a Python List
We use the adapter design pattern to define an ArrayStack class that uses an un-
derlying Python list for storage. (We choose the name ArrayStack to emphasize
that the underlying storage is inherently array based.) One question that remains is
what our code should do if a user calls pop or top when the stack is empty. Our
ADT suggests that an error occurs, but we must decide what type of error. When
pop is called on an empty Python list, it formally raises an IndexError, as lists are
index-based sequences. That choice does not seem appropriate for a stack, since
there is no assumption of indices. Instead, we can define a new exception class that
is more appropriate. Code Fragment 6.1 defines such an Empty class as a trivial
subclass of the Python Exception class.
class Empty(Exception):
”””Error attempting to access an element from an empty container.”””
pass
Code Fragment 6.1: Definition for an Empty exception class.
The formal definition for our ArrayStack class is given in Code Fragment 6.2.
The constructor establishes the member self. data as an initially empty Python list,
for internal storage. The rest of the public stack behaviors are implemented, using
the corresponding adaptation that was outlined in Table 6.1.
Example Usage
Below, we present an example of the use of our ArrayStack class, mirroring the
operations at the beginning of Example 6.3 on page 230.
S = ArrayStack( ) # contents: [ ]
S.push(5) # contents: [5]
S.push(3) # contents: [5, 3]
print(len(S)) # contents: [5, 3]; outputs 2
print(S.pop()) # contents: [5]; outputs 3
print(S.is empty()) # contents: [5]; outputs False
print(S.pop()) # contents: [ ]; outputs 5
print(S.is empty()) # contents: [ ]; outputs True
S.push(7) # contents: [7]
S.push(9) # contents: [7, 9]
print(S.top()) # contents: [7, 9]; outputs 9
S.push(4) # contents: [7, 9, 4]
print(len(S)) # contents: [7, 9, 4]; outputs 3
print(S.pop()) # contents: [7, 9]; outputs 4
S.push(6) # contents: [7, 9, 6]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-254-2048.jpg)
![6.1. Stacks 233
1 class ArrayStack:
2 ”””LIFO Stack implementation using a Python list as underlying storage.”””
3
4 def init (self):
5 ”””Create an empty stack.”””
6 self. data = [ ] # nonpublic list instance
7
8 def len (self):
9 ”””Return the number of elements in the stack.”””
10 return len(self. data)
11
12 def is empty(self):
13 ”””Return True if the stack is empty.”””
14 return len(self. data) == 0
15
16 def push(self, e):
17 ”””Add element e to the top of the stack.”””
18 self. data.append(e) # new item stored at end of list
19
20 def top(self):
21 ”””Return (but do not remove) the element at the top of the stack.
22
23 Raise Empty exception if the stack is empty.
24 ”””
25 if self.is empty():
26 raise Empty( Stack is empty )
27 return self. data[−1] # the last item in the list
28
29 def pop(self):
30 ”””Remove and return the element from the top of the stack (i.e., LIFO).
31
32 Raise Empty exception if the stack is empty.
33 ”””
34 if self.is empty():
35 raise Empty( Stack is empty )
36 return self. data.pop( ) # remove last item from list
Code Fragment 6.2: Implementing a stack using a Python list as storage.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-255-2048.jpg)
![236 Chapter 6. Stacks, Queues, and Deques
6.1.4 Matching Parentheses and HTML Tags
In this subsection, we explore two related applications of stacks, both of which
involve testing for pairs of matching delimiters. In our first application, we consider
arithmetic expressions that may contain various pairs of grouping symbols, such as
• Parentheses: “(” and “)”
• Braces: “{” and “}”
• Brackets: “[” and “]”
Each opening symbol must match its corresponding closing symbol. For example, a
left bracket, “[,” must match a corresponding right bracket, “],” as in the expression
[(5+x)-(y+z)]. The following examples further illustrate this concept:
• Correct: ()(()){([()])}
• Correct: ((()(()){([()])}))
• Incorrect: )(()){([()])}
• Incorrect: ({[])}
• Incorrect: (
We leave the precise definition of a matching group of symbols to Exercise R-6.6.
An Algorithm for Matching Delimiters
An important task when processing arithmetic expressions is to make sure their
delimiting symbols match up correctly. Code Fragment 6.4 presents a Python im-
plementation of such an algorithm. A discussion of the code follows.
1 def is matched(expr):
2 ”””Return True if all delimiters are properly match; False otherwise.”””
3 lefty = ({[ # opening delimiters
4 righty = )}] # respective closing delims
5 S = ArrayStack()
6 for c in expr:
7 if c in lefty:
8 S.push(c) # push left delimiter on stack
9 elif c in righty:
10 if S.is empty():
11 return False # nothing to match with
12 if righty.index(c) != lefty.index(S.pop()):
13 return False # mismatched
14 return S.is empty( ) # were all symbols matched?
Code Fragment 6.4: Function for matching delimiters in an arithmetic expression.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-258-2048.jpg)
![6.1. Stacks 237
We assume the input is a sequence of characters, such as [(5+x)-(y+z)] .
We perform a left-to-right scan of the original sequence, using a stack S to facilitate
the matching of grouping symbols. Each time we encounter an opening symbol,
we push that symbol onto S, and each time we encounter a closing symbol, we pop
a symbol from the stack S (assuming S is not empty), and check that these two
symbols form a valid pair. If we reach the end of the expression and the stack is
empty, then the original expression was properly matched. Otherwise, there must
be an opening delimiter on the stack without a matching symbol.
If the length of the original expression is n, the algorithm will make at most
n calls to push and n calls to pop. Those calls run in a total of O(n) time, even con-
sidering the amortized nature of the O(1) time bound for those methods. Given that
our selection of possible delimiters, ({[, has constant size, auxiliary tests such as
c in lefty and righty.index(c) each run in O(1) time. Combining these operations,
the matching algorithm on a sequence of length n runs in O(n) time.
Matching Tags in a Markup Language
Another application of matching delimiters is in the validation of markup languages
such as HTML or XML. HTML is the standard format for hyperlinked documents
on the Internet and XML is an extensible markup language used for a variety of
structured data sets. We show a sample HTML document and a possible rendering
in Figure 6.3.
body
center
h1 The Little Boat /h1
/center
p The storm tossed the little
boat like a cheap sneaker in an
old washing machine. The three
drunken fishermen were used to
such treatment, of course, but
not the tree salesman, who even as
a stowaway now felt that he
had overpaid for the voyage. /p
ol
li Will the salesman die? /li
li What color is the boat? /li
li And what about Naomi? /li
/ol
/body
The Little Boat
The storm tossed the little boat
like a cheap sneaker in an
old washing machine. The three
drunken fishermen were used to
such treatment, of course, but not
the tree salesman, who even as
a stowaway now felt that he had
overpaid for the voyage.
1. Will the salesman die?
2. What color is the boat?
3. And what about Naomi?
(a) (b)
Figure 6.3: Illustrating HTML tags. (a) An HTML document; (b) its rendering.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-259-2048.jpg)
![238 Chapter 6. Stacks, Queues, and Deques
In an HTML document, portions of text are delimited by HTML tags. A simple
opening HTML tag has the form “name” and the corresponding closing tag has
the form “/name”. For example, we see the body tag on the first line of
Figure 6.3(a), and the matching /body tag at the close of that document. Other
commonly used HTML tags that are used in this example include:
• body: document body
• h1: section header
• center: center justify
• p: paragraph
• ol: numbered (ordered) list
• li: list item
Ideally, an HTML document should have matching tags, although most browsers
tolerate a certain number of mismatching tags. In Code Fragment 6.5, we give a
Python function that matches tags in a string representing an HTML document. We
make a left-to-right pass through the raw string, using index j to track our progress
and the find method of the str class to locate the and characters that define
the tags. Opening tags are pushed onto the stack, and matched against closing tags
as they are popped from the stack, just as we did when matching delimiters in Code
Fragment 6.4. By similar analysis, this algorithm runs in O(n) time, where n is the
number of characters in the raw HTML source.
1 def is matched html(raw):
2 ”””Return True if all HTML tags are properly match; False otherwise.”””
3 S = ArrayStack()
4 j = raw.find( ) # find first ’’ character (if any)
5 while j != −1:
6 k = raw.find( , j+1) # find next ’’ character
7 if k == −1:
8 return False # invalid tag
9 tag = raw[j+1:k] # strip away
10 if not tag.startswith( / ): # this is opening tag
11 S.push(tag)
12 else: # this is closing tag
13 if S.is empty():
14 return False # nothing to match with
15 if tag[1:] != S.pop():
16 return False # mismatched delimiter
17 j = raw.find( , k+1) # find next ’’ character (if any)
18 return S.is empty( ) # were all opening tags matched?
Code Fragment 6.5: Function for testing if an HTML document has matching tags.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-260-2048.jpg)
![240 Chapter 6. Stacks, Queues, and Deques
6.2.1 The Queue Abstract Data Type
Formally, the queue abstract data type defines a collection that keeps objects in a
sequence, where element access and deletion are restricted to the first element in
the queue, and element insertion is restricted to the back of the sequence. This
restriction enforces the rule that items are inserted and deleted in a queue accord-
ing to the first-in, first-out (FIFO) principle. The queue abstract data type (ADT)
supports the following two fundamental methods for a queue Q:
Q.enqueue(e): Add element e to the back of queue Q.
Q.dequeue(): Remove and return the first element from queue Q;
an error occurs if the queue is empty.
The queue ADT also includes the following supporting methods (with first being
analogous to the stack’s top method):
Q.first(): Return a reference to the element at the front of queue Q,
without removing it; an error occurs if the queue is empty.
Q.is empty(): Return True if queue Q does not contain any elements.
len(Q): Return the number of elements in queue Q; in Python,
we implement this with the special method len .
By convention, we assume that a newly created queue is empty, and that there
is no a priori bound on the capacity of the queue. Elements added to the queue can
have arbitrary type.
Example 6.4: The following table shows a series of queue operations and their
effects on an initially empty queue Q of integers.
Operation Return Value first ← Q ← last
Q.enqueue(5) – [5]
Q.enqueue(3) – [5, 3]
len(Q) 2 [5, 3]
Q.dequeue() 5 [3]
Q.is empty() False [3]
Q.dequeue() 3 [ ]
Q.is empty() True [ ]
Q.dequeue() “error” [ ]
Q.enqueue(7) – [7]
Q.enqueue(9) – [7, 9]
Q.first() 7 [7, 9]
Q.enqueue(4) – [7, 9, 4]
len(Q) 3 [7, 9, 4]
Q.dequeue() 7 [9, 4]](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-262-2048.jpg)
![6.2. Queues 243
1 class ArrayQueue:
2 ”””FIFO queue implementation using a Python list as underlying storage.”””
3 DEFAULT CAPACITY = 10 # moderate capacity for all new queues
4
5 def init (self):
6 ”””Create an empty queue.”””
7 self. data = [None] ArrayQueue.DEFAULT CAPACITY
8 self. size = 0
9 self. front = 0
10
11 def len (self):
12 ”””Return the number of elements in the queue.”””
13 return self. size
14
15 def is empty(self):
16 ”””Return True if the queue is empty.”””
17 return self. size == 0
18
19 def first(self):
20 ”””Return (but do not remove) the element at the front of the queue.
21
22 Raise Empty exception if the queue is empty.
23 ”””
24 if self.is empty():
25 raise Empty( Queue is empty )
26 return self. data[self. front]
27
28 def dequeue(self):
29 ”””Remove and return the first element of the queue (i.e., FIFO).
30
31 Raise Empty exception if the queue is empty.
32 ”””
33 if self.is empty():
34 raise Empty( Queue is empty )
35 answer = self. data[self. front]
36 self. data[self. front] = None # help garbage collection
37 self. front = (self. front + 1) % len(self. data)
38 self. size −= 1
39 return answer
Code Fragment 6.6: Array-based implementation of a queue (continued in Code
Fragment 6.7).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-265-2048.jpg)
![244 Chapter 6. Stacks, Queues, and Deques
40 def enqueue(self, e):
41 ”””Add an element to the back of queue.”””
42 if self. size == len(self. data):
43 self. resize(2 len(self.data)) # double the array size
44 avail = (self. front + self. size) % len(self. data)
45 self. data[avail] = e
46 self. size += 1
47
48 def resize(self, cap): # we assume cap = len(self)
49 ”””Resize to a new list of capacity = len(self).”””
50 old = self. data # keep track of existing list
51 self. data = [None] cap # allocate list with new capacity
52 walk = self. front
53 for k in range(self. size): # only consider existing elements
54 self. data[k] = old[walk] # intentionally shift indices
55 walk = (1 + walk) % len(old) # use old size as modulus
56 self. front = 0 # front has been realigned
Code Fragment 6.7: Array-based implementation of a queue (continued from Code
Fragment 6.6).
The implementation of len and is empty are trivial, given knowledge of
the size. The implementation of the front method is also simple, as the front
index tells us precisely where the desired element is located within the data list,
assuming that list is not empty.
Adding and Removing Elements
The goal of the enqueue method is to add a new element to the back of the queue.
We need to determine the proper index at which to place the new element. Although
we do not explicitly maintain an instance variable for the back of the queue, we
compute the location of the next opening based on the formula:
avail = (self. front + self. size) % len(self. data)
Note that we are using the size of the queue as it exists prior to the addition of the
new element. For example, consider a queue with capacity 10, current size 3, and
first element at index 5. The three elements of such a queue are stored at indices 5,
6, and 7. The new element should be placed at index (front + size) = 8. In a case
with wrap-around, the use of the modular arithmetic achieves the desired circular
semantics. For example, if our hypothetical queue had 3 elements with the first at
index 8, our computation of (8+3) % 10 evaluates to 1, which is perfect since the
three existing elements occupy indices 8, 9, and 0.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-266-2048.jpg)
![6.2. Queues 245
When the dequeue method is called, the current value of self. front designates
the index of the value that is to be removed and returned. We keep a local refer-
ence to the element that will be returned, setting answer = self. data[self. front]
just prior to removing the reference to that object from the list, with the assignment
self. data[self. front] = None. Our reason for the assignment to None relates to
Python’s mechanism for reclaiming unused space. Internally, Python maintains a
count of the number of references that exist to each object. If that count reaches
zero, the object is effectively inaccessible, thus the system may reclaim that mem-
ory for future use. (For more details, see Section 15.1.2.) Since we are no longer
responsible for storing a dequeued element, we remove the reference to it from our
list so as to reduce that element’s reference count.
The second significant responsibility of the dequeue method is to update the
value of front to reflect the removal of the element, and the presumed promotion
of the second element to become the new first. In most cases, we simply want
to increment the index by one, but because of the possibility of a wrap-around
configuration, we rely on modular arithmetic as originally described on page 242.
Resizing the Queue
When enqueue is called at a time when the size of the queue equals the size of the
underlying list, we rely on a standard technique of doubling the storage capacity of
the underlying list. In this way, our approach is similar to the one used when we
implemented a DynamicArray in Section 5.3.1.
However, more care is needed in the queue’s resize utility than was needed in
the corresponding method of the DynamicArray class. After creating a temporary
reference to the old list of values, we allocate a new list that is twice the size and
copy references from the old list to the new list. While transferring the contents, we
intentionally realign the front of the queue with index 0 in the new array, as shown
in Figure 6.7. This realignment is not purely cosmetic. Since the modular arith-
metic depends on the size of the array, our state would be flawed had we transferred
each element to its same index in the new array.
E G H
I J K
E F G H I J K
F
f
1 2
f = 0
Figure 6.7: Resizing the queue, while realigning the front element with index 0.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-267-2048.jpg)
![248 Chapter 6. Stacks, Queues, and Deques
Example 6.5: The following table shows a series of operations and their effects
on an initially empty deque D of integers.
Operation Return Value Deque
D.add last(5) – [5]
D.add first(3) – [3, 5]
D.add first(7) – [7, 3, 5]
D.first() 7 [7, 3, 5]
D.delete last() 5 [7, 3]
len(D) 2 [7, 3]
D.delete last() 3 [7]
D.delete last() 7 [ ]
D.add first(6) – [6]
D.last() 6 [6]
D.add first(8) – [8, 6]
D.is empty() False [8, 6]
D.last() 6 [8, 6]
6.3.2 Implementing a Deque with a Circular Array
We can implement the deque ADT in much the same way as the ArrayQueue class
provided in Code Fragments 6.6 and 6.7 of Section 6.2.2 (so much so that we leave
the details of an ArrayDeque implementation to Exercise P-6.32). We recommend
maintaining the same three instance variables: data, size, and front. Whenever
we need to know the index of the back of the deque, or the first available slot
beyond the back of the deque, we use modular arithmetic for the computation. For
example, our implementation of the last() method uses the index
back = (self. front + self. size − 1) % len(self. data)
Our implementation of the ArrayDeque.add last method is essentially the same
as that for ArrayQueue.enqueue, including the reliance on a resize utility. Like-
wise, the implementation of the ArrayDeque.delete first method is the same as
ArrayQueue.dequeue. Implementations of add first and delete last use similar
techniques. One subtlety is that a call to add first may need to wrap around the
beginning of the array, so we rely on modular arithmetic to circularly decrement
the index, as
self. front = (self. front − 1) % len(self. data) # cyclic shift
The efficiency of an ArrayDeque is similar to that of an ArrayQueue, with all
operations having O(1) running time, but with that bound being amortized for op-
erations that may change the size of the underlying list.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-270-2048.jpg)
![6.3. Double-Ended Queues 249
6.3.3 Deques in the Python Collections Module
An implementation of a deque class is available in Python’s standard collections
module. A summary of the most commonly used behaviors of the collections.deque
class is given in Table 6.4. It uses more asymmetric nomenclature than our ADT.
Our Deque ADT collections.deque Description
len(D) len(D) number of elements
D.add first() D.appendleft() add to beginning
D.add last() D.append() add to end
D.delete first() D.popleft() remove from beginning
D.delete last() D.pop() remove from end
D.first() D[0] access first element
D.last() D[−1] access last element
D[j] access arbitrary entry by index
D[j] = val modify arbitrary entry by index
D.clear() clear all contents
D.rotate(k) circularly shift rightward k steps
D.remove(e) remove first matching element
D.count(e) count number of matches for e
Table 6.4: Comparison of our deque ADT and the collections.deque class.
The collections.deque interface was chosen to be consistent with established
naming conventions of Python’s list class, for which append and pop are presumed
to act at the end of the list. Therefore, appendleft and popleft designate an opera-
tion at the beginning of the list. The library deque also mimics a list in that it is an
indexed sequence, allowing arbitrary access or modification using the D[j] syntax.
The library deque constructor also supports an optional maxlen parameter to
force a fixed-length deque. However, if a call to append at either end is invoked
when the deque is full, it does not throw an error; instead, it causes one element to
be dropped from the opposite side. That is, calling appendleft when the deque is
full causes an implicit pop from the right side to make room for the new element.
The current Python distribution implements collections.deque with a hybrid ap-
proach that uses aspects of circular arrays, but organized into blocks that are them-
selves organized in a doubly linked list (a data structure that we will introduce in
the next chapter). The deque class is formally documented to guarantee O(1)-time
operations at either end, but O(n)-time worst-case operations when using index
notation near the middle of the deque.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-271-2048.jpg)
![250 Chapter 6. Stacks, Queues, and Deques
6.4 Exercises
For help with exercises, please visit the site, www.wiley.com/college/goodrich.
Reinforcement
R-6.1 What values are returned during the following series of stack operations, if
executed upon an initially empty stack? push(5), push(3), pop(), push(2),
push(8), pop(), pop(), push(9), push(1), pop(), push(7), push(6), pop(),
pop(), push(4), pop(), pop().
R-6.2 Suppose an initially empty stack S has executed a total of 25 push opera-
tions, 12 top operations, and 10 pop operations, 3 of which raised Empty
errors that were caught and ignored. What is the current size of S?
R-6.3 Implement a function with signature transfer(S, T) that transfers all ele-
ments from stack S onto stack T, so that the element that starts at the top
of S is the first to be inserted onto T, and the element at the bottom of S
ends up at the top of T.
R-6.4 Give a recursive method for removing all the elements from a stack.
R-6.5 Implement a function that reverses a list of elements by pushing them onto
a stack in one order, and writing them back to the list in reversed order.
R-6.6 Give a precise and complete definition of the concept of matching for
grouping symbols in an arithmetic expression. Your definition may be
recursive.
R-6.7 What values are returned during the following sequence of queue opera-
tions, if executed on an initially empty queue? enqueue(5), enqueue(3),
dequeue(), enqueue(2), enqueue(8), dequeue(), dequeue(), enqueue(9),
enqueue(1), dequeue(), enqueue(7), enqueue(6), dequeue(), dequeue(),
enqueue(4), dequeue(), dequeue().
R-6.8 Suppose an initially empty queue Q has executed a total of 32 enqueue
operations, 10 first operations, and 15 dequeue operations, 5 of which
raised Empty errors that were caught and ignored. What is the current
size of Q?
R-6.9 Had the queue of the previous problem been an instance of ArrayQueue
that used an initial array of capacity 30, and had its size never been greater
than 30, what would be the final value of the front instance variable?
R-6.10 Consider what happens if the loop in the ArrayQueue. resize method at
lines 53–55 of Code Fragment 6.7 had been implemented as:
for k in range(self. size):
self. data[k] = old[k] # rather than old[walk]
Give a clear explanation of what could go wrong.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-272-2048.jpg)
![252 Chapter 6. Stacks, Queues, and Deques
C-6.21 Show how to use a stack S and a queue Q to generate all possible subsets
of an n-element set T nonrecursively.
C-6.22 Postfix notation is an unambiguous way of writing an arithmetic expres-
sion without parentheses. It is defined so that if “(exp1)op(exp2)” is a
normal, fully parenthesized expression whose operation is op, the postfix
version of this is “pexp1 pexp2 op”, where pexp1 is the postfix version of
exp1 and pexp2 is the postfix version of exp2. The postfix version of a sin-
gle number or variable is just that number or variable. For example, the
postfix version of “((5+2)∗(8−3))/4” is “5 2 + 8 3 − ∗ 4 /”. Describe
a nonrecursive way of evaluating an expression in postfix notation.
C-6.23 Suppose you have three nonempty stacks R, S, and T. Describe a sequence
of operations that results in S storing all elements originally in T below all
of S’s original elements, with both sets of those elements in their original
order. The final configuration for R should be the same as its original
configuration. For example, if R = [1,2,3], S = [4,5], and T = [6,7,8,9],
the final configuration should have R = [1,2,3] and S = [6,7,8,9,4,5].
C-6.24 Describe how to implement the stack ADT using a single queue as an
instance variable, and only constant additional local memory within the
method bodies. What is the running time of the push(), pop(), and top()
methods for your design?
C-6.25 Describe how to implement the queue ADT using two stacks as instance
variables, such that all queue operations execute in amortized O(1) time.
Give a formal proof of the amortized bound.
C-6.26 Describe how to implement the double-ended queue ADT using two stacks
as instance variables. What are the running times of the methods?
C-6.27 Suppose you have a stack S containing n elements and a queue Q that is
initially empty. Describe how you can use Q to scan S to see if it contains a
certain element x, with the additional constraint that your algorithm must
return the elements back to S in their original order. You may only use S,
Q, and a constant number of other variables.
C-6.28 Modify the ArrayQueue implementation so that the queue’s capacity is
limited to maxlen elements, where maxlen is an optional parameter to the
constructor (that defaults to None). If enqueue is called when the queue
is at full capacity, throw a Full exception (defined similarly to Empty).
C-6.29 In certain applications of the queue ADT, it is common to repeatedly
dequeue an element, process it in some way, and then immediately en-
queue the same element. Modify the ArrayQueue implementation to in-
clude a rotate() method that has semantics identical to the combina-
tion, Q.enqueue(Q.dequeue()). However, your implementation should
be more efficient than making two separate calls (for example, because
there is no need to modify size).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-274-2048.jpg)
![254 Chapter 6. Stacks, Queues, and Deques
P-6.36 When a share of common stock of some company is sold, the capital
gain (or, sometimes, loss) is the difference between the share’s selling
price and the price originally paid to buy it. This rule is easy to under-
stand for a single share, but if we sell multiple shares of stock bought
over a long period of time, then we must identify the shares actually be-
ing sold. A standard accounting principle for identifying which shares of
a stock were sold in such a case is to use a FIFO protocol—the shares
sold are the ones that have been held the longest (indeed, this is the de-
fault method built into several personal finance software packages). For
example, suppose we buy 100 shares at $20 each on day 1, 20 shares at
$24 on day 2, 200 shares at $36 on day 3, and then sell 150 shares on day
4 at $30 each. Then applying the FIFO protocol means that of the 150
shares sold, 100 were bought on day 1, 20 were bought on day 2, and 30
were bought on day 3. The capital gain in this case would therefore be
100·10+20·6+30·(−6), or $940. Write a program that takes as input
a sequence of transactions of the form “buy x share(s) at y each”
or “sell x share(s) at y each,” assuming that the transactions oc-
cur on consecutive days and the values x and y are integers. Given this
input sequence, the output should be the total capital gain (or loss) for the
entire sequence, using the FIFO protocol to identify shares.
P-6.37 Design an ADT for a two-color, double-stack ADT that consists of two
stacks—one “red” and one “blue”—and has as its operations color-coded
versions of the regular stack ADT operations. For example, this ADT
should support both a red push operation and a blue push operation. Give
an efficient implementation of this ADT using a single array whose ca-
pacity is set at some value N that is assumed to always be larger than the
sizes of the red and blue stacks combined.
Chapter Notes
We were introduced to the approach of defining data structures first in terms of their ADTs
and then in terms of concrete implementations by the classic books by Aho, Hopcroft, and
Ullman [5, 6]. Exercises C-6.30, and C-6.31 are similar to interview questions said to be
from a well-known software company. For further study of abstract data types, see Liskov
and Guttag [71], Cardelli and Wegner [23], or Demurjian [33].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-276-2048.jpg)
![298 Chapter 7. Linked Lists
P-7.45 An array A is sparse if most of its entries are empty (i.e., None). A list
L can be used to implement such an array efficiently. In particular, for
each nonempty cell A[i], we can store an entry (i,e) in L, where e is the
element stored at A[i]. This approach allows us to represent A using O(m)
storage, where m is the number of nonempty entries in A. Provide such
a SparseArray class that minimally supports methods getitem (j) and
setitem (j, e) to provide standard indexing operations. Analyze the
efficiency of these methods.
P-7.46 Although we have used a doubly linked list to implement the positional
list ADT, it is possible to support the ADT with an array-based imple-
mentation. The key is to use the composition pattern and store a sequence
of position items, where each item stores an element as well as that ele-
ment’s current index in the array. Whenever an element’s place in the array
is changed, the recorded index in the position must be updated to match.
Given a complete class providing such an array-based implementation of
the positional list ADT. What is the efficiency of the various operations?
P-7.47 Implement a CardHand class that supports a person arranging a group of
cards in his or her hand. The simulator should represent the sequence of
cards using a single positional list ADT so that cards of the same suit are
kept together. Implement this strategy by means of four “fingers” into the
hand, one for each of the suits of hearts, clubs, spades, and diamonds,
so that adding a new card to the person’s hand or playing a correct card
from the hand can be done in constant time. The class should support the
following methods:
• add card(r, s): Add a new card with rank r and suit s to the hand.
• play(s): Remove and return a card of suit s from the player’s hand;
if there is no card of suit s, then remove and return an arbitrary card
from the hand.
• iter (): Iterate through all cards currently in the hand.
• all of suit(s): Iterate through all cards of suit s that are currently in
the hand.
Chapter Notes
A view of data structures as collections (and other principles of object-oriented design)
can be found in object-oriented design books by Booch [17], Budd [20], Goldberg and
Robson [42], and Liskov and Guttag [71]. Our positional list ADT is derived from the
“position” abstraction introduced by Aho, Hopcroft, and Ullman [6], and the list ADT of
Wood [104]. Implementations of linked lists are discussed by Knuth [64].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-320-2048.jpg)
![326 Chapter 8. Trees
The level numbering function f suggests a representation of a binary tree T
by means of an array-based structure A (such as a Python list), with the element
at position p of T stored at index f(p) of the array. We show an example of an
array-based representation of a binary tree in Figure 8.13.
/
4 2
0
2
1
3 4 5 6
12
11
8
7
3 1
+
×
9 5
−
+
0 6 12
1 2 3 4 5 7 8 9 10 11 13 14
5
× + + 4 − 2 3 1 9
/
Figure 8.13: Representation of a binary tree by means of an array.
One advantage of an array-based representation of a binary tree is that a posi-
tion p can be represented by the single integer f(p), and that position-based meth-
ods such as root, parent, left, and right can be implemented using simple arithmetic
operations on the number f(p). Based on our formula for the level numbering, the
left child of p has index 2f(p) + 1, the right child of p has index 2f(p) + 2, and
the parent of p has index (f(p)− 1)/2. We leave the details of a complete im-
plementation as an exercise (R-8.18).
The space usage of an array-based representation depends greatly on the shape
of the tree. Let n be the number of nodes of T, and let fM be the maximum value
of f(p) over all the nodes of T. The array A requires length N = 1 + fM, since
elements range from A[0] to A[ fM]. Note that A may have a number of empty cells
that do not refer to existing nodes of T. In fact, in the worst case, N = 2n − 1,
the justification of which is left as an exercise (R-8.16). In Section 9.3, we will
see a class of binary trees, called “heaps” for which N = n. Thus, in spite of the
worst-case space usage, there are applications for which the array representation
of a binary tree is space efficient. Still, for general binary trees, the exponential
worst-case space requirement of this representation is prohibitive.
Another drawback of an array representation is that some update operations for
trees cannot be efficiently supported. For example, deleting a node and promoting
its child takes O(n) time because it is not just the child that moves locations within
the array, but all descendants of that child.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-348-2048.jpg)
![8.4. Tree Traversal Algorithms 339
1 def preorder label(T, p, d, path):
2 ”””Print labeled representation of subtree of T rooted at p at depth d.”””
3 label = . .join(str(j+1) for j in path) # displayed labels are one-indexed
4 print(2 d + label, p.element())
5 path.append(0) # path entries are zero-indexed
6 for c in T.children(p):
7 preorder label(T, c, d+1, path) # child depth is d+1
8 path[−1] += 1
9 path.pop()
Code Fragment 8.24: Efficient recursion for printing an indented and labeled pre-
sentation of a preorder traversal.
Parenthetic Representations of a Tree
It is not possible to reconstruct a general tree, given only the preorder sequence
of elements, as in Figure 8.20(a). Some additional context is necessary for the
structure of the tree to be well defined. The use of indentation or numbered labels
provides such context, with a very human-friendly presentation. However, there
are more concise string representations of trees that are computer-friendly.
In this section, we explore one such representation. The parenthetic string
representation P(T) of tree T is recursively defined as follows. If T consists of a
single position p, then
P(T) = str(p.element()).
Otherwise, it is defined recursively as,
P(T) = str(p.element())+ ( +P(T1)+ , + ··· + , +P(Tk)+ )
where p is the root of T and T1,T2,...,Tk are the subtrees rooted at the children
of p, which are given in order if T is an ordered tree. We are using “+” here to
denote string concatenation. As an example, the parenthetic representation of the
tree of Figure 8.2 would appear as follows (line breaks are cosmetic):
Electronics R’Us (RD, Sales (Domestic, International (Canada,
S. America, Overseas (Africa, Europe, Asia, Australia))),
Purchasing, Manufacturing (TV, CD, Tuner))
Although the parenthetic representation is essentially a preorder traversal, we
cannot easily produce the additional punctuation using the formal implementation
of preorder, as given in Code Fragment 8.17. The opening parenthesis must be
produced just before the loop over a position’s children and the closing parenthesis
must be produced just after that loop. Furthermore, the separating commas must
be produced. The Python function parenthesize, shown in Code Fragment 8.25, is
a custom traversal that prints such a parenthetic string representation of a tree T.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-361-2048.jpg)
![8.4. Tree Traversal Algorithms 343
1 class EulerTour:
2 ”””Abstract base class for performing Euler tour of a tree.
3
4 hook previsit and hook postvisit may be overridden by subclasses.
5 ”””
6 def init (self, tree):
7 ”””Prepare an Euler tour template for given tree.”””
8 self. tree = tree
9
10 def tree(self):
11 ”””Return reference to the tree being traversed.”””
12 return self. tree
13
14 def execute(self):
15 ”””Perform the tour and return any result from post visit of root.”””
16 if len(self. tree) 0:
17 return self. tour(self. tree.root(), 0, [ ]) # start the recursion
18
19 def tour(self, p, d, path):
20 ”””Perform tour of subtree rooted at Position p.
21
22 p Position of current node being visited
23 d depth of p in the tree
24 path list of indices of children on path from root to p
25 ”””
26 self. hook previsit(p, d, path) # ”pre visit” p
27 results = [ ]
28 path.append(0) # add new index to end of path before recursion
29 for c in self. tree.children(p):
30 results.append(self. tour(c, d+1, path)) # recur on child s subtree
31 path[−1] += 1 # increment index
32 path.pop( ) # remove extraneous index from end of path
33 answer = self. hook postvisit(p, d, path, results) # ”post visit” p
34 return answer
35
36 def hook previsit(self, p, d, path): # can be overridden
37 pass
38
39 def hook postvisit(self, p, d, path, results): # can be overridden
40 pass
Code Fragment 8.28: An EulerTour base class providing a framework for perform-
ing Euler tour traversals of a tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-365-2048.jpg)
![8.4. Tree Traversal Algorithms 345
A labeled version of an indented, preorder presentation, akin to Code Frag-
ment 8.24, could be generated by the new subclass of EulerTour shown in Code
Fragment 8.30.
1 class PreorderPrintIndentedLabeledTour(EulerTour):
2 def hook previsit(self, p, d, path):
3 label = . .join(str(j+1) for j in path) # labels are one-indexed
4 print(2 d + label, p.element())
Code Fragment 8.30: A subclass of EulerTour that produces a labeled and indented,
preorder list of a tree’s elements.
To produce the parenthetic string representation, originally achieved with Code
Fragment 8.25, we define a subclass that overrides both the previsit and postvisit
hooks. Our new implementation is given in Code Fragment 8.31.
1 class ParenthesizeTour(EulerTour):
2 def hook previsit(self, p, d, path):
3 if path and path[−1] 0: # p follows a sibling
4 print( , , end= ) # so preface with comma
5 print(p.element(), end= ) # then print element
6 if not self.tree().is leaf(p): # if p has children
7 print( ( , end= ) # print opening parenthesis
8
9 def hook postvisit(self, p, d, path, results):
10 if not self.tree().is leaf(p): # if p has children
11 print( ) , end= ) # print closing parenthesis
Code Fragment 8.31: A subclass of EulerTour that prints a parenthetic string repre-
sentation of a tree.
Notice that in this implementation, we need to invoke a method on the tree instance
that is being traversed from within the hooks. The public tree() method of the
EulerTour class serves as an accessor for that tree.
Finally, the task of computing disk space, as originally implemented in Code
Fragment 8.26, can be performed quite easily with the EulerTour subclass shown
in Code Fragment 8.32. The postvisit result of the root will be returned by the call
to execute().
1 class DiskSpaceTour(EulerTour):
2 def hook postvisit(self, p, d, path, results):
3 # we simply add space associated with p to that of its subtrees
4 return p.element().space( ) + sum(results)
Code Fragment 8.32: A subclass of EulerTour that computes disk space for a tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-367-2048.jpg)
![346 Chapter 8. Trees
The Euler Tour Traversal of a Binary Tree
In Section 8.4.6, we introduced the concept of an Euler tour traversal of a general
graph, using the template method pattern in designing the EulerTour class. That
class provided methods hook previsit and hook postvisit that could be overrid-
den to customize a tour. In Code Fragment 8.33 we provide a BinaryEulerTour
specialization that includes an additional hook invisit that is called once for each
position—after its left subtree is traversed, but before its right subtree is traversed.
Our implementation of BinaryEulerTour replaces the original tour utility to
specialize to the case in which a node has at most two children. If a node has only
one child, a tour differentiates between whether that is a left child or a right child,
with the “in visit” taking place after the visit of a sole left child, but before the visit
of a sole right child. In the case of a leaf, the three hooks are called in succession.
1 class BinaryEulerTour(EulerTour):
2 ”””Abstract base class for performing Euler tour of a binary tree.
3
4 This version includes an additional hook invisit that is called after the tour
5 of the left subtree (if any), yet before the tour of the right subtree (if any).
6
7 Note: Right child is always assigned index 1 in path, even if no left sibling.
8 ”””
9 def tour(self, p, d, path):
10 results = [None, None] # will update with results of recursions
11 self. hook previsit(p, d, path) # ”pre visit” for p
12 if self. tree.left(p) is not None: # consider left child
13 path.append(0)
14 results[0] = self. tour(self. tree.left(p), d+1, path)
15 path.pop()
16 self. hook invisit(p, d, path) # ”in visit” for p
17 if self. tree.right(p) is not None: # consider right child
18 path.append(1)
19 results[1] = self. tour(self. tree.right(p), d+1, path)
20 path.pop()
21 answer = self. hook postvisit(p, d, path, results) # ”post visit” p
22 return answer
23
24 def hook invisit(self, p, d, path): pass # can be overridden
Code Fragment 8.33: A BinaryEulerTour base class providing a specialized tour for
binary trees. The original EulerTour base class was given in Code Fragment 8.28.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-368-2048.jpg)
![8.5. Case Study: An Expression Tree 349
1 class ExpressionTree(LinkedBinaryTree):
2 ”””An arithmetic expression tree.”””
3
4 def init (self, token, left=None, right=None):
5 ”””Create an expression tree.
6
7 In a single parameter form, token should be a leaf value (e.g., 42 ),
8 and the expression tree will have that value at an isolated node.
9
10 In a three-parameter version, token should be an operator,
11 and left and right should be existing ExpressionTree instances
12 that become the operands for the binary operator.
13 ”””
14 super(). init ( ) # LinkedBinaryTree initialization
15 if not isinstance(token, str):
16 raise TypeError( Token must be a string )
17 self. add root(token) # use inherited, nonpublic method
18 if left is not None: # presumably three-parameter form
19 if token not in +-*x/ :
20 raise ValueError( token must be valid operator )
21 self. attach(self.root(), left, right) # use inherited, nonpublic method
22
23 def str (self):
24 ”””Return string representation of the expression.”””
25 pieces = [ ] # sequence of piecewise strings to compose
26 self. parenthesize recur(self.root(), pieces)
27 return .join(pieces)
28
29 def parenthesize recur(self, p, result):
30 ”””Append piecewise representation of p s subtree to resulting list.”””
31 if self.is leaf(p):
32 result.append(str(p.element())) # leaf value as a string
33 else:
34 result.append( ( ) # opening parenthesis
35 self. parenthesize recur(self.left(p), result) # left subtree
36 result.append(p.element()) # operator
37 self. parenthesize recur(self.right(p), result) # right subtree
38 result.append( ) ) # closing parenthesis
Code Fragment 8.35: The beginning of an ExpressionTree class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-371-2048.jpg)
![8.5. Case Study: An Expression Tree 351
Building an Expression Tree
The constructor for the ExpressionTree class, from Code Fragment 8.35, provides
basic functionality for combining existing trees to build larger expression trees.
However, the question still remains how to construct a tree that represents an ex-
pression for a given string, such as (((3+1)x4)/((9-5)+2)) .
To automate this process, we rely on a bottom-up construction algorithm, as-
suming that a string can first be tokenized so that multidigit numbers are treated
atomically (see Exercise R-8.30), and that the expression is fully parenthesized.
The algorithm uses a stack S while scanning tokens of the input expression E to
find values, operators, and right parentheses. (Left parentheses are ignored.)
• When we see an operator ◦, we push that string on the stack.
• When we see a literal value v, we create a single-node expression tree T
storing v, and push T on the stack.
• When we see a right parenthesis, ) , we pop the top three items from the
stack S, which represent a subexpression (E1 ◦ E2). We then construct a
tree T using trees for E1 and E2 as subtrees of the root storing ◦, and push
the resulting tree T back on the stack.
We repeat this until the expression E has been processed, at which time the top
element on the stack is the expression tree for E. The total running time is O(n).
An implementation of this algorithm is given in Code Fragment 8.38 in the form
of a stand-alone function named build expression tree, which produces and returns
an appropriate ExpressionTree instance, assuming the input has been tokenized.
1 def build expression tree(tokens):
2 ”””Returns an ExpressionTree based upon by a tokenized expression.”””
3 S = [ ] # we use Python list as stack
4 for t in tokens:
5 if t in +-x*/ : # t is an operator symbol
6 S.append(t) # push the operator symbol
7 elif t not in () : # consider t to be a literal
8 S.append(ExpressionTree(t)) # push trivial tree storing value
9 elif t == ) : # compose a new tree from three constituent parts
10 right = S.pop( ) # right subtree as per LIFO
11 op = S.pop( ) # operator symbol
12 left = S.pop( ) # left subtree
13 S.append(ExpressionTree(op, left, right)) # repush tree
14 # we ignore a left parenthesis
15 return S.pop()
Code Fragment 8.38: Implementation of a build expression tree that produces an
ExpressionTree from a sequence of tokens representing an arithmetic expression.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-373-2048.jpg)
![354 Chapter 8. Trees
R-8.22 In what order are positions visited during a postorder traversal of the tree
of Figure 8.8?
R-8.23 Let T be an ordered tree with more than one node. Is it possible that the
preorder traversal of T visits the nodes in the same order as the postorder
traversal of T? If so, give an example; otherwise, explain why this cannot
occur. Likewise, is it possible that the preorder traversal of T visits the
nodes in the reverse order of the postorder traversal of T? If so, give an
example; otherwise, explain why this cannot occur.
R-8.24 Answer the previous question for the case when T is a proper binary tree
with more than one node.
R-8.25 Consider the example of a breadth-first traversal given in Figure 8.17.
Using the annotated numbers from that figure, describe the contents of
the queue before each pass of the while loop in Code Fragment 8.14. To
get started, the queue has contents {1} before the first pass, and contents
{2,3,4} before the second pass.
R-8.26 The collections.deque class supports an extend method that adds a col-
lection of elements to the end of the queue at once. Reimplement the
breadthfirst method of the Tree class to take advantage of this feature.
R-8.27 Give the output of the function parenthesize(T, T.root()), as described
in Code Fragment 8.25, when T is the tree of Figure 8.8.
R-8.28 What is the running time of parenthesize(T, T.root()), as given in Code
Fragment 8.25, for a tree T with n nodes?
R-8.29 Describe, in pseudo-code, an algorithm for computing the number of de-
scendants of each node of a binary tree. The algorithm should be based
on the Euler tour traversal.
R-8.30 The build expression tree method of the ExpressionTree class requires
input that is an iterable of string tokens. We used a convenient exam-
ple, (((3+1)x4)/((9-5)+2)) , in which each character is its own to-
ken, so that the string itself sufficed as input to build expression tree.
In general, a string, such as (35 + 14) , must be explicitly tokenized
into list [ ( , 35 , + , 14 , ) ] so as to ignore whitespace and to
recognize multidigit numbers as a single token. Write a utility method,
tokenize(raw), that returns such a list of tokens for a raw string.
Creativity
C-8.31 Define the internal path length, I(T), of a tree T to be the sum of the
depths of all the internal positions in T. Likewise, define the external path
length, E(T), of a tree T to be the sum of the depths of all the external
positions in T. Show that if T is a proper binary tree with n positions, then
E(T) = I(T)+n−1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-376-2048.jpg)
![Chapter Notes 361
P-8.68 Write a program that can play Tic-Tac-Toe effectively. (See Section 5.6.)
To do this, you will need to create a game tree T, which is a tree where
each position corresponds to a game configuration, which, in this case,
is a representation of the Tic-Tac-Toe board. (See Section 8.4.2.) The
root corresponds to the initial configuration. For each internal position p
in T, the children of p correspond to the game states we can reach from
p’s game state in a single legal move for the appropriate player, A (the
first player) or B (the second player). Positions at even depths correspond
to moves for A and positions at odd depths correspond to moves for B.
Leaves are either final game states or are at a depth beyond which we do
not want to explore. We score each leaf with a value that indicates how
good this state is for player A. In large games, like chess, we have to use a
heuristic scoring function, but for small games, like Tic-Tac-Toe, we can
construct the entire game tree and score leaves as +1, 0, −1, indicating
whether player A has a win, draw, or lose in that configuration. A good
algorithm for choosing moves is minimax. In this algorithm, we assign a
score to each internal position p in T, such that if p represents A’s turn, we
compute p’s score as the maximum of the scores of p’s children (which
corresponds to A’s optimal play from p). If an internal node p represents
B’s turn, then we compute p’s score as the minimum of the scores of p’s
children (which corresponds to B’s optimal play from p).
P-8.69 Implement the tree ADT using the binary tree representation described in
Exercise C-8.43. You may adapt the LinkedBinaryTree implementation.
P-8.70 Write a program that takes as input a general tree T and a position p of T
and converts T to another tree with the same set of position adjacencies,
but now with p as its root.
Chapter Notes
Discussions of the classic preorder, inorder, and postorder tree traversal methods can be
found in Knuth’s Fundamental Algorithms book [64]. The Euler tour traversal technique
comes from the parallel algorithms community; it is introduced by Tarjan and Vishkin [93]
and is discussed by JáJá [54] and by Karp and Ramachandran [58]. The algorithm for
drawing a tree is generally considered to be a part of the “folklore” of graph-drawing al-
gorithms. The reader interested in graph drawing is referred to the book by Di Battista,
Eades, Tamassia, and Tollis [34] and the survey by Tamassia and Liotta [92]. The puzzle
in Exercise R-8.12 was communicated by Micha Sharir.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-383-2048.jpg)
![9.3. Heaps 371
(14,E)
(5,A) (6,Z)
(20,B)
(7,Q)
(9,F)
(15,K)
(11,S)
(16,X) (25,J) (13,W)
(12,H)
(4,C)
Figure 9.1: Example of a heap storing 13 entries with integer keys. The last position
is the one storing entry (13,W).
The tree in Figure 9.1 is complete because levels 0, 1, and 2 are full, and the six
nodes in level 3 are in the six leftmost possible positions at that level. In formalizing
what we mean by the leftmost possible positions, we refer to the discussion of level
numbering from Section 8.3.2, in the context of an array-based representation of a
binary tree. (In fact, in Section 9.3.3 we will discuss the use of an array to represent
a heap.) A complete binary tree with n elements is one that has positions with level
numbering 0 through n − 1. For example, in an array-based representation of the
above tree, its 13 entries would be stored consecutively from A[0] to A[12].
The Height of a Heap
Let h denote the height of T. Insisting that T be complete also has an important
consequence, as shown in Proposition 9.2.
Proposition 9.2: A heap T storing n entries has height h = logn.
Justification: From the fact that T is complete, we know that the number of
nodes in levels 0 through h−1 of T is precisely 1+2+4+···+2h−1 = 2h −1, and
that the number of nodes in level h is at least 1 and at most 2h. Therefore
n ≥ 2h
−1+1 = 2h
and n ≤ 2h
−1+2h
= 2h+1
−1.
By taking the logarithm of both sides of inequality 2h ≤ n, we see that height
h ≤ logn. By rearranging terms and taking the logarithm of both sides of inequality
n ≤ 2h+1 − 1, we see that log(n + 1) − 1 ≤ h. Since h is an integer, these two
inequalities imply that h = logn.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-393-2048.jpg)
![376 Chapter 9. Priority Queues
9.3.3 Array-Based Representation of a Complete Binary Tree
The array-based representation of a binary tree (Section 8.3.2) is especially suitable
for a complete binary tree T. We recall that in this implementation, the elements
of T are stored in an array-based list A such that the element at position p in T is
stored in A with index equal to the level number f(p) of p, defined as follows:
• If p is the root of T, then f(p) = 0.
• If p is the left child of position q, then f(p) = 2f(q)+1.
• If p is the right child of position q, then f(p) = 2f(q)+2.
With this implementation, the elements of T have contiguous indices in the range
[0,n−1] and the last position of T is always at index n−1, where n is the number
of positions of T. For example, Figure 9.4 illustrates the array-based representation
of the heap structure originally portrayed in Figure 9.1.
0 1 2 3 4 5 6 7 8 9 10 11 12
(4,C) (6,Z) (15,K) (9,F) (7,Q) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (8,W)
(5,A)
Figure 9.4: An array-based representation of the heap from Figure 9.1.
Implementing a priority queue using an array-based heap representation allows
us to avoid some complexities of a node-based tree structure. In particular, the add
and remove min operations of a priority queue both depend on locating the last
index of a heap of size n. With the array-based representation, the last position
is at index n− 1 of the array. Locating the last position of a complete binary tree
implemented with a linked structure requires more effort. (See Exercise C-9.34.)
If the size of a priority queue is not known in advance, use of an array-based
representation does introduce the need to dynamically resize the array on occasion,
as is done with a Python list. The space usage of such an array-based representation
of a complete binary tree with n nodes is O(n), and the time bounds of methods for
adding or removing elements become amortized. (See Section 5.3.1.)
9.3.4 Python Heap Implementation
We provide a Python implementation of a heap-based priority queue in Code Frag-
ments 9.4 and 9.5. We use an array-based representation, maintaining a Python list
of item composites. Although we do not formally use the binary tree ADT, Code
Fragment 9.4 includes nonpublic utility functions that compute the level numbering
of a parent or child of another. This allows us to describe the rest of our algorithms
using tree-like terminology of parent, left, and right. However, the relevant vari-
ables are integer indexes (not “position” objects). We use recursion to implement
the repetition in the upheap and downheap utilities.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-398-2048.jpg)
![9.3. Heaps 377
1 class HeapPriorityQueue(PriorityQueueBase): # base class defines Item
2 ”””A min-oriented priority queue implemented with a binary heap.”””
3 #------------------------------ nonpublic behaviors ------------------------------
4 def parent(self, j):
5 return (j−1) // 2
6
7 def left(self, j):
8 return 2 j + 1
9
10 def right(self, j):
11 return 2 j + 2
12
13 def has left(self, j):
14 return self. left(j) len(self. data) # index beyond end of list?
15
16 def has right(self, j):
17 return self. right(j) len(self. data) # index beyond end of list?
18
19 def swap(self, i, j):
20 ”””Swap the elements at indices i and j of array.”””
21 self. data[i], self. data[j] = self. data[j], self. data[i]
22
23 def upheap(self, j):
24 parent = self. parent(j)
25 if j 0 and self. data[j] self. data[parent]:
26 self. swap(j, parent)
27 self. upheap(parent) # recur at position of parent
28
29 def downheap(self, j):
30 if self. has left(j):
31 left = self. left(j)
32 small child = left # although right may be smaller
33 if self. has right(j):
34 right = self. right(j)
35 if self. data[right] self. data[left]:
36 small child = right
37 if self. data[small child] self. data[j]:
38 self. swap(j, small child)
39 self. downheap(small child) # recur at position of small child
Code Fragment 9.4: An implementation of a priority queue using an array-based
heap (continued in Code Fragment 9.5). The extends the PriorityQueueBase class
from Code Fragment 9.1.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-399-2048.jpg)
![378 Chapter 9. Priority Queues
40 #------------------------------ public behaviors ------------------------------
41 def init (self):
42 ”””Create a new empty Priority Queue.”””
43 self. data = [ ]
44
45 def len (self):
46 ”””Return the number of items in the priority queue.”””
47 return len(self. data)
48
49 def add(self, key, value):
50 ”””Add a key-value pair to the priority queue.”””
51 self. data.append(self. Item(key, value))
52 self. upheap(len(self. data) − 1) # upheap newly added position
53
54 def min(self):
55 ”””Return but do not remove (k,v) tuple with minimum key.
56
57 Raise Empty exception if empty.
58 ”””
59 if self.is empty():
60 raise Empty( Priority queue is empty. )
61 item = self. data[0]
62 return (item. key, item. value)
63
64 def remove min(self):
65 ”””Remove and return (k,v) tuple with minimum key.
66
67 Raise Empty exception if empty.
68 ”””
69 if self.is empty():
70 raise Empty( Priority queue is empty. )
71 self. swap(0, len(self. data) − 1) # put minimum item at the end
72 item = self. data.pop( ) # and remove it from the list;
73 self. downheap(0) # then fix new root
74 return (item. key, item. value)
Code Fragment 9.5: An implementation of a priority queue using an array-based
heap (continued from Code Fragment 9.4).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-400-2048.jpg)
![382 Chapter 9. Priority Queues
Python Implementation of a Bottom-Up Heap Construction
Implementing a bottom-up heap construction is quite easy, given the existence of
a “down-heap” utility function. The “merging” of two equally sized heaps that
are subtrees of a common position p, as described in the opening of this section,
can be accomplished simply by down-heaping p’s entry. For example, that is what
happened to the key 14 in going from Figure 9.5(f) to (g).
With our array-based representation of a heap, if we initially store all n items in
arbitrary order within the array, we can implement the bottom-up heap construction
process with a single loop that makes a call to downheap from each position of
the tree, as long as those calls are ordered starting with the deepest level and ending
with the root of the tree. In fact, that loop can start with the deepest nonleaf, since
there is no effect when down-heap is called at a leaf position.
In Code Fragment 9.6, we augment the original HeapPriorityQueue class from
Section 9.3.4 to provide support for the bottom-up construction of an initial col-
lection. We introduce a nonpublic utility method, heapify, that calls downheap
on each nonleaf position, beginning with the deepest and concluding with a call at
the root of the tree. We have redesigned the constructor of the class to accept an
optional parameter that can be any sequence of (k,v) tuples. Rather than initializing
self. data to an empty list, we use a list comprehension syntax (see Section 1.9.2)
to create an initial list of item composites based on the given contents. We de-
clare an empty sequence as the default parameter value so that the default syntax
HeapPriorityQueue() continues to result in an empty priority queue.
def init (self, contents=()):
”””Create a new priority queue.
By default, queue will be empty. If contents is given, it should be as an
iterable sequence of (k,v) tuples specifying the initial contents.
”””
self. data = [ self. Item(k,v) for k,v in contents ] # empty by default
if len(self. data) 1:
self. heapify()
def heapify(self):
start = self. parent(len(self) − 1) # start at PARENT of last leaf
for j in range(start, −1, −1): # going to and including the root
self. downheap(j)
Code Fragment 9.6: Revision to the HeapPriorityQueue class of Code Frag-
ments 9.4 and 9.5 to support a linear-time construction given an initial sequence
of entries.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-404-2048.jpg)
![384 Chapter 9. Priority Queues
9.3.7 Python’s heapq Module
Python’s standard distribution includes a heapq module that provides support for
heap-based priority queues. That module does not provide any priority queue class;
instead it provides functions that allow a standard Python list to be managed as a
heap. Its model is essentially the same as our own, with n elements stored in list
cells L[0] through L[n − 1], based on the level-numbering indices with the small-
est element at the root in L[0]. We note that heapq does not separately manage
associated values; elements serve as their own key.
The heapq module supports the following functions, all of which presume that
existing list L satisfies the heap-order property prior to the call:
heappush(L, e): Push element e onto list L and restore the heap-order
property. The function executes in O(logn) time.
heappop(L): Pop and return the element with smallest value from list L,
and reestablish the heap-order property. The operation
executes in O(logn) time.
heappushpop(L, e): Push element e on list L and then pop and return the
smallest item. The time is O(logn), but it is slightly more
efficient than separate calls to push and pop because the
size of the list never changes. If the newly pushed el-
ement becomes the smallest, it is immediately returned.
Otherwise, the new element takes the place of the popped
element at the root and a down-heap is performed.
heapreplace(L, e): Similar to heappushpop, but equivalent to the pop be-
ing performed before the push (in other words, the new
element cannot be returned as the smallest). Again, the
time is O(logn), but it is more efficient that two separate
operations.
The module supports additional functions that operate on sequences that do not
previously satisfy the heap-order property.
heapify(L): Transform unordered list to satisfy the heap-order prop-
erty. This executes in O(n) time by using the bottom-up
construction algorithm.
nlargest(k, iterable): Produce a list of the k largest values from a given iterable.
This can be implemented to run in O(n + klogn) time,
where we use n to denote the length of the iterable (see
Exercise C-9.42).
nsmallest(k, iterable): Produce a list of the k smallest values from a given it-
erable. This can be implemented to run in O(n+klogn)
time, using similar technique as with nlargest.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-406-2048.jpg)
![9.5. Adaptable Priority Queues 393
ride the remove min method because the only change in behavior for the adaptable
priority queue is again provided by the overridden swap method.
The update and remove methods provide the core new functionality for the
adaptable priority queue. We perform robust checking of the validity of a locator
that is sent by a caller (although in the interest of space, our displayed code does
not do preliminary type-checking to ensure that the parameter is indeed a Locator
instance). To ensure that a locator is associated with a current element of the given
priority queue, we examine the index that is encapsulated within the locator object,
and then verify that the entry of the list at that index is the very same locator.
In conclusion, the adaptable priority queue provides the same asymptotic effi-
ciency and space usage as the nonadaptive version, and provides logarithmic per-
formance for the new locator-based update and remove methods. A summary of
the performance is given in Table 9.4.
1 class AdaptableHeapPriorityQueue(HeapPriorityQueue):
2 ”””A locator-based priority queue implemented with a binary heap.”””
3
4 #------------------------------ nested Locator class ------------------------------
5 class Locator(HeapPriorityQueue. Item):
6 ”””Token for locating an entry of the priority queue.”””
7 slots = _index # add index as additional field
8
9 def init (self, k, v, j):
10 super(). init (k,v)
11 self. index = j
12
13 #------------------------------ nonpublic behaviors ------------------------------
14 # override swap to record new indices
15 def swap(self, i, j):
16 super(). swap(i,j) # perform the swap
17 self. data[i]. index = i # reset locator index (post-swap)
18 self. data[j]. index = j # reset locator index (post-swap)
19
20 def bubble(self, j):
21 if j 0 and self. data[j] self. data[self. parent(j)]:
22 self. upheap(j)
23 else:
24 self. downheap(j)
Code Fragment 9.8: An implementation of an adaptable priority queue (continued
in Code Fragment 9.9). This extends the HeapPriorityQueue class of Code Frag-
ments 9.4 and 9.5](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-415-2048.jpg)
![394 Chapter 9. Priority Queues
25 def add(self, key, value):
26 ”””Add a key-value pair.”””
27 token = self.Locator(key, value, len(self. data)) # initiaize locator index
28 self. data.append(token)
29 self. upheap(len(self. data) − 1)
30 return token
31
32 def update(self, loc, newkey, newval):
33 ”””Update the key and value for the entry identified by Locator loc.”””
34 j = loc. index
35 if not (0 = j len(self) and self. data[j] is loc):
36 raise ValueError( Invalid locator )
37 loc. key = newkey
38 loc. value = newval
39 self. bubble(j)
40
41 def remove(self, loc):
42 ”””Remove and return the (k,v) pair identified by Locator loc.”””
43 j = loc. index
44 if not (0 = j len(self) and self. data[j] is loc):
45 raise ValueError( Invalid locator )
46 if j == len(self) − 1: # item at last position
47 self. data.pop( ) # just remove it
48 else:
49 self. swap(j, len(self)−1) # swap item to the last position
50 self. data.pop( ) # remove it from the list
51 self. bubble(j) # fix item displaced by the swap
52 return (loc. key, loc. value)
Code Fragment 9.9: An implementation of an adaptable priority queue (continued
from Code Fragment 9.8).
Operation Running Time
len(P), P.is empty(), P.min() O(1)
P.add(k,v) O(logn)∗
P.update(loc, k, v) O(logn)
P.remove(loc) O(logn)∗
P.remove min() O(logn)∗
∗amortized with dynamic array
Table 9.4: Running times of the methods of an adaptable priority queue, P, of size n,
realized by means of our array-based heap representation. The space requirement
is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-416-2048.jpg)
![400 Chapter 9. Priority Queues
P-9.55 Write a program that can process a sequence of stock buy and sell orders
as described in Exercise C-9.50.
P-9.56 Let S be a set of n points in the plane with distinct integer x- and y-
coordinates. Let T be a complete binary tree storing the points from S
at its external nodes, such that the points are ordered left to right by in-
creasing x-coordinates. For each node v in T, let S(v) denote the subset of
S consisting of points stored in the subtree rooted at v. For the root r of
T, define top(r) to be the point in S = S(r) with maximum y-coordinate.
For every other node v, define top(r) to be the point in S with highest y-
coordinate in S(v) that is not also the highest y-coordinate in S(u), where
u is the parent of v in T (if such a point exists). Such labeling turns T into
a priority search tree. Describe a linear-time algorithm for turning T into
a priority search tree. Implement this approach.
P-9.57 One of the main applications of priority queues is in operating systems—
for scheduling jobs on a CPU. In this project you are to build a program
that schedules simulated CPU jobs. Your program should run in a loop,
each iteration of which corresponds to a time slice for the CPU. Each job
is assigned a priority, which is an integer between −20 (highest priority)
and 19 (lowest priority), inclusive. From among all jobs waiting to be pro-
cessed in a time slice, the CPU must work on a job with highest priority.
In this simulation, each job will also come with a length value, which is an
integer between 1 and 100, inclusive, indicating the number of time slices
that are needed to process this job. For simplicity, you may assume jobs
cannot be interrupted—once it is scheduled on the CPU, a job runs for a
number of time slices equal to its length. Your simulator must output the
name of the job running on the CPU in each time slice and must process
a sequence of commands, one per time slice, each of which is of the form
“add job name with length n and priority p” or “no new job this slice”.
P-9.58 Develop a Python implementation of an adaptable priority queue that is
based on an unsorted list and supports location-aware entries.
Chapter Notes
Knuth’s book on sorting and searching [65] describes the motivation and history for the
selection-sort, insertion-sort, and heap-sort algorithms. The heap-sort algorithm is due
to Williams [103], and the linear-time heap construction algorithm is due to Floyd [39].
Additional algorithms and analyses for heaps and heap-sort variations can be found in
papers by Bentley [15], Carlsson [24], Gonnet and Munro [45], McDiarmid and Reed [74],
and Schaffer and Sedgewick [88].](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-422-2048.jpg)
![402 Chapter 10. Maps, Hash Tables, and Skip Lists
10.1 Maps and Dictionaries
Python’s dict class is arguably the most significant data structure in the language. It
represents an abstraction known as a dictionary in which unique keys are mapped
to associated values. Because of the relationship they express between keys and
values, dictionaries are commonly known as associative arrays or maps. In this
book, we use the term dictionary when specifically discussing Python’s dict class,
and the term map when discussing the more general notion of the abstract data type.
As a simple example, Figure 10.1 illustrates a map from the names of countries
to their associated units of currency.
Rupee
Turkey Spain China United States India
Greece
Lira Euro Yuan Dollar
Figure 10.1: A map from countries (the keys) to their units of currency (the values).
We note that the keys (the country names) are assumed to be unique, but the values
(the currency units) are not necessarily unique. For example, we note that Spain
and Greece both use the euro for currency. Maps use an array-like syntax for in-
dexing, such as currency[ Greece ] to access a value associated with a given key
or currency[ Greece ] = Drachma to remap it to a new value. Unlike a stan-
dard array, indices for a map need not be consecutive nor even numeric. Common
applications of maps include the following.
• A university’s information system relies on some form of a student ID as a
key that is mapped to that student’s associated record (such as the student’s
name, address, and course grades) serving as the value.
• The domain-name system (DNS) maps a host name, such as www.wiley.com,
to an Internet-Protocol (IP) address, such as 208.215.179.146.
• A social media site typically relies on a (nonnumeric) username as a key that
can be efficiently mapped to a particular user’s associated information.
• A computer graphics system may map a color name, such as turquoise ,
to the triple of numbers that describes the color’s RGB (red-green-blue) rep-
resentation, such as (64,224,208).
• Python uses a dictionary to represent each namespace, mapping an identifying
string, such as pi , to an associated object, such as 3.14159.
In this chapter and the next we demonstrate that a map may be implemented so
that a search for a key, and its associated value, can be performed very efficiently,
thereby supporting fast lookup in such applications.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-424-2048.jpg)
![10.1. Maps and Dictionaries 403
10.1.1 The Map ADT
In this section, we introduce the map ADT, and define its behaviors to be consistent
with those of Python’s built-in dict class. We begin by listing what we consider the
most significant five behaviors of a map M as follows:
M[k]: Return the value v associated with key k in map M, if
one exists; otherwise raise a KeyError. In Python, this is
implemented with the special method getitem .
M[k] = v: Associate value v with key k in map M, replacing the ex-
isting value if the map already contains an item with key
equal to k. In Python, this is implemented with the special
method setitem .
del M[k]: Remove from map M the item with key equal to k; if M
has no such item, then raise a KeyError. In Python, this is
implemented with the special method delitem .
len(M): Return the number of items in map M. In Python, this is
implemented with the special method len .
iter(M): The default iteration for a map generates a sequence of
keys in the map. In Python, this is implemented with the
special method iter , and it allows loops of the form,
for k in M.
We have highlighted the above five behaviors because they demonstrate the core
functionality of a map—namely, the ability to query, add, modify, or delete a key-
value pair, and the ability to report all such pairs. For additional convenience,
map M should also support the following behaviors:
k in M: Return True if the map contains an item with key k. In
Python, this is implemented with the special contains
method.
M.get(k, d=None): Return M[k] if key k exists in the map; otherwise return
default value d. This provides a form to query M[k] with-
out risk of a KeyError.
M.setdefault(k, d): If key k exists in the map, simply return M[k]; if key k
does not exist, set M[k] = d and return that value.
M.pop(k, d=None): Remove the item associated with key k from the map and
return its associated value v. If key k is not in the map,
return default value d (or raise KeyError if parameter d is
None).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-425-2048.jpg)
![404 Chapter 10. Maps, Hash Tables, and Skip Lists
M.popitem(): Remove an arbitrary key-value pair from the map, and re-
turn a (k,v) tuple representing the removed pair. If map is
empty, raise a KeyError.
M.clear(): Remove all key-value pairs from the map.
M.keys(): Return a set-like view of all keys of M.
M.values(): Return a set-like view of all values of M.
M.items(): Return a set-like view of (k,v) tuples for all entries of M.
M.update(M2): Assign M[k] = v for every (k,v) pair in map M2.
M == M2: Return True if maps M and M2 have identical key-value
associations.
M != M2: Return True if maps M and M2 do not have identical key-
value associations.
Example 10.1: In the following, we show the effect of a series of operations on
an initially empty map storing items with integer keys and single-character values.
We use the literal syntax for Python’s dict class to describe the map contents.
Operation Return Value Map
len(M) 0 { }
M[ K ] = 2 – { K : 2}
M[ B ] = 4 – { K : 2, B : 4}
M[ U ] = 2 – { K : 2, B : 4, U : 2}
M[ V ] = 8 – { K : 2, B : 4, U : 2, V : 8}
M[ K ] = 9 – { K : 9, B : 4, U : 2, V : 8}
M[ B ] 4 { K : 9, B : 4, U : 2, V : 8}
M[ X ] KeyError { K : 9, B : 4, U : 2, V : 8}
M.get( F ) None { K : 9, B : 4, U : 2, V : 8}
M.get( F , 5) 5 { K : 9, B : 4, U : 2, V : 8}
M.get( K , 5) 9 { K : 9, B : 4, U : 2, V : 8}
len(M) 4 { K : 9, B : 4, U : 2, V : 8}
del M[ V ] – { K : 9, B : 4, U : 2}
M.pop( K ) 9 { B : 4, U : 2}
M.keys() B , U { B : 4, U : 2}
M.values() 4, 2 { B : 4, U : 2}
M.items() ( B , 4), ( U , 2) { B : 4, U : 2}
M.setdefault( B , 1) 4 { B : 4, U : 2}
M.setdefault( A , 1) 1 { A : 1, B : 4, U : 2}
M.popitem() ( B , 4) { A : 1, U : 2}](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-426-2048.jpg)
![10.1. Maps and Dictionaries 405
10.1.2 Application: Counting Word Frequencies
As a case study for using a map, consider the problem of counting the number
of occurrences of words in a document. This is a standard task when performing a
statistical analysis of a document, for example, when categorizing an email or news
article. A map is an ideal data structure to use here, for we can use words as keys
and word counts as values. We show such an application in Code Fragment 10.1.
We break apart the original document using a combination of file and string
methods that results in a loop over a lowercased version of all whitespace separated
pieces of the document. We omit all nonalphabetic characters so that parentheses,
apostrophes, and other such punctuation are not considered part of a word.
In terms of map operations, we begin with an empty Python dictionary named
freq. During the first phase of the algorithm, we execute the command
freq[word] = 1 + freq.get(word, 0)
for each word occurrence. We use the get method on the right-hand side because the
current word might not exist in the dictionary; the default value of 0 is appropriate
in that case.
During the second phase of the algorithm, after the full document has been pro-
cessed, we examine the contents of the frequency map, looping over freq.items()
to determine which word has the most occurrences.
1 freq = { }
2 for piece in open(filename).read().lower().split():
3 # only consider alphabetic characters within this piece
4 word = .join(c for c in piece if c.isalpha())
5 if word: # require at least one alphabetic character
6 freq[word] = 1 + freq.get(word, 0)
7
8 max word =
9 max count = 0
10 for (w,c) in freq.items(): # (key, value) tuples represent (word, count)
11 if c max count:
12 max word = w
13 max count = c
14 print( The most frequent word is , max word)
15 print( Its number of occurrences is , max count)
Code Fragment 10.1: A program for counting word frequencies in a document, and
reporting the most frequent word. We use Python’s dict class for the map. We
convert the input to lowercase and ignore any nonalphabetic characters.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-427-2048.jpg)
![406 Chapter 10. Maps, Hash Tables, and Skip Lists
10.1.3 Python’s MutableMapping Abstract Base Class
Section 2.4.3 provides an introduction to the concept of an abstract base class
and the role of such classes in Python’s collections module. Methods that are de-
clared to be abstract in such a base class must be implemented by concrete sub-
classes. However, an abstract base class may provide concrete implementation of
other methods that depend upon use of the presumed abstract methods. (This is an
example of the template method design pattern.)
The collections module provides two abstract base classes that are relevant to
our current discussion: the Mapping and MutableMapping classes. The Mapping
class includes all nonmutating methods supported by Python’s dict class, while the
MutableMapping class extends that to include the mutating methods. What we
define as the map ADT in Section 10.1.1 is akin to the MutableMapping abstract
base class in Python’s collections module.
The significance of these abstract base classes is that they provide a framework
to assist in creating a user-defined map class. In particular, the MutableMapping
class provides concrete implementations for all behaviors other than the first five
outlined in Section 10.1.1: getitem , setitem , delitem , len , and
iter . As we implement the map abstraction with various data structures, as
long as we provide the five core behaviors, we can inherit all other derived behav-
iors by simply declaring MutableMapping as a parent class.
To better understand the MutableMapping class, we provide a few examples of
how concrete behaviors can be derived from the five core abstractions. For example,
the contains method, supporting the syntax k in M, could be implemented by
making a guarded attempt to retrieve self[k] to determine if the key exists.
def contains (self, k):
try:
self[k] # access via getitem (ignore result)
return True
except KeyError:
return False # attempt failed
A similar approach might be used to provide the logic of the setdefault method.
def setdefault(self, k, d):
try:
return self[k] # if getitem succeeds, return value
except KeyError: # otherwise:
self[k] = d # set default value with setitem
return d # and return that newly assigned value
We leave as exercises the implementations of the remaining concrete methods of
the MutableMapping class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-428-2048.jpg)
![10.1. Maps and Dictionaries 409
1 class UnsortedTableMap(MapBase):
2 ”””Map implementation using an unordered list.”””
3
4 def init (self):
5 ”””Create an empty map.”””
6 self. table = [ ] # list of Item’s
7
8 def getitem (self, k):
9 ”””Return value associated with key k (raise KeyError if not found).”””
10 for item in self. table:
11 if k == item. key:
12 return item. value
13 raise KeyError( Key Error: + repr(k))
14
15 def setitem (self, k, v):
16 ”””Assign value v to key k, overwriting existing value if present.”””
17 for item in self. table:
18 if k == item. key: # Found a match:
19 item. value = v # reassign value
20 return # and quit
21 # did not find match for key
22 self. table.append(self. Item(k,v))
23
24 def delitem (self, k):
25 ”””Remove item associated with key k (raise KeyError if not found).”””
26 for j in range(len(self. table)):
27 if k == self. table[j]. key: # Found a match:
28 self. table.pop(j) # remove item
29 return # and quit
30 raise KeyError( Key Error: + repr(k))
31
32 def len (self):
33 ”””Return number of items in the map.”””
34 return len(self. table)
35
36 def iter (self):
37 ”””Generate iteration of the map s keys.”””
38 for item in self. table:
39 yield item. key # yield the KEY
Code Fragment 10.3: An implementation of a map using a Python list as an unsorted
table. Parent class MapBase is given in Code Fragment 10.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-431-2048.jpg)
![410 Chapter 10. Maps, Hash Tables, and Skip Lists
10.2 Hash Tables
In this section, we introduce one of the most practical data structures for imple-
menting a map, and the one that is used by Python’s own implementation of the
dict class. This structure is known as a hash table.
Intuitively, a map M supports the abstraction of using keys as indices with a
syntax such as M[k]. As a mental warm-up, consider a restricted setting in which
a map with n items uses keys that are known to be integers in a range from 0 to
N −1 for some N ≥ n. In this case, we can represent the map using a lookup table
of length N, as diagrammed in Figure 10.3.
0 1 2 3 4 5 6 7 8 9 10
D Z C Q
Figure 10.3: A lookup table with length 11 for a map containing items (1,D), (3,Z),
(6,C), and (7,Q).
In this representation, we store the value associated with key k at index k of the
table (presuming that we have a distinct way to represent an empty slot). Basic map
operations of getitem , setitem , and delitem can be implemented in
O(1) worst-case time.
There are two challenges in extending this framework to the more general set-
ting of a map. First, we may not wish to devote an array of length N if it is the case
that N n. Second, we do not in general require that a map’s keys be integers.
The novel concept for a hash table is the use of a hash function to map general
keys to corresponding indices in a table. Ideally, keys will be well distributed in the
range from 0 to N −1 by a hash function, but in practice there may be two or more
distinct keys that get mapped to the same index. As a result, we will conceptualize
our table as a bucket array, as shown in Figure 10.4, in which each bucket may
manage a collection of items that are sent to a specific index by the hash function.
(To save space, an empty bucket may be replaced by None.)
0 1 2 3 4 5 6 7 8 9 10
(1,D) (25,C)
(3,F)
(14,Z)
(39,C)
(6,A) (7,Q)
Figure 10.4: A bucket array of capacity 11 with items (1,D), (25,C), (3,F), (14,Z),
(6,A), (39,C), and (7,Q), using a simple hash function.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-432-2048.jpg)
![10.2. Hash Tables 411
10.2.1 Hash Functions
The goal of a hash function, h, is to map each key k to an integer in the range
[0,N − 1], where N is the capacity of the bucket array for a hash table. Equipped
with such a hash function, h, the main idea of this approach is to use the hash
function value, h(k), as an index into our bucket array, A, instead of the key k
(which may not be appropriate for direct use as an index). That is, we store the
item (k,v) in the bucket A[h(k)].
If there are two or more keys with the same hash value, then two different items
will be mapped to the same bucket in A. In this case, we say that a collision has
occurred. To be sure, there are ways of dealing with collisions, which we will
discuss later, but the best strategy is to try to avoid them in the first place. We say
that a hash function is “good” if it maps the keys in our map so as to sufficiently
minimize collisions. For practical reasons, we also would like a hash function to
be fast and easy to compute.
It is common to view the evaluation of a hash function, h(k), as consisting of
two portions—a hash code that maps a key k to an integer, and a compression
function that maps the hash code to an integer within a range of indices, [0,N −1],
for a bucket array. (See Figure 10.5.)
-1
hash code
1 2
0
-2
... ...
compression function
1 2
0 N-1
...
Arbitrary Objects
Figure 10.5: Two parts of a hash function: a hash code and a compression function.
The advantage of separating the hash function into two such components is that
the hash code portion of that computation is independent of a specific hash table
size. This allows the development of a general hash code for each object that can
be used for a hash table of any size; only the compression function depends upon
the table size. This is particularly convenient, because the underlying bucket array
for a hash table may be dynamically resized, depending on the number of items
currently stored in the map. (See Section 10.2.3.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-433-2048.jpg)
![412 Chapter 10. Maps, Hash Tables, and Skip Lists
Hash Codes
The first action that a hash function performs is to take an arbitrary key k in our
map and compute an integer that is called the hash code for k; this integer need not
be in the range [0,N −1], and may even be negative. We desire that the set of hash
codes assigned to our keys should avoid collisions as much as possible. For if the
hash codes of our keys cause collisions, then there is no hope for our compression
function to avoid them. In this subsection, we begin by discussing the theory of
hash codes. Following that, we discuss practical implementations of hash codes in
Python.
Treating the Bit Representation as an Integer
To begin, we note that, for any data type X that is represented using at most as many
bits as our integer hash codes, we can simply take as a hash code for X an integer
interpretation of its bits. For example, the hash code for key 314 could simply be
314. The hash code for a floating-point number such as 3.14 could be based upon
an interpretation of the bits of the floating-point representation as an integer.
For a type whose bit representation is longer than a desired hash code, the above
scheme is not immediately applicable. For example, Python relies on 32-bit hash
codes. If a floating-point number uses a 64-bit representation, its bits cannot be
viewed directly as a hash code. One possibility is to use only the high-order 32 bits
(or the low-order 32 bits). This hash code, of course, ignores half of the information
present in the original key, and if many of the keys in our map only differ in these
bits, then they will collide using this simple hash code.
A better approach is to combine in some way the high-order and low-order por-
tions of a 64-bit key to form a 32-bit hash code, which takes all the original bits
into consideration. A simple implementation is to add the two components as 32-
bit numbers (ignoring overflow), or to take the exclusive-or of the two components.
These approaches of combining components can be extended to any object x whose
binary representation can be viewed as an n-tuple (x0,x1,...,xn−1) of 32-bit inte-
gers, for example, by forming a hash code for x as ∑n−1
i=0 xi, or as x0 ⊕x1 ⊕···⊕xn−1,
where the ⊕ symbol represents the bitwise exclusive-or operation (which is ˆ in
Python).
Polynomial Hash Codes
The summation and exclusive-or hash codes, described above, are not good choices
for character strings or other variable-length objects that can be viewed as tuples of
the form (x0,x1,...,xn−1), where the order of the xi’s is significant. For example,
consider a 16-bit hash code for a character string s that sums the Unicode values
of the characters in s. This hash code unfortunately produces lots of unwanted](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-434-2048.jpg)
![416 Chapter 10. Maps, Hash Tables, and Skip Lists
Compression Functions
The hash code for a key k will typically not be suitable for immediate use with a
bucket array, because the integer hash code may be negative or may exceed the ca-
pacity of the bucket array. Thus, once we have determined an integer hash code for
a key object k, there is still the issue of mapping that integer into the range [0,N −1].
This computation, known as a compression function, is the second action per-
formed as part of an overall hash function. A good compression function is one
that minimizes the number of collisions for a given set of distinct hash codes.
The Division Method
A simple compression function is the division method, which maps an integer i to
i mod N,
where N, the size of the bucket array, is a fixed positive integer. Additionally, if we
take N to be a prime number, then this compression function helps “spread out” the
distribution of hashed values. Indeed, if N is not prime, then there is greater risk
that patterns in the distribution of hash codes will be repeated in the distribution of
hash values, thereby causing collisions. For example, if we insert keys with hash
codes {200,205,210,215,220,... ,600} into a bucket array of size 100, then each
hash code will collide with three others. But if we use a bucket array of size 101,
then there will be no collisions. If a hash function is chosen well, it should ensure
that the probability of two different keys getting hashed to the same bucket is 1/N.
Choosing N to be a prime number is not always enough, however, for if there is
a repeated pattern of hash codes of the form pN + q for several different p’s, then
there will still be collisions.
The MAD Method
A more sophisticated compression function, which helps eliminate repeated pat-
terns in a set of integer keys, is the Multiply-Add-and-Divide (or “MAD”) method.
This method maps an integer i to
[(ai+b) mod p] mod N,
where N is the size of the bucket array, p is a prime number larger than N, and a
and b are integers chosen at random from the interval [0, p− 1], with a 0. This
compression function is chosen in order to eliminate repeated patterns in the set of
hash codes and get us closer to having a “good” hash function, that is, one such that
the probability any two different keys collide is 1/N. This good behavior would be
the same as we would have if these keys were “thrown” into A uniformly at random.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-438-2048.jpg)
![10.2. Hash Tables 417
10.2.2 Collision-Handling Schemes
The main idea of a hash table is to take a bucket array, A, and a hash function, h, and
use them to implement a map by storing each item (k,v) in the “bucket” A[h(k)].
This simple idea is challenged, however, when we have two distinct keys, k1 and k2,
such that h(k1) = h(k2). The existence of such collisions prevents us from simply
inserting a new item (k,v) directly into the bucket A[h(k)]. It also complicates our
procedure for performing insertion, search, and deletion operations.
Separate Chaining
A simple and efficient way for dealing with collisions is to have each bucket A[ j]
store its own secondary container, holding items (k,v) such that h(k) = j. A natural
choice for the secondary container is a small map instance implemented using a list,
as described in Section 10.1.5. This collision resolution rule is known as separate
chaining, and is illustrated in Figure 10.6.
A
1 2 3 4 5 6 7 8 9 10
0 11 12
12
38
25
90
54
28
41
36
18 10
Figure 10.6: A hash table of size 13, storing 10 items with integer keys, with colli-
sions resolved by separate chaining. The compression function is h(k) = k mod 13.
For simplicity, we do not show the values associated with the keys.
In the worst case, operations on an individual bucket take time proportional to
the size of the bucket. Assuming we use a good hash function to index the n items
of our map in a bucket array of capacity N, the expected size of a bucket is n/N.
Therefore, if given a good hash function, the core map operations run in O( n/N).
The ratio λ = n/N, called the load factor of the hash table, should be bounded by
a small constant, preferably below 1. As long as λ is O(1), the core operations on
the hash table run in O(1) expected time.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-439-2048.jpg)
![418 Chapter 10. Maps, Hash Tables, and Skip Lists
Open Addressing
The separate chaining rule has many nice properties, such as affording simple im-
plementations of map operations, but it nevertheless has one slight disadvantage:
It requires the use of an auxiliary data structure—a list—to hold items with collid-
ing keys. If space is at a premium (for example, if we are writing a program for a
small handheld device), then we can use the alternative approach of always storing
each item directly in a table slot. This approach saves space because no auxiliary
structures are employed, but it requires a bit more complexity to deal with colli-
sions. There are several variants of this approach, collectively referred to as open
addressing schemes, which we discuss next. Open addressing requires that the load
factor is always at most 1 and that items are stored directly in the cells of the bucket
array itself.
Linear Probing and Its Variants
A simple method for collision handling with open addressing is linear probing.
With this approach, if we try to insert an item (k,v) into a bucket A[ j] that is already
occupied, where j = h(k), then we next try A[(j +1) mod N]. If A[(j +1) mod N]
is also occupied, then we try A[(j + 2) mod N], and so on, until we find an empty
bucket that can accept the new item. Once this bucket is located, we simply in-
sert the item there. Of course, this collision resolution strategy requires that we
change the implementation when searching for an existing key—the first step of all
getitem , setitem , or delitem operations. In particular, to attempt
to locate an item with key equal to k, we must examine consecutive slots, starting
from A[h(k)], until we either find an item with that key or we find an empty bucket.
(See Figure 10.7.) The name “linear probing” comes from the fact that accessing a
cell of the bucket array can be viewed as a “probe.”
26
1 2 3 4 5 6 7 8 9 10
0
New element with
key = 15 to be inserted
Must probe 4 times
before finding empty slot
5 37 16 21
13
Figure 10.7: Insertion into a hash table with integer keys using linear probing. The
hash function is h(k) = k mod 11. Values associated with keys are not shown.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-440-2048.jpg)
![10.2. Hash Tables 419
To implement a deletion, we cannot simply remove a found item from its slot
in the array. For example, after the insertion of key 15 portrayed in Figure 10.7,
if the item with key 37 were trivially deleted, a subsequent search for 15 would
fail because that search would start by probing at index 4, then index 5, and then
index 6, at which an empty cell is found. A typical way to get around this diffi-
culty is to replace a deleted item with a special “available” marker object. With
this special marker possibly occupying spaces in our hash table, we modify our
search algorithm so that the search for a key k will skip over cells containing the
available marker and continue probing until reaching the desired item or an empty
bucket (or returning back to where we started from). Additionally, our algorithm
for setitem should remember an available cell encountered during the search
for k, since this is a valid place to put a new item (k,v), if no existing item is found.
Although use of an open addressing scheme can save space, linear probing
suffers from an additional disadvantage. It tends to cluster the items of a map into
contiguous runs, which may even overlap (particularly if more than half of the cells
in the hash table are occupied). Such contiguous runs of occupied hash cells cause
searches to slow down considerably.
Another open addressing strategy, known as quadratic probing, iteratively tries
the buckets A[(h(k)+ f(i)) mod N], for i = 0,1,2,..., where f(i) = i2, until finding
an empty bucket. As with linear probing, the quadratic probing strategy compli-
cates the removal operation, but it does avoid the kinds of clustering patterns that
occur with linear probing. Nevertheless, it creates its own kind of clustering, called
secondary clustering, where the set of filled array cells still has a non-uniform
pattern, even if we assume that the original hash codes are distributed uniformly.
When N is prime and the bucket array is less than half full, the quadratic probing
strategy is guaranteed to find an empty slot. However, this guarantee is not valid
once the table becomes at least half full, or if N is not chosen as a prime number;
we explore the cause of this type of clustering in an exercise (C-10.36).
An open addressing strategy that does not cause clustering of the kind produced
by linear probing or the kind produced by quadratic probing is the double hashing
strategy. In this approach, we choose a secondary hash function, h, and if h maps
some key k to a bucket A[h(k)] that is already occupied, then we iteratively try
the buckets A[(h(k) + f(i)) mod N] next, for i = 1,2,3,..., where f(i) = i · h(k).
In this scheme, the secondary hash function is not allowed to evaluate to zero; a
common choice is h(k) = q−(k mod q), for some prime number q N. Also, N
should be a prime.
Another approach to avoid clustering with open addressing is to iteratively try
buckets A[(h(k) + f(i)) mod N] where f(i) is based on a pseudo-random number
generator, providing a repeatable, but somewhat arbitrary, sequence of subsequent
probes that depends upon bits of the original hash code. This is the approach cur-
rently used by Python’s dictionary class.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-441-2048.jpg)
![10.2. Hash Tables 421
Operation List Hash Table
expected worst case
getitem O(n) O(1) O(n)
setitem O(n) O(1) O(n)
delitem O(n) O(1) O(n)
len O(1) O(1) O(1)
iter O(n) O(n) O(n)
Table 10.2: Comparison of the running times of the methods of a map realized by
means of an unsorted list (as in Section 10.1.5) or a hash table. We let n denote
the number of items in the map, and we assume that the bucket array supporting
the hash table is maintained such that its capacity is proportional to the number of
items in the map.
In practice, hash tables are among the most efficient means for implementing
a map, and it is essentially taken for granted by programmers that their core oper-
ations run in constant time. Python’s dict class is implemented with hashing, and
the Python interpreter relies on dictionaries to retrieve an object that is referenced
by an identifier in a given namespace. (See Sections 1.10 and 2.5.) The basic com-
mand c = a + b involves two calls to getitem in the dictionary for the local
namespace to retrieve the values identified as a and b, and a call to setitem
to store the result associated with name c in that namespace. In our own algorithm
analysis, we simply presume that such dictionary operations run in constant time,
independent of the number of entries in the namespace. (Admittedly, the number
of entries in a typical namespace can almost surely be bounded by a constant.)
In a 2003 academic paper [31], researchers discuss the possibility of exploiting
a hash table’s worst-case performance to cause a denial-of-service (DoS) attack
of Internet technologies. For many published algorithms that compute hash codes,
they note that an attacker could precompute a very large number of moderate-length
strings that all hash to the identical 32-bit hash code. (Recall that by any of the
hashing schemes we describe, other than double hashing, if two keys are mapped
to the same hash code, they will be inseparable in the collision resolution.)
In late 2011, another team of researchers demonstrated an implementation of
just such an attack [61]. Web servers allow a series of key-value parameters to be
embedded in a URL using a syntax such as ?key1=val1key2=val2key3=val3.
Typically, those key-value pairs are immediately stored in a map by the server,
and a limit is placed on the length and number of such parameters presuming that
storage time in the map will be linear in the number of entries. If all keys were
to collide, that storage requires quadratic time (causing the server to perform an
inordinate amount of work). In spring of 2012, Python developers distributed a
security patch that introduces randomization into the computation of hash codes
for strings, making it less tractable to reverse engineer a set of colliding strings.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-443-2048.jpg)
![10.2. Hash Tables 423
1 class HashMapBase(MapBase):
2 ”””Abstract base class for map using hash-table with MAD compression.”””
3
4 def init (self, cap=11, p=109345121):
5 ”””Create an empty hash-table map.”””
6 self. table = cap [ None ]
7 self. n = 0 # number of entries in the map
8 self. prime = p # prime for MAD compression
9 self. scale = 1 + randrange(p−1) # scale from 1 to p-1 for MAD
10 self. shift = randrange(p) # shift from 0 to p-1 for MAD
11
12 def hash function(self, k):
13 return (hash(k) self. scale + self. shift) % self. prime % len(self. table)
14
15 def len (self):
16 return self. n
17
18 def getitem (self, k):
19 j = self. hash function(k)
20 return self. bucket getitem(j, k) # may raise KeyError
21
22 def setitem (self, k, v):
23 j = self. hash function(k)
24 self. bucket setitem(j, k, v) # subroutine maintains self. n
25 if self. n len(self. table) // 2: # keep load factor = 0.5
26 self. resize(2 len(self. table) − 1) # number 2ˆx - 1 is often prime
27
28 def delitem (self, k):
29 j = self. hash function(k)
30 self. bucket delitem(j, k) # may raise KeyError
31 self. n −= 1
32
33 def resize(self, c): # resize bucket array to capacity c
34 old = list(self.items()) # use iteration to record existing items
35 self. table = c [None] # then reset table to desired capacity
36 self. n = 0 # n recomputed during subsequent adds
37 for (k,v) in old:
38 self[k] = v # reinsert old key-value pair
Code Fragment 10.4: A base class for our hash table implementations, extending
our MapBase class from Code Fragment 10.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-445-2048.jpg)
![424 Chapter 10. Maps, Hash Tables, and Skip Lists
Separate Chaining
Code Fragment 10.5 provides a concrete implementation of a hash table with sepa-
rate chaining, in the form of the ChainHashMap class. To represent a single bucket,
it relies on an instance of the UnsortedTableMap class from Code Fragment 10.3.
The first three methods in the class use index j to access the potential bucket in
the bucket array, and a check for the special case in which that table entry is None.
The only time we need a new bucket structure is when bucket setitem is called on
an otherwise empty slot. The remaining functionality relies on map behaviors that
are already supported by the individual UnsortedTableMap instances. We need a
bit of forethought to determine whether the application of setitem on the chain
causes a net increase in the size of the map (that is, whether the given key is new).
1 class ChainHashMap(HashMapBase):
2 ”””Hash map implemented with separate chaining for collision resolution.”””
3
4 def bucket getitem(self, j, k):
5 bucket = self. table[j]
6 if bucket is None:
7 raise KeyError( Key Error: + repr(k)) # no match found
8 return bucket[k] # may raise KeyError
9
10 def bucket setitem(self, j, k, v):
11 if self. table[j] is None:
12 self. table[j] = UnsortedTableMap( ) # bucket is new to the table
13 oldsize = len(self. table[j])
14 self. table[j][k] = v
15 if len(self. table[j]) oldsize: # key was new to the table
16 self. n += 1 # increase overall map size
17
18 def bucket delitem(self, j, k):
19 bucket = self. table[j]
20 if bucket is None:
21 raise KeyError( Key Error: + repr(k)) # no match found
22 del bucket[k] # may raise KeyError
23
24 def iter (self):
25 for bucket in self. table:
26 if bucket is not None: # a nonempty slot
27 for key in bucket:
28 yield key
Code Fragment 10.5: Concrete hash map class with separate chaining.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-446-2048.jpg)
![10.2. Hash Tables 425
Linear Probing
Our implementation of a ProbeHashMap class, using open addressing with linear
probing, is given in Code Fragments 10.6 and 10.7. In order to support deletions,
we use a technique described in Section 10.2.2 in which we place a special marker
in a table location at which an item has been deleted, so that we can distinguish
between it and a location that has always been empty. In our implementation, we
declare a class-level attribute, AVAIL, as a sentinel. (We use an instance of the
built-in object class because we do not care about any behaviors of the sentinel,
just our ability to differentiate it from other objects.)
The most challenging aspect of open addressing is to properly trace the series
of probes when collisions occur during an insertion or search for an item. To this
end, we define a nonpublic utility, find slot, that searches for an item with key k
in “bucket” j (that is, where j is the index returned by the hash function for key k).
1 class ProbeHashMap(HashMapBase):
2 ”””Hash map implemented with linear probing for collision resolution.”””
3 AVAIL = object( ) # sentinal marks locations of previous deletions
4
5 def is available(self, j):
6 ”””Return True if index j is available in table.”””
7 return self. table[j] is None or self. table[j] is ProbeHashMap. AVAIL
8
9 def find slot(self, j, k):
10 ”””Search for key k in bucket at index j.
11
12 Return (success, index) tuple, described as follows:
13 If match was found, success is True and index denotes its location.
14 If no match found, success is False and index denotes first available slot.
15 ”””
16 firstAvail = None
17 while True:
18 if self. is available(j):
19 if firstAvail is None:
20 firstAvail = j # mark this as first avail
21 if self. table[j] is None:
22 return (False, firstAvail) # search has failed
23 elif k == self. table[j]. key:
24 return (True, j) # found a match
25 j = (j + 1) % len(self. table) # keep looking (cyclically)
Code Fragment 10.6: Concrete ProbeHashMap class that uses linear probing for
collision resolution (continued in Code Fragment 10.7).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-447-2048.jpg)
![426 Chapter 10. Maps, Hash Tables, and Skip Lists
26 def bucket getitem(self, j, k):
27 found, s = self. find slot(j, k)
28 if not found:
29 raise KeyError( Key Error: + repr(k)) # no match found
30 return self. table[s]. value
31
32 def bucket setitem(self, j, k, v):
33 found, s = self. find slot(j, k)
34 if not found:
35 self. table[s] = self. Item(k,v) # insert new item
36 self. n += 1 # size has increased
37 else:
38 self. table[s]. value = v # overwrite existing
39
40 def bucket delitem(self, j, k):
41 found, s = self. find slot(j, k)
42 if not found:
43 raise KeyError( Key Error: + repr(k)) # no match found
44 self. table[s] = ProbeHashMap. AVAIL # mark as vacated
45
46 def iter (self):
47 for j in range(len(self. table)): # scan entire table
48 if not self. is available(j):
49 yield self. table[j]. key
Code Fragment 10.7: Concrete ProbeHashMap class that uses linear probing for
collision resolution (continued from Code Fragment 10.6).
The three primary map operations each rely on the find slot utility. When at-
tempting to retrieve the value associated with a given key, we must continue probing
until we find the key, or until we reach a table slot with the None value. We cannot
stop the search upon reaching an AVAIL sentinel, because it represents a location
that may have been filled when the desired item was once inserted.
When a key-value pair is being assigned in the map, we must attempt to find
an existing item with the given key, so that we might overwrite its value, before
adding a new item to the map. Therefore, we must search beyond any occurrences
of the AVAIL sentinel when inserting. However, if no match is found, we prefer to
repurpose the first slot marked with AVAIL, if any, when placing the new element
in the table. The find slot method enacts this logic, continuing the search until a
truly empty slot, but returning the index of the first available slot for an insertion.
When deleting an existing item within bucket delitem, we intentionally set
the table entry to the AVAIL sentinel in accordance with our strategy.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-448-2048.jpg)
![430 Chapter 10. Maps, Hash Tables, and Skip Lists
1 class SortedTableMap(MapBase):
2 ”””Map implementation using a sorted table.”””
3
4 #----------------------------- nonpublic behaviors -----------------------------
5 def find index(self, k, low, high):
6 ”””Return index of the leftmost item with key greater than or equal to k.
7
8 Return high + 1 if no such item qualifies.
9
10 That is, j will be returned such that:
11 all items of slice table[low:j] have key k
12 all items of slice table[j:high+1] have key = k
13 ”””
14 if high low:
15 return high + 1 # no element qualifies
16 else:
17 mid = (low + high) // 2
18 if k == self. table[mid]. key:
19 return mid # found exact match
20 elif k self. table[mid]. key:
21 return self. find index(k, low, mid − 1) # Note: may return mid
22 else:
23 return self. find index(k, mid + 1, high) # answer is right of mid
24
25 #----------------------------- public behaviors -----------------------------
26 def init (self):
27 ”””Create an empty map.”””
28 self. table = [ ]
29
30 def len (self):
31 ”””Return number of items in the map.”””
32 return len(self. table)
33
34 def getitem (self, k):
35 ”””Return value associated with key k (raise KeyError if not found).”””
36 j = self. find index(k, 0, len(self. table) − 1)
37 if j == len(self. table) or self. table[j]. key != k:
38 raise KeyError( Key Error: + repr(k))
39 return self. table[j]. value
Code Fragment 10.8: An implementation of a SortedTableMap class (continued in
Code Fragments 10.9 and 10.10).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-452-2048.jpg)
![10.3. Sorted Maps 431
40 def setitem (self, k, v):
41 ”””Assign value v to key k, overwriting existing value if present.”””
42 j = self. find index(k, 0, len(self. table) − 1)
43 if j len(self. table) and self. table[j]. key == k:
44 self. table[j]. value = v # reassign value
45 else:
46 self. table.insert(j, self. Item(k,v)) # adds new item
47
48 def delitem (self, k):
49 ”””Remove item associated with key k (raise KeyError if not found).”””
50 j = self. find index(k, 0, len(self. table) − 1)
51 if j == len(self. table) or self. table[j]. key != k:
52 raise KeyError( Key Error: + repr(k))
53 self. table.pop(j) # delete item
54
55 def iter (self):
56 ”””Generate keys of the map ordered from minimum to maximum.”””
57 for item in self. table:
58 yield item. key
59
60 def reversed (self):
61 ”””Generate keys of the map ordered from maximum to minimum.”””
62 for item in reversed(self. table):
63 yield item. key
64
65 def find min(self):
66 ”””Return (key,value) pair with minimum key (or None if empty).”””
67 if len(self. table) 0:
68 return (self. table[0]. key, self. table[0]. value)
69 else:
70 return None
71
72 def find max(self):
73 ”””Return (key,value) pair with maximum key (or None if empty).”””
74 if len(self. table) 0:
75 return (self. table[−1]. key, self. table[−1]. value)
76 else:
77 return None
Code Fragment 10.9: An implementation of a SortedTableMap class (together with
Code Fragments 10.8 and 10.10).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-453-2048.jpg)
![432 Chapter 10. Maps, Hash Tables, and Skip Lists
78 def find ge(self, k):
79 ”””Return (key,value) pair with least key greater than or equal to k.”””
80 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
81 if j len(self. table):
82 return (self. table[j]. key, self. table[j]. value)
83 else:
84 return None
85
86 def find lt(self, k):
87 ”””Return (key,value) pair with greatest key strictly less than k.”””
88 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
89 if j 0:
90 return (self. table[j−1]. key, self. table[j−1]. value) # Note use of j-1
91 else:
92 return None
93
94 def find gt(self, k):
95 ”””Return (key,value) pair with least key strictly greater than k.”””
96 j = self. find index(k, 0, len(self. table) − 1) # j s key = k
97 if j len(self. table) and self. table[j]. key == k:
98 j += 1 # advanced past match
99 if j len(self. table):
100 return (self. table[j]. key, self. table[j]. value)
101 else:
102 return None
103
104 def find range(self, start, stop):
105 ”””Iterate all (key,value) pairs such that start = key stop.
106
107 If start is None, iteration begins with minimum key of map.
108 If stop is None, iteration continues through the maximum key of map.
109 ”””
110 if start is None:
111 j = 0
112 else:
113 j = self. find index(start, 0, len(self. table)−1) # find first result
114 while j len(self. table) and (stop is None or self. table[j]. key stop):
115 yield (self. table[j]. key, self. table[j]. value)
116 j += 1
Code Fragment 10.10: An implementation of a SortedTableMap class (continued
from Code Fragments 10.9 and 10.10). We omit the find le method due to space.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-454-2048.jpg)
![10.3. Sorted Maps 433
Analysis
We conclude by analyzing the performance of our SortedTableMap implementa-
tion. A summary of the running times for all methods of the sorted map ADT
(including the traditional map operations) is given in Table 10.3. It should be clear
that the len , find min, and find max methods run in O(1) time, and that iter-
ating the keys of the table in either direction can be peformed in O(n) time.
The analysis for the various forms of search all depend on the fact that a binary
search on a table with n entries runs in O(logn) time. This claim was originally
shown as Proposition 4.2 in Section 4.2, and that analysis clearly applies to our
find index method as well. We therefore claim an O(logn) worst-case running
time for methods getitem , find lt, find gt, find le, and find ge. Each of these
makes a single call to find index, followed by a constant number of additional
steps to determine the appropriate answer based on the index. The analysis of
find range is a bit more interesting. It begins with a binary search to find the first
item within the range (if any). After that, it executes a loop that takes O(1) time per
iteration to report subsequent values until reaching the end of the range. If there are
s items reported in the range, the total running time is O(s+logn).
In contrast to the efficient search operations, update operations for a sorted table
may take considerable time. Although binary search can help identify the index at
which an update occurs, both insertions and deletions require, in the worst case, that
linearly many existing elements be shifted in order to maintain the sorted order of
the table. Specifically, the potential call to table.insert from within setitem
and table.pop from within delitem lead to O(n) worst-case time. (See the
discussion of corresponding operations of the list class in Section 5.4.1.)
In conclusion, sorted tables are primarily used in situations where we expect
many searches but relatively few updates.
Operation Running Time
len(M) O(1)
k in M O(logn)
M[k] = v O(n) worst case; O(logn) if existing k
del M[k] O(n) worst case
M.find min(), M.find max() O(1)
M.find lt(k), M.find gt(k)
O(logn)
M.find le(k), M.find ge(k)
M.find range(start, stop) O(s+logn) where s items are reported
iter(M), reversed(M) O(n)
Table 10.3: Performance of a sorted map, as implemented with SortedTableMap.
We use n to denote the number of items in the map at the time the operation is
performed. The space requirement is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-455-2048.jpg)
![436 Chapter 10. Maps, Hash Tables, and Skip Lists
Maintaining a Maxima Set with a Sorted Map
We can store the set of maxima pairs in a sorted map, M, so that the cost is the
key field and performance (speed) is the value field. We can then implement opera-
tions add(c, p), which adds a new cost-performance pair (c, p), and best(c), which
returns the best pair with cost at most c, as shown in Code Fragment 10.11.
1 class CostPerformanceDatabase:
2 ”””Maintain a database of maximal (cost,performance) pairs.”””
3
4 def init (self):
5 ”””Create an empty database.”””
6 self. M = SortedTableMap( ) # or a more efficient sorted map
7
8 def best(self, c):
9 ”””Return (cost,performance) pair with largest cost not exceeding c.
10
11 Return None if there is no such pair.
12 ”””
13 return self. M.find le(c)
14
15 def add(self, c, p):
16 ”””Add new entry with cost c and performance p.”””
17 # determine if (c,p) is dominated by an existing pair
18 other = self. M.find le(c) # other is at least as cheap as c
19 if other is not None and other[1] = p: # if its performance is as good,
20 return # (c,p) is dominated, so ignore
21 self. M[c] = p # else, add (c,p) to database
22 # and now remove any pairs that are dominated by (c,p)
23 other = self. M.find gt(c) # other more expensive than c
24 while other is not None and other[1] = p:
25 del self. M[other[0]]
26 other = self. M.find gt(c)
Code Fragment 10.11: An implementation of a class maintaining a set of maxima
cost-performance pairs using a sorted map.
Unfortunately, if we implement M using the SortedTableMap, the add behavior
has O(n) worst-case running time. If, on the other hand, we implement M using
a skip list, which we next describe, we can perform best(c) queries in O(logn)
expected time and add(c, p) updates in O((1+ r)logn) expected time, where r is
the number of points removed.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-458-2048.jpg)
![10.4. Skip Lists 439
10.4.1 Search and Update Operations in a Skip List
The skip-list structure affords simple map search and update algorithms. In fact,
all of the skip-list search and update algorithms are based on an elegant SkipSearch
method that takes a key k and finds the position p of the item in list S0 that has the
largest key less than or equal to k (which is possibly −∞).
Searching in a Skip List
Suppose we are given a search key k. We begin the SkipSearch method by setting
a position variable p to the topmost, left position in the skip list S, called the start
position of S. That is, the start position is the position of Sh storing the special
entry with key −∞. We then perform the following steps (see Figure 10.11), where
key(p) denotes the key of the item at position p:
1. If S.below(p) is None, then the search terminates—we are at the bottom and
have located the item in S with the largest key less than or equal to the search
key k. Otherwise, we drop down to the next lower level in the present tower
by setting p = S.below(p).
2. Starting at position p, we move p forward until it is at the rightmost position
on the present level such that key(p) ≤ k. We call this the scan forward step.
Note that such a position always exists, since each level contains the keys
+∞ and −∞. It may be that p remains where it started after we perform
such a forward scan for this level.
3. Return to step 1.
55
S1
S2
S3
S4
S5
+∞
+∞
+∞
+∞
+∞
+∞
-∞
-∞
-∞ 12
12
-∞
17
17 25
25
20
17 31 38 39
-∞
-∞ 17
17 25
25 31
31 38 44
44 50
55
55
55
S0
Figure 10.11: Example of a search in a skip list. The positions examined when
searching for key 50 are highlighted.
We give a pseudo-code description of the skip-list search algorithm, SkipSearch,
in Code Fragment 10.12. Given this method, the map operation M[k] is performed
by computing p = SkipSearch(k) and testing whether or not key(p) = k. If these
two keys are equal, we return the associated value; otherwise, we raise a KeyError.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-461-2048.jpg)
![440 Chapter 10. Maps, Hash Tables, and Skip Lists
Algorithm SkipSearch(k):
Input: A search key k
Output: Position p in the bottom list S0 with the largest key such that key(p) ≤ k
p = start {begin at start position}
while below(p) = None do
p = below(p) {drop down}
while k ≥ key(next(p)) do
p = next(p) {scan forward}
return p.
Code Fragment 10.12: Algorthm to search a skip list S for key k.
As it turns out, the expected running time of algorithm SkipSearch on a skip list
with n entries is O(logn). We postpone the justification of this fact, however, until
after we discuss the implementation of the update methods for skip lists. Navigation
starting at the position identified by SkipSearch(k) can be easily used to provide the
additional forms of searches in the sorted map ADT (e.g., find gt, find range).
Insertion in a Skip List
The execution of the map operation M[k] = v begins with a call to SkipSearch(k).
This gives us the position p of the bottom-level item with the largest key less than or
equal to k (note that p may hold the special item with key −∞). If key(p) = k, the
associated value is overwritten with v. Otherwise, we need to create a new tower for
item (k,v). We insert (k,v) immediately after position p within S0. After inserting
the new item at the bottom level, we use randomization to decide the height of the
tower for the new item. We “flip” a coin, and if the flip comes up tails, then we stop
here. Else (the flip comes up heads), we backtrack to the previous (next higher)
level and insert (k,v) in this level at the appropriate position. We again flip a coin;
if it comes up heads, we go to the next higher level and repeat. Thus, we continue
to insert the new item (k,v) in lists until we finally get a flip that comes up tails.
We link together all the references to the new item (k,v) created in this process to
create its tower. A coin flip can be simulated with Python’s built-in pseudo-random
number generator from the random module by calling randrange(2), which returns
0 or 1, each with probability 1/2.
We give the insertion algorithm for a skip list S in Code Fragment 10.13 and
we illustrate it in Figure 10.12. The algorithm uses an insertAfterAbove(p,q,(k,v))
method that inserts a position storing the item (k,v) after position p (on the same
level as p) and above position q, returning the new position r (and setting internal
references so that next, prev, above, and below methods will work correctly for p,
q, and r). The expected running time of the insertion algorithm on a skip list with
n entries is O(logn), which we show in Section 10.4.2.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-462-2048.jpg)
![442 Chapter 10. Maps, Hash Tables, and Skip Lists
Removal in a Skip List
Like the search and insertion algorithms, the removal algorithm for a skip list is
quite simple. In fact, it is even easier than the insertion algorithm. That is, to per-
form the map operation del M[k] we begin by executing method SkipSearch(k).
If the position p stores an entry with key different from k, we raise a KeyError.
Otherwise, we remove p and all the positions above p, which are easily accessed
by using above operations to climb up the tower of this entry in S starting at posi-
tion p. While removing levels of the tower, we reestablish links between the hor-
izontal neighbors of each removed position. The removal algorithm is illustrated
in Figure 10.13 and a detailed description of it is left as an exercise (R-10.24). As
we show in the next subsection, deletion operation in a skip list with n entries has
O(logn) expected running time.
Before we give this analysis, however, there are some minor improvements to
the skip-list data structure we would like to discuss. First, we do not actually need
to store references to values at the levels of the skip list above the bottom level,
because all that is needed at these levels are references to keys. In fact, we can
more efficiently represent a tower as a single object, storing the key-value pair,
and maintaining j previous references and j next references if the tower reaches
level Sj. Second, for the horizontal axes, it is possible to keep the list singly linked,
storing only the next references. We can perform insertions and removals in strictly
a top-down, scan-forward fashion. We explore the details of this optimization in
Exercise C-10.44. Neither of these optimizations improve the asymptotic perfor-
mance of skip lists by more than a constant factor, but these improvements can,
nevertheless, be meaningful in practice. In fact, experimental evidence suggests
that optimized skip lists are faster in practice than AVL trees and other balanced
search trees, which are discussed in Chapter 11.
31
S5
S4
S3
S2
S1
-∞
-∞
-∞ 12
12
-∞
17
17 25
25 31
31
42
55
50
55
+∞
+∞
+∞
+∞
+∞
-∞
-∞
17
38
38 39 42
42
42
44
44
55
55
+∞
17
17
20 25
25
S0
Figure 10.13: Removal of the entry with key 25 from the skip list of Figure 10.12.
The positions visited after the search for the position of S0 holding the entry are
highlighted. The positions removed are drawn with dashed lines.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-464-2048.jpg)
![10.4. Skip Lists 445
So we have yet to bound the number of scan-forward steps we make. Let ni be
the number of keys examined while scanning forward at level i. Observe that, after
the key at the starting position, each additional key examined in a scan-forward at
level i cannot also belong to level i + 1. If any of these keys were on the previous
level, we would have encountered them in the previous scan-forward step. Thus,
the probability that any key is counted in ni is 1/2. Therefore, the expected value of
ni is exactly equal to the expected number of times we must flip a fair coin before
it comes up heads. This expected value is 2. Hence, the expected amount of time
spent scanning forward at any level i is O(1). Since S has O(logn) levels with high
probability, a search in S takes expected time O(logn). By a similar analysis, we
can show that the expected running time of an insertion or a removal is O(logn).
Space Usage in a Skip List
Finally, let us turn to the space requirement of a skip list S with n entries. As we
observed above, the expected number of positions at level i is n/2i, which means
that the expected total number of positions in S is
h
∑
i=0
n
2i
= n
h
∑
i=0
1
2i
.
Using Proposition 3.5 on geometric summations, we have
h
∑
i=0
1
2i
=
1
2
h+1
−1
1
2 −1
= 2·
1−
1
2h+1
2 for all h ≥ 0.
Hence, the expected space requirement of S is O(n).
Table 10.4 summarizes the performance of a sorted map realized by a skip list.
Operation Running Time
len(M) O(1)
k in M O(logn) expected
M[k] = v O(logn) expected
del M[k] O(logn) expected
M.find min(), M.find max() O(1)
M.find lt(k), M.find gt(k)
O(logn) expected
M.find le(k), M.find ge(k)
M.find range(start, stop) O(s+logn) expected, with s items reported
iter(M), reversed(M) O(n)
Table 10.4: Performance of a sorted map implemented with a skip list. We use n to
denote the number of entries in the dictionary at the time the operation is performed.
The expected space requirement is O(n).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-467-2048.jpg)
![10.5. Sets, Multisets, and Multimaps 451
1 class MultiMap:
2 ”””A multimap class built upon use of an underlying map for storage.”””
3 MapType = dict # Map type; can be redefined by subclass
4
5 def init (self):
6 ”””Create a new empty multimap instance.”””
7 self. map = self. MapType( ) # create map instance for storage
8 self. n = 0
9
10 def iter (self):
11 ”””Iterate through all (k,v) pairs in multimap.”””
12 for k,secondary in self. map.items():
13 for v in secondary:
14 yield (k,v)
15
16 def add(self, k, v):
17 ”””Add pair (k,v) to multimap.”””
18 container = self. map.setdefault(k, [ ]) # create empty list, if needed
19 container.append(v)
20 self. n += 1
21
22 def pop(self, k):
23 ”””Remove and return arbitrary (k,v) with key k (or raise KeyError).”””
24 secondary = self. map[k] # may raise KeyError
25 v = secondary.pop()
26 if len(secondary) == 0:
27 del self. map[k] # no pairs left
28 self. n −= 1
29 return (k, v)
30
31 def find(self, k):
32 ”””Return arbitrary (k,v) pair with given key (or raise KeyError).”””
33 secondary = self. map[k] # may raise KeyError
34 return (k, secondary[0])
35
36 def find all(self, k):
37 ”””Generate iteration of all (k,v) pairs with given key.”””
38 secondary = self. map.get(k, [ ]) # empty list, by default
39 for v in secondary:
40 yield (k,v)
Code Fragment 10.17: An implementation of a MultiMap using a dict for storage.
The len method, which returns self. n, is omitted from this listing.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-473-2048.jpg)
![10.6. Exercises 453
R-10.12 What is the result of Exercise R-10.9 when collisions are handled by dou-
ble hashing using the secondary hash function h(k) = 7−(k mod 7)?
R-10.13 What is the worst-case time for putting n entries in an initially empty hash
table, with collisions resolved by chaining? What is the best case?
R-10.14 Show the result of rehashing the hash table shown in Figure 10.6 into a
table of size 19 using the new hash function h(k) = 3k mod 17.
R-10.15 Our HashMapBase class maintains a load factor λ ≤ 0.5. Reimplement
that class to allow the user to specify the maximum load, and adjust the
concrete subclasses accordingly.
R-10.16 Give a pseudo-code description of an insertion into a hash table that uses
quadratic probing to resolve collisions, assuming we also use the trick of
replacing deleted entries with a special “deactivated entry” object.
R-10.17 Modify our ProbeHashMap to use quadratic probing.
R-10.18 Explain why a hash table is not suited to implement a sorted map.
R-10.19 Describe how a sorted list implemented as a doubly linked list could be
used to implement the sorted map ADT.
R-10.20 What is the worst-case asymptotic running time for performing n deletions
from a SortedTableMap instance that initially contains 2n entries?
R-10.21 Consider the following variant of the find index method from Code Frag-
ment 10.8, in the context of the SortedTableMap class:
def find index(self, k, low, high):
if high low:
return high + 1
else:
mid = (low + high) // 2
if self. table[mid]. key k:
return self. find index(k, mid + 1, high)
else:
return self. find index(k, low, mid − 1)
Does this always produce the same result as the original version? Justify
your answer.
R-10.22 What is the expected running time of the methods for maintaining a max-
ima set if we insert n pairs such that each pair has lower cost and perfor-
mance than one before it? What is contained in the sorted map at the end
of this series of operations? What if each pair had a lower cost and higher
performance than the one before it?
R-10.23 Draw an example skip list S that results from performing the following
series of operations on the skip list shown in Figure 10.13: del S[38],
S[48] = x , S[24] = y , del S[55]. Record your coin flips, as well.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-475-2048.jpg)
![454 Chapter 10. Maps, Hash Tables, and Skip Lists
R-10.24 Give a pseudo-code description of the delitem map operation when
using a skip list.
R-10.25 Give a concrete implementation of the pop method, in the context of a
MutableSet abstract base class, that relies only on the five core set behav-
iors described in Section 10.5.2.
R-10.26 Give a concrete implementation of the isdisjoint method in the context
of the MutableSet abstract base class, relying only on the five primary
abstract methods of that class. Your algorithm should run in O(min(n,m))
where n and m denote the respective cardinalities of the two sets.
R-10.27 What abstraction would you use to manage a database of friends’ birth-
days in order to support efficient queries such as “find all friends whose
birthday is today” and “find the friend who will be the next to celebrate a
birthday”?
Creativity
C-10.28 On page 406 of Section 10.1.3, we give an implementation of the method
setdefault as it might appear in the MutableMapping abstract base class.
While that method accomplishes the goal in a general fashion, its effi-
ciency is less than ideal. In particular, when the key is new, there will be
a failed search due to the initial use of getitem , and then a subse-
quent insertion via setitem . For a concrete implementation, such as
the UnsortedTableMap, this is twice the work because a complete scan
of the table will take place during the failed getitem , and then an-
other complete scan of the table takes place due to the implementation of
setitem . A better solution is for the UnsortedTableMap class to over-
ride setdefault to provide a direct solution that performs a single search.
Give such an implementation of UnsortedTableMap.setdefault.
C-10.29 Repeat Exercise C-10.28 for the ProbeHashMap class.
C-10.30 Repeat Exercise C-10.28 for the ChainHashMap class.
C-10.31 For an ideal compression function, the capacity of the bucket array for a
hash table should be a prime number. Therefore, we consider the problem
of locating a prime number in a range [M,2M]. Implement a method for
finding such a prime by using the sieve algorithm. In this algorithm, we
allocate a 2M cell Boolean array A, such that cell i is associated with the
integer i. We then initialize the array cells to all be “true” and we “mark
off” all the cells that are multiples of 2, 3, 5, 7, and so on. This process
can stop after it reaches a number larger than
√
2M. (Hint: Consider a
bootstrapping method for finding the primes up to
√
2M.)](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-476-2048.jpg)
![10.6. Exercises 455
C-10.32 Perform experiments on our ChainHashMap and ProbeHashMap classes
to measure its efficiency using random key sets and varying limits on the
load factor (see Exercise R-10.15).
C-10.33 Our implementation of separate chaining in ChainHashMap conserves
memory by representing empty buckets in the table as None, rather than
as empty instances of a secondary structure. Because many of these buck-
ets will hold a single item, a better optimization is to have those slots of
the table directly reference the Item instance, and to reserve use of sec-
ondary containers for buckets that have two or more items. Modify our
implementation to provide this additional optimization.
C-10.34 Computing a hash code can be expensive, especially for lengthy keys. In
our hash table implementations, we compute the hash code when first in-
serting an item, and recompute each item’s hash code each time we resize
our table. Python’s dict class makes an interesting trade-off. The hash
code is computed once, when an item is inserted, and the hash code is
stored as an extra field of the item composite, so that it need not be recom-
puted. Reimplement our HashTableBase class to use such an approach.
C-10.35 Describe how to perform a removal from a hash table that uses linear
probing to resolve collisions where we do not use a special marker to
represent deleted elements. That is, we must rearrange the contents so that
it appears that the removed entry was never inserted in the first place.
C-10.36 The quadratic probing strategy has a clustering problem related to the way
it looks for open slots. Namely, when a collision occurs at bucket h(k), it
checks buckets A[(h(k)+i2) mod N], for i = 1,2,...,N −1.
a. Show that i2 mod N will assume at most (N + 1)/2 distinct values,
for N prime, as i ranges from 1 to N − 1. As a part of this justifica-
tion, note that i2 mod N = (N −i)2 mod N for all i.
b. A better strategy is to choose a prime N such that N mod 4 = 3 and
then to check the buckets A[(h(k) ± i2) mod N] as i ranges from 1
to (N − 1)/2, alternating between plus and minus. Show that this
alternate version is guaranteed to check every bucket in A.
C-10.37 Refactor our ProbeHashMap design so that the sequence of secondary
probes for collision resolution can be more easily customized. Demon-
strate your new framework by providing separate concrete subclasses for
linear probing and quadratic probing.
C-10.38 Design a variation of binary search for performing the multimap opera-
tion find all(k) implemented with a sorted search table that includes du-
plicates, and show that it runs in time O(s+logn), where n is the number
of elements in the dictionary and s is the number of items with given key k.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-477-2048.jpg)
![458 Chapter 10. Maps, Hash Tables, and Skip Lists
Chapter Notes
Hashing is a well-studied technique. The reader interested in further study is encouraged
to explore the book by Knuth [65], as well as the book by Vitter and Chen [100]. Skip
lists were introduced by Pugh [86]. Our analysis of skip lists is a simplification of a pre-
sentation given by Motwani and Raghavan [80]. For a more in-depth analysis of skip lists,
please see the various research papers on skip lists that have appeared in the data structures
literature [59, 81, 84]. Exercise C-10.36 was contributed by James Lee.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-480-2048.jpg)
![460 Chapter 11. Search Trees
11.1 Binary Search Trees
In Chapter 8 we introduced the tree data structure and demonstrated a variety of
applications. One important use is as a search tree (as described on page 332). In
this chapter, we use a search tree structure to efficiently implement a sorted map.
The three most fundamental methods of a map M (see Section 10.1.1) are:
M[k]: Return the value v associated with key k in map M, if one exists;
otherwise raise a KeyError; implemented with getitem method.
M[k] = v: Associate value v with key k in map M, replacing the existing value
if the map already contains an item with key equal to k; implemented
with setitem method.
del M[k]: Remove from map M the item with key equal to k; if M has no such
item, then raise a KeyError; implemented with delitem method.
The sorted map ADT includes additional functionality (see Section 10.3), guar-
anteeing that an iteration reports keys in sorted order, and supporting additional
searches such as find gt(k) and find range(start, stop).
Binary trees are an excellent data structure for storing items of a map, assuming
we have an order relation defined on the keys. In this context, a binary search tree
is a binary tree T with each position p storing a key-value pair (k,v) such that:
• Keys stored in the left subtree of p are less than k.
• Keys stored in the right subtree of p are greater than k.
An example of such a binary search tree is given in Figure 11.1. As a matter of
convenience, we will not diagram the values associated with keys in this chapter,
since those values do not affect the placement of items within a search tree.
80
82
54
28 93
65
32 97
44
88
17
8
76
29
Figure 11.1: A binary search tree with integer keys. We omit the display of associ-
ated values in this chapter, since they are not relevant to the order of items within a
search tree.](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-482-2048.jpg)
![11.1. Binary Search Trees 465
11.1.3 Insertions and Deletions
Algorithms for inserting or deleting entries of a binary search tree are fairly straight-
forward, although not trivial.
Insertion
The map command M[k] = v, as supported by the setitem method, begins
with a search for key k (assuming the map is nonempty). If found, that item’s
existing value is reassigned. Otherwise, a node for the new item can be inserted
into the underlying tree T in place of the empty subtree that was reached at the end
of the failed search. The binary search tree property is sustained by that placement
(note that it is placed exactly where a search would expect it). Pseudo-code for
such a TreeInsert algorithm is given in in Code Fragment 11.3.
Algorithm TreeInsert(T, k, v):
Input: A search key k to be associated with value v
p = TreeSearch(T,T.root(),k)
if k == p.key() then
Set p’s value to v
else if k p.key() then
add node with item (k,v) as left child of p
else
add node with item (k,v) as right child of p
Code Fragment 11.3: Algorithm for inserting a key-value pair into a map that is
represented as a binary search tree.
An example of insertion into a binary search tree is shown in Figure 11.4.
82
54
28 93
65
32 97
44
88
17
8
80
76
29
68
54
28 93
65
32 97
44
88
17
8
80
76
82
29
(a) (b)
Figure 11.4: Insertion of an item with key 68 into the search tree of Figure 11.2.
Finding the position to insert is shown in (a), and the resulting tree is shown in (b).](https://image.slidesharecdn.com/datastructuresandalgorithmsinpython-240604093259-ae9bf9ac/75/Data-structures-and-Algorithms-in-Python-pdf-487-2048.jpg)
More Related Content
![[pgday.Seoul 2022] POSTGRES 테스트코드로 기여하기 - 이동욱](https://cdn.slidesharecdn.com/ss_thumbnails/postgres-221114014538-b9df2ddf-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![[pgday.Seoul 2022] 서비스개편시 PostgreSQL 도입기 - 진소린 & 김태정](https://cdn.slidesharecdn.com/ss_thumbnails/postgresql-221121085744-f0fb1a8a-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Similar to Data structures and Algorithms in Python.pdf
![Table of contents [data structure and algorithmic thinking with python]](https://cdn.slidesharecdn.com/ss_thumbnails/tableofcontentsdatastructureandalgorithmicthinkingwithpython-150401104714-conversion-gate01-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
![Sample chapters [data structure and algorithmic thinking with python]](https://cdn.slidesharecdn.com/ss_thumbnails/samplechaptersdatastructureandalgorithmicthinkingwithpython-150401104825-conversion-gate01-thumbnail.jpg?width=600ounds&width=560&fit=bounds)
Data structures and Algorithms in Python.pdf
- 3. Data Structures and Algorithmsin Python Michael T. Goodrich Department of Computer Science University of California, Irvine Roberto Tamassia Department of Computer Science Brown University Michael H. Goldwasser Department of Mathematics and Computer Science Saint Louis University
- 4. VP & PUBLISHERDon Fowley EXECUTIVE EDITOR Beth Lang Golub EDITORIAL PROGRAM ASSISTANT Katherine Willis MARKETING MANAGER Christopher Ruel DESIGNER Kenji Ngieng SENIOR PRODUCTION MANAGER Janis Soo ASSOCIATE PRODUCTION MANAGER Joyce Poh This book was set in L aTEX by the authors. Printed and bound by Courier Westford. The cover was printed by Courier Westford. This book is printed on acid free paper. Founded in 1807, John Wiley & Sons, Inc. has been a valued source of knowledge and understanding for more than 200 years, helping people around the world meet their needs and fulfill their aspirations. Our company is built on a foundation of principles that include responsibility to the communities we serve and where we live and work. In 2008, we launched a Corporate Citizenship Initiative, a global effort to address the environmental, social, economic, and ethical challenges we face in our business. Among the issues we are addressing are carbon impact, paper specifications and procurement, ethical conduct within our business and among our vendors, and community and charitable support. For more information, please visit our website: www.wiley.com/go/citizenship. Copyright © 2013 John Wiley & Sons, Inc. All rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc. 222 Rosewood Drive, Danvers, MA 01923, website www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030-5774, (201)748-6011, fax (201)748-6008, website http://www.wiley.com/go/permissions. Evaluation copies are provided to qualified academics and professionals for review purposes only, for use in their courses during the next academic year. These copies are licensed and may not be sold or transferred to a third party. Upon completion of the review period, please return the evaluation copy to Wiley. Return instructions and a free of charge return mailing label are available at www.wiley.com/go/returnlabel. If you have chosen to adopt this textbook for use in your course, please accept this book as your complimentary desk copy. Outside of the United States, please contact your local sales representative. Printed in the United States of America 10 9 8 7 6 5 4 3 2 1
- 5. To Karen, Paul,Anna, and Jack – Michael T. Goodrich To Isabel – Roberto Tamassia To Susan, Calista, and Maya – Michael H. Goldwasser
- 7. Preface The design andanalysis of efficient data structures has long been recognized as a vital subject in computing and is part of the core curriculum of computer science and computer engineering undergraduate degrees. Data Structures and Algorithms in Python provides an introduction to data structures and algorithms, including their design, analysis, and implementation. This book is designed for use in a beginning- level data structures course, or in an intermediate-level introduction to algorithms course. We discuss its use for such courses in more detail later in this preface. To promote the development of robust and reusable software, we have tried to take a consistent object-oriented viewpoint throughout this text. One of the main ideas of the object-oriented approach is that data should be presented as being en- capsulated with the methods that access and modify them. That is, rather than simply viewing data as a collection of bytes and addresses, we think of data ob- jects as instances of an abstract data type (ADT), which includes a repertoire of methods for performing operations on data objects of this type. We then empha- size that there may be several different implementation strategies for a particular ADT, and explore the relative pros and cons of these choices. We provide complete Python implementations for almost all data structures and algorithms discussed, and we introduce important object-oriented design patterns as means to organize those implementations into reusable components. Desired outcomes for readers of our book include that: • They have knowledge of the most common abstractions for data collections (e.g., stacks, queues, lists, trees, maps). • They understand algorithmic strategies for producing efficient realizations of common data structures. • They can analyze algorithmic performance, both theoretically and experi- mentally, and recognize common trade-offs between competing strategies. • They can wisely use existing data structures and algorithms found in modern programming language libraries. • They have experience working with concrete implementations for most foun- dational data structures and algorithms. • They can apply data structures and algorithms to solve complex problems. In support of the last goal, we present many example applications of data structures throughout the book, including the processing of file systems, matching of tags in structured formats such as HTML, simple cryptography, text frequency analy- sis, automated geometric layout, Huffman coding, DNA sequence alignment, and search engine indexing. v
- 8. vi Preface Book Features Thisbook is based upon the book Data Structures and Algorithms in Java by Goodrich and Tamassia, and the related Data Structures and Algorithms in C++ by Goodrich, Tamassia, and Mount. However, this book is not simply a translation of those other books to Python. In adapting the material for this book, we have significantly redesigned the organization and content of the book as follows: • The code base has been entirely redesigned to take advantage of the features of Python, such as use of generators for iterating elements of a collection. • Many algorithms that were presented as pseudo-code in the Java and C++ versions are directly presented as complete Python code. • In general, ADTs are defined to have consistent interface with Python’s built- in data types and those in Python’s collections module. • Chapter 5 provides an in-depth exploration of the dynamic array-based un- derpinnings of Python’s built-in list, tuple, and str classes. New Appendix A serves as an additional reference regarding the functionality of the str class. • Over 450 illustrations have been created or revised. • New and revised exercises bring the overall total number to 750. Online Resources This book is accompanied by an extensive set of online resources, which can be found at the following Web site: www.wiley.com/college/goodrich Students are encouraged to use this site along with the book, to help with exer- cises and increase understanding of the subject. Instructors are likewise welcome to use the site to help plan, organize, and present their course materials. Included on this Web site is a collection of educational aids that augment the topics of this book, for both students and instructors. Because of their added value, some of these online resources are password protected. For all readers, and especially for students, we include the following resources: • All the Python source code presented in this book. • PDF handouts of Powerpoint slides (four-per-page) provided to instructors. • A database of hints to all exercises, indexed by problem number. For instructors using this book, we include the following additional teaching aids: • Solutions to hundreds of the book’s exercises. • Color versions of all figures and illustrations from the book. • Slides in Powerpoint and PDF (one-per-page) format. The slides are fully editable, so as to allow an instructor using this book full free- dom in customizing his or her presentations. All the online resources are provided at no extra charge to any instructor adopting this book for his or her course.
- 9. Preface vii Contents andOrganization The chapters for this book are organized to provide a pedagogical path that starts with the basics of Python programming and object-oriented design. We then add foundational techniques like algorithm analysis and recursion. In the main portion of the book, we present fundamental data structures and algorithms, concluding with a discussion of memory management (that is, the architectural underpinnings of data structures). Specifically, the chapters for this book are organized as follows: 1. Python Primer 2. Object-Oriented Programming 3. Algorithm Analysis 4. Recursion 5. Array-Based Sequences 6. Stacks, Queues, and Deques 7. Linked Lists 8. Trees 9. Priority Queues 10. Maps, Hash Tables, and Skip Lists 11. Search Trees 12. Sorting and Selection 13. Text Processing 14. Graph Algorithms 15. Memory Management and B-Trees A. Character Strings in Python B. Useful Mathematical Facts A more detailed table of contents follows this preface, beginning on page xi. Prerequisites We assume that the reader is at least vaguely familiar with a high-level program- ming language, such as C, C++, Python, or Java, and that he or she understands the main constructs from such a high-level language, including: • Variables and expressions. • Decision structures (such as if-statements and switch-statements). • Iteration structures (for loops and while loops). • Functions (whether stand-alone or object-oriented methods). For readers who are familiar with these concepts, but not with how they are ex- pressed in Python, we provide a primer on the Python language in Chapter 1. Still, this book is primarily a data structures book, not a Python book; hence, it does not give a comprehensive treatment of Python.
- 10. viii Preface We delaytreatment of object-oriented programming in Python until Chapter 2. This chapter is useful for those new to Python, and for those who may be familiar with Python, yet not with object-oriented programming. In terms of mathematical background, we assume the reader is somewhat famil- iar with topics from high-school mathematics. Even so, in Chapter 3, we discuss the seven most-important functions for algorithm analysis. In fact, sections that use something other than one of these seven functions are considered optional, and are indicated with a star (). We give a summary of other useful mathematical facts, including elementary probability, in Appendix B. Relation to Computer Science Curriculum To assist instructors in designing a course in the context of the IEEE/ACM 2013 Computing Curriculum, the following table describes curricular knowledge units that are covered within this book. Knowledge Unit Relevant Material AL/Basic Analysis Chapter 3 and Sections 4.2 12.2.4 AL/Algorithmic Strategies Sections 12.2.1, 13.2.1, 13.3, 13.4.2 AL/Fundamental Data Structures and Algorithms Sections 4.1.3, 5.5.2, 9.4.1, 9.3, 10.2, 11.1, 13.2, Chapter 12 much of Chapter 14 AL/Advanced Data Structures Sections 5.3, 10.4, 11.2 through 11.6, 12.3.1, 13.5, 14.5.1, 15.3 AR/Memory System Organization and Architecture Chapter 15 DS/Sets, Relations and Functions Sections 10.5.1, 10.5.2, 9.4 DS/Proof Techniques Sections 3.4, 4.2, 5.3.2, 9.3.6, 12.4.1 DS/Basics of Counting Sections 2.4.2, 6.2.2, 12.2.4, 8.2.2 Appendix B DS/Graphs and Trees Much of Chapters 8 and 14 DS/Discrete Probability Sections 1.11.1, 10.2, 10.4.2, 12.3.1 PL/Object-Oriented Programming Much of the book, yet especially Chapter 2 and Sections 7.4, 9.5.1, 10.1.3, 11.2.1 PL/Functional Programming Section 1.10 SDF/Algorithms and Design Sections 2.1, 3.3, 12.2.1 SDF/Fundamental Programming Concepts Chapters 1 4 SDF/Fundamental Data Structures Chapters 6 7, Appendix A, and Sections 1.2.1, 5.2, 5.4, 9.1, 10.1 SDF/Developmental Methods Sections 1.7 2.2 SE/Software Design Sections 2.1 2.1.3 Mapping IEEE/ACM 2013 Computing Curriculum knowledge units to coverage in this book.
- 11. Preface ix About theAuthors Michael Goodrich received his Ph.D. in Computer Science from Purdue University in 1987. He is currently a Chancellor’s Professor in the Department of Computer Science at University of California, Irvine. Previously, he was a professor at Johns Hopkins University. He is a Fulbright Scholar and a Fellow of the American As- sociation for the Advancement of Science (AAAS), Association for Computing Machinery (ACM), and Institute of Electrical and Electronics Engineers (IEEE). He is a recipient of the IEEE Computer Society Technical Achievement Award, the ACM Recognition of Service Award, and the Pond Award for Excellence in Undergraduate Teaching. Roberto Tamassia received his Ph.D. in Electrical and Computer Engineering from the University of Illinois at Urbana-Champaign in 1988. He is the Plastech Professor of Computer Science and the Chair of the Department of Computer Sci- ence at Brown University. He is also the Director of Brown’s Center for Geometric Computing. His research interests include information security, cryptography, anal- ysis, design, and implementation of algorithms, graph drawing and computational geometry. He is a Fellow of the American Association for the Advancement of Science (AAAS), Association for Computing Machinery (ACM) and Institute for Electrical and Electronic Engineers (IEEE). He is also a recipient of the Technical Achievement Award from the IEEE Computer Society. Michael Goldwasser received his Ph.D. in Computer Science from Stanford University in 1997. He is currently a Professor in the Department of Mathematics and Computer Science at Saint Louis University and the Director of their Com- puter Science program. Previously, he was a faculty member in the Department of Computer Science at Loyola University Chicago. His research interests focus on the design and implementation of algorithms, having published work involving approximation algorithms, online computation, computational biology, and compu- tational geometry. He is also active in the computer science education community. Additional Books by These Authors • M.T. Goodrich and R. Tamassia, Data Structures and Algorithms in Java, Wiley. • M.T. Goodrich, R. Tamassia, and D.M. Mount, Data Structures and Algorithms in C++, Wiley. • M.T. Goodrich and R. Tamassia, Algorithm Design: Foundations, Analysis, and Internet Examples, Wiley. • M.T. Goodrich and R. Tamassia, Introduction to Computer Security, Addison- Wesley. • M.H. Goldwasser and D. Letscher, Object-Oriented Programming in Python, Prentice Hall.
- 12. x Preface Acknowledgments We havedepended greatly upon the contributions of many individuals as part of the development of this book. We begin by acknowledging the wonderful team at Wiley. We are grateful to our editor, Beth Golub, for her enthusiastic support of this project, from beginning to end. The efforts of Elizabeth Mills and Katherine Willis were critical in keeping the project moving, from its early stages as an initial proposal, through the extensive peer review process. We greatly appreciate the attention to detail demonstrated by Julie Kennedy, the copyeditor for this book. Finally, many thanks are due to Joyce Poh for managing the final months of the production process. We are truly indebted to the outside reviewers and readers for their copious comments, emails, and constructive criticism, which were extremely useful in writ- ing this edition. We therefore thank the following reviewers for their comments and suggestions: Claude Anderson (Rose Hulman Institute of Technology), Alistair Campbell (Hamilton College), Barry Cohen (New Jersey Institute of Technology), Robert Franks (Central College), Andrew Harrington (Loyola University Chicago), Dave Musicant (Carleton College), and Victor Norman (Calvin College). We wish to particularly acknowledge Claude for going above and beyond the call of duty, providing us with an enumeration of 400 detailed corrections or suggestions. We thank David Mount, of University of Maryland, for graciously sharing the wisdom gained from his experience with the C++ version of this text. We are grate- ful to Erin Chambers and David Letscher, of Saint Louis University, for their intan- gible contributions during many hallway conversations about the teaching of data structures, and to David for comments on early versions of the Python code base for this book. We thank David Zampino, a student at Loyola University Chicago, for his feedback while using a draft of this book during an independent study course, and to Andrew Harrington for supervising David’s studies. We also wish to reiterate our thanks to the many research collaborators and teaching assistants whose feedback shaped the previous Java and C++ versions of this material. The benefits of those contributions carry forward to this book. Finally, we would like to warmly thank Susan Goldwasser, Isabel Cruz, Karen Goodrich, Giuseppe Di Battista, Franco Preparata, Ioannis Tollis, and our parents for providing advice, encouragement, and support at various stages of the prepa- ration of this book, and Calista and Maya Goldwasser for offering their advice regarding the artistic merits of many illustrations. More importantly, we thank all of these people for reminding us that there are things in life beyond writing books. Michael T. Goodrich Roberto Tamassia Michael H. Goldwasser
- 13. Contents Preface . .. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v 1 Python Primer 1 1.1 Python Overview . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 The Python Interpreter . . . . . . . . . . . . . . . . . . 2 1.1.2 Preview of a Python Program . . . . . . . . . . . . . . 3 1.2 Objects in Python . . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Identifiers, Objects, and the Assignment Statement . . . 4 1.2.2 Creating and Using Objects . . . . . . . . . . . . . . . . 6 1.2.3 Python’s Built-In Classes . . . . . . . . . . . . . . . . . 7 1.3 Expressions, Operators, and Precedence . . . . . . . . . . . 12 1.3.1 Compound Expressions and Operator Precedence . . . . 17 1.4 Control Flow . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.4.1 Conditionals . . . . . . . . . . . . . . . . . . . . . . . . 18 1.4.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.5 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.5.1 Information Passing . . . . . . . . . . . . . . . . . . . . 24 1.5.2 Python’s Built-In Functions . . . . . . . . . . . . . . . . 28 1.6 Simple Input and Output . . . . . . . . . . . . . . . . . . . . 30 1.6.1 Console Input and Output . . . . . . . . . . . . . . . . 30 1.6.2 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 1.7 Exception Handling . . . . . . . . . . . . . . . . . . . . . . . 33 1.7.1 Raising an Exception . . . . . . . . . . . . . . . . . . . 34 1.7.2 Catching an Exception . . . . . . . . . . . . . . . . . . 36 1.8 Iterators and Generators . . . . . . . . . . . . . . . . . . . . 39 1.9 Additional Python Conveniences . . . . . . . . . . . . . . . . 42 1.9.1 Conditional Expressions . . . . . . . . . . . . . . . . . . 42 1.9.2 Comprehension Syntax . . . . . . . . . . . . . . . . . . 43 1.9.3 Packing and Unpacking of Sequences . . . . . . . . . . 44 1.10 Scopes and Namespaces . . . . . . . . . . . . . . . . . . . . 46 1.11 Modules and the Import Statement . . . . . . . . . . . . . . 48 1.11.1 Existing Modules . . . . . . . . . . . . . . . . . . . . . 49 1.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 xi
- 14. xii Contents 2 Object-OrientedProgramming 56 2.1 Goals, Principles, and Patterns . . . . . . . . . . . . . . . . 57 2.1.1 Object-Oriented Design Goals . . . . . . . . . . . . . . 57 2.1.2 Object-Oriented Design Principles . . . . . . . . . . . . 58 2.1.3 Design Patterns . . . . . . . . . . . . . . . . . . . . . . 61 2.2 Software Development . . . . . . . . . . . . . . . . . . . . . 62 2.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 2.2.2 Pseudo-Code . . . . . . . . . . . . . . . . . . . . . . . 64 2.2.3 Coding Style and Documentation . . . . . . . . . . . . . 64 2.2.4 Testing and Debugging . . . . . . . . . . . . . . . . . . 67 2.3 Class Definitions . . . . . . . . . . . . . . . . . . . . . . . . . 69 2.3.1 Example: CreditCard Class . . . . . . . . . . . . . . . . 69 2.3.2 Operator Overloading and Python’s Special Methods . . 74 2.3.3 Example: Multidimensional Vector Class . . . . . . . . . 77 2.3.4 Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . 79 2.3.5 Example: Range Class . . . . . . . . . . . . . . . . . . . 80 2.4 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 2.4.1 Extending the CreditCard Class . . . . . . . . . . . . . . 83 2.4.2 Hierarchy of Numeric Progressions . . . . . . . . . . . . 87 2.4.3 Abstract Base Classes . . . . . . . . . . . . . . . . . . . 93 2.5 Namespaces and Object-Orientation . . . . . . . . . . . . . 96 2.5.1 Instance and Class Namespaces . . . . . . . . . . . . . . 96 2.5.2 Name Resolution and Dynamic Dispatch . . . . . . . . . 100 2.6 Shallow and Deep Copying . . . . . . . . . . . . . . . . . . . 101 2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 3 Algorithm Analysis 109 3.1 Experimental Studies . . . . . . . . . . . . . . . . . . . . . . 111 3.1.1 Moving Beyond Experimental Analysis . . . . . . . . . . 113 3.2 The Seven Functions Used in This Book . . . . . . . . . . . 115 3.2.1 Comparing Growth Rates . . . . . . . . . . . . . . . . . 122 3.3 Asymptotic Analysis . . . . . . . . . . . . . . . . . . . . . . . 123 3.3.1 The “Big-Oh” Notation . . . . . . . . . . . . . . . . . . 123 3.3.2 Comparative Analysis . . . . . . . . . . . . . . . . . . . 128 3.3.3 Examples of Algorithm Analysis . . . . . . . . . . . . . 130 3.4 Simple Justification Techniques . . . . . . . . . . . . . . . . 137 3.4.1 By Example . . . . . . . . . . . . . . . . . . . . . . . . 137 3.4.2 The “Contra” Attack . . . . . . . . . . . . . . . . . . . 137 3.4.3 Induction and Loop Invariants . . . . . . . . . . . . . . 138 3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
- 15. Contents xiii 4 Recursion148 4.1 Illustrative Examples . . . . . . . . . . . . . . . . . . . . . . 150 4.1.1 The Factorial Function . . . . . . . . . . . . . . . . . . 150 4.1.2 Drawing an English Ruler . . . . . . . . . . . . . . . . . 152 4.1.3 Binary Search . . . . . . . . . . . . . . . . . . . . . . . 155 4.1.4 File Systems . . . . . . . . . . . . . . . . . . . . . . . . 157 4.2 Analyzing Recursive Algorithms . . . . . . . . . . . . . . . . 161 4.3 Recursion Run Amok . . . . . . . . . . . . . . . . . . . . . . 165 4.3.1 Maximum Recursive Depth in Python . . . . . . . . . . 168 4.4 Further Examples of Recursion . . . . . . . . . . . . . . . . . 169 4.4.1 Linear Recursion . . . . . . . . . . . . . . . . . . . . . . 169 4.4.2 Binary Recursion . . . . . . . . . . . . . . . . . . . . . 174 4.4.3 Multiple Recursion . . . . . . . . . . . . . . . . . . . . 175 4.5 Designing Recursive Algorithms . . . . . . . . . . . . . . . . 177 4.6 Eliminating Tail Recursion . . . . . . . . . . . . . . . . . . . 178 4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180 5 Array-Based Sequences 183 5.1 Python’s Sequence Types . . . . . . . . . . . . . . . . . . . . 184 5.2 Low-Level Arrays . . . . . . . . . . . . . . . . . . . . . . . . . 185 5.2.1 Referential Arrays . . . . . . . . . . . . . . . . . . . . . 187 5.2.2 Compact Arrays in Python . . . . . . . . . . . . . . . . 190 5.3 Dynamic Arrays and Amortization . . . . . . . . . . . . . . . 192 5.3.1 Implementing a Dynamic Array . . . . . . . . . . . . . . 195 5.3.2 Amortized Analysis of Dynamic Arrays . . . . . . . . . . 197 5.3.3 Python’s List Class . . . . . . . . . . . . . . . . . . . . 201 5.4 Efficiency of Python’s Sequence Types . . . . . . . . . . . . 202 5.4.1 Python’s List and Tuple Classes . . . . . . . . . . . . . 202 5.4.2 Python’s String Class . . . . . . . . . . . . . . . . . . . 208 5.5 Using Array-Based Sequences . . . . . . . . . . . . . . . . . 210 5.5.1 Storing High Scores for a Game . . . . . . . . . . . . . 210 5.5.2 Sorting a Sequence . . . . . . . . . . . . . . . . . . . . 214 5.5.3 Simple Cryptography . . . . . . . . . . . . . . . . . . . 216 5.6 Multidimensional Data Sets . . . . . . . . . . . . . . . . . . 219 5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 6 Stacks, Queues, and Deques 228 6.1 Stacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 6.1.1 The Stack Abstract Data Type . . . . . . . . . . . . . . 230 6.1.2 Simple Array-Based Stack Implementation . . . . . . . . 231 6.1.3 Reversing Data Using a Stack . . . . . . . . . . . . . . 235 6.1.4 Matching Parentheses and HTML Tags . . . . . . . . . 236
- 16. xiv Contents 6.2 Queues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 6.2.1 The Queue Abstract Data Type . . . . . . . . . . . . . 240 6.2.2 Array-Based Queue Implementation . . . . . . . . . . . 241 6.3 Double-Ended Queues . . . . . . . . . . . . . . . . . . . . . . 247 6.3.1 The Deque Abstract Data Type . . . . . . . . . . . . . 247 6.3.2 Implementing a Deque with a Circular Array . . . . . . . 248 6.3.3 Deques in the Python Collections Module . . . . . . . . 249 6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250 7 Linked Lists 255 7.1 Singly Linked Lists . . . . . . . . . . . . . . . . . . . . . . . . 256 7.1.1 Implementing a Stack with a Singly Linked List . . . . . 261 7.1.2 Implementing a Queue with a Singly Linked List . . . . . 264 7.2 Circularly Linked Lists . . . . . . . . . . . . . . . . . . . . . . 266 7.2.1 Round-Robin Schedulers . . . . . . . . . . . . . . . . . 267 7.2.2 Implementing a Queue with a Circularly Linked List . . . 268 7.3 Doubly Linked Lists . . . . . . . . . . . . . . . . . . . . . . . 270 7.3.1 Basic Implementation of a Doubly Linked List . . . . . . 273 7.3.2 Implementing a Deque with a Doubly Linked List . . . . 275 7.4 The Positional List ADT . . . . . . . . . . . . . . . . . . . . 277 7.4.1 The Positional List Abstract Data Type . . . . . . . . . 279 7.4.2 Doubly Linked List Implementation . . . . . . . . . . . . 281 7.5 Sorting a Positional List . . . . . . . . . . . . . . . . . . . . 285 7.6 Case Study: Maintaining Access Frequencies . . . . . . . . 286 7.6.1 Using a Sorted List . . . . . . . . . . . . . . . . . . . . 286 7.6.2 Using a List with the Move-to-Front Heuristic . . . . . . 289 7.7 Link-Based vs. Array-Based Sequences . . . . . . . . . . . . 292 7.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 8 Trees 299 8.1 General Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . 300 8.1.1 Tree Definitions and Properties . . . . . . . . . . . . . . 301 8.1.2 The Tree Abstract Data Type . . . . . . . . . . . . . . 305 8.1.3 Computing Depth and Height . . . . . . . . . . . . . . . 308 8.2 Binary Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . 311 8.2.1 The Binary Tree Abstract Data Type . . . . . . . . . . . 313 8.2.2 Properties of Binary Trees . . . . . . . . . . . . . . . . 315 8.3 Implementing Trees . . . . . . . . . . . . . . . . . . . . . . . 317 8.3.1 Linked Structure for Binary Trees . . . . . . . . . . . . . 317 8.3.2 Array-Based Representation of a Binary Tree . . . . . . 325 8.3.3 Linked Structure for General Trees . . . . . . . . . . . . 327 8.4 Tree Traversal Algorithms . . . . . . . . . . . . . . . . . . . 328
- 17. Contents xv 8.4.1 Preorderand Postorder Traversals of General Trees . . . 328 8.4.2 Breadth-First Tree Traversal . . . . . . . . . . . . . . . 330 8.4.3 Inorder Traversal of a Binary Tree . . . . . . . . . . . . 331 8.4.4 Implementing Tree Traversals in Python . . . . . . . . . 333 8.4.5 Applications of Tree Traversals . . . . . . . . . . . . . . 337 8.4.6 Euler Tours and the Template Method Pattern . . . . 341 8.5 Case Study: An Expression Tree . . . . . . . . . . . . . . . . 348 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352 9 Priority Queues 362 9.1 The Priority Queue Abstract Data Type . . . . . . . . . . . 363 9.1.1 Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . 363 9.1.2 The Priority Queue ADT . . . . . . . . . . . . . . . . . 364 9.2 Implementing a Priority Queue . . . . . . . . . . . . . . . . 365 9.2.1 The Composition Design Pattern . . . . . . . . . . . . . 365 9.2.2 Implementation with an Unsorted List . . . . . . . . . . 366 9.2.3 Implementation with a Sorted List . . . . . . . . . . . . 368 9.3 Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 9.3.1 The Heap Data Structure . . . . . . . . . . . . . . . . . 370 9.3.2 Implementing a Priority Queue with a Heap . . . . . . . 372 9.3.3 Array-Based Representation of a Complete Binary Tree . 376 9.3.4 Python Heap Implementation . . . . . . . . . . . . . . . 376 9.3.5 Analysis of a Heap-Based Priority Queue . . . . . . . . . 379 9.3.6 Bottom-Up Heap Construction . . . . . . . . . . . . . 380 9.3.7 Python’s heapq Module . . . . . . . . . . . . . . . . . . 384 9.4 Sorting with a Priority Queue . . . . . . . . . . . . . . . . . 385 9.4.1 Selection-Sort and Insertion-Sort . . . . . . . . . . . . . 386 9.4.2 Heap-Sort . . . . . . . . . . . . . . . . . . . . . . . . . 388 9.5 Adaptable Priority Queues . . . . . . . . . . . . . . . . . . . 390 9.5.1 Locators . . . . . . . . . . . . . . . . . . . . . . . . . . 390 9.5.2 Implementing an Adaptable Priority Queue . . . . . . . 391 9.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395 10 Maps, Hash Tables, and Skip Lists 401 10.1 Maps and Dictionaries . . . . . . . . . . . . . . . . . . . . . 402 10.1.1 The Map ADT . . . . . . . . . . . . . . . . . . . . . . 403 10.1.2 Application: Counting Word Frequencies . . . . . . . . . 405 10.1.3 Python’s MutableMapping Abstract Base Class . . . . . 406 10.1.4 Our MapBase Class . . . . . . . . . . . . . . . . . . . . 407 10.1.5 Simple Unsorted Map Implementation . . . . . . . . . . 408 10.2 Hash Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . 410 10.2.1 Hash Functions . . . . . . . . . . . . . . . . . . . . . . 411
- 18. xvi Contents 10.2.2 Collision-HandlingSchemes . . . . . . . . . . . . . . . . 417 10.2.3 Load Factors, Rehashing, and Efficiency . . . . . . . . . 420 10.2.4 Python Hash Table Implementation . . . . . . . . . . . 422 10.3 Sorted Maps . . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.1 Sorted Search Tables . . . . . . . . . . . . . . . . . . . 428 10.3.2 Two Applications of Sorted Maps . . . . . . . . . . . . 434 10.4 Skip Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 10.4.1 Search and Update Operations in a Skip List . . . . . . 439 10.4.2 Probabilistic Analysis of Skip Lists . . . . . . . . . . . 443 10.5 Sets, Multisets, and Multimaps . . . . . . . . . . . . . . . . 446 10.5.1 The Set ADT . . . . . . . . . . . . . . . . . . . . . . . 446 10.5.2 Python’s MutableSet Abstract Base Class . . . . . . . . 448 10.5.3 Implementing Sets, Multisets, and Multimaps . . . . . . 450 10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452 11 Search Trees 459 11.1 Binary Search Trees . . . . . . . . . . . . . . . . . . . . . . . 460 11.1.1 Navigating a Binary Search Tree . . . . . . . . . . . . . 461 11.1.2 Searches . . . . . . . . . . . . . . . . . . . . . . . . . . 463 11.1.3 Insertions and Deletions . . . . . . . . . . . . . . . . . . 465 11.1.4 Python Implementation . . . . . . . . . . . . . . . . . . 468 11.1.5 Performance of a Binary Search Tree . . . . . . . . . . . 473 11.2 Balanced Search Trees . . . . . . . . . . . . . . . . . . . . . 475 11.2.1 Python Framework for Balancing Search Trees . . . . . . 478 11.3 AVL Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 11.3.1 Update Operations . . . . . . . . . . . . . . . . . . . . 483 11.3.2 Python Implementation . . . . . . . . . . . . . . . . . . 488 11.4 Splay Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 11.4.1 Splaying . . . . . . . . . . . . . . . . . . . . . . . . . . 490 11.4.2 When to Splay . . . . . . . . . . . . . . . . . . . . . . . 494 11.4.3 Python Implementation . . . . . . . . . . . . . . . . . . 496 11.4.4 Amortized Analysis of Splaying . . . . . . . . . . . . 497 11.5 (2,4) Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 11.5.1 Multiway Search Trees . . . . . . . . . . . . . . . . . . 502 11.5.2 (2,4)-Tree Operations . . . . . . . . . . . . . . . . . . . 505 11.6 Red-Black Trees . . . . . . . . . . . . . . . . . . . . . . . . . 512 11.6.1 Red-Black Tree Operations . . . . . . . . . . . . . . . . 514 11.6.2 Python Implementation . . . . . . . . . . . . . . . . . . 525 11.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
- 19. Contents xvii 12 Sortingand Selection 536 12.1 Why Study Sorting Algorithms? . . . . . . . . . . . . . . . . 537 12.2 Merge-Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . 538 12.2.1 Divide-and-Conquer . . . . . . . . . . . . . . . . . . . . 538 12.2.2 Array-Based Implementation of Merge-Sort . . . . . . . 543 12.2.3 The Running Time of Merge-Sort . . . . . . . . . . . . 544 12.2.4 Merge-Sort and Recurrence Equations . . . . . . . . . 546 12.2.5 Alternative Implementations of Merge-Sort . . . . . . . 547 12.3 Quick-Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . 550 12.3.1 Randomized Quick-Sort . . . . . . . . . . . . . . . . . . 557 12.3.2 Additional Optimizations for Quick-Sort . . . . . . . . . 559 12.4 Studying Sorting through an Algorithmic Lens . . . . . . . 562 12.4.1 Lower Bound for Sorting . . . . . . . . . . . . . . . . . 562 12.4.2 Linear-Time Sorting: Bucket-Sort and Radix-Sort . . . . 564 12.5 Comparing Sorting Algorithms . . . . . . . . . . . . . . . . . 567 12.6 Python’s Built-In Sorting Functions . . . . . . . . . . . . . . 569 12.6.1 Sorting According to a Key Function . . . . . . . . . . . 569 12.7 Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 571 12.7.1 Prune-and-Search . . . . . . . . . . . . . . . . . . . . . 571 12.7.2 Randomized Quick-Select . . . . . . . . . . . . . . . . . 572 12.7.3 Analyzing Randomized Quick-Select . . . . . . . . . . . 573 12.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 574 13 Text Processing 581 13.1 Abundance of Digitized Text . . . . . . . . . . . . . . . . . . 582 13.1.1 Notations for Strings and the Python str Class . . . . . . 583 13.2 Pattern-Matching Algorithms . . . . . . . . . . . . . . . . . 584 13.2.1 Brute Force . . . . . . . . . . . . . . . . . . . . . . . . 584 13.2.2 The Boyer-Moore Algorithm . . . . . . . . . . . . . . . 586 13.2.3 The Knuth-Morris-Pratt Algorithm . . . . . . . . . . . . 590 13.3 Dynamic Programming . . . . . . . . . . . . . . . . . . . . . 594 13.3.1 Matrix Chain-Product . . . . . . . . . . . . . . . . . . . 594 13.3.2 DNA and Text Sequence Alignment . . . . . . . . . . . 597 13.4 Text Compression and the Greedy Method . . . . . . . . . 601 13.4.1 The Huffman Coding Algorithm . . . . . . . . . . . . . 602 13.4.2 The Greedy Method . . . . . . . . . . . . . . . . . . . . 603 13.5 Tries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 604 13.5.1 Standard Tries . . . . . . . . . . . . . . . . . . . . . . . 604 13.5.2 Compressed Tries . . . . . . . . . . . . . . . . . . . . . 608 13.5.3 Suffix Tries . . . . . . . . . . . . . . . . . . . . . . . . 610 13.5.4 Search Engine Indexing . . . . . . . . . . . . . . . . . . 612
- 20. xviii Contents 13.6 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 613 14 Graph Algorithms 619 14.1 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 620 14.1.1 The Graph ADT . . . . . . . . . . . . . . . . . . . . . . 626 14.2 Data Structures for Graphs . . . . . . . . . . . . . . . . . . . 627 14.2.1 Edge List Structure . . . . . . . . . . . . . . . . . . . . 628 14.2.2 Adjacency List Structure . . . . . . . . . . . . . . . . . 630 14.2.3 Adjacency Map Structure . . . . . . . . . . . . . . . . . 632 14.2.4 Adjacency Matrix Structure . . . . . . . . . . . . . . . . 633 14.2.5 Python Implementation . . . . . . . . . . . . . . . . . . 634 14.3 Graph Traversals . . . . . . . . . . . . . . . . . . . . . . . . . 638 14.3.1 Depth-First Search . . . . . . . . . . . . . . . . . . . . 639 14.3.2 DFS Implementation and Extensions . . . . . . . . . . . 644 14.3.3 Breadth-First Search . . . . . . . . . . . . . . . . . . . 648 14.4 Transitive Closure . . . . . . . . . . . . . . . . . . . . . . . . 651 14.5 Directed Acyclic Graphs . . . . . . . . . . . . . . . . . . . . 655 14.5.1 Topological Ordering . . . . . . . . . . . . . . . . . . . 655 14.6 Shortest Paths . . . . . . . . . . . . . . . . . . . . . . . . . . 659 14.6.1 Weighted Graphs . . . . . . . . . . . . . . . . . . . . . 659 14.6.2 Dijkstra’s Algorithm . . . . . . . . . . . . . . . . . . . . 661 14.7 Minimum Spanning Trees . . . . . . . . . . . . . . . . . . . . 670 14.7.1 Prim-Jarnı́k Algorithm . . . . . . . . . . . . . . . . . . 672 14.7.2 Kruskal’s Algorithm . . . . . . . . . . . . . . . . . . . . 676 14.7.3 Disjoint Partitions and Union-Find Structures . . . . . . 681 14.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686 15 Memory Management and B-Trees 697 15.1 Memory Management . . . . . . . . . . . . . . . . . . . . . . 698 15.1.1 Memory Allocation . . . . . . . . . . . . . . . . . . . . 699 15.1.2 Garbage Collection . . . . . . . . . . . . . . . . . . . . 700 15.1.3 Additional Memory Used by the Python Interpreter . . . 703 15.2 Memory Hierarchies and Caching . . . . . . . . . . . . . . . 705 15.2.1 Memory Systems . . . . . . . . . . . . . . . . . . . . . 705 15.2.2 Caching Strategies . . . . . . . . . . . . . . . . . . . . 706 15.3 External Searching and B-Trees . . . . . . . . . . . . . . . . 711 15.3.1 (a,b) Trees . . . . . . . . . . . . . . . . . . . . . . . . . 712 15.3.2 B-Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 714 15.4 External-Memory Sorting . . . . . . . . . . . . . . . . . . . . 715 15.4.1 Multiway Merging . . . . . . . . . . . . . . . . . . . . . 716 15.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
- 21. Contents xix A CharacterStrings in Python 721 B Useful Mathematical Facts 725 Bibliography 732 Index 737
- 23. Chapter 1 Python Primer Contents 1.1Python Overview . . . . . . . . . . . . . . . . . . . . . . . . 2 1.1.1 The Python Interpreter . . . . . . . . . . . . . . . . . . . 2 1.1.2 Preview of a Python Program . . . . . . . . . . . . . . . 3 1.2 Objects in Python . . . . . . . . . . . . . . . . . . . . . . . 4 1.2.1 Identifiers, Objects, and the Assignment Statement . . . . 4 1.2.2 Creating and Using Objects . . . . . . . . . . . . . . . . . 6 1.2.3 Python’s Built-In Classes . . . . . . . . . . . . . . . . . . 7 1.3 Expressions, Operators, and Precedence . . . . . . . . . . . 12 1.3.1 Compound Expressions and Operator Precedence . . . . . 17 1.4 Control Flow . . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.4.1 Conditionals . . . . . . . . . . . . . . . . . . . . . . . . . 18 1.4.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 1.5 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 1.5.1 Information Passing . . . . . . . . . . . . . . . . . . . . . 24 1.5.2 Python’s Built-In Functions . . . . . . . . . . . . . . . . . 28 1.6 Simple Input and Output . . . . . . . . . . . . . . . . . . . 30 1.6.1 Console Input and Output . . . . . . . . . . . . . . . . . 30 1.6.2 Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 1.7 Exception Handling . . . . . . . . . . . . . . . . . . . . . . 33 1.7.1 Raising an Exception . . . . . . . . . . . . . . . . . . . . 34 1.7.2 Catching an Exception . . . . . . . . . . . . . . . . . . . 36 1.8 Iterators and Generators . . . . . . . . . . . . . . . . . . . 39 1.9 Additional Python Conveniences . . . . . . . . . . . . . . . 42 1.9.1 Conditional Expressions . . . . . . . . . . . . . . . . . . . 42 1.9.2 Comprehension Syntax . . . . . . . . . . . . . . . . . . . 43 1.9.3 Packing and Unpacking of Sequences . . . . . . . . . . . 44 1.10 Scopes and Namespaces . . . . . . . . . . . . . . . . . . . 46 1.11 Modules and the Import Statement . . . . . . . . . . . . . 48 1.11.1 Existing Modules . . . . . . . . . . . . . . . . . . . . . . 49 1.12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
- 24. 2 Chapter 1.Python Primer 1.1 Python Overview Building data structures and algorithms requires that we communicate detailed in- structions to a computer. An excellent way to perform such communications is using a high-level computer language, such as Python. The Python programming language was originally developed by Guido van Rossum in the early 1990s, and has since become a prominently used language in industry and education. The sec- ond major version of the language, Python 2, was released in 2000, and the third major version, Python 3, released in 2008. We note that there are significant in- compatibilities between Python 2 and Python 3. This book is based on Python 3 (more specifically, Python 3.1 or later). The latest version of the language is freely available at www.python.org, along with documentation and tutorials. In this chapter, we provide an overview of the Python programming language, and we continue this discussion in the next chapter, focusing on object-oriented principles. We assume that readers of this book have prior programming experi- ence, although not necessarily using Python. This book does not provide a com- plete description of the Python language (there are numerous language references for that purpose), but it does introduce all aspects of the language that are used in code fragments later in this book. 1.1.1 The Python Interpreter Python is formally an interpreted language. Commands are executed through a piece of software known as the Python interpreter. The interpreter receives a com- mand, evaluates that command, and reports the result of the command. While the interpreter can be used interactively (especially when debugging), a programmer typically defines a series of commands in advance and saves those commands in a plain text file known as source code or a script. For Python, source code is conven- tionally stored in a file named with the .py suffix (e.g., demo.py). On most operating systems, the Python interpreter can be started by typing python from the command line. By default, the interpreter starts in interactive mode with a clean workspace. Commands from a predefined script saved in a file (e.g., demo.py) are executed by invoking the interpreter with the filename as an argument (e.g., python demo.py), or using an additional -i flag in order to execute a script and then enter interactive mode (e.g., python -i demo.py). Many integrated development environments (IDEs) provide richer software development platforms for Python, including one named IDLE that is included with the standard Python distribution. IDLE provides an embedded text-editor with support for displaying and editing Python code, and a basic debugger, allowing step-by-step execution of a program while examining key variable values.
- 25. 1.1. Python Overview3 1.1.2 Preview of a Python Program As a simple introduction, Code Fragment 1.1 presents a Python program that com- putes the grade-point average (GPA) for a student based on letter grades that are entered by a user. Many of the techniques demonstrated in this example will be discussed in the remainder of this chapter. At this point, we draw attention to a few high-level issues, for readers who are new to Python as a programming language. Python’s syntax relies heavily on the use of whitespace. Individual statements are typically concluded with a newline character, although a command can extend to another line, either with a concluding backslash character (), or if an opening delimiter has not yet been closed, such as the { character in defining value map. Whitespace is also key in delimiting the bodies of control structures in Python. Specifically, a block of code is indented to designate it as the body of a control structure, and nested control structures use increasing amounts of indentation. In Code Fragment 1.1, the body of the while loop consists of the subsequent 8 lines, including a nested conditional structure. Comments are annotations provided for human readers, yet ignored by the Python interpreter. The primary syntax for comments in Python is based on use of the # character, which designates the remainder of the line as a comment. print( Welcome to the GPA calculator. ) print( Please enter all your letter grades, one per line. ) print( Enter a blank line to designate the end. ) # map from letter grade to point value points = { A+ :4.0, A :4.0, A- :3.67, B+ :3.33, B :3.0, B- :2.67, C+ :2.33, C :2.0, C :1.67, D+ :1.33, D :1.0, F :0.0} num courses = 0 total points = 0 done = False while not done: grade = input( ) # read line from user if grade == : # empty line was entered done = True elif grade not in points: # unrecognized grade entered print(Unknown grade {0} being ignored.format(grade)) else: num courses += 1 total points += points[grade] if num courses 0: # avoid division by zero print( Your GPA is {0:.3} .format(total points / num courses)) Code Fragment 1.1: A Python program that computes a grade-point average (GPA).
- 26. 4 Chapter 1.Python Primer 1.2 Objects in Python Python is an object-oriented language and classes form the basis for all data types. In this section, we describe key aspects of Python’s object model, and we intro- duce Python’s built-in classes, such as the int class for integers, the float class for floating-point values, and the str class for character strings. A more thorough presentation of object-orientation is the focus of Chapter 2. 1.2.1 Identifiers, Objects, and the Assignment Statement The most important of all Python commands is an assignment statement, such as temperature = 98.6 This command establishes temperature as an identifier (also known as a name), and then associates it with the object expressed on the right-hand side of the equal sign, in this case a floating-point object with value 98.6. We portray the outcome of this assignment in Figure 1.1. float 98.6 temperature Figure 1.1: The identifier temperature references an instance of the float class having value 98.6. Identifiers Identifiers in Python are case-sensitive, so temperature and Temperature are dis- tinct names. Identifiers can be composed of almost any combination of letters, numerals, and underscore characters (or more general Unicode characters). The primary restrictions are that an identifier cannot begin with a numeral (thus 9lives is an illegal name), and that there are 33 specially reserved words that cannot be used as identifiers, as shown in Table 1.1. Reserved Words False as continue else from in not return yield None assert def except global is or try True break del finally if lambda pass while and class elif for import nonlocal raise with Table 1.1: A listing of the reserved words in Python. These names cannot be used as identifiers.
- 27. 1.2. Objects inPython 5 For readers familiar with other programming languages, the semantics of a Python identifier is most similar to a reference variable in Java or a pointer variable in C++. Each identifier is implicitly associated with the memory address of the object to which it refers. A Python identifier may be assigned to a special object named None, serving a similar purpose to a null reference in Java or C++. Unlike Java and C++, Python is a dynamically typed language, as there is no advance declaration associating an identifier with a particular data type. An iden- tifier can be associated with any type of object, and it can later be reassigned to another object of the same (or different) type. Although an identifier has no de- clared type, the object to which it refers has a definite type. In our first example, the characters 98.6 are recognized as a floating-point literal, and thus the identifier temperature is associated with an instance of the float class having that value. A programmer can establish an alias by assigning a second identifier to an existing object. Continuing with our earlier example, Figure 1.2 portrays the result of a subsequent assignment, original = temperature. float 98.6 original temperature Figure 1.2: Identifiers temperature and original are aliases for the same object. Once an alias has been established, either name can be used to access the under- lying object. If that object supports behaviors that affect its state, changes enacted through one alias will be apparent when using the other alias (because they refer to the same object). However, if one of the names is reassigned to a new value using a subsequent assignment statement, that does not affect the aliased object, rather it breaks the alias. Continuing with our concrete example, we consider the command: temperature = temperature + 5.0 The execution of this command begins with the evaluation of the expression on the right-hand side of the = operator. That expression, temperature + 5.0, is eval- uated based on the existing binding of the name temperature, and so the result has value 103.6, that is, 98.6 + 5.0. That result is stored as a new floating-point instance, and only then is the name on the left-hand side of the assignment state- ment, temperature, (re)assigned to the result. The subsequent configuration is dia- grammed in Figure 1.3. Of particular note, this last command had no effect on the value of the existing float instance that identifier original continues to reference. 98.6 float 103.6 temperature original float Figure 1.3: The temperature identifier has been assigned to a new value, while original continues to refer to the previously existing value.
- 28. 6 Chapter 1.Python Primer 1.2.2 Creating and Using Objects Instantiation The process of creating a new instance of a class is known as instantiation. In general, the syntax for instantiating an object is to invoke the constructor of a class. For example, if there were a class named Widget, we could create an instance of that class using a syntax such as w = Widget(), assuming that the constructor does not require any parameters. If the constructor does require parameters, we might use a syntax such as Widget(a, b, c) to construct a new instance. Many of Python’s built-in classes (discussed in Section 1.2.3) support what is known as a literal form for designating new instances. For example, the command temperature = 98.6 results in the creation of a new instance of the float class; the term 98.6 in that expression is a literal form. We discuss further cases of Python literals in the coming section. From a programmer’s perspective, yet another way to indirectly create a new instance of a class is to call a function that creates and returns such an instance. For example, Python has a built-in function named sorted (see Section 1.5.2) that takes a sequence of comparable elements as a parameter and returns a new instance of the list class containing those elements in sorted order. Calling Methods Python supports traditional functions (see Section 1.5) that are invoked with a syn- tax such as sorted(data), in which case data is a parameter sent to the function. Python’s classes may also define one or more methods (also known as member functions), which are invoked on a specific instance of a class using the dot (“.”) operator. For example, Python’s list class has a method named sort that can be invoked with a syntax such as data.sort(). This particular method rearranges the contents of the list so that they are sorted. The expression to the left of the dot identifies the object upon which the method is invoked. Often, this will be an identifier (e.g., data), but we can use the dot op- erator to invoke a method upon the immediate result of some other operation. For example, if response identifies a string instance (we will discuss strings later in this section), the syntax response.lower().startswith( y ) first evaluates the method call, response.lower(), which itself returns a new string instance, and then the startswith( y ) method is called on that intermediate string. When using a method of a class, it is important to understand its behavior. Some methods return information about the state of an object, but do not change that state. These are known as accessors. Other methods, such as the sort method of the list class, do change the state of an object. These methods are known as mutators or update methods.
- 29. 1.2. Objects inPython 7 1.2.3 Python’s Built-In Classes Table 1.2 provides a summary of commonly used, built-in classes in Python; we take particular note of which classes are mutable and which are immutable. A class is immutable if each object of that class has a fixed value upon instantiation that cannot subsequently be changed. For example, the float class is immutable. Once an instance has been created, its value cannot be changed (although an identifier referencing that object can be reassigned to a different value). Class Description Immutable? bool Boolean value int integer (arbitrary magnitude) float floating-point number list mutable sequence of objects tuple immutable sequence of objects str character string set unordered set of distinct objects frozenset immutable form of set class dict associative mapping (aka dictionary) Table 1.2: Commonly used built-in classes for Python In this section, we provide an introduction to these classes, discussing their purpose and presenting several means for creating instances of the classes. Literal forms (such as 98.6) exist for most of the built-in classes, and all of the classes support a traditional constructor form that creates instances that are based upon one or more existing values. Operators supported by these classes are described in Section 1.3. More detailed information about these classes can be found in later chapters as follows: lists and tuples (Chapter 5); strings (Chapters 5 and 13, and Appendix A); sets and dictionaries (Chapter 10). The bool Class The bool class is used to manipulate logical (Boolean) values, and the only two instances of that class are expressed as the literals True and False. The default constructor, bool(), returns False, but there is no reason to use that syntax rather than the more direct literal form. Python allows the creation of a Boolean value from a nonboolean type using the syntax bool(foo) for value foo. The interpretation depends upon the type of the parameter. Numbers evaluate to False if zero, and True if nonzero. Sequences and other container types, such as strings and lists, evaluate to False if empty and True if nonempty. An important application of this interpretation is the use of a nonboolean value as a condition in a control structure.
- 30. 8 Chapter 1.Python Primer The int Class The int and float classes are the primary numeric types in Python. The int class is designed to represent integer values with arbitrary magnitude. Unlike Java and C++, which support different integral types with different precisions (e.g., int, short, long), Python automatically chooses the internal representation for an in- teger based upon the magnitude of its value. Typical literals for integers include 0, 137, and −23. In some contexts, it is convenient to express an integral value using binary, octal, or hexadecimal. That can be done by using a prefix of the number 0 and then a character to describe the base. Example of such literals are respectively 0b1011, 0o52, and 0x7f. The integer constructor, int(), returns value 0 by default. But this constructor can be used to construct an integer value based upon an existing value of another type. For example, if f represents a floating-point value, the syntax int(f) produces the truncated value of f. For example, both int(3.14) and int(3.99) produce the value 3, while int(−3.9) produces the value −3. The constructor can also be used to parse a string that is presumed to represent an integral value (such as one en- tered by a user). If s represents a string, then int(s) produces the integral value that string represents. For example, the expression int( 137 ) produces the inte- ger value 137. If an invalid string is given as a parameter, as in int( hello ), a ValueError is raised (see Section 1.7 for discussion of Python’s exceptions). By de- fault, the string must use base 10. If conversion from a different base is desired, that base can be indicated as a second, optional, parameter. For example, the expression int( 7f , 16) evaluates to the integer 127. The float Class The float class is the sole floating-point type in Python, using a fixed-precision representation. Its precision is more akin to a double in Java or C++, rather than those languages’ float type. We have already discussed a typical literal form, 98.6. We note that the floating-point equivalent of an integral number can be expressed directly as 2.0. Technically, the trailing zero is optional, so some programmers might use the expression 2. to designate this floating-point literal. One other form of literal for floating-point values uses scientific notation. For example, the literal 6.022e23 represents the mathematical value 6.022×1023. The constructor form of float() returns 0.0. When given a parameter, the con- structor attempts to return the equivalent floating-point value. For example, the call float(2) returns the floating-point value 2.0. If the parameter to the constructor is a string, as with float( 3.14 ), it attempts to parse that string as a floating-point value, raising a ValueError as an exception.
- 31. 1.2. Objects inPython 9 Sequence Types: The list, tuple, and str Classes The list, tuple, and str classes are sequence types in Python, representing a col- lection of values in which the order is significant. The list class is the most general, representing a sequence of arbitrary objects (akin to an “array” in other languages). The tuple class is an immutable version of the list class, benefiting from a stream- lined internal representation. The str class is specially designed for representing an immutable sequence of text characters. We note that Python does not have a separate class for characters; they are just strings with length one. The list Class A list instance stores a sequence of objects. A list is a referential structure, as it technically stores a sequence of references to its elements (see Figure 1.4). El- ements of a list may be arbitrary objects (including the None object). Lists are array-based sequences and are zero-indexed, thus a list of length n has elements indexed from 0 to n−1 inclusive. Lists are perhaps the most used container type in Python and they will be extremely central to our study of data structures and algo- rithms. They have many valuable behaviors, including the ability to dynamically expand and contract their capacities as needed. In this chapter, we will discuss only the most basic properties of lists. We revisit the inner working of all of Python’s sequence types as the focus of Chapter 5. Python uses the characters [ ] as delimiters for a list literal, with [ ] itself being an empty list. As another example, [ red , green , blue ] is a list containing three string instances. The contents of a list literal need not be expressed as literals; if identifiers a and b have been established, then syntax [a, b] is legitimate. The list() constructor produces an empty list by default. However, the construc- tor will accept any parameter that is of an iterable type. We will discuss iteration further in Section 1.8, but examples of iterable types include all of the standard con- tainer types (e.g., strings, list, tuples, sets, dictionaries). For example, the syntax list( hello ) produces a list of individual characters, [ h , e , l , l , o ]. Because an existing list is itself iterable, the syntax backup = list(data) can be used to construct a new list instance referencing the same contents as the original. 3 4 5 6 7 0 1 2 10 9 8 primes: 13 19 23 29 31 7 5 3 2 11 17 Figure 1.4: Python’s internal representation of a list of integers, instantiated as prime = [2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31]. The implicit indices of the ele- ments are shown below each entry.
- 32. 10 Chapter 1.Python Primer The tuple Class The tuple class provides an immutable version of a sequence, and therefore its instances have an internal representation that may be more streamlined than that of a list. While Python uses the [ ] characters to delimit a list, parentheses delimit a tuple, with () being an empty tuple. There is one important subtlety. To express a tuple of length one as a literal, a comma must be placed after the element, but within the parentheses. For example, (17,) is a one-element tuple. The reason for this requirement is that, without the trailing comma, the expression (17) is viewed as a simple parenthesized numeric expression. The str Class Python’s str class is specifically designed to efficiently represent an immutable sequence of characters, based upon the Unicode international character set. Strings have a more compact internal representation than the referential lists and tuples, as portrayed in Figure 1.5. 0 A S M P L E 3 4 5 1 2 Figure 1.5: A Python string, which is an indexed sequence of characters. String literals can be enclosed in single quotes, as in hello , or double quotes, as in hello. This choice is convenient, especially when using an- other of the quotation characters as an actual character in the sequence, as in Don t worry. Alternatively, the quote delimiter can be designated using a backslash as a so-called escape character, as in Don t worry . Because the backslash has this purpose, the backslash must itself be escaped to occur as a natu- ral character of the string literal, as in C:Python , for a string that would be displayed as C:Python. Other commonly escaped characters are n for newline and t for tab. Unicode characters can be included, such as 20u20AC for the string 20 . Python also supports using the delimiter or to begin and end a string literal. The advantage of such triple-quoted strings is that newline characters can be embedded naturally (rather than escaped as n). This can greatly improve the readability of long, multiline strings in source code. For example, at the beginning of Code Fragment 1.1, rather than use separate print statements for each line of introductory output, we can use a single print statement, as follows: print(”””Welcome to the GPA calculator. Please enter all your letter grades, one per line. Enter a blank line to designate the end.”””)
- 33. 1.2. Objects inPython 11 The set and frozenset Classes Python’s set class represents the mathematical notion of a set, namely a collection of elements, without duplicates, and without an inherent order to those elements. The major advantage of using a set, as opposed to a list, is that it has a highly optimized method for checking whether a specific element is contained in the set. This is based on a data structure known as a hash table (which will be the primary topic of Chapter 10). However, there are two important restrictions due to the algorithmic underpinnings. The first is that the set does not maintain the elements in any particular order. The second is that only instances of immutable types can be added to a Python set. Therefore, objects such as integers, floating-point numbers, and character strings are eligible to be elements of a set. It is possible to maintain a set of tuples, but not a set of lists or a set of sets, as lists and sets are mutable. The frozenset class is an immutable form of the set type, so it is legal to have a set of frozensets. Python uses curly braces { and } as delimiters for a set, for example, as {17} or { red , green , blue }. The exception to this rule is that { } does not represent an empty set; for historical reasons, it represents an empty dictionary (see next paragraph). Instead, the constructor syntax set() produces an empty set. If an iterable parameter is sent to the constructor, then the set of distinct elements is produced. For example, set( hello ) produces { h , e , l , o }. The dict Class Python’s dict class represents a dictionary, or mapping, from a set of distinct keys to associated values. For example, a dictionary might map from unique student ID numbers, to larger student records (such as the student’s name, address, and course grades). Python implements a dict using an almost identical approach to that of a set, but with storage of the associated values. A dictionary literal also uses curly braces, and because dictionaries were intro- duced in Python prior to sets, the literal form { } produces an empty dictionary. A nonempty dictionary is expressed using a comma-separated series of key:value pairs. For example, the dictionary { ga : Irish , de : German } maps ga to Irish and de to German . The constructor for the dict class accepts an existing mapping as a parameter, in which case it creates a new dictionary with identical associations as the existing one. Alternatively, the constructor accepts a sequence of key-value pairs as a pa- rameter, as in dict(pairs) with pairs = [( ga , Irish ), ( de , German )].
- 34. 12 Chapter 1.Python Primer 1.3 Expressions, Operators, and Precedence In the previous section, we demonstrated how names can be used to identify ex- isting objects, and how literals and constructors can be used to create instances of built-in classes. Existing values can be combined into larger syntactic expressions using a variety of special symbols and keywords known as operators. The seman- tics of an operator depends upon the type of its operands. For example, when a and b are numbers, the syntax a + b indicates addition, while if a and b are strings, the operator indicates concatenation. In this section, we describe Python’s opera- tors in various contexts of the built-in types. We continue, in Section 1.3.1, by discussing compound expressions, such as a + b c, which rely on the evaluation of two or more operations. The order in which the operations of a compound expression are evaluated can affect the overall value of the expression. For this reason, Python defines a specific order of precedence for evaluating operators, and it allows a programmer to override this order by using explicit parentheses to group subexpressions. Logical Operators Python supports the following keyword operators for Boolean values: not unary negation and conditional and or conditional or The and and or operators short-circuit, in that they do not evaluate the second operand if the result can be determined based on the value of the first operand. This feature is useful when constructing Boolean expressions in which we first test that a certain condition holds (such as a reference not being None), and then test a condition that could have otherwise generated an error condition had the prior test not succeeded. Equality Operators Python supports the following operators to test two notions of equality: is same identity is not different identity == equivalent != not equivalent The expression a is b evaluates to True, precisely when identifiers a and b are aliases for the same object. The expression a == b tests a more general notion of equivalence. If identifiers a and b refer to the same object, then a == b should also evaluate to True. Yet a == b also evaluates to True when the identifiers refer to
- 35. 1.3. Expressions, Operators,and Precedence 13 different objects that happen to have values that are deemed equivalent. The precise notion of equivalence depends on the data type. For example, two strings are con- sidered equivalent if they match character for character. Two sets are equivalent if they have the same contents, irrespective of order. In most programming situations, the equivalence tests == and != are the appropriate operators; use of is and is not should be reserved for situations in which it is necessary to detect true aliasing. Comparison Operators Data types may define a natural order via the following operators: less than = less than or equal to greater than = greater than or equal to These operators have expected behavior for numeric types, and are defined lexi- cographically, and case-sensitively, for strings. An exception is raised if operands have incomparable types, as with 5 hello . Arithmetic Operators Python supports the following arithmetic operators: + addition − subtraction multiplication / true division // integer division % the modulo operator The use of addition, subtraction, and multiplication is straightforward, noting that if both operands have type int, then the result is an int as well; if one or both operands have type float, the result will be a float. Python takes more care in its treatment of division. We first consider the case in which both operands have type int, for example, the quantity 27 divided by 4. In mathematical notation, 27 ÷ 4 = 63 4 = 6.75. In Python, the / operator designates true division, returning the floating-point result of the computation. Thus, 27 / 4 results in the float value 6.75. Python supports the pair of opera- tors // and % to perform the integral calculations, with expression 27 // 4 evalu- ating to int value 6 (the mathematical floor of the quotient), and expression 27 % 4 evaluating to int value 3, the remainder of the integer division. We note that lan- guages such as C, C++, and Java do not support the // operator; instead, the / op- erator returns the truncated quotient when both operands have integral type, and the result of true division when at least one operand has a floating-point type.
- 36. 14 Chapter 1.Python Primer Python carefully extends the semantics of // and % to cases where one or both operands are negative. For the sake of notation, let us assume that variables n and m represent respectively the dividend and divisor of a quotient n m, and that q = n // m and r = n % m. Python guarantees that q m + r will equal n. We already saw an example of this identity with positive operands, as 6∗4+ 3 = 27. When the divisor m is positive, Python further guarantees that 0 ≤ r m. As a consequence, we find that −27 // 4 evaluates to −7 and −27 % 4 evaluates to 1, as (−7)∗4+ 1 = −27. When the divisor is negative, Python guarantees that m r ≤ 0. As an example, 27 // −4 is −7 and 27 % −4 is −1, satisfying the identity 27 = (−7)∗(−4)+(−1). The conventions for the // and % operators are even extended to floating- point operands, with the expression q = n // m being the integral floor of the quotient, and r = n % m being the “remainder” to ensure that q m + r equals n. For example, 8.2 // 3.14 evaluates to 2.0 and 8.2 % 3.14 evaluates to 1.92, as 2.0∗3.14+1.92 = 8.2. Bitwise Operators Python provides the following bitwise operators for integers: ∼ bitwise complement (prefix unary operator) bitwise and | bitwise or ˆ bitwise exclusive-or shift bits left, filling in with zeros shift bits right, filling in with sign bit Sequence Operators Each of Python’s built-in sequence types (str, tuple, and list) support the following operator syntaxes: s[j] element at index j s[start:stop] slice including indices [start,stop) s[start:stop:step] slice including indices start, start + step, start + 2 step, . . . , up to but not equalling or stop s + t concatenation of sequences k s shorthand for s + s + s + ... (k times) val in s containment check val not in s non-containment check Python relies on zero-indexing of sequences, thus a sequence of length n has ele- ments indexed from 0 to n− 1 inclusive. Python also supports the use of negative indices, which denote a distance from the end of the sequence; index −1 denotes the last element, index −2 the second to last, and so on. Python uses a slicing
- 37. 1.3. Expressions, Operators,and Precedence 15 notation to describe subsequences of a sequence. Slices are described as half-open intervals, with a start index that is included, and a stop index that is excluded. For example, the syntax data[3:8] denotes a subsequence including the five indices: 3,4,5,6,7. An optional “step” value, possibly negative, can be indicated as a third parameter of the slice. If a start index or stop index is omitted in the slicing nota- tion, it is presumed to designate the respective extreme of the original sequence. Because lists are mutable, the syntax s[j] = val can be used to replace an ele- ment at a given index. Lists also support a syntax, del s[j], that removes the desig- nated element from the list. Slice notation can also be used to replace or delete a sublist. The notation val in s can be used for any of the sequences to see if there is an element equivalent to val in the sequence. For strings, this syntax can be used to check for a single character or for a larger substring, as with amp in example . All sequences define comparison operations based on lexicographic order, per- forming an element by element comparison until the first difference is found. For example, [5, 6, 9] [5, 7] because of the entries at index 1. Therefore, the follow- ing operations are supported by sequence types: s == t equivalent (element by element) s != t not equivalent s t lexicographically less than s = t lexicographically less than or equal to s t lexicographically greater than s = t lexicographically greater than or equal to Operators for Sets and Dictionaries Sets and frozensets support the following operators: key in s containment check key not in s non-containment check s1 == s2 s1 is equivalent to s2 s1 != s2 s1 is not equivalent to s2 s1 = s2 s1 is subset of s2 s1 s2 s1 is proper subset of s2 s1 = s2 s1 is superset of s2 s1 s2 s1 is proper superset of s2 s1 | s2 the union of s1 and s2 s1 s2 the intersection of s1 and s2 s1 − s2 the set of elements in s1 but not s2 s1 ˆ s2 the set of elements in precisely one of s1 or s2 Note well that sets do not guarantee a particular order of their elements, so the comparison operators, such as , are not lexicographic; rather, they are based on the mathematical notion of a subset. As a result, the comparison operators define
- 38. 16 Chapter 1.Python Primer a partial order, but not a total order, as disjoint sets are neither “less than,” “equal to,” or “greater than” each other. Sets also support many fundamental behaviors through named methods (e.g., add, remove); we will explore their functionality more fully in Chapter 10. Dictionaries, like sets, do not maintain a well-defined order on their elements. Furthermore, the concept of a subset is not typically meaningful for dictionaries, so the dict class does not support operators such as . Dictionaries support the notion of equivalence, with d1 == d2 if the two dictionaries contain the same set of key- value pairs. The most widely used behavior of dictionaries is accessing a value associated with a particular key k with the indexing syntax, d[k]. The supported operators are as follows: d[key] value associated with given key d[key] = value set (or reset) the value associated with given key del d[key] remove key and its associated value from dictionary key in d containment check key not in d non-containment check d1 == d2 d1 is equivalent to d2 d1 != d2 d1 is not equivalent to d2 Dictionaries also support many useful behaviors through named methods, which we explore more fully in Chapter 10. Extended Assignment Operators Python supports an extended assignment operator for most binary operators, for example, allowing a syntax such as count += 5. By default, this is a shorthand for the more verbose count = count + 5. For an immutable type, such as a number or a string, one should not presume that this syntax changes the value of the existing object, but instead that it will reassign the identifier to a newly constructed value. (See discussion of Figure 1.3.) However, it is possible for a type to redefine such semantics to mutate the object, as the list class does for the += operator. alpha = [1, 2, 3] beta = alpha # an alias for alpha beta += [4, 5] # extends the original list with two more elements beta = beta + [6, 7] # reassigns beta to a new list [1, 2, 3, 4, 5, 6, 7] print(alpha) # will be [1, 2, 3, 4, 5] This example demonstrates the subtle difference between the list semantics for the syntax beta += foo versus beta = beta + foo.
- 39. 1.3. Expressions, Operators,and Precedence 17 1.3.1 Compound Expressions and Operator Precedence Programming languages must have clear rules for the order in which compound expressions, such as 5 + 2 3, are evaluated. The formal order of precedence for operators in Python is given in Table 1.3. Operators in a category with higher precedence will be evaluated before those with lower precedence, unless the expres- sion is otherwise parenthesized. Therefore, we see that Python gives precedence to multiplication over addition, and therefore evaluates the expression 5 + 2 3 as 5 + (2 3), with value 11, but the parenthesized expression (5 + 2) 3 evalu- ates to value 21. Operators within a category are typically evaluated from left to right, thus 5 − 2 + 3 has value 6. Exceptions to this rule include that unary oper- ators and exponentiation are evaluated from right to left. Python allows a chained assignment, such as x = y = 0, to assign multiple identifiers to the rightmost value. Python also allows the chaining of comparison operators. For example, the expression 1 = x + y = 10 is evaluated as the compound (1 = x + y) and (x + y = 10), but without computing the inter- mediate value x + y twice. Operator Precedence Type Symbols 1 member access expr.member 2 function/method calls expr(...) container subscripts/slices expr[...] 3 exponentiation 4 unary operators +expr, −expr, ˜expr 5 multiplication, division , /, //, % 6 addition, subtraction +, − 7 bitwise shifting , 8 bitwise-and 9 bitwise-xor ˆ 10 bitwise-or | 11 comparisons is, is not, ==, !=, , =, , = containment in, not in 12 logical-not not expr 13 logical-and and 14 logical-or or 15 conditional val1 if cond else val2 16 assignments =, +=, −=, =, etc. Table 1.3: Operator precedence in Python, with categories ordered from highest precedence to lowest precedence. When stated, we use expr to denote a literal, identifier, or result of a previously evaluated expression. All operators without explicit mention of expr are binary operators, with syntax expr1 operator expr2.
- 40. 18 Chapter 1.Python Primer 1.4 Control Flow In this section, we review Python’s most fundamental control structures: condi- tional statements and loops. Common to all control structures is the syntax used in Python for defining blocks of code. The colon character is used to delimit the beginning of a block of code that acts as a body for a control structure. If the body can be stated as a single executable statement, it can technically placed on the same line, to the right of the colon. However, a body is more typically typeset as an indented block starting on the line following the colon. Python relies on the inden- tation level to designate the extent of that block of code, or any nested blocks of code within. The same principles will be applied when designating the body of a function (see Section 1.5), and the body of a class (see Section 2.3). 1.4.1 Conditionals Conditional constructs (also known as if statements) provide a way to execute a chosen block of code based on the run-time evaluation of one or more Boolean expressions. In Python, the most general form of a conditional is written as follows: if first condition: first body elif second condition: second body elif third condition: third body else: fourth body Each condition is a Boolean expression, and each body contains one or more com- mands that are to be executed conditionally. If the first condition succeeds, the first body will be executed; no other conditions or bodies are evaluated in that case. If the first condition fails, then the process continues in similar manner with the evaluation of the second condition. The execution of this overall construct will cause precisely one of the bodies to be executed. There may be any number of elif clauses (including zero), and the final else clause is optional. As described on page 7, nonboolean types may be evaluated as Booleans with intuitive meanings. For example, if response is a string that was entered by a user, and we want to condition a behavior on this being a nonempty string, we may write if response: as a shorthand for the equivalent, if response != :
- 41. 1.4. Control Flow19 As a simple example, a robot controller might have the following logic: if door is closed: open door() advance() Notice that the final command, advance(), is not indented and therefore not part of the conditional body. It will be executed unconditionally (although after opening a closed door). We may nest one control structure within another, relying on indentation to make clear the extent of the various bodies. Revisiting our robot example, here is a more complex control that accounts for unlocking a closed door. if door is closed: if door is locked: unlock door() open door() advance() The logic expressed by this example can be diagrammed as a traditional flowchart, as portrayed in Figure 1.6. open door() False door is closed advance() door is locked unlock door() True False True Figure 1.6: A flowchart describing the logic of nested conditional statements.
- 42. 20 Chapter 1.Python Primer 1.4.2 Loops Python offers two distinct looping constructs. A while loop allows general repeti- tion based upon the repeated testing of a Boolean condition. A for loop provides convenient iteration of values from a defined series (such as characters of a string, elements of a list, or numbers within a given range). We discuss both forms in this section. While Loops The syntax for a while loop in Python is as follows: while condition: body As with an if statement, condition can be an arbitrary Boolean expression, and body can be an arbitrary block of code (including nested control structures). The execution of a while loop begins with a test of the Boolean condition. If that condi- tion evaluates to True, the body of the loop is performed. After each execution of the body, the loop condition is retested, and if it evaluates to True, another iteration of the body is performed. When the conditional test evaluates to False (assuming it ever does), the loop is exited and the flow of control continues just beyond the body of the loop. As an example, here is a loop that advances an index through a sequence of characters until finding an entry with value X or reaching the end of the sequence. j = 0 while j len(data) and data[j] != X : j += 1 The len function, which we will introduce in Section 1.5.2, returns the length of a sequence such as a list or string. The correctness of this loop relies on the short- circuiting behavior of the and operator, as described on page 12. We intention- ally test j len(data) to ensure that j is a valid index, prior to accessing element data[j]. Had we written that compound condition with the opposite order, the eval- uation of data[j] would eventually raise an IndexError when X is not found. (See Section 1.7 for discussion of exceptions.) As written, when this loop terminates, variable j’s value will be the index of the leftmost occurrence of X , if found, or otherwise the length of the sequence (which is recognizable as an invalid index to indicate failure of the search). It is worth noting that this code behaves correctly, even in the special case when the list is empty, as the condition j len(data) will initially fail and the body of the loop will never be executed.
- 43. 1.4. Control Flow21 For Loops Python’s for-loop syntax is a more convenient alternative to a while loop when iterating through a series of elements. The for-loop syntax can be used on any type of iterable structure, such as a list, tuple str, set, dict, or file (we will discuss iterators more formally in Section 1.8). Its general syntax appears as follows. for element in iterable: body # body may refer to element as an identifier For readers familiar with Java, the semantics of Python’s for loop is similar to the “for each” loop style introduced in Java 1.5. As an instructive example of such a loop, we consider the task of computing the sum of a list of numbers. (Admittedly, Python has a built-in function, sum, for this purpose.) We perform the calculation with a for loop as follows, assuming that data identifies the list: total = 0 for val in data: total += val # note use of the loop variable, val The loop body executes once for each element of the data sequence, with the iden- tifier, val, from the for-loop syntax assigned at the beginning of each pass to a respective element. It is worth noting that val is treated as a standard identifier. If the element of the original data happens to be mutable, the val identifier can be used to invoke its methods. But a reassignment of identifier val to a new value has no affect on the original data, nor on the next iteration of the loop. As a second classic example, we consider the task of finding the maximum value in a list of elements (again, admitting that Python’s built-in max function already provides this support). If we can assume that the list, data, has at least one element, we could implement this task as follows: biggest = data[0] # as we assume nonempty list for val in data: if val biggest: biggest = val Although we could accomplish both of the above tasks with a while loop, the for-loop syntax had an advantage of simplicity, as there is no need to manage an explicit index into the list nor to author a Boolean loop condition. Furthermore, we can use a for loop in cases for which a while loop does not apply, such as when iterating through a collection, such as a set, that does not support any direct form of indexing.
- 44. 22 Chapter 1.Python Primer Index-Based For Loops The simplicity of a standard for loop over the elements of a list is wonderful; how- ever, one limitation of that form is that we do not know where an element resides within the sequence. In some applications, we need knowledge of the index of an element within the sequence. For example, suppose that we want to know where the maximum element in a list resides. Rather than directly looping over the elements of the list in that case, we prefer to loop over all possible indices of the list. For this purpose, Python provides a built-in class named range that generates integer sequences. (We will discuss generators in Section 1.8.) In simplest form, the syntax range(n) generates the series of n values from 0 to n − 1. Conveniently, these are precisely the series of valid indices into a sequence of length n. Therefore, a standard Python idiom for looping through the series of indices of a data sequence uses a syntax, for j in range(len(data)): In this case, identifier j is not an element of the data—it is an integer. But the expression data[j] can be used to retrieve the respective element. For example, we can find the index of the maximum element of a list as follows: big index = 0 for j in range(len(data)): if data[j] data[big index]: big index = j Break and Continue Statements Python supports a break statement that immediately terminate a while or for loop when executed within its body. More formally, if applied within nested control structures, it causes the termination of the most immediately enclosing loop. As a typical example, here is code that determines whether a target value occurs in a data set: found = False for item in data: if item == target: found = True break Python also supports a continue statement that causes the current iteration of a loop body to stop, but with subsequent passes of the loop proceeding as expected. We recommend that the break and continue statements be used sparingly. Yet, there are situations in which these commands can be effectively used to avoid in- troducing overly complex logical conditions.
- 45. 1.5. Functions 23 1.5Functions In this section, we explore the creation of and use of functions in Python. As we did in Section 1.2.2, we draw a distinction between functions and methods. We use the general term function to describe a traditional, stateless function that is in- voked without the context of a particular class or an instance of that class, such as sorted(data). We use the more specific term method to describe a member function that is invoked upon a specific object using an object-oriented message passing syn- tax, such as data.sort(). In this section, we only consider pure functions; methods will be explored with more general object-oriented principles in Chapter 2. We begin with an example to demonstrate the syntax for defining functions in Python. The following function counts the number of occurrences of a given target value within any form of iterable data set. def count(data, target): n = 0 for item in data: if item == target: # found a match n += 1 return n The first line, beginning with the keyword def, serves as the function’s signature. This establishes a new identifier as the name of the function (count, in this exam- ple), and it establishes the number of parameters that it expects, as well as names identifying those parameters (data and target, in this example). Unlike Java and C++, Python is a dynamically typed language, and therefore a Python signature does not designate the types of those parameters, nor the type (if any) of a return value. Those expectations should be stated in the function’s documentation (see Section 2.2.3) and can be enforced within the body of the function, but misuse of a function will only be detected at run-time. The remainder of the function definition is known as the body of the func- tion. As is the case with control structures in Python, the body of a function is typically expressed as an indented block of code. Each time a function is called, Python creates a dedicated activation record that stores information relevant to the current call. This activation record includes what is known as a namespace (see Section 1.10) to manage all identifiers that have local scope within the current call. The namespace includes the function’s parameters and any other identifiers that are defined locally within the body of the function. An identifier in the local scope of the function caller has no relation to any identifier with the same name in the caller’s scope (although identifiers in different scopes may be aliases to the same object). In our first example, the identifier n has scope that is local to the function call, as does the identifier item, which is established as the loop variable.
- 46. 24 Chapter 1.Python Primer Return Statement A return statement is used within the body of a function to indicate that the func- tion should immediately cease execution, and that an expressed value should be returned to the caller. If a return statement is executed without an explicit argu- ment, the None value is automatically returned. Likewise, None will be returned if the flow of control ever reaches the end of a function body without having executed a return statement. Often, a return statement will be the final command within the body of the function, as was the case in our earlier example of a count function. However, there can be multiple return statements in the same function, with con- ditional logic controlling which such command is executed, if any. As a further example, consider the following function that tests if a value exists in a sequence. def contains(data, target): for item in target: if item == target: # found a match return True return False If the conditional within the loop body is ever satisfied, the return True statement is executed and the function immediately ends, with True designating that the target value was found. Conversely, if the for loop reaches its conclusion without ever finding the match, the final return False statement will be executed. 1.5.1 Information Passing To be a successful programmer, one must have clear understanding of the mech- anism in which a programming language passes information to and from a func- tion. In the context of a function signature, the identifiers used to describe the expected parameters are known as formal parameters, and the objects sent by the caller when invoking the function are the actual parameters. Parameter passing in Python follows the semantics of the standard assignment statement. When a function is invoked, each identifier that serves as a formal parameter is assigned, in the function’s local scope, to the respective actual parameter that is provided by the caller of the function. For example, consider the following call to our count function from page 23: prizes = count(grades, A ) Just before the function body is executed, the actual parameters, grades and A , are implicitly assigned to the formal parameters, data and target, as follows: data = grades target = A
- 47. 1.5. Functions 25 Theseassignment statements establish identifier data as an alias for grades and target as a name for the string literal A . (See Figure 1.7.) ... str A data target grades list Figure 1.7: A portrayal of parameter passing in Python, for the function call count(grades, A ). Identifiers data and target are formal parameters defined within the local scope of the count function. The communication of a return value from the function back to the caller is similarly implemented as an assignment. Therefore, with our sample invocation of prizes = count(grades, A ), the identifier prizes in the caller’s scope is assigned to the object that is identified as n in the return statement within our function body. An advantage to Python’s mechanism for passing information to and from a function is that objects are not copied. This ensures that the invocation of a function is efficient, even in a case where a parameter or return value is a complex object. Mutable Parameters Python’s parameter passing model has additional implications when a parameter is a mutable object. Because the formal parameter is an alias for the actual parameter, the body of the function may interact with the object in ways that change its state. Considering again our sample invocation of the count function, if the body of the function executes the command data.append( F ), the new entry is added to the end of the list identified as data within the function, which is one and the same as the list known to the caller as grades. As an aside, we note that reassigning a new value to a formal parameter with a function body, such as by setting data = [ ], does not alter the actual parameter; such a reassignment simply breaks the alias. Our hypothetical example of a count method that appends a new element to a list lacks common sense. There is no reason to expect such a behavior, and it would be quite a poor design to have such an unexpected effect on the parameter. There are, however, many legitimate cases in which a function may be designed (and clearly documented) to modify the state of a parameter. As a concrete example, we present the following implementation of a method named scale that’s primary purpose is to multiply all entries of a numeric data set by a given factor. def scale(data, factor): for j in range(len(data)): data[j] = factor
- 48. 26 Chapter 1.Python Primer Default Parameter Values Python provides means for functions to support more than one possible calling signature. Such a function is said to be polymorphic (which is Greek for “many forms”). Most notably, functions can declare one or more default values for pa- rameters, thereby allowing the caller to invoke a function with varying numbers of actual parameters. As an artificial example, if a function is declared with signature def foo(a, b=15, c=27): there are three parameters, the last two of which offer default values. A caller is welcome to send three actual parameters, as in foo(4, 12, 8), in which case the de- fault values are not used. If, on the other hand, the caller only sends one parameter, foo(4), the function will execute with parameters values a=4, b=15, c=27. If a caller sends two parameters, they are assumed to be the first two, with the third be- ing the default. Thus, foo(8, 20) executes with a=8, b=20, c=27. However, it is illegal to define a function with a signature such as bar(a, b=15, c) with b having a default value, yet not the subsequent c; if a default parameter value is present for one parameter, it must be present for all further parameters. As a more motivating example for the use of a default parameter, we revisit the task of computing a student’s GPA (see Code Fragment 1.1). Rather than as- sume direct input and output with the console, we prefer to design a function that computes and returns a GPA. Our original implementation uses a fixed mapping from each letter grade (such as a B−) to a corresponding point value (such as 2.67). While that point system is somewhat common, it may not agree with the system used by all schools. (For example, some may assign an A+ grade a value higher than 4.0.) Therefore, we design a compute gpa function, given in Code Fragment 1.2, which allows the caller to specify a custom mapping from grades to values, while offering the standard point system as a default. def compute gpa(grades, points={ A+ :4.0, A :4.0, A- :3.67, B+ :3.33, B :3.0, B- :2.67, C+ :2.33, C :2.0, C :1.67, D+ :1.33, D :1.0, F :0.0}): num courses = 0 total points = 0 for g in grades: if g in points: # a recognizable grade num courses += 1 total points += points[g] return total points / num courses Code Fragment 1.2: A function that computes a student’s GPA with a point value system that can be customized as an optional parameter.
- 49. 1.5. Functions 27 Asan additional example of an interesting polymorphic function, we consider Python’s support for range. (Technically, this is a constructor for the range class, but for the sake of this discussion, we can treat it as a pure function.) Three calling syntaxes are supported. The one-parameter form, range(n), generates a sequence of integers from 0 up to but not including n. A two-parameter form, range(start,stop) generates integers from start up to, but not including, stop. A three-parameter form, range(start, stop, step), generates a similar range as range(start, stop), but with increments of size step rather than 1. This combination of forms seems to violate the rules for default parameters. In particular, when a single parameter is sent, as in range(n), it serves as the stop value (which is the second parameter); the value of start is effectively 0 in that case. However, this effect can be achieved with some sleight of hand, as follows: def range(start, stop=None, step=1): if stop is None: stop = start start = 0 ... From a technical perspective, when range(n) is invoked, the actual parameter n will be assigned to formal parameter start. Within the body, if only one parameter is received, the start and stop values are reassigned to provide the desired semantics. Keyword Parameters The traditional mechanism for matching the actual parameters sent by a caller, to the formal parameters declared by the function signature is based on the concept of positional arguments. For example, with signature foo(a=10, b=20, c=30), parameters sent by the caller are matched, in the given order, to the formal param- eters. An invocation of foo(5) indicates that a=5, while b and c are assigned their default values. Python supports an alternate mechanism for sending a parameter to a function known as a keyword argument. A keyword argument is specified by explicitly assigning an actual parameter to a formal parameter by name. For example, with the above definition of function foo, a call foo(c=5) will invoke the function with parameters a=10, b=20, c=5. A function’s author can require that certain parameters be sent only through the keyword-argument syntax. We never place such a restriction in our own function definitions, but we will see several important uses of keyword-only parameters in Python’s standard libraries. As an example, the built-in max function accepts a keyword parameter, coincidentally named key, that can be used to vary the notion of “maximum” that is used.
- 50. 28 Chapter 1.Python Primer By default, max operates based upon the natural order of elements according to the operator for that type. But the maximum can be computed by comparing some other aspect of the elements. This is done by providing an auxiliary function that converts a natural element to some other value for the sake of comparison. For example, if we are interested in finding a numeric value with magnitude that is maximal (i.e., considering −35 to be larger than +20), we can use the calling syn- tax max(a, b, key=abs). In this case, the built-in abs function is itself sent as the value associated with the keyword parameter key. (Functions are first-class objects in Python; see Section 1.10.) When max is called in this way, it will compare abs(a) to abs(b), rather than a to b. The motivation for the keyword syntax as an alternate to positional arguments is important in the case of max. This function is polymor- phic in the number of arguments, allowing a call such as max(a,b,c,d); therefore, it is not possible to designate a key function as a traditional positional element. Sorting functions in Python also support a similar key parameter for indicating a nonstandard order. (We explore this further in Section 9.4 and in Section 12.6.1, when discussing sorting algorithms). 1.5.2 Python’s Built-In Functions Table 1.4 provides an overview of common functions that are automatically avail- able in Python, including the previously discussed abs, max, and range. When choosing names for the parameters, we use identifiers x, y, z for arbitrary numeric types, k for an integer, and a, b, and c for arbitrary comparable types. We use the identifier, iterable, to represent an instance of any iterable type (e.g., str, list, tuple, set, dict); we will discuss iterators and iterable data types in Section 1.8. A sequence represents a more narrow category of indexable classes, including str, list, and tuple, but neither set nor dict. Most of the entries in Table 1.4 can be categorized according to their functionality as follows: Input/Output: print, input, and open will be more fully explained in Section 1.6. Character Encoding: ord and chr relate characters and their integer code points. For example, ord( A ) is 65 and chr(65) is A . Mathematics: abs, divmod, pow, round, and sum provide common mathematical functionality; an additional math module will be introduced in Section 1.11. Ordering: max and min apply to any data type that supports a notion of compar- ison, or to any collection of such values. Likewise, sorted can be used to produce an ordered list of elements drawn from any existing collection. Collections/Iterations: range generates a new sequence of numbers; len reports the length of any existing collection; functions reversed, all, any, and map oper- ate on arbitrary iterations as well; iter and next provide a general framework for iteration through elements of a collection, and are discussed in Section 1.8.
- 51. 1.5. Functions 29 CommonBuilt-In Functions Calling Syntax Description abs(x) Return the absolute value of a number. all(iterable) Return True if bool(e) is True for each element e. any(iterable) Return True if bool(e) is True for at least one element e. chr(integer) Return a one-character string with the given Unicode code point. divmod(x, y) Return (x // y, x % y) as tuple, if x and y are integers. hash(obj) Return an integer hash value for the object (see Chapter 10). id(obj) Return the unique integer serving as an “identity” for the object. input(prompt) Return a string from standard input; the prompt is optional. isinstance(obj, cls) Determine if obj is an instance of the class (or a subclass). iter(iterable) Return a new iterator object for the parameter (see Section 1.8). len(iterable) Return the number of elements in the given iteration. map(f, iter1, iter2, ...) Return an iterator yielding the result of function calls f(e1, e2, ...) for respective elements e1 ∈ iter1,e2 ∈ iter2,... max(iterable) Return the largest element of the given iteration. max(a, b, c, ...) Return the largest of the arguments. min(iterable) Return the smallest element of the given iteration. min(a, b, c, ...) Return the smallest of the arguments. next(iterator) Return the next element reported by the iterator (see Section 1.8). open(filename, mode) Open a file with the given name and access mode. ord(char) Return the Unicode code point of the given character. pow(x, y) Return the value xy (as an integer if x and y are integers); equivalent to x y. pow(x, y, z) Return the value (xy mod z) as an integer. print(obj1, obj2, ...) Print the arguments, with separating spaces and trailing newline. range(stop) Construct an iteration of values 0, 1, ..., stop−1. range(start, stop) Construct an iteration of values start, start+1, ..., stop−1. range(start, stop, step) Construct an iteration of values start, start+step, start+2 step, ... reversed(sequence) Return an iteration of the sequence in reverse. round(x) Return the nearest int value (a tie is broken toward the even value). round(x, k) Return the value rounded to the nearest 10−k (return-type matches x). sorted(iterable) Return a list containing elements of the iterable in sorted order. sum(iterable) Return the sum of the elements in the iterable (must be numeric). type(obj) Return the class to which the instance obj belongs. Table 1.4: Commonly used built-in function in Python.
- 52. 30 Chapter 1.Python Primer 1.6 Simple Input and Output In this section, we address the basics of input and output in Python, describing stan- dard input and output through the user console, and Python’s support for reading and writing text files. 1.6.1 Console Input and Output The print Function The built-in function, print, is used to generate standard output to the console. In its simplest form, it prints an arbitrary sequence of arguments, separated by spaces, and followed by a trailing newline character. For example, the command print( maroon , 5) outputs the string maroon 5n . Note that arguments need not be string instances. A nonstring argument x will be displayed as str(x). Without any arguments, the command print() outputs a single newline character. The print function can be customized through the use of the following keyword parameters (see Section 1.5 for a discussion of keyword parameters): • By default, the print function inserts a separating space into the output be- tween each pair of arguments. The separator can be customized by providing a desired separating string as a keyword parameter, sep. For example, colon- separated output can be produced as print(a, b, c, sep= : ). The separating string need not be a single character; it can be a longer string, and it can be the empty string, sep= , causing successive arguments to be directly con- catenated. • By default, a trailing newline is output after the final argument. An alterna- tive trailing string can be designated using a keyword parameter, end. Des- ignating the empty string end= suppresses all trailing characters. • By default, the print function sends its output to the standard console. How- ever, output can be directed to a file by indicating an output file stream (see Section 1.6.2) using file as a keyword parameter. The input Function The primary means for acquiring information from the user console is a built-in function named input. This function displays a prompt, if given as an optional pa- rameter, and then waits until the user enters some sequence of characters followed by the return key. The formal return value of the function is the string of characters that were entered strictly before the return key (i.e., no newline character exists in the returned string).
- 53. 1.6. Simple Inputand Output 31 When reading a numeric value from the user, a programmer must use the input function to get the string of characters, and then use the int or float syntax to construct the numeric value that character string represents. That is, if a call to response = input() reports that the user entered the characters, 2013 , the syntax int(response) could be used to produce the integer value 2013. It is quite common to combine these operations with a syntax such as year = int(input( In what year were you born? )) if we assume that the user will enter an appropriate response. (In Section 1.7 we discuss error handling in such a situation.) Because input returns a string as its result, use of that function can be combined with the existing functionality of the string class, as described in Appendix A. For example, if the user enters multiple pieces of information on the same line, it is common to call the split method on the result, as in reply = input( Enter x and y, separated by spaces: ) pieces = reply.split( ) # returns a list of strings, as separated by spaces x = float(pieces[0]) y = float(pieces[1]) A Sample Program Here is a simple, but complete, program that demonstrates the use of the input and print functions. The tools for formatting the final output is discussed in Ap- pendix A. age = int(input( Enter your age in years: )) max heart rate = 206.9 − (0.67 age) # as per Med Sci Sports Exerc. target = 0.65 max heart rate print( Your target fat-burning heart rate is , target) 1.6.2 Files Files are typically accessed in Python beginning with a call to a built-in function, named open, that returns a proxy for interactions with the underlying file. For example, the command, fp = open( sample.txt ), attempts to open a file named sample.txt, returning a proxy that allows read-only access to the text file. The open function accepts an optional second parameter that determines the access mode. The default mode is r for reading. Other common modes are w for writing to the file (causing any existing file with that name to be overwritten), or a for appending to the end of an existing file. Although we focus on use of text files, it is possible to work with binary files, using access modes such as rb or wb .
- 54. 32 Chapter 1.Python Primer When processing a file, the proxy maintains a current position within the file as an offset from the beginning, measured in number of bytes. When opening a file with mode r or w , the position is initially 0; if opened in append mode, a , the position is initially at the end of the file. The syntax fp.close() closes the file associated with proxy fp, ensuring that any written contents are saved. A summary of methods for reading and writing a file is given in Table 1.5 Calling Syntax Description fp.read() Return the (remaining) contents of a readable file as a string. fp.read(k) Return the next k bytes of a readable file as a string. fp.readline() Return (remainder of) the current line of a readable file as a string. fp.readlines() Return all (remaining) lines of a readable file as a list of strings. for line in fp: Iterate all (remaining) lines of a readable file. fp.seek(k) Change the current position to be at the kth byte of the file. fp.tell() Return the current position, measured as byte-offset from the start. fp.write(string) Write given string at current position of the writable file. fp.writelines(seq) Write each of the strings of the given sequence at the current position of the writable file. This command does not insert any newlines, beyond those that are embedded in the strings. print(..., file=fp) Redirect output of print function to the file. Table 1.5: Behaviors for interacting with a text file via a file proxy (named fp). Reading from a File The most basic command for reading via a proxy is the read method. When invoked on file proxy fp, as fp.read(k), the command returns a string representing the next k bytes of the file, starting at the current position. Without a parameter, the syntax fp.read() returns the remaining contents of the file in entirety. For convenience, files can be read a line at a time, using the readline method to read one line, or the readlines method to return a list of all remaining lines. Files also support the for-loop syntax, with iteration being line by line (e.g., for line in fp:). Writing to a File When a file proxy is writable, for example, if created with access mode w or a , text can be written using methods write or writelines. For example, if we de- fine fp = open( results.txt , w ), the syntax fp.write( Hello World.n ) writes a single line to the file with the given string. Note well that write does not explicitly add a trailing newline, so desired newline characters must be embedded directly in the string parameter. Recall that the output of the print method can be redirected to a file using a keyword parameter, as described in Section 1.6.
- 55. 1.7. Exception Handling33 1.7 Exception Handling Exceptions are unexpected events that occur during the execution of a program. An exception might result from a logical error or an unanticipated situation. In Python, exceptions (also known as errors) are objects that are raised (or thrown) by code that encounters an unexpected circumstance. The Python interpreter can also raise an exception should it encounter an unexpected condition, like running out of memory. A raised error may be caught by a surrounding context that “handles” the exception in an appropriate fashion. If uncaught, an exception causes the interpreter to stop executing the program and to report an appropriate message to the console. In this section, we examine the most common error types in Python, the mechanism for catching and handling errors that have been raised, and the syntax for raising errors from within user-defined blocks of code. Common Exception Types Python includes a rich hierarchy of exception classes that designate various cate- gories of errors; Table 1.6 shows many of those classes. The Exception class serves as a base class for most other error types. An instance of the various subclasses encodes details about a problem that has occurred. Several of these errors may be raised in exceptional cases by behaviors introduced in this chapter. For example, use of an undefined identifier in an expression causes a NameError, and errant use of the dot notation, as in foo.bar(), will generate an AttributeError if object foo does not support a member named bar. Class Description Exception A base class for most error types AttributeError Raised by syntax obj.foo, if obj has no member named foo EOFError Raised if “end of file” reached for console or file input IOError Raised upon failure of I/O operation (e.g., opening file) IndexError Raised if index to sequence is out of bounds KeyError Raised if nonexistent key requested for set or dictionary KeyboardInterrupt Raised if user types ctrl-C while program is executing NameError Raised if nonexistent identifier used StopIteration Raised by next(iterator) if no element; see Section 1.8 TypeError Raised when wrong type of parameter is sent to a function ValueError Raised when parameter has invalid value (e.g., sqrt(−5)) ZeroDivisionError Raised when any division operator used with 0 as divisor Table 1.6: Common exception classes in Python
- 56. 34 Chapter 1.Python Primer Sending the wrong number, type, or value of parameters to a function is another common cause for an exception. For example, a call to abs( hello ) will raise a TypeError because the parameter is not numeric, and a call to abs(3, 5) will raise a TypeError because one parameter is expected. A ValueError is typically raised when the correct number and type of parameters are sent, but a value is illegitimate for the context of the function. For example, the int constructor accepts a string, as with int( 137 ), but a ValueError is raised if that string does not represent an integer, as with int( 3.14 ) or int( hello ). Python’s sequence types (e.g., list, tuple, and str) raise an IndexError when syntax such as data[k] is used with an integer k that is not a valid index for the given sequence (as described in Section 1.2.3). Sets and dictionaries raise a KeyError when an attempt is made to access a nonexistent element. 1.7.1 Raising an Exception An exception is thrown by executing the raise statement, with an appropriate in- stance of an exception class as an argument that designates the problem. For exam- ple, if a function for computing a square root is sent a negative value as a parameter, it can raise an exception with the command: raise ValueError( x cannot be negative ) This syntax raises a newly created instance of the ValueError class, with the error message serving as a parameter to the constructor. If this exception is not caught within the body of the function, the execution of the function immediately ceases and the exception is propagated to the calling context (and possibly beyond). When checking the validity of parameters sent to a function, it is customary to first verify that a parameter is of an appropriate type, and then to verify that it has an appropriate value. For example, the sqrt function in Python’s math library performs error-checking that might be implemented as follows: def sqrt(x): if not isinstance(x, (int, float)): raise TypeError( x must be numeric ) elif x 0: raise ValueError( x cannot be negative ) # do the real work here... Checking the type of an object can be performed at run-time using the built-in function, isinstance. In simplest form, isinstance(obj, cls) returns True if object, obj, is an instance of class, cls, or any subclass of that type. In the above example, a more general form is used with a tuple of allowable types indicated with the second parameter. After confirming that the parameter is numeric, the function enforces an expectation that the number be nonnegative, raising a ValueError otherwise.
- 57. 1.7. Exception Handling35 How much error-checking to perform within a function is a matter of debate. Checking the type and value of each parameter demands additional execution time and, if taken to an extreme, seems counter to the nature of Python. Consider the built-in sum function, which computes a sum of a collection of numbers. An im- plementation with rigorous error-checking might be written as follows: def sum(values): if not isinstance(values, collections.Iterable): raise TypeError( parameter must be an iterable type ) total = 0 for v in values: if not isinstance(v, (int, float)): raise TypeError( elements must be numeric ) total = total+ v return total The abstract base class, collections.Iterable, includes all of Python’s iterable con- tainers types that guarantee support for the for-loop syntax (e.g., list, tuple, set); we discuss iterables in Section 1.8, and the use of modules, such as collections, in Section 1.11. Within the body of the for loop, each element is verified as numeric before being added to the total. A far more direct and clear implementation of this function can be written as follows: def sum(values): total = 0 for v in values: total = total + v return total Interestingly, this simple implementation performs exactly like Python’s built-in version of the function. Even without the explicit checks, appropriate exceptions are raised naturally by the code. In particular, if values is not an iterable type, the attempt to use the for-loop syntax raises a TypeError reporting that the object is not iterable. In the case when a user sends an iterable type that includes a nonnumer- ical element, such as sum([3.14, oops ]), a TypeError is naturally raised by the evaluation of expression total + v. The error message unsupported operand type(s) for +: ’float’ and ’str’ should be sufficiently informative to the caller. Perhaps slightly less obvious is the error that results from sum([ alpha , beta ]). It will technically report a failed attempt to add an int and str, due to the initial evaluation of total + alpha , when total has been initialized to 0. In the remainder of this book, we tend to favor the simpler implementations in the interest of clean presentation, performing minimal error-checking in most situations.
- 58. 36 Chapter 1.Python Primer 1.7.2 Catching an Exception There are several philosophies regarding how to cope with possible exceptional cases when writing code. For example, if a division x/y is to be computed, there is clear risk that a ZeroDivisionError will be raised when variable y has value 0. In an ideal situation, the logic of the program may dictate that y has a nonzero value, thereby removing the concern for error. However, for more complex code, or in a case where the value of y depends on some external input to the program, there remains some possibility of an error. One philosophy for managing exceptional cases is to “look before you leap.” The goal is to entirely avoid the possibility of an exception being raised through the use of a proactive conditional test. Revisiting our division example, we might avoid the offending situation by writing: if y != 0: ratio = x / y else: ... do something else ... A second philosophy, often embraced by Python programmers, is that “it is easier to ask for forgiveness than it is to get permission.” This quote is attributed to Grace Hopper, an early pioneer in computer science. The sentiment is that we need not spend extra execution time safeguarding against every possible excep- tional case, as long as there is a mechanism for coping with a problem after it arises. In Python, this philosophy is implemented using a try-except control struc- ture. Revising our first example, the division operation can be guarded as follows: try: ratio = x / y except ZeroDivisionError: ... do something else ... In this structure, the “try” block is the primary code to be executed. Although it is a single command in this example, it can more generally be a larger block of indented code. Following the try-block are one or more “except” cases, each with an identified error type and an indented block of code that should be executed if the designated error is raised within the try-block. The relative advantage of using a try-except structure is that the non-exceptional case runs efficiently, without extraneous checks for the exceptional condition. How- ever, handling the exceptional case requires slightly more time when using a try- except structure than with a standard conditional statement. For this reason, the try-except clause is best used when there is reason to believe that the exceptional case is relatively unlikely, or when it is prohibitively expensive to proactively eval- uate a condition to avoid the exception.
- 59. 1.7. Exception Handling37 Exception handling is particularly useful when working with user input, or when reading from or writing to files, because such interactions are inherently less predictable. In Section 1.6.2, we suggest the syntax, fp = open( sample.txt ), for opening a file with read access. That command may raise an IOError for a vari- ety of reasons, such as a non-existent file, or lack of sufficient privilege for opening a file. It is significantly easier to attempt the command and catch the resulting error than it is to accurately predict whether the command will succeed. We continue by demonstrating a few other forms of the try-except syntax. Ex- ceptions are objects that can be examined when caught. To do so, an identifier must be established with a syntax as follows: try: fp = open( sample.txt ) except IOError as e: print( Unable to open the file: , e) In this case, the name, e, denotes the instance of the exception that was thrown, and printing it causes a detailed error message to be displayed (e.g., “file not found”). A try-statement may handle more than one type of exception. For example, consider the following command from Section 1.6.1: age = int(input( Enter your age in years: )) This command could fail for a variety of reasons. The call to input will raise an EOFError if the console input fails. If the call to input completes successfully, the int constructor raises a ValueError if the user has not entered characters represent- ing a valid integer. If we want to handle two or more types of errors in the same way, we can use a single except-statement, as in the following example: age = −1 # an initially invalid choice while age = 0: try: age = int(input( Enter your age in years: )) if age = 0: print( Your age must be positive ) except (ValueError, EOFError): print( Invalid response ) We use the tuple, (ValueError, EOFError), to designate the types of errors that we wish to catch with the except-clause. In this implementation, we catch either error, print a response, and continue with another pass of the enclosing while loop. We note that when an error is raised within the try-block, the remainder of that body is immediately skipped. In this example, if the exception arises within the call to input, or the subsequent call to the int constructor, the assignment to age never occurs, nor the message about needing a positive value. Because the value of age
- 60. 38 Chapter 1.Python Primer will be unchanged, the while loop will continue. If we preferred to have the while loop continue without printing the Invalid response message, we could have written the exception-clause as except (ValueError, EOFError): pass The keyword, pass, is a statement that does nothing, yet it can serve syntactically as a body of a control structure. In this way, we quietly catch the exception, thereby allowing the surrounding while loop to continue. In order to provide different responses to different types of errors, we may use two or more except-clauses as part of a try-structure. In our previous example, an EOFError suggests a more insurmountable error than simply an errant value being entered. In that case, we might wish to provide a more specific error message, or perhaps to allow the exception to interrupt the loop and be propagated to a higher context. We could implement such behavior as follows: age = −1 # an initially invalid choice while age = 0: try: age = int(input( Enter your age in years: )) if age = 0: print( Your age must be positive ) except ValueError: print( That is an invalid age specification ) except EOFError: print( There was an unexpected error reading input. ) raise # let s re-raise this exception In this implementation, we have separate except-clauses for the ValueError and EOFError cases. The body of the clause for handling an EOFError relies on another technique in Python. It uses the raise statement without any subsequent argument, to re-raise the same exception that is currently being handled. This allows us to provide our own response to the exception, and then to interrupt the while loop and propagate the exception upward. In closing, we note two additional features of try-except structures in Python. It is permissible to have a final except-clause without any identified error types, using syntax except:, to catch any other exceptions that occurred. However, this technique should be used sparingly, as it is difficult to suggest how to handle an error of an unknown type. A try-statement can have a finally clause, with a body of code that will always be executed in the standard or exceptional cases, even when an uncaught or re-raised exception occurs. That block is typically used for critical cleanup work, such as closing an open file.
- 61. 1.8. Iterators andGenerators 39 1.8 Iterators and Generators In Section 1.4.2, we introduced the for-loop syntax beginning as: for element in iterable: and we noted that there are many types of objects in Python that qualify as being iterable. Basic container types, such as list, tuple, and set, qualify as iterable types. Furthermore, a string can produce an iteration of its characters, a dictionary can produce an iteration of its keys, and a file can produce an iteration of its lines. User- defined types may also support iteration. In Python, the mechanism for iteration is based upon the following conventions: • An iterator is an object that manages an iteration through a series of values. If variable, i, identifies an iterator object, then each call to the built-in function, next(i), produces a subsequent element from the underlying series, with a StopIteration exception raised to indicate that there are no further elements. • An iterable is an object, obj, that produces an iterator via the syntax iter(obj). By these definitions, an instance of a list is an iterable, but not itself an iterator. With data = [1, 2, 4, 8], it is not legal to call next(data). However, an iterator object can be produced with syntax, i = iter(data), and then each subsequent call to next(i) will return an element of that list. The for-loop syntax in Python simply automates this process, creating an iterator for the give iterable, and then repeatedly calling for the next element until catching the StopIteration exception. More generally, it is possible to create multiple iterators based upon the same iterable object, with each iterator maintaining its own state of progress. However, iterators typically maintain their state with indirect reference back to the original collection of elements. For example, calling iter(data) on a list instance produces an instance of the list iterator class. That iterator does not store its own copy of the list of elements. Instead, it maintains a current index into the original list, represent- ing the next element to be reported. Therefore, if the contents of the original list are modified after the iterator is constructed, but before the iteration is complete, the iterator will be reporting the updated contents of the list. Python also supports functions and classes that produce an implicit iterable se- ries of values, that is, without constructing a data structure to store all of its values at once. For example, the call range(1000000) does not return a list of numbers; it returns a range object that is iterable. This object generates the million values one at a time, and only as needed. Such a lazy evaluation technique has great advan- tage. In the case of range, it allows a loop of the form, for j in range(1000000):, to execute without setting aside memory for storing one million values. Also, if such a loop were to be interrupted in some fashion, no time will have been spent computing unused values of the range.
- 62. 40 Chapter 1.Python Primer We see lazy evaluation used in many of Python’s libraries. For example, the dictionary class supports methods keys(), values(), and items(), which respec- tively produce a “view” of all keys, values, or (key,value) pairs within a dictionary. None of these methods produces an explicit list of results. Instead, the views that are produced are iterable objects based upon the actual contents of the dictionary. An explicit list of values from such an iteration can be immediately constructed by calling the list class constructor with the iteration as a parameter. For example, the syntax list(range(1000)) produces a list instance with values from 0 to 999, while the syntax list(d.values()) produces a list that has elements based upon the current values of dictionary d. We can similarly construct a tuple or set instance based upon a given iterable. Generators In Section 2.3.4, we will explain how to define a class whose instances serve as iterators. However, the most convenient technique for creating iterators in Python is through the use of generators. A generator is implemented with a syntax that is very similar to a function, but instead of returning values, a yield statement is executed to indicate each element of the series. As an example, consider the goal of determining all factors of a positive integer. For example, the number 100 has factors 1, 2, 4, 5, 10, 20, 25, 50, 100. A traditional function might produce and return a list containing all factors, implemented as: def factors(n): # traditional function that computes factors results = [ ] # store factors in a new list for k in range(1,n+1): if n % k == 0: # divides evenly, thus k is a factor results.append(k) # add k to the list of factors return results # return the entire list In contrast, an implementation of a generator for computing those factors could be implemented as follows: def factors(n): # generator that computes factors for k in range(1,n+1): if n % k == 0: # divides evenly, thus k is a factor yield k # yield this factor as next result Notice use of the keyword yield rather than return to indicate a result. This indi- cates to Python that we are defining a generator, rather than a traditional function. It is illegal to combine yield and return statements in the same implementation, other than a zero-argument return statement to cause a generator to end its execution. If a programmer writes a loop such as for factor in factors(100):, an instance of our generator is created. For each iteration of the loop, Python executes our procedure
- 63. 1.8. Iterators andGenerators 41 until a yield statement indicates the next value. At that point, the procedure is tem- porarily interrupted, only to be resumed when another value is requested. When the flow of control naturally reaches the end of our procedure (or a zero-argument return statement), a StopIteration exception is automatically raised. Although this particular example uses a single yield statement in the source code, a generator can rely on multiple yield statements in different constructs, with the generated series determined by the natural flow of control. For example, we can greatly improve the efficiency of our generator for computing factors of a number, n, by only test- ing values up to the square root of that number, while reporting the factor n//k that is associated with each k (unless n//k equals k). We might implement such a generator as follows: def factors(n): # generator that computes factors k = 1 while k k n: # while k sqrt(n) if n % k == 0: yield k yield n // k k += 1 if k k == n: # special case if n is perfect square yield k We should note that this generator differs from our first version in that the factors are not generated in strictly increasing order. For example, factors(100) generates the series 1,100,2,50,4,25,5,20,10. In closing, we wish to emphasize the benefits of lazy evaluation when using a generator rather than a traditional function. The results are only computed if re- quested, and the entire series need not reside in memory at one time. In fact, a generator can effectively produce an infinite series of values. As an example, the Fibonacci numbers form a classic mathematical sequence, starting with value 0, then value 1, and then each subsequent value being the sum of the two preceding values. Hence, the Fibonacci series begins as: 0,1,1,2,3,5,8,13,.... The follow- ing generator produces this infinite series. def fibonacci(): a = 0 b = 1 while True: # keep going... yield a # report value, a, during this pass future = a + b a = b # this will be next value reported b = future # and subsequently this
- 64. 42 Chapter 1.Python Primer 1.9 Additional Python Conveniences In this section, we introduce several features of Python that are particularly conve- nient for writing clean, concise code. Each of these syntaxes provide functionality that could otherwise be accomplished using functionality that we have introduced earlier in this chapter. However, at times, the new syntax is a more clear and direct expression of the logic. 1.9.1 Conditional Expressions Python supports a conditional expression syntax that can replace a simple control structure. The general syntax is an expression of the form: expr1 if condition else expr2 This compound expression evaluates to expr1 if the condition is true, and otherwise evaluates to expr2. For those familiar with Java or C++, this is equivalent to the syntax, condition ? expr1 : expr2, in those languages. As an example, consider the goal of sending the absolute value of a variable, n, to a function (and without relying on the built-in abs function, for the sake of ex- ample). Using a traditional control structure, we might accomplish this as follows: if n = 0: param = n else: param = −n result = foo(param) # call the function With the conditional expression syntax, we can directly assign a value to variable, param, as follows: param = n if n = 0 else −n # pick the appropriate value result = foo(param) # call the function In fact, there is no need to assign the compound expression to a variable. A condi- tional expression can itself serve as a parameter to the function, written as follows: result = foo(n if n = 0 else −n) Sometimes, the mere shortening of source code is advantageous because it avoids the distraction of a more cumbersome control structure. However, we rec- ommend that a conditional expression be used only when it improves the readability of the source code, and when the first of the two options is the more “natural” case, given its prominence in the syntax. (We prefer to view the alternative value as more exceptional.)
- 65. 1.9. Additional PythonConveniences 43 1.9.2 Comprehension Syntax A very common programming task is to produce one series of values based upon the processing of another series. Often, this task can be accomplished quite simply in Python using what is known as a comprehension syntax. We begin by demon- strating list comprehension, as this was the first form to be supported by Python. Its general form is as follows: [ expression for value in iterable if condition ] We note that both expression and condition may depend on value, and that the if-clause is optional. The evaluation of the comprehension is logically equivalent to the following traditional control structure for computing a resulting list: result = [ ] for value in iterable: if condition: result.append(expression) As a concrete example, a list of the squares of the numbers from 1 to n, that is [1,4,9,16,25,... ,n2], can be created by traditional means as follows: squares = [ ] for k in range(1, n+1): squares.append(k k) With list comprehension, this logic is expressed as follows: squares = [k k for k in range(1, n+1)] As a second example, Section 1.8 introduced the goal of producing a list of factors for an integer n. That task is accomplished with the following list comprehension: factors = [k for k in range(1,n+1) if n % k == 0] Python supports similar comprehension syntaxes that respectively produce a set, generator, or dictionary. We compare those syntaxes using our example for producing the squares of numbers. [ k k for k in range(1, n+1) ] list comprehension { k k for k in range(1, n+1) } set comprehension ( k k for k in range(1, n+1) ) generator comprehension { k : k k for k in range(1, n+1) } dictionary comprehension The generator syntax is particularly attractive when results do not need to be stored in memory. For example, to compute the sum of the first n squares, the genera- tor syntax, total = sum(k k for k in range(1, n+1)), is preferred to the use of an explicitly instantiated list comprehension as the parameter.
- 66. 44 Chapter 1.Python Primer 1.9.3 Packing and Unpacking of Sequences Python provides two additional conveniences involving the treatment of tuples and other sequence types. The first is rather cosmetic. If a series of comma-separated expressions are given in a larger context, they will be treated as a single tuple, even if no enclosing parentheses are provided. For example, the assignment data = 2, 4, 6, 8 results in identifier, data, being assigned to the tuple (2, 4, 6, 8). This behavior is called automatic packing of a tuple. One common use of packing in Python is when returning multiple values from a function. If the body of a function executes the command, return x, y it will be formally returning a single object that is the tuple (x, y). As a dual to the packing behavior, Python can automatically unpack a se- quence, allowing one to assign a series of individual identifiers to the elements of sequence. As an example, we can write a, b, c, d = range(7, 11) which has the effect of assigning a=7, b=8, c=9, and d=10, as those are the four values in the sequence returned by the call to range. For this syntax, the right-hand side expression can be any iterable type, as long as the number of variables on the left-hand side is the same as the number of elements in the iteration. This technique can be used to unpack tuples returned by a function. For exam- ple, the built-in function, divmod(a, b), returns the pair of values (a // b, a % b) associated with an integer division. Although the caller can consider the return value to be a single tuple, it is possible to write quotient, remainder = divmod(a, b) to separately identify the two entries of the returned tuple. This syntax can also be used in the context of a for loop, when iterating over a sequence of iterables, as in for x, y in [ (7, 2), (5, 8), (6, 4) ]: In this example, there will be three iterations of the loop. During the first pass, x=7 and y=2, and so on. This style of loop is quite commonly used to iterate through key-value pairs that are returned by the items() method of the dict class, as in: for k, v in mapping.items():
- 67. 1.9. Additional PythonConveniences 45 Simultaneous Assignments The combination of automatic packing and unpacking forms a technique known as simultaneous assignment, whereby we explicitly assign a series of values to a series of identifiers, using a syntax: x, y, z = 6, 2, 5 In effect, the right-hand side of this assignment is automatically packed into a tuple, and then automatically unpacked with its elements assigned to the three identifiers on the left-hand side. When using a simultaneous assignment, all of the expressions are evaluated on the right-hand side before any of the assignments are made to the left-hand variables. This is significant, as it provides a convenient means for swapping the values associated with two variables: j, k = k, j With this command, j will be assigned to the old value of k, and k will be assigned to the old value of j. Without simultaneous assignment, a swap typically requires more delicate use of a temporary variable, such as temp = j j = k k = temp With the simultaneous assignment, the unnamed tuple representing the packed val- ues on the right-hand side implicitly serves as the temporary variable when per- forming such a swap. The use of simultaneous assignments can greatly simplify the presentation of code. As an example, we reconsider the generator on page 41 that produces the Fibonacci series. The original code requires separate initialization of variables a and b to begin the series. Within each pass of the loop, the goal was to reassign a and b, respectively, to the values of b and a+b. At the time, we accomplished this with brief use of a third variable. With simultaneous assignments, that generator can be implemented more directly as follows: def fibonacci(): a, b = 0, 1 while True: yield a a, b = b, a+b
- 68. 46 Chapter 1.Python Primer 1.10 Scopes and Namespaces When computing a sum with the syntax x + y in Python, the names x and y must have been previously associated with objects that serve as values; a NameError will be raised if no such definitions are found. The process of determining the value associated with an identifier is known as name resolution. Whenever an identifier is assigned to a value, that definition is made with a specific scope. Top-level assignments are typically made in what is known as global scope. Assignments made within the body of a function typically have scope that is local to that function call. Therefore, an assignment, x = 5, within a function has no effect on the identifier, x, in the broader scope. Each distinct scope in Python is represented using an abstraction known as a namespace. A namespace manages all identifiers that are currently defined in a given scope. Figure 1.8 portrays two namespaces, one being that of a caller to our count function from Section 1.5, and the other being the local namespace during the execution of that function. A- str A str CS float 3.56 int 2 item data grades major gpa target n list str B+ str A- str Figure 1.8: A portrayal of the two namespaces associated with a user’s call count(grades, A ), as defined in Section 1.5. The left namespace is the caller’s and the right namespace represents the local scope of the function. Python implements a namespace with its own dictionary that maps each iden- tifying string (e.g., n ) to its associated value. Python provides several ways to examine a given namespace. The function, dir, reports the names of the identifiers in a given namespace (i.e., the keys of the dictionary), while the function, vars, returns the full dictionary. By default, calls to dir() and vars() report on the most locally enclosing namespace in which they are executed.
- 69. 1.10. Scopes andNamespaces 47 When an identifier is indicated in a command, Python searches a series of namespaces in the process of name resolution. First, the most locally enclosing scope is searched for a given name. If not found there, the next outer scope is searched, and so on. We will continue our examination of namespaces, in Sec- tion 2.5, when discussing Python’s treatment of object-orientation. We will see that each object has its own namespace to store its attributes, and that classes each have a namespace as well. First-Class Objects In the terminology of programming languages, first-class objects are instances of a type that can be assigned to an identifier, passed as a parameter, or returned by a function. All of the data types we introduced in Section 1.2.3, such as int and list, are clearly first-class types in Python. In Python, functions and classes are also treated as first-class objects. For example, we could write the following: scream = print # assign name ’scream’ to the function denoted as ’print’ scream( Hello ) # call that function In this case, we have not created a new function, we have simply defined scream as an alias for the existing print function. While there is little motivation for pre- cisely this example, it demonstrates the mechanism that is used by Python to al- low one function to be passed as a parameter to another. On page 28, we noted that the built-in function, max, accepts an optional keyword parameter to specify a non-default order when computing a maximum. For example, a caller can use the syntax, max(a, b, key=abs), to determine which value has the larger absolute value. Within the body of that function, the formal parameter, key, is an identifier that will be assigned to the actual parameter, abs. In terms of namespaces, an assignment such as scream = print, introduces the identifier, scream, into the current namespace, with its value being the object that represents the built-in function, print. The same mechanism is applied when a user- defined function is declared. For example, our count function from Section 1.5 beings with the following syntax: def count(data, target): ... Such a declaration introduces the identifier, count, into the current namespace, with the value being a function instance representing its implementation. In similar fashion, the name of a newly defined class is associated with a representation of that class as its value. (Class definitions will be introduced in the next chapter.)
- 70. 48 Chapter 1.Python Primer 1.11 Modules and the Import Statement We have already introduced many functions (e.g., max) and classes (e.g., list) that are defined within Python’s built-in namespace. Depending on the version of Python, there are approximately 130–150 definitions that were deemed significant enough to be included in that built-in namespace. Beyond the built-in definitions, the standard Python distribution includes per- haps tens of thousands of other values, functions, and classes that are organized in additional libraries, known as modules, that can be imported from within a pro- gram. As an example, we consider the math module. While the built-in namespace includes a few mathematical functions (e.g., abs, min, max, round), many more are relegated to the math module (e.g., sin, cos, sqrt). That module also defines approximate values for the mathematical constants, pi and e. Python’s import statement loads definitions from a module into the current namespace. One form of an import statement uses a syntax such as the following: from math import pi, sqrt This command adds both pi and sqrt, as defined in the math module, into the cur- rent namespace, allowing direct use of the identifier, pi, or a call of the function, sqrt(2). If there are many definitions from the same module to be imported, an asterisk may be used as a wild card, as in, from math import , but this form should be used sparingly. The danger is that some of the names defined in the mod- ule may conflict with names already in the current namespace (or being imported from another module), and the import causes the new definitions to replace existing ones. Another approach that can be used to access many definitions from the same module is to import the module itself, using a syntax such as: import math Formally, this adds the identifier, math, to the current namespace, with the module as its value. (Modules are also first-class objects in Python.) Once imported, in- dividual definitions from the module can be accessed using a fully-qualified name, such as math.pi or math.sqrt(2). Creating a New Module To create a new module, one simply has to put the relevant definitions in a file named with a .py suffix. Those definitions can be imported from any other .py file within the same project directory. For example, if we were to put the definition of our count function (see Section 1.5) into a file named utility.py, we could import that function using the syntax, from utility import count.
- 71. 1.11. Modules andthe Import Statement 49 It is worth noting that top-level commands with the module source code are executed when the module is first imported, almost as if the module were its own script. There is a special construct for embedding commands within the module that will be executed if the module is directly invoked as a script, but not when the module is imported from another script. Such commands should be placed in a body of a conditional statement of the following form, if name == __main__ : Using our hypothetical utility.py module as an example, such commands will be executed if the interpreter is started with a command python utility.py, but not when the utility module is imported into another context. This approach is often used to embed what are known as unit tests within the module; we will discuss unit testing further in Section 2.2.4. 1.11.1 Existing Modules Table 1.7 provides a summary of a few available modules that are relevant to a study of data structures. We have already discussed the math module briefly. In the remainder of this section, we highlight another module that is particularly important for some of the data structures and algorithms that we will study later in this book. Existing Modules Module Name Description array Provides compact array storage for primitive types. collections Defines additional data structures and abstract base classes involving collections of objects. copy Defines general functions for making copies of objects. heapq Provides heap-based priority queue functions (see Section 9.3.7). math Defines common mathematical constants and functions. os Provides support for interactions with the operating system. random Provides random number generation. re Provides support for processing regular expressions. sys Provides additional level of interaction with the Python interpreter. time Provides support for measuring time, or delaying a program. Table 1.7: Some existing Python modules relevant to data structures and algorithms. Pseudo-Random Number Generation Python’s random module provides the ability to generate pseudo-random numbers, that is, numbers that are statistically random (but not necessarily truly random). A pseudo-random number generator uses a deterministic formula to generate the
- 72. 50 Chapter 1.Python Primer next number in a sequence based upon one or more past numbers that it has gen- erated. Indeed, a simple yet popular pseudo-random number generator chooses its next number based solely on the most recently chosen number and some additional parameters using the following formula. next = (a*current + b) % n; where a, b, and n are appropriately chosen integers. Python uses a more advanced technique known as a Mersenne twister. It turns out that the sequences generated by these techniques can be proven to be statistically uniform, which is usually good enough for most applications requiring random numbers, such as games. For applications, such as computer security settings, where one needs unpredictable random sequences, this kind of formula should not be used. Instead, one should ideally sample from a source that is actually random, such as radio static coming from outer space. Since the next number in a pseudo-random generator is determined by the pre- vious number(s), such a generator always needs a place to start, which is called its seed. The sequence of numbers generated for a given seed will always be the same. One common trick to get a different sequence each time a program is run is to use a seed that will be different for each run. For example, we could use some timed input from a user or the current system time in milliseconds. Python’s random module provides support for pseudo-random number gener- ation by defining a Random class; instances of that class serve as generators with independent state. This allows different aspects of a program to rely on their own pseudo-random number generator, so that calls to one generator do not affect the sequence of numbers produced by another. For convenience, all of the methods supported by the Random class are also supported as stand-alone functions of the random module (essentially using a single generator instance for all top-level calls). Syntax Description seed(hashable) Initializes the pseudo-random number generator based upon the hash value of the parameter random() Returns a pseudo-random floating-point value in the interval [0.0,1.0). randint(a,b) Returns a pseudo-random integer in the closed interval [a,b]. randrange(start, stop, step) Returns a pseudo-random integer in the standard Python range indicated by the parameters. choice(seq) Returns an element of the given sequence chosen pseudo-randomly. shuffle(seq) Reorders the elements of the given sequence pseudo-randomly. Table 1.8: Methods supported by instances of the Random class, and as top-level functions of the random module.
- 73. 1.12. Exercises 51 1.12Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-1.1 Write a short Python function, is multiple(n, m), that takes two integer values and returns True if n is a multiple of m, that is, n = mi for some integer i, and False otherwise. R-1.2 Write a short Python function, is even(k), that takes an integer value and returns True if k is even, and False otherwise. However, your function cannot use the multiplication, modulo, or division operators. R-1.3 Write a short Python function, minmax(data), that takes a sequence of one or more numbers, and returns the smallest and largest numbers, in the form of a tuple of length two. Do not use the built-in functions min or max in implementing your solution. R-1.4 Write a short Python function that takes a positive integer n and returns the sum of the squares of all the positive integers smaller than n. R-1.5 Give a single command that computes the sum from Exercise R-1.4, rely- ing on Python’s comprehension syntax and the built-in sum function. R-1.6 Write a short Python function that takes a positive integer n and returns the sum of the squares of all the odd positive integers smaller than n. R-1.7 Give a single command that computes the sum from Exercise R-1.6, rely- ing on Python’s comprehension syntax and the built-in sum function. R-1.8 Python allows negative integers to be used as indices into a sequence, such as a string. If string s has length n, and expression s[k] is used for in- dex −n ≤ k 0, what is the equivalent index j ≥ 0 such that s[j] references the same element? R-1.9 What parameters should be sent to the range constructor, to produce a range with values 50, 60, 70, 80? R-1.10 What parameters should be sent to the range constructor, to produce a range with values 8, 6, 4, 2, 0, −2, −4, −6, −8? R-1.11 Demonstrate how to use Python’s list comprehension syntax to produce the list [1, 2, 4, 8, 16, 32, 64, 128, 256]. R-1.12 Python’s random module includes a function choice(data) that returns a random element from a non-empty sequence. The random module in- cludes a more basic function randrange, with parameterization similar to the built-in range function, that return a random choice from the given range. Using only the randrange function, implement your own version of the choice function.
- 74. 52 Chapter 1.Python Primer Creativity C-1.13 Write a pseudo-code description of a function that reverses a list of n integers, so that the numbers are listed in the opposite order than they were before, and compare this method to an equivalent Python function for doing the same thing. C-1.14 Write a short Python function that takes a sequence of integer values and determines if there is a distinct pair of numbers in the sequence whose product is odd. C-1.15 Write a Python function that takes a sequence of numbers and determines if all the numbers are different from each other (that is, they are distinct). C-1.16 In our implementation of the scale function (page 25), the body of the loop executes the command data[j] = factor. We have discussed that numeric types are immutable, and that use of the = operator in this context causes the creation of a new instance (not the mutation of an existing instance). How is it still possible, then, that our implementation of scale changes the actual parameter sent by the caller? C-1.17 Had we implemented the scale function (page 25) as follows, does it work properly? def scale(data, factor): for val in data: val = factor Explain why or why not. C-1.18 Demonstrate how to use Python’s list comprehension syntax to produce the list [0, 2, 6, 12, 20, 30, 42, 56, 72, 90]. C-1.19 Demonstrate how to use Python’s list comprehension syntax to produce the list [ a , b , c , ..., z ], but without having to type all 26 such characters literally. C-1.20 Python’s random module includes a function shuffle(data) that accepts a list of elements and randomly reorders the elements so that each possi- ble order occurs with equal probability. The random module includes a more basic function randint(a, b) that returns a uniformly random integer from a to b (including both endpoints). Using only the randint function, implement your own version of the shuffle function. C-1.21 Write a Python program that repeatedly reads lines from standard input until an EOFError is raised, and then outputs those lines in reverse order (a user can indicate end of input by typing ctrl-D).
- 75. 1.12. Exercises 53 C-1.22Write a short Python program that takes two arrays a and b of length n storing int values, and returns the dot product of a and b. That is, it returns an array c of length n such that c[i] = a[i]·b[i], for i = 0,...,n−1. C-1.23 Give an example of a Python code fragment that attempts to write an ele- ment to a list based on an index that may be out of bounds. If that index is out of bounds, the program should catch the exception that results, and print the following error message: “Don’t try buffer overflow attacks in Python!” C-1.24 Write a short Python function that counts the number of vowels in a given character string. C-1.25 Write a short Python function that takes a string s, representing a sentence, and returns a copy of the string with all punctuation removed. For exam- ple, if given the string Let s try, Mike., this function would return Lets try Mike. C-1.26 Write a short program that takes as input three integers, a, b, and c, from the console and determines if they can be used in a correct arithmetic formula (in the given order), like “a+b = c,” “a = b−c,” or “a∗b = c.” C-1.27 In Section 1.8, we provided three different implementations of a generator that computes factors of a given integer. The third of those implementa- tions, from page 41, was the most efficient, but we noted that it did not yield the factors in increasing order. Modify the generator so that it reports factors in increasing order, while maintaining its general performance ad- vantages. C-1.28 The p-norm of a vector v = (v1,v2,...,vn) in n-dimensional space is de- fined as v = p vp 1 +vp 2 +···+vp n. For the special case of p = 2, this results in the traditional Euclidean norm, which represents the length of the vector. For example, the Eu- clidean norm of a two-dimensional vector with coordinates (4,3) has a Euclidean norm of √ 42 +32 = √ 16+9 = √ 25 = 5. Give an implemen- tation of a function named norm such that norm(v, p) returns the p-norm value of v and norm(v) returns the Euclidean norm of v. You may assume that v is a list of numbers.
- 76. 54 Chapter 1.Python Primer Projects P-1.29 Write a Python program that outputs all possible strings formed by using the characters c , a , t , d , o , and g exactly once. P-1.30 Write a Python program that can take a positive integer greater than 2 as input and write out the number of times one must repeatedly divide this number by 2 before getting a value less than 2. P-1.31 Write a Python program that can “make change.” Your program should take two numbers as input, one that is a monetary amount charged and the other that is a monetary amount given. It should then return the number of each kind of bill and coin to give back as change for the difference between the amount given and the amount charged. The values assigned to the bills and coins can be based on the monetary system of any current or former government. Try to design your program so that it returns as few bills and coins as possible. P-1.32 Write a Python program that can simulate a simple calculator, using the console as the exclusive input and output device. That is, each input to the calculator, be it a number, like 12.34 or 1034, or an operator, like + or =, can be done on a separate line. After each such input, you should output to the Python console what would be displayed on your calculator. P-1.33 Write a Python program that simulates a handheld calculator. Your pro- gram should process input from the Python console representing buttons that are “pushed,” and then output the contents of the screen after each op- eration is performed. Minimally, your calculator should be able to process the basic arithmetic operations and a reset/clear operation. P-1.34 A common punishment for school children is to write out a sentence mul- tiple times. Write a Python stand-alone program that will write out the following sentence one hundred times: “I will never spam my friends again.” Your program should number each of the sentences and it should make eight different random-looking typos. P-1.35 The birthday paradox says that the probability that two people in a room will have the same birthday is more than half, provided n, the number of people in the room, is more than 23. This property is not really a paradox, but many people find it surprising. Design a Python program that can test this paradox by a series of experiments on randomly generated birthdays, which test this paradox for n = 5,10,15,20,... ,100. P-1.36 Write a Python program that inputs a list of words, separated by white- space, and outputs how many times each word appears in the list. You need not worry about efficiency at this point, however, as this topic is something that will be addressed later in this book.
- 77. Chapter Notes 55 ChapterNotes The official Python Web site (http://www.python.org) has a wealth of information, in- cluding a tutorial and full documentation of the built-in functions, classes, and standard modules. The Python interpreter is itself a useful reference, as the interactive command help(foo) provides documentation for any function, class, or module that foo identifies. Books providing an introduction to programming in Python include titles authored by Campbell et al. [22], Cedar [25], Dawson [32], Goldwasser and Letscher [43], Lutz [72], Perkovic [82], and Zelle [105]. More complete reference books on Python include titles by Beazley [12], and Summerfield [91].
- 78. Chapter 2 Object-Oriented Programming Contents 2.1Goals, Principles, and Patterns . . . . . . . . . . . . . . . . 57 2.1.1 Object-Oriented Design Goals . . . . . . . . . . . . . . . 57 2.1.2 Object-Oriented Design Principles . . . . . . . . . . . . . 58 2.1.3 Design Patterns . . . . . . . . . . . . . . . . . . . . . . . 61 2.2 Software Development . . . . . . . . . . . . . . . . . . . . 62 2.2.1 Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 2.2.2 Pseudo-Code . . . . . . . . . . . . . . . . . . . . . . . . 64 2.2.3 Coding Style and Documentation . . . . . . . . . . . . . . 64 2.2.4 Testing and Debugging . . . . . . . . . . . . . . . . . . . 67 2.3 Class Definitions . . . . . . . . . . . . . . . . . . . . . . . . 69 2.3.1 Example: CreditCard Class . . . . . . . . . . . . . . . . . 69 2.3.2 Operator Overloading and Python’s Special Methods . . . 74 2.3.3 Example: Multidimensional Vector Class . . . . . . . . . . 77 2.3.4 Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . 79 2.3.5 Example: Range Class . . . . . . . . . . . . . . . . . . . . 80 2.4 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 2.4.1 Extending the CreditCard Class . . . . . . . . . . . . . . . 83 2.4.2 Hierarchy of Numeric Progressions . . . . . . . . . . . . . 87 2.4.3 Abstract Base Classes . . . . . . . . . . . . . . . . . . . . 93 2.5 Namespaces and Object-Orientation . . . . . . . . . . . . . 96 2.5.1 Instance and Class Namespaces . . . . . . . . . . . . . . . 96 2.5.2 Name Resolution and Dynamic Dispatch . . . . . . . . . . 100 2.6 Shallow and Deep Copying . . . . . . . . . . . . . . . . . . 101 2.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
- 79. 2.1. Goals, Principles,and Patterns 57 2.1 Goals, Principles, and Patterns As the name implies, the main “actors” in the object-oriented paradigm are called objects. Each object is an instance of a class. Each class presents to the outside world a concise and consistent view of the objects that are instances of this class, without going into too much unnecessary detail or giving others access to the inner workings of the objects. The class definition typically specifies instance variables, also known as data members, that the object contains, as well as the methods, also known as member functions, that the object can execute. This view of computing is intended to fulfill several goals and incorporate several design principles, which we discuss in this chapter. 2.1.1 Object-Oriented Design Goals Software implementations should achieve robustness, adaptability, and reusabil- ity. (See Figure 2.1.) Robustness Adaptability Reusability Figure 2.1: Goals of object-oriented design. Robustness Every good programmer wants to develop software that is correct, which means that a program produces the right output for all the anticipated inputs in the program’s application. In addition, we want software to be robust, that is, capable of handling unexpected inputs that are not explicitly defined for its application. For example, if a program is expecting a positive integer (perhaps representing the price of an item) and instead is given a negative integer, then the program should be able to recover gracefully from this error. More importantly, in life-critical applications, where a software error can lead to injury or loss of life, software that is not robust could be deadly. This point was driven home in the late 1980s in accidents involv- ing Therac-25, a radiation-therapy machine, which severely overdosed six patients between 1985 and 1987, some of whom died from complications resulting from their radiation overdose. All six accidents were traced to software errors.
- 80. 58 Chapter 2.Object-Oriented Programming Adaptability Modern software applications, such as Web browsers and Internet search engines, typically involve large programs that are used for many years. Software, there- fore, needs to be able to evolve over time in response to changing conditions in its environment. Thus, another important goal of quality software is that it achieves adaptability (also called evolvability). Related to this concept is portability, which is the ability of software to run with minimal change on different hardware and operating system platforms. An advantage of writing software in Python is the portability provided by the language itself. Reusability Going hand in hand with adaptability is the desire that software be reusable, that is, the same code should be usable as a component of different systems in various applications. Developing quality software can be an expensive enterprise, and its cost can be offset somewhat if the software is designed in a way that makes it easily reusable in future applications. Such reuse should be done with care, however, for one of the major sources of software errors in the Therac-25 came from inappropri- ate reuse of Therac-20 software (which was not object-oriented and not designed for the hardware platform used with the Therac-25). 2.1.2 Object-Oriented Design Principles Chief among the principles of the object-oriented approach, which are intended to facilitate the goals outlined above, are the following (see Figure 2.2): • Modularity • Abstraction • Encapsulation Modularity Abstraction Encapsulation Figure 2.2: Principles of object-oriented design.
- 81. 2.1. Goals, Principles,and Patterns 59 Modularity Modern software systems typically consist of several different components that must interact correctly in order for the entire system to work properly. Keeping these interactions straight requires that these different components be well orga- nized. Modularity refers to an organizing principle in which different components of a software system are divided into separate functional units. As a real-world analogy, a house or apartment can be viewed as consisting of several interacting units: electrical, heating and cooling, plumbing, and structural. Rather than viewing these systems as one giant jumble of wires, vents, pipes, and boards, the organized architect designing a house or apartment will view them as separate modules that interact in well-defined ways. In so doing, he or she is using modularity to bring a clarity of thought that provides a natural way of organizing functions into distinct manageable units. In like manner, using modularity in a software system can also provide a pow- erful organizing framework that brings clarity to an implementation. In Python, we have already seen that a module is a collection of closely related functions and classes that are defined together in a single file of source code. Python’s standard libraries include, for example, the math module, which provides definitions for key mathematical constants and functions, and the os module, which provides support for interacting with the operating system. The use of modularity helps support the goals listed in Section 2.1.1. Robust- ness is greatly increased because it is easier to test and debug separate components before they are integrated into a larger software system. Furthermore, bugs that per- sist in a complete system might be traced to a particular component, which can be fixed in relative isolation. The structure imposed by modularity also helps enable software reusability. If software modules are written in a general way, the modules can be reused when related need arises in other contexts. This is particularly rel- evant in a study of data structures, which can typically be designed with sufficient abstraction and generality to be reused in many applications. Abstraction The notion of abstraction is to distill a complicated system down to its most funda- mental parts. Typically, describing the parts of a system involves naming them and explaining their functionality. Applying the abstraction paradigm to the design of data structures gives rise to abstract data types (ADTs). An ADT is a mathematical model of a data structure that specifies the type of data stored, the operations sup- ported on them, and the types of parameters of the operations. An ADT specifies what each operation does, but not how it does it. We will typically refer to the collective set of behaviors supported by an ADT as its public interface.
- 82. 60 Chapter 2.Object-Oriented Programming As a programming language, Python provides a great deal of latitude in regard to the specification of an interface. Python has a tradition of treating abstractions implicitly using a mechanism known as duck typing. As an interpreted and dy- namically typed language, there is no “compile time” checking of data types in Python, and no formal requirement for declarations of abstract base classes. In- stead programmers assume that an object supports a set of known behaviors, with the interpreter raising a run-time error if those assumptions fail. The description of this as “duck typing” comes from an adage attributed to poet James Whitcomb Riley, stating that “when I see a bird that walks like a duck and swims like a duck and quacks like a duck, I call that bird a duck.” More formally, Python supports abstract data types using a mechanism known as an abstract base class (ABC). An abstract base class cannot be instantiated (i.e., you cannot directly create an instance of that class), but it defines one or more common methods that all implementations of the abstraction must have. An ABC is realized by one or more concrete classes that inherit from the abstract base class while providing implementations for those method declared by the ABC. Python’s abc module provides formal support for ABCs, although we omit such declarations for simplicity. We will make use of several existing abstract base classes coming from Python’s collections module, which includes definitions for several common data structure ADTs, and concrete implementations of some of those abstractions. Encapsulation Another important principle of object-oriented design is encapsulation. Different components of a software system should not reveal the internal details of their respective implementations. One of the main advantages of encapsulation is that it gives one programmer freedom to implement the details of a component, without concern that other programmers will be writing code that intricately depends on those internal decisions. The only constraint on the programmer of a component is to maintain the public interface for the component, as other programmers will be writing code that depends on that interface. Encapsulation yields robustness and adaptability, for it allows the implementation details of parts of a program to change without adversely affecting other parts, thereby making it easier to fix bugs or add new functionality with relatively local changes to a component. Throughout this book, we will adhere to the principle of encapsulation, making clear which aspects of a data structure are assumed to be public and which are assumed to be internal details. With that said, Python provides only loose support for encapsulation. By convention, names of members of a class (both data members and member functions) that start with a single underscore character (e.g., secret) are assumed to be nonpublic and should not be relied upon. Those conventions are reinforced by the intentional omission of those members from automatically generated documentation.
- 83. 2.1. Goals, Principles,and Patterns 61 2.1.3 Design Patterns Object-oriented design facilitates reusable, robust, and adaptable software. De- signing good code takes more than simply understanding object-oriented method- ologies, however. It requires the effective use of object-oriented design techniques. Computing researchers and practitioners have developed a variety of organiza- tional concepts and methodologies for designing quality object-oriented software that is concise, correct, and reusable. Of special relevance to this book is the con- cept of a design pattern, which describes a solution to a “typical” software design problem. A pattern provides a general template for a solution that can be applied in many different situations. It describes the main elements of a solution in an abstract way that can be specialized for a specific problem at hand. It consists of a name, which identifies the pattern; a context, which describes the scenarios for which this pattern can be applied; a template, which describes how the pattern is applied; and a result, which describes and analyzes what the pattern produces. We present several design patterns in this book, and we show how they can be consistently applied to implementations of data structures and algorithms. These design patterns fall into two groups—patterns for solving algorithm design prob- lems and patterns for solving software engineering problems. The algorithm design patterns we discuss include the following: • Recursion (Chapter 4) • Amortization (Sections 5.3 and 11.4) • Divide-and-conquer (Section 12.2.1) • Prune-and-search, also known as decrease-and-conquer (Section 12.7.1) • Brute force (Section 13.2.1) • Dynamic programming (Section 13.3). • The greedy method (Sections 13.4.2, 14.6.2, and 14.7) Likewise, the software engineering design patterns we discuss include: • Iterator (Sections 1.8 and 2.3.4) • Adapter (Section 6.1.2) • Position (Sections 7.4 and 8.1.2) • Composition (Sections 7.6.1, 9.2.1, and 10.1.4) • Template method (Sections 2.4.3, 8.4.6, 10.1.3, 10.5.2, and 11.2.1) • Locator (Section 9.5.1) • Factory method (Section 11.2.1) Rather than explain each of these concepts here, however, we introduce them throughout the text as noted above. For each pattern, be it for algorithm engineering or software engineering, we explain its general use and we illustrate it with at least one concrete example.
- 84. 62 Chapter 2.Object-Oriented Programming 2.2 Software Development Traditional software development involves several phases. Three major steps are: 1. Design 2. Implementation 3. Testing and Debugging In this section, we briefly discuss the role of these phases, and we introduce sev- eral good practices for programming in Python, including coding style, naming conventions, formal documentation, and unit testing. 2.2.1 Design For object-oriented programming, the design step is perhaps the most important phase in the process of developing software. For it is in the design step that we decide how to divide the workings of our program into classes, we decide how these classes will interact, what data each will store, and what actions each will perform. Indeed, one of the main challenges that beginning programmers face is deciding what classes to define to do the work of their program. While general prescriptions are hard to come by, there are some rules of thumb that we can apply when determining how to design our classes: • Responsibilities: Divide the work into different actors, each with a different responsibility. Try to describe responsibilities using action verbs. These actors will form the classes for the program. • Independence: Define the work for each class to be as independent from other classes as possible. Subdivide responsibilities between classes so that each class has autonomy over some aspect of the program. Give data (as in- stance variables) to the class that has jurisdiction over the actions that require access to this data. • Behaviors: Define the behaviors for each class carefully and precisely, so that the consequences of each action performed by a class will be well un- derstood by other classes that interact with it. These behaviors will define the methods that this class performs, and the set of behaviors for a class are the interface to the class, as these form the means for other pieces of code to interact with objects from the class. Defining the classes, together with their instance variables and methods, are key to the design of an object-oriented program. A good programmer will naturally develop greater skill in performing these tasks over time, as experience teaches him or her to notice patterns in the requirements of a program that match patterns that he or she has seen before.
- 85. 2.2. Software Development63 A common tool for developing an initial high-level design for a project is the use of CRC cards. Class-Responsibility-Collaborator (CRC) cards are simple in- dex cards that subdivide the work required of a program. The main idea behind this tool is to have each card represent a component, which will ultimately become a class in the program. We write the name of each component on the top of an index card. On the left-hand side of the card, we begin writing the responsibilities for this component. On the right-hand side, we list the collaborators for this compo- nent, that is, the other components that this component will have to interact with to perform its duties. The design process iterates through an action/actor cycle, where we first iden- tify an action (that is, a responsibility), and we then determine an actor (that is, a component) that is best suited to perform that action. The design is complete when we have assigned all actions to actors. In using index cards for this process (rather than larger pieces of paper), we are relying on the fact that each component should have a small set of responsibilities and collaborators. Enforcing this rule helps keep the individual classes manageable. As the design takes form, a standard approach to explain and document the design is the use of UML (Unified Modeling Language) diagrams to express the organization of a program. UML diagrams are a standard visual notation to express object-oriented software designs. Several computer-aided tools are available to build UML diagrams. One type of UML figure is known as a class diagram. An example of such a diagram is given in Figure 2.3, for a class that represents a consumer credit card. The diagram has three portions, with the first designating the name of the class, the second designating the recommended instance variables, and the third designating the recommended methods of the class. In Section 2.2.3, we discuss our naming conventions, and in Section 2.3.1, we provide a complete implementation of a Python CreditCard class based on this design. Class: Fields: Behaviors: make payment(amount) customer account get customer() get bank() bank get account() balance limit get balance() get limit() CreditCard charge(price) Figure 2.3: Class diagram for a proposed CreditCard class.
- 86. 64 Chapter 2.Object-Oriented Programming 2.2.2 Pseudo-Code As an intermediate step before the implementation of a design, programmers are often asked to describe algorithms in a way that is intended for human eyes only. Such descriptions are called pseudo-code. Pseudo-code is not a computer program, but is more structured than usual prose. It is a mixture of natural language and high-level programming constructs that describe the main ideas behind a generic implementation of a data structure or algorithm. Because pseudo-code is designed for a human reader, not a computer, we can communicate high-level ideas, without being burdened with low-level implementation details. At the same time, we should not gloss over important steps. Like many forms of human communication, finding the right balance is an important skill that is refined through practice. In this book, we rely on a pseudo-code style that we hope will be evident to Python programmers, yet with a mix of mathematical notations and English prose. For example, we might use the phrase “indicate an error” rather than a formal raise statement. Following conventions of Python, we rely on indentation to indicate the extent of control structures and on an indexing notation in which entries of a sequence A with length n are indexed from A[0] to A[n− 1]. However, we choose to enclose comments within curly braces { like these } in our pseudo-code, rather than using Python’s # character. 2.2.3 Coding Style and Documentation Programs should be made easy to read and understand. Good programmers should therefore be mindful of their coding style, and develop a style that communicates the important aspects of a program’s design for both humans and computers. Con- ventions for coding style tend to vary between different programming communities. The official Style Guide for Python Code is available online at http://www.python.org/dev/peps/pep-0008/ The main principles that we adopt are as follows: • Python code blocks are typically indented by 4 spaces. However, to avoid having our code fragments overrun the book’s margins, we use 2 spaces for each level of indentation. It is strongly recommended that tabs be avoided, as tabs are displayed with differing widths across systems, and tabs and spaces are not viewed as identical by the Python interpreter. Many Python-aware editors will automatically replace tabs with an appropriate number of spaces.
- 87. 2.2. Software Development65 • Use meaningful names for identifiers. Try to choose names that can be read aloud, and choose names that reflect the action, responsibility, or data each identifier is naming. ◦ Classes (other than Python’s built-in classes) should have a name that serves as a singular noun, and should be capitalized (e.g., Date rather than date or Dates). When multiple words are concatenated to form a class name, they should follow the so-called “CamelCase” convention in which the first letter of each word is capitalized (e.g., CreditCard). ◦ Functions, including member functions of a class, should be lowercase. If multiple words are combined, they should be separated by under- scores (e.g., make payment). The name of a function should typically be a verb that describes its affect. However, if the only purpose of the function is to return a value, the function name may be a noun that describes the value (e.g., sqrt rather than calculate sqrt). ◦ Names that identify an individual object (e.g., a parameter, instance variable, or local variable) should be a lowercase noun (e.g., price). Occasionally, we stray from this rule when using a single uppercase letter to designate the name of a data structures (such as tree T). ◦ Identifiers that represent a value considered to be a constant are tradi- tionally identified using all capital letters and with underscores to sep- arate words (e.g., MAX SIZE). Recall from our discussion of encapsulation that identifiers in any context that begin with a single leading underscore (e.g., secret) are intended to suggest that they are only for “internal” use to a class or module, and not part of a public interface. • Use comments that add meaning to a program and explain ambiguous or confusing constructs. In-line comments are good for quick explanations; they are indicated in Python following the # character, as in if n % 2 == 1: # n is odd Multiline block comments are good for explaining more complex code sec- tions. In Python, these are technically multiline string literals, typically de- limited with triple quotes (”””), which have no effect when executed. In the next section, we discuss the use of block comments for documentation.
- 88. 66 Chapter 2.Object-Oriented Programming Documentation Python provides integrated support for embedding formal documentation directly in source code using a mechanism known as a docstring. Formally, any string literal that appears as the first statement within the body of a module, class, or function (including a member function of a class) will be considered to be a docstring. By convention, those string literals should be delimited within triple quotes (”””). As an example, our version of the scale function from page 25 could be documented as follows: def scale(data, factor): ”””Multiply all entries of numeric data list by the given factor.””” for j in range(len(data)): data[j] = factor It is common to use the triple-quoted string delimiter for a docstring, even when the string fits on a single line, as in the above example. More detailed docstrings should begin with a single line that summarizes the purpose, followed by a blank line, and then further details. For example, we might more clearly document the scale function as follows: def scale(data, factor): ”””Multiply all entries of numeric data list by the given factor. data an instance of any mutable sequence type (such as a list) containing numeric elements factor a number that serves as the multiplicative factor for scaling ””” for j in range(len(data)): data[j] = factor A docstring is stored as a field of the module, function, or class in which it is declared. It serves as documentation and can be retrieved in a variety of ways. For example, the command help(x), within the Python interpreter, produces the documentation associated with the identified object x. An external tool named pydoc is distributed with Python and can be used to generate formal documentation as text or as a Web page. Guidelines for authoring useful docstrings are available at: http://www.python.org/dev/peps/pep-0257/ In this book, we will try to present docstrings when space allows. Omitted docstrings can be found in the online version of our source code.
- 89. 2.2. Software Development67 2.2.4 Testing and Debugging Testing is the process of experimentally checking the correctness of a program, while debugging is the process of tracking the execution of a program and discov- ering the errors in it. Testing and debugging are often the most time-consuming activity in the development of a program. Testing A careful testing plan is an essential part of writing a program. While verifying the correctness of a program over all possible inputs is usually infeasible, we should aim at executing the program on a representative subset of inputs. At the very minimum, we should make sure that every method of a class is tested at least once (method coverage). Even better, each code statement in the program should be executed at least once (statement coverage). Programs often tend to fail on special cases of the input. Such cases need to be carefully identified and tested. For example, when testing a method that sorts (that is, puts in order) a sequence of integers, we should consider the following inputs: • The sequence has zero length (no elements). • The sequence has one element. • All the elements of the sequence are the same. • The sequence is already sorted. • The sequence is reverse sorted. In addition to special inputs to the program, we should also consider special conditions for the structures used by the program. For example, if we use a Python list to store data, we should make sure that boundary cases, such as inserting or removing at the beginning or end of the list, are properly handled. While it is essential to use handcrafted test suites, it is also advantageous to run the program on a large collection of randomly generated inputs. The random module in Python provides several means for generating random numbers, or for randomizing the order of collections. The dependencies among the classes and functions of a program induce a hi- erarchy. Namely, a component A is above a component B in the hierarchy if A depends upon B, such as when function A calls function B, or function A relies on a parameter that is an instance of class B. There are two main testing strategies, top-down and bottom-up, which differ in the order in which components are tested. Top-down testing proceeds from the top to the bottom of the program hierarchy. It is typically used in conjunction with stubbing, a boot-strapping technique that replaces a lower-level component with a stub, a replacement for the component that simulates the functionality of the original. For example, if function A calls function B to get the first line of a file, when testing A we can replace B with a stub that returns a fixed string.
- 90. 68 Chapter 2.Object-Oriented Programming Bottom-up testing proceeds from lower-level components to higher-level com- ponents. For example, bottom-level functions, which do not invoke other functions, are tested first, followed by functions that call only bottom-level functions, and so on. Similarly a class that does not depend upon any other classes can be tested before another class that depends on the former. This form of testing is usually described as unit testing, as the functionality of a specific component is tested in isolation of the larger software project. If used properly, this strategy better isolates the cause of errors to the component being tested, as lower-level components upon which it relies should have already been thoroughly tested. Python provides several forms of support for automated testing. When func- tions or classes are defined in a module, testing for that module can be embedded in the same file. The mechanism for doing so was described in Section 1.11. Code that is shielded in a conditional construct of the form if name == __main__ : # perform tests... will be executed when Python is invoked directly on that module, but not when the module is imported for use in a larger software project. It is common to put tests in such a construct to test the functionality of the functions and classes specifically defined in that module. More robust support for automation of unit testing is provided by Python’s unittest module. This framework allows the grouping of individual test cases into larger test suites, and provides support for executing those suites, and reporting or analyzing the results of those tests. As software is maintained, the act of regression testing is used, whereby all previous tests are re-executed to ensure that changes to the software do not introduce new bugs in previously tested components. Debugging The simplest debugging technique consists of using print statements to track the values of variables during the execution of the program. A problem with this ap- proach is that eventually the print statements need to be removed or commented out, so they are not executed when the software is finally released. A better approach is to run the program within a debugger, which is a special- ized environment for controlling and monitoring the execution of a program. The basic functionality provided by a debugger is the insertion of breakpoints within the code. When the program is executed within the debugger, it stops at each breakpoint. While the program is stopped, the current value of variables can be inspected. The standard Python distribution includes a module named pdb, which provides debugging support directly within the interpreter. Most IDEs for Python, such as IDLE, provide debugging environments with graphical user interfaces.
- 91. 2.3. Class Definitions69 2.3 Class Definitions A class serves as the primary means for abstraction in object-oriented program- ming. In Python, every piece of data is represented as an instance of some class. A class provides a set of behaviors in the form of member functions (also known as methods), with implementations that are common to all instances of that class. A class also serves as a blueprint for its instances, effectively determining the way that state information for each instance is represented in the form of attributes (also known as fields, instance variables, or data members). 2.3.1 Example: CreditCard Class As a first example, we provide an implementation of a CreditCard class based on the design we introduced in Figure 2.3 of Section 2.2.1. The instances defined by the CreditCard class provide a simple model for traditional credit cards. They have identifying information about the customer, bank, account number, credit limit, and current balance. The class restricts charges that would cause a card’s balance to go over its spending limit, but it does not charge interest or late payments (we revisit such themes in Section 2.4.1). Our code begins in Code Fragment 2.1 and continues in Code Fragment 2.2. The construct begins with the keyword, class, followed by the name of the class, a colon, and then an indented block of code that serves as the body of the class. The body includes definitions for all methods of the class. These methods are defined as functions, using techniques introduced in Section 1.5, yet with a special parameter, named self, that serves to identify the particular instance upon which a member is invoked. The self Identifier In Python, the self identifier plays a key role. In the context of the CreditCard class, there can presumably be many different CreditCard instances, and each must maintain its own balance, its own credit limit, and so on. Therefore, each instance stores its own instance variables to reflect its current state. Syntactically, self identifies the instance upon which a method is invoked. For example, assume that a user of our class has a variable, my card, that identifies an instance of the CreditCard class. When the user calls my card.get balance(), identifier self, within the definition of the get balance method, refers to the card known as my card by the caller. The expression, self. balance refers to an instance variable, named balance, stored as part of that particular credit card’s state.
- 92. 70 Chapter 2.Object-Oriented Programming 1 class CreditCard: 2 ”””A consumer credit card.””” 3 4 def init (self, customer, bank, acnt, limit): 5 ”””Create a new credit card instance. 6 7 The initial balance is zero. 8 9 customer the name of the customer (e.g., John Bowman ) 10 bank the name of the bank (e.g., California Savings ) 11 acnt the acount identifier (e.g., 5391 0375 9387 5309 ) 12 limit credit limit (measured in dollars) 13 ””” 14 self. customer = customer 15 self. bank = bank 16 self. account = acnt 17 self. limit = limit 18 self. balance = 0 19 20 def get customer(self): 21 ”””Return name of the customer.””” 22 return self. customer 23 24 def get bank(self): 25 ”””Return the bank s name.””” 26 return self. bank 27 28 def get account(self): 29 ”””Return the card identifying number (typically stored as a string).””” 30 return self. account 31 32 def get limit(self): 33 ”””Return current credit limit.””” 34 return self. limit 35 36 def get balance(self): 37 ”””Return current balance.””” 38 return self. balance Code Fragment 2.1: The beginning of the CreditCard class definition (continued in Code Fragment 2.2).
- 93. 2.3. Class Definitions71 39 def charge(self, price): 40 ”””Charge given price to the card, assuming sufficient credit limit. 41 42 Return True if charge was processed; False if charge was denied. 43 ””” 44 if price + self. balance self. limit: # if charge would exceed limit, 45 return False # cannot accept charge 46 else: 47 self. balance += price 48 return True 49 50 def make payment(self, amount): 51 ”””Process customer payment that reduces balance.””” 52 self. balance −= amount Code Fragment 2.2: The conclusion of the CreditCard class definition (continued from Code Fragment 2.1). These methods are indented within the class definition. We draw attention to the difference between the method signature as declared within the class versus that used by a caller. For example, from a user’s perspec- tive we have seen that the get balance method takes zero parameters, yet within the class definition, self is an explicit parameter. Likewise, the charge method is declared within the class having two parameters (self and price), even though this method is called with one parameter, for example, as my card.charge(200). The interpretter automatically binds the instance upon which the method is invoked to the self parameter. The Constructor A user can create an instance of the CreditCard class using a syntax as: cc = CreditCard( John Doe, 1st Bank , 5391 0375 9387 5309 , 1000) Internally, this results in a call to the specially named init method that serves as the constructor of the class. Its primary responsibility is to establish the state of a newly created credit card object with appropriate instance variables. In the case of the CreditCard class, each object maintains five instance variables, which we name: customer, bank, account, limit, and balance. The initial values for the first four of those five are provided as explicit parameters that are sent by the user when instantiating the credit card, and assigned within the body of the construc- tor. For example, the command, self. customer = customer, assigns the instance variable self. customer to the parameter customer; note that because customer is unqualified on the right-hand side, it refers to the parameter in the local namespace.
- 94. 72 Chapter 2.Object-Oriented Programming Encapsulation By the conventions described in Section 2.2.3, a single leading underscore in the name of a data member, such as balance, implies that it is intended as nonpublic. Users of a class should not directly access such members. As a general rule, we will treat all data members as nonpublic. This allows us to better enforce a consistent state for all instances. We can provide accessors, such as get balance, to provide a user of our class read-only access to a trait. If we wish to allow the user to change the state, we can provide appropriate update methods. In the context of data structures, encapsulating the internal representation allows us greater flexibility to redesign the way a class works, perhaps to improve the efficiency of the structure. Additional Methods The most interesting behaviors in our class are charge and make payment. The charge function typically adds the given price to the credit card balance, to reflect a purchase of said price by the customer. However, before accepting the charge, our implementation verifies that the new purchase would not cause the balance to exceed the credit limit. The make payment charge reflects the customer sending payment to the bank for the given amount, thereby reducing the balance on the card. We note that in the command, self. balance −= amount, the expression self. balance is qualified with the self identifier because it represents an instance variable of the card, while the unqualified amount represents the local parameter. Error Checking Our implementation of the CreditCard class is not particularly robust. First, we note that we did not explicitly check the types of the parameters to charge and make payment, nor any of the parameters to the constructor. If a user were to make a call such as visa.charge( candy ), our code would presumably crash when at- tempting to add that parameter to the current balance. If this class were to be widely used in a library, we might use more rigorous techniques to raise a TypeError when facing such misuse (see Section 1.7). Beyond the obvious type errors, our implementation may be susceptible to log- ical errors. For example, if a user were allowed to charge a negative price, such as visa.charge(−300), that would serve to lower the customer’s balance. This pro- vides a loophole for lowering a balance without making a payment. Of course, this might be considered valid usage if modeling the credit received when a cus- tomer returns merchandise to a store. We will explore some such issues with the CreditCard class in the end-of-chapter exercises.
- 95. 2.3. Class Definitions73 Testing the Class In Code Fragment 2.3, we demonstrate some basic usage of the CreditCard class, inserting three cards into a list named wallet. We use loops to make some charges and payments, and use various accessors to print results to the console. These tests are enclosed within a conditional, if name == __main__ :, so that they can be embedded in the source code with the class definition. Using the terminology of Section 2.2.4, these tests provide method coverage, as each of the methods is called at least once, but it does not provide statement coverage, as there is never a case in which a charge is rejected due to the credit limit. This is not a particular advanced from of testing as the output of the given tests must be manually audited in order to determine whether the class behaved as expected. Python has tools for more formal testing (see discussion of the unittest module in Section 2.2.4), so that resulting values can be automatically compared to the predicted outcomes, with output generated only when an error is detected. 53 if name == __main__ : 54 wallet = [ ] 55 wallet.append(CreditCard( John Bowman , California Savings , 56 5391 0375 9387 5309 , 2500) ) 57 wallet.append(CreditCard( John Bowman , California Federal , 58 3485 0399 3395 1954 , 3500) ) 59 wallet.append(CreditCard( John Bowman , California Finance , 60 5391 0375 9387 5309 , 5000) ) 61 62 for val in range(1, 17): 63 wallet[0].charge(val) 64 wallet[1].charge(2 val) 65 wallet[2].charge(3 val) 66 67 for c in range(3): 68 print( Customer = , wallet[c].get customer()) 69 print( Bank = , wallet[c].get bank()) 70 print( Account = , wallet[c].get account()) 71 print( Limit = , wallet[c].get limit()) 72 print( Balance = , wallet[c].get balance()) 73 while wallet[c].get balance( ) 100: 74 wallet[c].make payment(100) 75 print( New balance = , wallet[c].get balance()) 76 print() Code Fragment 2.3: Testing the CreditCard class.
- 96. 74 Chapter 2.Object-Oriented Programming 2.3.2 Operator Overloading and Python’s Special Methods Python’s built-in classes provide natural semantics for many operators. For ex- ample, the syntax a + b invokes addition for numeric types, yet concatenation for sequence types. When defining a new class, we must consider whether a syntax like a + b should be defined when a or b is an instance of that class. By default, the + operator is undefined for a new class. However, the author of a class may provide a definition using a technique known as operator overload- ing. This is done by implementing a specially named method. In particular, the + operator is overloaded by implementing a method named add , which takes the right-hand operand as a parameter and which returns the result of the expres- sion. That is, the syntax, a + b, is converted to a method call on object a of the form, a. add (b). Similar specially named methods exist for other operators. Table 2.1 provides a comprehensive list of such methods. When a binary operator is applied to two instances of different types, as in 3 love me , Python gives deference to the class of the left operand. In this example, it would effectively check if the int class provides a sufficient definition for how to multiply an instance by a string, via the mul method. However, if that class does not implement such a behavior, Python checks the class defini- tion for the right-hand operand, in the form of a special method named rmul (i.e., “right multiply”). This provides a way for a new user-defined class to support mixed operations that involve an instance of an existing class (given that the exist- ing class would presumably not have defined a behavior involving this new class). The distinction between mul and rmul also allows a class to define dif- ferent semantics in cases, such as matrix multiplication, in which an operation is noncommutative (that is, A x may differ from x A). Non-Operator Overloads In addition to traditional operator overloading, Python relies on specially named methods to control the behavior of various other functionality, when applied to user-defined classes. For example, the syntax, str(foo), is formally a call to the constructor for the string class. Of course, if the parameter is an instance of a user- defined class, the original authors of the string class could not have known how that instance should be portrayed. So the string constructor calls a specially named method, foo. str (), that must return an appropriate string representation. Similar special methods are used to determine how to construct an int, float, or bool based on a parameter from a user-defined class. The conversion to a Boolean value is particularly important, because the syntax, if foo:, can be used even when foo is not formally a Boolean value (see Section 1.4.1). For a user-defined class, that condition is evaluated by the special method foo. bool ().
- 97. 2.3. Class Definitions75 Common Syntax Special Method Form a + b a. add (b); alternatively b. radd (a) a − b a. sub (b); alternatively b. rsub (a) a b a. mul (b); alternatively b. rmul (a) a / b a. truediv (b); alternatively b. rtruediv (a) a // b a. floordiv (b); alternatively b. rfloordiv (a) a % b a. mod (b); alternatively b. rmod (a) a b a. pow (b); alternatively b. rpow (a) a b a. lshift (b); alternatively b. rlshift (a) a b a. rshift (b); alternatively b. rrshift (a) a b a. and (b); alternatively b. rand (a) a ˆ b a. xor (b); alternatively b. rxor (a) a | b a. or (b); alternatively b. ror (a) a += b a. iadd (b) a −= b a. isub (b) a = b a. imul (b) ... ... +a a. pos () −a a. neg () ˜a a. invert () abs(a) a. abs () a b a. lt (b) a = b a. le (b) a b a. gt (b) a = b a. ge (b) a == b a. eq (b) a != b a. ne (b) v in a a. contains (v) a[k] a. getitem (k) a[k] = v a. setitem (k,v) del a[k] a. delitem (k) a(arg1, arg2, ...) a. call (arg1, arg2, ...) len(a) a. len () hash(a) a. hash () iter(a) a. iter () next(a) a. next () bool(a) a. bool () float(a) a. float () int(a) a. int () repr(a) a. repr () reversed(a) a. reversed () str(a) a. str () Table 2.1: Overloaded operations, implemented with Python’s special methods.
- 98. 76 Chapter 2.Object-Oriented Programming Several other top-level functions rely on calling specially named methods. For example, the standard way to determine the size of a container type is by calling the top-level len function. Note well that the calling syntax, len(foo), is not the traditional method-calling syntax with the dot operator. However, in the case of a user-defined class, the top-level len function relies on a call to a specially named len method of that class. That is, the call len(foo) is evaluated through a method call, foo. len (). When developing data structures, we will routinely define the len method to return a measure of the size of the structure. Implied Methods As a general rule, if a particular special method is not implemented in a user-defined class, the standard syntax that relies upon that method will raise an exception. For example, evaluating the expression, a + b, for instances of a user-defined class without add or radd will raise an error. However, there are some operators that have default definitions provided by Python, in the absence of special methods, and there are some operators whose definitions are derived from others. For example, the bool method, which supports the syntax if foo:, has default semantics so that every object other than None is evaluated as True. However, for container types, the len method is typically defined to return the size of the container. If such a method exists, then the evaluation of bool(foo) is interpreted by default to be True for instances with nonzero length, and False for instances with zero length, allowing a syntax such as if waitlist: to be used to test whether there are one or more entries in the waitlist. In Section 2.3.4, we will discuss Python’s mechanism for providing iterators for collections via the special method, iter . With that said, if a container class provides implementations for both len and getitem , a default iteration is provided automatically (using means we describe in Section 2.3.4). Furthermore, once an iterator is defined, default functionality of contains is provided. In Section 1.3 we drew attention to the distinction between expression a is b and expression a == b, with the former evaluating whether identifiers a and b are aliases for the same object, and the latter testing a notion of whether the two iden- tifiers reference equivalent values. The notion of “equivalence” depends upon the context of the class, and semantics is defined with the eq method. However, if no implementation is given for eq , the syntax a == b is legal with semantics of a is b, that is, an instance is equivalent to itself and no others. We should caution that some natural implications are not automatically pro- vided by Python. For example, the eq method supports syntax a == b, but providing that method does not affect the evaluation of syntax a != b. (The ne method should be provided, typically returning not (a == b) as a result.) Simi- larly, providing a lt method supports syntax a b, and indirectly b a, but providing both lt and eq does not imply semantics for a = b.
- 99. 2.3. Class Definitions77 2.3.3 Example: Multidimensional Vector Class To demonstrate the use of operator overloading via special methods, we provide an implementation of a Vector class, representing the coordinates of a vector in a multidimensional space. For example, in a three-dimensional space, we might wish to represent a vector with coordinates 5,−2, 3 . Although it might be tempting to directly use a Python list to represent those coordinates, a list does not provide an appropriate abstraction for a geometric vector. In particular, if using lists, the ex- pression [5, −2, 3] + [1, 4, 2] results in the list [5, −2, 3, 1, 4, 2]. When working with vectors, if u = 5,−2, 3 and v = 1, 4, 2 , one would expect the expression, u + v, to return a three-dimensional vector with coordinates 6, 2, 5 . We therefore define a Vector class, in Code Fragment 2.4, that provides a better abstraction for the notion of a geometric vector. Internally, our vector relies upon an instance of a list, named coords, as its storage mechanism. By keeping the internal list encapsulated, we can enforce the desired public interface for instances of our class. A demonstration of supported behaviors includes the following: v = Vector(5) # construct five-dimensional 0, 0, 0, 0, 0 v[1] = 23 # 0, 23, 0, 0, 0 (based on use of setitem ) v[−1] = 45 # 0, 23, 0, 0, 45 (also via setitem ) print(v[4]) # print 45 (via getitem ) u = v + v # 0, 46, 0, 0, 90 (via add ) print(u) # print 0, 46, 0, 0, 90 total = 0 for entry in v: # implicit iteration via len and getitem total += entry We implement many of the behaviors by trivially invoking a similar behavior on the underlying list of coordinates. However, our implementation of add is customized. Assuming the two operands are vectors with the same length, this method creates a new vector and sets the coordinates of the new vector to be equal to the respective sum of the operands’ elements. It is interesting to note that the class definition, as given in Code Fragment 2.4, automatically supports the syntax u = v + [5, 3, 10, −2, 1], resulting in a new vector that is the element-by-element “sum” of the first vector and the list in- stance. This is a result of Python’s polymorphism. Literally, “polymorphism” means “many forms.” Although it is tempting to think of the other parameter of our add method as another Vector instance, we never declared it as such. Within the body, the only behaviors we rely on for parameter other is that it sup- ports len(other) and access to other[j]. Therefore, our code executes when the right-hand operand is a list of numbers (with matching length).
- 100. 78 Chapter 2.Object-Oriented Programming 1 class Vector: 2 ”””Represent a vector in a multidimensional space.””” 3 4 def init (self, d): 5 ”””Create d-dimensional vector of zeros.””” 6 self. coords = [0] d 7 8 def len (self): 9 ”””Return the dimension of the vector.””” 10 return len(self. coords) 11 12 def getitem (self, j): 13 ”””Return jth coordinate of vector.””” 14 return self. coords[j] 15 16 def setitem (self, j, val): 17 ”””Set jth coordinate of vector to given value.””” 18 self. coords[j] = val 19 20 def add (self, other): 21 ”””Return sum of two vectors.””” 22 if len(self) != len(other): # relies on len method 23 raise ValueError( dimensions must agree ) 24 result = Vector(len(self)) # start with vector of zeros 25 for j in range(len(self)): 26 result[j] = self[j] + other[j] 27 return result 28 29 def eq (self, other): 30 ”””Return True if vector has same coordinates as other.””” 31 return self. coords == other. coords 32 33 def ne (self, other): 34 ”””Return True if vector differs from other.””” 35 return not self == other # rely on existing eq definition 36 37 def str (self): 38 ”””Produce string representation of vector.””” 39 return + str(self. coords)[1:−1] + # adapt list representation Code Fragment 2.4: Definition of a simple Vector class.
- 101. 2.3. Class Definitions79 2.3.4 Iterators Iteration is an important concept in the design of data structures. We introduced Python’s mechanism for iteration in Section 1.8. In short, an iterator for a collec- tion provides one key behavior: It supports a special method named next that returns the next element of the collection, if any, or raises a StopIteration exception to indicate that there are no further elements. Fortunately, it is rare to have to directly implement an iterator class. Our pre- ferred approach is the use of the generator syntax (also described in Section 1.8), which automatically produces an iterator of yielded values. Python also helps by providing an automatic iterator implementation for any class that defines both len and getitem . To provide an instructive exam- ple of a low-level iterator, Code Fragment 2.5 demonstrates just such an iterator class that works on any collection that supports both len and getitem . This class can be instantiated as SequenceIterator(data). It operates by keeping an internal reference to the data sequence, as well as a current index into the sequence. Each time next is called, the index is incremented, until reaching the end of the sequence. 1 class SequenceIterator: 2 ”””An iterator for any of Python s sequence types.””” 3 4 def init (self, sequence): 5 ”””Create an iterator for the given sequence.””” 6 self. seq = sequence # keep a reference to the underlying data 7 self. k = −1 # will increment to 0 on first call to next 8 9 def next (self): 10 ”””Return the next element, or else raise StopIteration error.””” 11 self. k += 1 # advance to next index 12 if self. k len(self. seq): 13 return(self. seq[self. k]) # return the data element 14 else: 15 raise StopIteration( ) # there are no more elements 16 17 def iter (self): 18 ”””By convention, an iterator must return itself as an iterator.””” 19 return self Code Fragment 2.5: An iterator class for any sequence type.
- 102. 80 Chapter 2.Object-Oriented Programming 2.3.5 Example: Range Class As the final example for this section, we develop our own implementation of a class that mimics Python’s built-in range class. Before introducing our class, we discuss the history of the built-in version. Prior to Python 3 being released, range was implemented as a function, and it returned a list instance with elements in the specified range. For example, range(2, 10, 2) returned the list [2, 4, 6, 8]. However, a typical use of the function was to support a for-loop syntax, such as for k in range(10000000). Unfortunately, this caused the instantiation and initial- ization of a list with the range of numbers. That was an unnecessarily expensive step, in terms of both time and memory usage. The mechanism used to support ranges in Python 3 is entirely different (to be fair, the “new” behavior existed in Python 2 under the name xrange). It uses a strategy known as lazy evaluation. Rather than creating a new list instance, range is a class that can effectively represent the desired range of elements without ever storing them explicitly in memory. To better explore the built-in range class, we recommend that you create an instance as r = range(8, 140, 5). The result is a relatively lightweight object, an instance of the range class, that has only a few behaviors. The syntax len(r) will report the number of elements that are in the given range (27, in our example). A range also supports the getitem method, so that syntax r[15] reports the sixteenth element in the range (as r[0] is the first element). Because the class supports both len and getitem , it inherits automatic support for iteration (see Section 2.3.4), which is why it is possible to execute a for loop over a range. At this point, we are ready to demonstrate our own version of such a class. Code Fragment 2.6 provides a class we name Range (so as to clearly differentiate it from built-in range). The biggest challenge in the implementation is properly computing the number of elements that belong in the range, given the parameters sent by the caller when constructing a range. By computing that value in the constructor, and storing it as self. length, it becomes trivial to return it from the len method. To properly implement a call to getitem (k), we simply take the starting value of the range plus k times the step size (i.e., for k=0, we return the start value). There are a few subtleties worth examining in the code: • To properly support optional parameters, we rely on the technique described on page 27, when discussing a functional version of range. • We compute the number of elements in the range as max(0, (stop − start + step − 1) // step) It is worth testing this formula for both positive and negative step sizes. • The getitem method properly supports negative indices by converting an index −k to len(self)−k before computing the result.
- 103. 2.3. Class Definitions81 1 class Range: 2 ”””A class that mimic s the built-in range class.””” 3 4 def init (self, start, stop=None, step=1): 5 ”””Initialize a Range instance. 6 7 Semantics is similar to built-in range class. 8 ””” 9 if step == 0: 10 raise ValueError( step cannot be 0 ) 11 12 if stop is None: # special case of range(n) 13 start, stop = 0, start # should be treated as if range(0,n) 14 15 # calculate the effective length once 16 self. length = max(0, (stop − start + step − 1) // step) 17 18 # need knowledge of start and step (but not stop) to support getitem 19 self. start = start 20 self. step = step 21 22 def len (self): 23 ”””Return number of entries in the range.””” 24 return self. length 25 26 def getitem (self, k): 27 ”””Return entry at index k (using standard interpretation if negative).””” 28 if k 0: 29 k += len(self) # attempt to convert negative index 30 31 if not 0 = k self. length: 32 raise IndexError( index out of range ) 33 34 return self. start + k self. step Code Fragment 2.6: Our own implementation of a Range class.
- 104. 82 Chapter 2.Object-Oriented Programming 2.4 Inheritance A natural way to organize various structural components of a software package is in a hierarchical fashion, with similar abstract definitions grouped together in a level-by-level manner that goes from specific to more general as one traverses up the hierarchy. An example of such a hierarchy is shown in Figure 2.4. Using mathematical notations, the set of houses is a subset of the set of buildings, but a superset of the set of ranches. The correspondence between levels is often referred to as an “is a” relationship, as a house is a building, and a ranch is a house. Building Low-rise Apartment High-rise Apartment Two-story House Ranch Skyscraper Commercial Building House Apartment Figure 2.4: An example of an “is a” hierarchy involving architectural buildings. A hierarchical design is useful in software development, as common function- ality can be grouped at the most general level, thereby promoting reuse of code, while differentiated behaviors can be viewed as extensions of the general case, In object-oriented programming, the mechanism for a modular and hierarchical orga- nization is a technique known as inheritance. This allows a new class to be defined based upon an existing class as the starting point. In object-oriented terminology, the existing class is typically described as the base class, parent class, or super- class, while the newly defined class is known as the subclass or child class. There are two ways in which a subclass can differentiate itself from its su- perclass. A subclass may specialize an existing behavior by providing a new im- plementation that overrides an existing method. A subclass may also extend its superclass by providing brand new methods.
- 105. 2.4. Inheritance 83 Python’sException Hierarchy Another example of a rich inheritance hierarchy is the organization of various ex- ception types in Python. We introduced many of those classes in Section 1.7, but did not discuss their relationship with each other. Figure 2.5 illustrates a (small) portion of that hierarchy. The BaseException class is the root of the entire hierar- chy, while the more specific Exception class includes most of the error types that we have discussed. Programmers are welcome to define their own special exception classes to denote errors that may occur in the context of their application. Those user-defined exception types should be declared as subclasses of Exception. ValueError Exception KeyboardInterrupt SystemExit BaseException IndexError KeyError ZeroDivisionError LookupError ArithmeticError Figure 2.5: A portion of Python’s hierarchy of exception types. 2.4.1 Extending the CreditCard Class To demonstrate the mechanisms for inheritance in Python, werevisit the CreditCard class of Section 2.3, implementing a subclass that, for lack of a better name, we name PredatoryCreditCard. The new class will differ from the original in two ways: (1) if an attempted charge is rejected because it would have exceeded the credit limit, a $5 fee will be charged, and (2) there will be a mechanism for assess- ing a monthly interest charge on the outstanding balance, based upon an Annual Percentage Rate (APR) specified as a constructor parameter. In accomplishing this goal, we demonstrate the techniques of specialization and extension. To charge a fee for an invalid charge attempt, we override the existing charge method, thereby specializing it to provide the new functionality (although the new version takes advantage of a call to the overridden version). To provide support for charging interest, we extend the class with a new method named process month.
- 106. 84 Chapter 2.Object-Oriented Programming Class: Fields: Behaviors: Class: Fields: Behaviors: process month() apr customer account get customer() get bank() bank get account() balance limit get balance() get limit() charge(price) make payment(amount) PredatoryCreditCard CreditCard charge(price) Figure 2.6: Diagram of an inheritance relationship. Figure 2.6 provides an overview of our use of inheritance in designing the new PredatoryCreditCard class, and Code Fragment 2.7 gives a complete Python im- plementation of that class. To indicate that the new class inherits from the existing CreditCard class, our definition begins with the syntax, class PredatoryCreditCard(CreditCard). The body of the new class provides three member functions: init , charge, and process month. The init constructor serves a very similar role to the original CreditCard constructor, except that for our new class, there is an extra parameter to specify the annual percentage rate. The body of our new constructor relies upon making a call to the inherited constructor to perform most of the initialization (in fact, everything other than the recording of the percentage rate). The mechanism for calling the inherited constructor relies on the syntax, super(). Specifically, at line 15 the command super(). init (customer, bank, acnt, limit) calls the init method that was inherited from the CreditCard superclass. Note well that this method only accepts four parameters. We record the APR value in a new field named apr. In similar fashion, our PredatoryCreditCard class provides a new implemen- tation of the charge method that overrides the inherited method. Yet, our imple- mentation of the new method relies on a call to the inherited method, with syntax super().charge(price) at line 24. The return value of that call designates whether
- 107. 2.4. Inheritance 85 1class PredatoryCreditCard(CreditCard): 2 ”””An extension to CreditCard that compounds interest and fees.””” 3 4 def init (self, customer, bank, acnt, limit, apr): 5 ”””Create a new predatory credit card instance. 6 7 The initial balance is zero. 8 9 customer the name of the customer (e.g., John Bowman ) 10 bank the name of the bank (e.g., California Savings ) 11 acnt the acount identifier (e.g., 5391 0375 9387 5309 ) 12 limit credit limit (measured in dollars) 13 apr annual percentage rate (e.g., 0.0825 for 8.25% APR) 14 ””” 15 super(). init (customer, bank, acnt, limit) # call super constructor 16 self. apr = apr 17 18 def charge(self, price): 19 ”””Charge given price to the card, assuming sufficient credit limit. 20 21 Return True if charge was processed. 22 Return False and assess 5 fee if charge is denied. 23 ””” 24 success = super().charge(price) # call inherited method 25 if not success: 26 self. balance += 5 # assess penalty 27 return success # caller expects return value 28 29 def process month(self): 30 ”””Assess monthly interest on outstanding balance.””” 31 if self. balance 0: 32 # if positive balance, convert APR to monthly multiplicative factor 33 monthly factor = pow(1 + self. apr, 1/12) 34 self. balance = monthly factor Code Fragment 2.7: A subclass of CreditCard that assesses interest and fees.
- 108. 86 Chapter 2.Object-Oriented Programming the charge was successful. We examine that return value to decide whether to as- sess a fee, and in turn we return that value to the caller of method, so that the new version of charge has a similar outward interface as the original. The process month method is a new behavior, so there is no inherited version upon which to rely. In our model, this method should be invoked by the bank, once each month, to add new interest charges to the customer’s balance. The most challenging aspect in implementing this method is making sure we have working knowledge of how an annual percentage rate translates to a monthly rate. We do not simply divide the annual rate by twelve to get a monthly rate (that would be too predatory, as it would result in a higher APR than advertised). The correct com- putation is to take the twelfth-root of 1 + self. apr, and use that as a multiplica- tive factor. For example, if the APR is 0.0825 (representing 8.25%), we compute 12 √ 1.0825 ≈ 1.006628, and therefore charge 0.6628% interest per month. In this way, each $100 of debt will amass $8.25 of compounded interest in a year. Protected Members Our PredatoryCreditCard subclass directly accesses the data member self. balance, which was established by the parent CreditCard class. The underscored name, by convention, suggests that this is a nonpublic member, so we might ask if it is okay that we access it in this fashion. While general users of the class should not be doing so, our subclass has a somewhat privileged relationship with the superclass. Several object-oriented languages (e.g., Java, C++) draw a distinction for nonpub- lic members, allowing declarations of protected or private access modes. Members that are declared as protected are accessible to subclasses, but not to the general public, while members that are declared as private are not accessible to either. In this respect, we are using balance as if it were protected (but not private). Python does not support formal access control, but names beginning with a sin- gle underscore are conventionally akin to protected, while names beginning with a double underscore (other than special methods) are akin to private. In choosing to use protected data, we have created a dependency in that our PredatoryCreditCard class might be compromised if the author of the CreditCard class were to change the internal design. Note that we could have relied upon the public get balance() method to retrieve the current balance within the process month method. But the current design of the CreditCard class does not afford an effective way for a sub- class to change the balance, other than by direct manipulation of the data member. It may be tempting to use charge to add fees or interest to the balance. However, that method does not allow the balance to go above the customer’s credit limit, even though a bank would presumably let interest compound beyond the credit limit, if warranted. If we were to redesign the original CreditCard class, we might add a nonpublic method, set balance, that could be used by subclasses to affect a change without directly accessing the data member balance.
- 109. 2.4. Inheritance 87 2.4.2Hierarchy of Numeric Progressions As a second example of the use of inheritance, we develop a hierarchy of classes for iterating numeric progressions. A numeric progression is a sequence of numbers, where each number depends on one or more of the previous numbers. For example, an arithmetic progression determines the next number by adding a fixed constant to the previous value, and a geometric progression determines the next number by multiplying the previous value by a fixed constant. In general, a progression requires a first value, and a way of identifying a new value based on one or more previous values. To maximize reusability of code, we develop a hierarchy of classes stemming from a general base class that we name Progression (see Figure 2.7). Technically, the Progression class produces the progression of whole numbers: 0, 1, 2, . . .. However, this class is designed to serve as the base class for other progression types, providing as much common functionality as possible, and thereby minimizing the burden on the subclasses. FibonacciProgression Progression ArithmeticProgression GeometricProgression Figure 2.7: Our hierarchy of progression classes. Our implementation of the basic Progression class is provided in Code Frag- ment 2.8. The constructor for this class accepts a starting value for the progression (0 by default), and initializes a data member, self. current, to that value. The Progression class implements the conventions of a Python iterator (see Section 2.3.4), namely the special next and iter methods. If a user of the class creates a progression as seq = Progression(), each call to next(seq) will return a subsequent element of the progression sequence. It would also be possi- ble to use a for-loop syntax, for value in seq:, although we note that our default progression is defined as an infinite sequence. To better separate the mechanics of the iterator convention from the core logic of advancing the progression, our framework relies on a nonpublic method named advance to update the value of the self. current field. In the default implementa- tion, advance adds one to the current value, but our intent is that subclasses will override advance to provide a different rule for computing the next entry. For convenience, the Progression class also provides a utility method, named print progression, that displays the next n values of the progression.
- 110. 88 Chapter 2.Object-Oriented Programming 1 class Progression: 2 ”””Iterator producing a generic progression. 3 4 Default iterator produces the whole numbers 0, 1, 2, ... 5 ””” 6 7 def init (self, start=0): 8 ”””Initialize current to the first value of the progression.””” 9 self. current = start 10 11 def advance(self): 12 ”””Update self. current to a new value. 13 14 This should be overridden by a subclass to customize progression. 15 16 By convention, if current is set to None, this designates the 17 end of a finite progression. 18 ””” 19 self. current += 1 20 21 def next (self): 22 ”””Return the next element, or else raise StopIteration error.””” 23 if self. current is None: # our convention to end a progression 24 raise StopIteration() 25 else: 26 answer = self. current # record current value to return 27 self. advance( ) # advance to prepare for next time 28 return answer # return the answer 29 30 def iter (self): 31 ”””By convention, an iterator must return itself as an iterator.””” 32 return self 33 34 def print progression(self, n): 35 ”””Print next n values of the progression.””” 36 print( .join(str(next(self)) for j in range(n))) Code Fragment 2.8: A general numeric progression class.
- 111. 2.4. Inheritance 89 AnArithmetic Progression Class Our first example of a specialized progression is an arithmetic progression. While the default progression increases its value by one in each step, an arithmetic pro- gression adds a fixed constant to one term of the progression to produce the next. For example, using an increment of 4 for an arithmetic progression that starts at 0 results in the sequence 0,4,8,12,... . Code Fragment 2.9 presents our implementation of an ArithmeticProgression class, which relies on Progression as its base class. The constructor for this new class accepts both an increment value and a starting value as parameters, although default values for each are provided. By our convention, ArithmeticProgression(4) produces the sequence 0,4,8,12,... , and ArithmeticProgression(4, 1) produces the sequence 1,5,9,13,... . The body of the ArithmeticProgression constructor calls the super constructor to initialize the current data member to the desired start value. Then it directly establishes the new increment data member for the arithmetic progression. The only remaining detail in our implementation is to override the advance method so as to add the increment to the current value. 1 class ArithmeticProgression(Progression): # inherit from Progression 2 ”””Iterator producing an arithmetic progression.””” 3 4 def init (self, increment=1, start=0): 5 ”””Create a new arithmetic progression. 6 7 increment the fixed constant to add to each term (default 1) 8 start the first term of the progression (default 0) 9 ””” 10 super(). init (start) # initialize base class 11 self. increment = increment 12 13 def advance(self): # override inherited version 14 ”””Update current value by adding the fixed increment.””” 15 self. current += self. increment Code Fragment 2.9: A class that produces an arithmetic progression.
- 112. 90 Chapter 2.Object-Oriented Programming A Geometric Progression Class Our second example of a specialized progression is a geometric progression, in which each value is produced by multiplying the preceding value by a fixed con- stant, known as the base of the geometric progression. The starting point of a ge- ometric progression is traditionally 1, rather than 0, because multiplying 0 by any factor results in 0. As an example, a geometric progression with base 2 proceeds as 1,2,4,8,16,... . Code Fragment 2.10 presents our implementation of a GeometricProgression class. The constructor uses 2 as a default base and 1 as a default starting value, but either of those can be varied using optional parameters. 1 class GeometricProgression(Progression): # inherit from Progression 2 ”””Iterator producing a geometric progression.””” 3 4 def init (self, base=2, start=1): 5 ”””Create a new geometric progression. 6 7 base the fixed constant multiplied to each term (default 2) 8 start the first term of the progression (default 1) 9 ””” 10 super(). init (start) 11 self. base = base 12 13 def advance(self): # override inherited version 14 ”””Update current value by multiplying it by the base value.””” 15 self. current = self. base Code Fragment 2.10: A class that produces a geometric progression. A Fibonacci Progression Class As our final example, we demonstrate how to use our progression framework to produce a Fibonacci progression. We originally discussed the Fibonacci series on page 41 in the context of generators. Each value of a Fibonacci series is the sum of the two most recent values. To begin the series, the first two values are conventionally 0 and 1, leading to the Fibonacci series 0,1,1,2,3,5,8,... . More generally, such a series can be generated from any two starting values. For example, if we start with values 4 and 6, the series proceeds as 4,6,10,16,26,42,... .
- 113. 2.4. Inheritance 91 1class FibonacciProgression(Progression): 2 ”””Iterator producing a generalized Fibonacci progression.””” 3 4 def init (self, first=0, second=1): 5 ”””Create a new fibonacci progression. 6 7 first the first term of the progression (default 0) 8 second the second term of the progression (default 1) 9 ””” 10 super(). init (first) # start progression at first 11 self. prev = second − first # fictitious value preceding the first 12 13 def advance(self): 14 ”””Update current value by taking sum of previous two.””” 15 self. prev, self. current = self. current, self. prev + self. current Code Fragment 2.11: A class that produces a Fibonacci progression. We use our progression framework to define a new FibonacciProgression class, as shown in Code Fragment 2.11. This class is markedly different from those for the arithmetic and geometric progressions because we cannot determine the next value of a Fibonacci series solely from the current one. We must maintain knowledge of the two most recent values. The base Progression class already provides storage of the most recent value as the current data member. Our FibonacciProgression class introduces a new member, named prev, to store the value that proceeded the current one. With both previous values stored, the implementation of advance is relatively straightforward. (We use a simultaneous assignment similar to that on page 45.) However, the question arises as to how to initialize the previous value in the con- structor. The desired first and second values are provided as parameters to the constructor. The first should be stored as current so that it becomes the first one that is reported. Looking ahead, once the first value is reported, we will do an assignment to set the new current value (which will be the second value re- ported), equal to the first value plus the “previous.” By initializing the previous value to (second − first), the initial advancement will set the new current value to first + (second − first) = second, as desired. Testing Our Progressions To complete our presentation, Code Fragment 2.12 provides a unit test for all of our progression classes, and Code Fragment 2.13 shows the output of that test.
- 114. 92 Chapter 2.Object-Oriented Programming if name == __main__ : print( Default progression: ) Progression().print progression(10) print( Arithmetic progression with increment 5: ) ArithmeticProgression(5).print progression(10) print( Arithmetic progression with increment 5 and start 2: ) ArithmeticProgression(5, 2).print progression(10) print( Geometric progression with default base: ) GeometricProgression().print progression(10) print( Geometric progression with base 3: ) GeometricProgression(3).print progression(10) print( Fibonacci progression with default start values: ) FibonacciProgression().print progression(10) print( Fibonacci progression with start values 4 and 6: ) FibonacciProgression(4, 6).print progression(10) Code Fragment 2.12: Unit tests for our progression classes. Default progression: 0 1 2 3 4 5 6 7 8 9 Arithmetic progression with increment 5: 0 5 10 15 20 25 30 35 40 45 Arithmetic progression with increment 5 and start 2: 2 7 12 17 22 27 32 37 42 47 Geometric progression with default base: 1 2 4 8 16 32 64 128 256 512 Geometric progression with base 3: 1 3 9 27 81 243 729 2187 6561 19683 Fibonacci progression with default start values: 0 1 1 2 3 5 8 13 21 34 Fibonacci progression with start values 4 and 6: 4 6 10 16 26 42 68 110 178 288 Code Fragment 2.13: Output of the unit tests from Code Fragment 2.12.
- 115. 2.4. Inheritance 93 2.4.3Abstract Base Classes When defining a group of classes as part of an inheritance hierarchy, one technique for avoiding repetition of code is to design a base class with common function- ality that can be inherited by other classes that need it. As an example, the hi- erarchy from Section 2.4.2 includes a Progression class, which serves as a base class for three distinct subclasses: ArithmeticProgression, GeometricProgression, and FibonacciProgression. Although it is possible to create an instance of the Progression base class, there is little value in doing so because its behavior is sim- ply a special case of an ArithmeticProgression with increment 1. The real purpose of the Progression class was to centralize the implementations of behaviors that other progressions needed, thereby streamlining the code that is relegated to those subclasses. In classic object-oriented terminology, we say a class is an abstract base class if its only purpose is to serve as a base class through inheritance. More formally, an abstract base class is one that cannot be directly instantiated, while a concrete class is one that can be instantiated. By this definition, our Progression class is technically concrete, although we essentially designed it as an abstract base class. In statically typed languages such as Java and C++, an abstract base class serves as a formal type that may guarantee one or more abstract methods. This provides support for polymorphism, as a variable may have an abstract base class as its de- clared type, even though it refers to an instance of a concrete subclass. Because there are no declared types in Python, this kind of polymorphism can be accom- plished without the need for a unifying abstract base class. For this reason, there is not as strong a tradition of defining abstract base classes in Python, although Python’s abc module provides support for defining a formal abstract base class. Our reason for focusing on abstract base classes in our study of data structures is that Python’s collections module provides several abstract base classes that assist when defining custom data structures that share a common interface with some of Python’s built-in data structures. These rely on an object-oriented software design pattern known as the template method pattern. The template method pattern is when an abstract base class provides concrete behaviors that rely upon calls to other abstract behaviors. In that way, as soon as a subclass provides definitions for the missing abstract behaviors, the inherited concrete behaviors are well defined. As a tangible example, the collections.Sequence abstract base class defines be- haviors common to Python’s list, str, and tuple classes, as sequences that sup- port element access via an integer index. More so, the collections.Sequence class provides concrete implementations of methods, count, index, and contains that can be inherited by any class that provides concrete implementations of both len and getitem . For the purpose of illustration, we provide a sample implementation of such a Sequence abstract base class in Code Fragment 2.14.
- 116. 94 Chapter 2.Object-Oriented Programming 1 from abc import ABCMeta, abstractmethod # need these definitions 2 3 class Sequence(metaclass=ABCMeta): 4 ”””Our own version of collections.Sequence abstract base class.””” 5 6 @abstractmethod 7 def len (self): 8 ”””Return the length of the sequence.””” 9 10 @abstractmethod 11 def getitem (self, j): 12 ”””Return the element at index j of the sequence.””” 13 14 def contains (self, val): 15 ”””Return True if val found in the sequence; False otherwise.””” 16 for j in range(len(self)): 17 if self[j] == val: # found match 18 return True 19 return False 20 21 def index(self, val): 22 ”””Return leftmost index at which val is found (or raise ValueError).””” 23 for j in range(len(self)): 24 if self[j] == val: # leftmost match 25 return j 26 raise ValueError( value not in sequence ) # never found a match 27 28 def count(self, val): 29 ”””Return the number of elements equal to given value.””” 30 k = 0 31 for j in range(len(self)): 32 if self[j] == val: # found a match 33 k += 1 34 return k Code Fragment 2.14: An abstract base class akin to collections.Sequence. This implementation relies on two advanced Python techniques. The first is that we declare the ABCMeta class of the abc module as a metaclass of our Sequence class. A metaclass is different from a superclass, in that it provides a template for the class definition itself. Specifically, the ABCMeta declaration assures that the constructor for the class raises an error.
- 117. 2.4. Inheritance 95 Thesecond advanced technique is the use of the @abstractmethod decorator immediately before the len and getitem methods are declared. That de- clares these two particular methods to be abstract, meaning that we do not provide an implementation within our Sequence base class, but that we expect any concrete subclasses to support those two methods. Python enforces this expectation, by dis- allowing instantiation for any subclass that does not override the abstract methods with concrete implementations. The rest of the Sequence class definition provides tangible implementations for other behaviors, under the assumption that the abstract len and getitem methods will exist in a concrete subclass. If you carefully examine the source code, the implementations of methods contains , index, and count do not rely on any assumption about the self instances, other than that syntax len(self) and self[j] are supported (by special methods len and getitem , respectively). Support for iteration is automatic as well, as described in Section 2.3.4. In the remainder of this book, we omit the formality of using the abc module. If we need an “abstract” base class, we simply document the expectation that sub- classes provide assumed functionality, without technical declaration of the methods as abstract. But we will make use of the wonderful abstract base classes that are defined within the collections module (such as Sequence). To use such a class, we need only rely on standard inheritance techniques. For example, our Range class, from Code Fragment 2.6 of Section 2.3.5, is an example of a class that supports the len and getitem methods. But that class does not support methods count or index. Had we originally declared it with Sequence as a superclass, then it would also inherit the count and index methods. The syntax for such a declaration would begin as: class Range(collections.Sequence): Finally, we emphasize that if a subclass provides its own implementation of an inherited behaviors from a base class, the new definition overrides the inherited one. This technique can be used when we have the ability to provide a more effi- cient implementation for a behavior than is achieved by the generic approach. As an example, the general implementation of contains for a sequence is based on a loop used to search for the desired value. For our Range class, there is an opportunity for a more efficient determination of containment. For example, it is evident that the expression, 100000 in Range(0, 2000000, 100), should evalu- ate to True, even without examining the individual elements of the range, because the range starts with zero, has an increment of 100, and goes until 2 million; it must include 100000, as that is a multiple of 100 that is between the start and stop values. Exercise C-2.27 explores the goal of providing an implementation of Range. contains that avoids the use of a (time-consuming) loop.
- 118. 96 Chapter 2.Object-Oriented Programming 2.5 Namespaces and Object-Orientation A namespace is an abstraction that manages all of the identifiers that are defined in a particular scope, mapping each name to its associated value. In Python, functions, classes, and modules are all first-class objects, and so the “value” associated with an identifier in a namespace may in fact be a function, class, or module. In Section 1.10 we explored Python’s use of namespaces to manage identifiers that are defined with global scope, versus those defined within the local scope of a function call. In this section, we discuss the important role of namespaces in Python’s management of object-orientation. 2.5.1 Instance and Class Namespaces We begin by exploring what is known as the instance namespace, which man- ages attributes specific to an individual object. For example, each instance of our CreditCard class maintains a distinct balance, a distinct account number, a distinct credit limit, and so on (even though some instances may coincidentally have equiv- alent balances, or equivalent credit limits). Each credit card will have a dedicated instance namespace to manage such values. There is a separate class namespace for each class that has been defined. This namespace is used to manage members that are to be shared by all instances of a class, or used without reference to any particular instance. For example, the make payment method of the CreditCard class from Section 2.3 is not stored independently by each instance of that class. That member function is stored within the namespace of the CreditCard class. Based on our definition from Code Fragments 2.1 and 2.2, the CreditCard class namespace includes the functions: init , get customer, get bank, get account, get balance, get limit, charge, and make payment. Our PredatoryCreditCard class has its own namespace, con- taining the three methods we defined for that subclass: init , charge, and process month. Figure 2.8 provides a portrayal of three such namespaces: a class namespace containing methods of the CreditCard class, another class namespace with meth- ods of the PredatoryCreditCard class, and finally a single instance namespace for a sample instance of the PredatoryCreditCard class. We note that there are two different definitions of a function named charge, one in the CreditCard class, and then the overriding method in the PredatoryCreditCard class. In similar fashion, there are two distinct init implementations. However, process month is a name that is only defined within the scope of the PredatoryCreditCard class. The instance namespace includes all data members for the instance (including the apr member that is established by the PredatoryCreditCard constructor).
- 119. 2.5. Namespaces andObject-Orientation 97 get bank get account make payment get balance get limit charge init function function function function function function function get customer function charge init function function process month function bank account balance limit apr 1234.56 2500 John Bowman California Savings 5391 0375 9387 5309 customer 0.0825 (a) (b) (c) Figure 2.8: Conceptual view of three namespaces: (a) the class namespace for CreditCard; (b) the class namespace for PredatoryCreditCard; (c) the instance namespace for a PredatoryCreditCard object. How Entries Are Established in a Namespace It is important to understand why a member such as balance resides in a credit card’s instance namespace, while a member such as make payment resides in the class namespace. The balance is established within the init method when a new credit card instance is constructed. The original assignment uses the syntax, self. balance = 0, where self is an identifier for the newly constructed instance. The use of self as a qualifier for self. balance in such an assignment causes the balance identifier to be added directly to the instance namespace. When inheritance is used, there is still a single instance namespace per object. For example, when an instance of the PredatoryCreditCard class is constructed, the apr attribute as well as attributes such as balance and limit all reside in that instance’s namespace, because all are assigned using a qualified syntax, such as self. apr. A class namespace includes all declarations that are made directly within the body of the class definition. For example, our CreditCard class definition included the following structure: class CreditCard: def make payment(self, amount): ... Because the make payment function is declared within the scope of the CreditCard class, that function becomes associated with the name make payment within the CreditCard class namespace. Although member functions are the most typical types of entries that are declared in a class namespace, we next discuss how other types of data values, or even other classes can be declared within a class namespace.
- 120. 98 Chapter 2.Object-Oriented Programming Class Data Members A class-level data member is often used when there is some value, such as a con- stant, that is to be shared by all instances of a class. In such a case, it would be unnecessarily wasteful to have each instance store that value in its instance namespace. As an example, we revisit the PredatoryCreditCard introduced in Sec- tion 2.4.1. That class assesses a $5 fee if an attempted charge is denied because of the credit limit. Our choice of $5 for the fee was somewhat arbitrary, and our coding style would be better if we used a named variable rather than embedding the literal value in our code. Often, the amount of such a fee is determined by the bank’s policy and does not vary for each customer. In that case, we could define and use a class data member as follows: class PredatoryCreditCard(CreditCard): OVERLIMIT FEE = 5 # this is a class-level member def charge(self, price): success = super().charge(price) if not success: self. balance += PredatoryCreditCard.OVERLIMIT FEE return success The data member, OVERLIMIT FEE, is entered into the PredatoryCreditCard class namespace because that assignment takes place within the immediate scope of the class definition, and without any qualifying identifier. Nested Classes It is also possible to nest one class definition within the scope of another class. This is a useful construct, which we will exploit several times in this book in the implementation of data structures. This can be done by using a syntax such as class A: # the outer class class B: # the nested class ... In this case, class B is the nested class. The identifier B is entered into the name- space of class A associated with the newly defined class. We note that this technique is unrelated to the concept of inheritance, as class B does not inherit from class A. Nesting one class in the scope of another makes clear that the nested class exists for support of the outer class. Furthermore, it can help reduce potential name conflicts, because it allows for a similarly named class to exist in another context. For example, we will later introduce a data structure known as a linked list and will define a nested node class to store the individual components of the list. We will also introduce a data structure known as a tree that depends upon its own nested
- 121. 2.5. Namespaces andObject-Orientation 99 node class. These two structures rely on different node definitions, and by nesting those within the respective container classes, we avoid ambiguity. Another advantage of one class being nested as a member of another is that it allows for a more advanced form of inheritance in which a subclass of the outer class overrides the definition of its nested class. We will make use of that technique in Section 11.2.1 when specializing the nodes of a tree structure. Dictionaries and the slots Declaration By default, Python represents each namespace with an instance of the built-in dict class (see Section 1.2.3) that maps identifying names in that scope to the associated objects. While a dictionary structure supports relatively efficient name lookups, it requires additional memory usage beyond the raw data that it stores (we will explore the data structure used to implement dictionaries in Chapter 10). Python provides a more direct mechanism for representing instance namespaces that avoids the use of an auxiliary dictionary. To use the streamlined representation for all instances of a class, that class definition must provide a class-level member named slots that is assigned to a fixed sequence of strings that serve as names for instance variables. For example, with our CreditCard class, we would declare the following: class CreditCard: slots = _customer , _bank , _account , _balance , _limit In this example, the right-hand side of the assignment is technically a tuple (see discussion of automatic packing of tuples in Section 1.9.3). When inheritance is used, if the base class declares slots , a subclass must also declare slots to avoid creation of instance dictionaries. The declaration in the subclass should only include names of supplemental methods that are newly introduced. For example, our PredatoryCreditCard declaration would include the following declaration: class PredatoryCreditCard(CreditCard): slots = _apr # in addition to the inherited members We could choose to use the slots declaration to streamline every class in this book. However, we do not do so because such rigor would be atypical for Python programs. With that said, there are a few classes in this book for which we expect to have a large number of instances, each representing a lightweight construct. For example, when discussing nested classes, we suggest linked lists and trees as data structures that are often comprised of a large number of individual nodes. To promote greater efficiency in memory usage, we will use an explicit slots declaration in any nested classes for which we expect many instances.
- 122. 100 Chapter 2.Object-Oriented Programming 2.5.2 Name Resolution and Dynamic Dispatch In the previous section, we discussed various namespaces, and the mechanism for establishing entries in those namespaces. In this section, we examine the process that is used when retrieving a name in Python’s object-oriented framework. When the dot operator syntax is used to access an existing member, such as obj.foo, the Python interpreter begins a name resolution process, described as follows: 1. The instance namespace is searched; if the desired name is found, its associ- ated value is used. 2. Otherwise the class namespace, for the class to which the instance belongs, is searched; if the name is found, its associated value is used. 3. If the name was not found in the immediate class namespace, the search con- tinues upward through the inheritance hierarchy, checking the class name- space for each ancestor (commonly by checking the superclass class, then its superclass class, and so on). The first time the name is found, its associate value is used. 4. If the name has still not been found, an AttributeError is raised. As a tangible example, let us assume that mycard identifies an instance of the PredatoryCreditCard class. Consider the following possible usage patterns. • mycard. balance (or equivalently, self. balance from within a method body): the balance method is found within the instance namespace for mycard. • mycard.process month(): the search begins in the instance namespace, but the name process month is not found in that namespace. As a result, the PredatoryCreditCard class namespace is searched; in this case, the name is found and that method is called. • mycard.make payment(200): the search for the name, make payment, fails in the instance namespace and in the PredatoryCreditCard namespace. The name is resolved in the namespace for superclass CreditCard and thus the inherited method is called. • mycard.charge(50): the search for name charge fails in the instance name- space. The next namespace checked is for the PredatoryCreditCard class, because that is the true type of the instance. There is a definition for a charge function in that class, and so that is the one that is called. In the last case shown, notice that the existence of a charge function in the PredatoryCreditCard class has the effect of overriding the version of that function that exists in the CreditCard namespace. In traditional object-oriented terminol- ogy, Python uses what is known as dynamic dispatch (or dynamic binding) to determine, at run-time, which implementation of a function to call based upon the type of the object upon which it is invoked. This is in contrast to some languages that use static dispatching, making a compile-time decision as to which version of a function to call, based upon the declared type of a variable.
- 123. 2.6. Shallow andDeep Copying 101 2.6 Shallow and Deep Copying In Chapter 1, we emphasized that an assignment statement foo = bar makes the name foo an alias for the object identified as bar. In this section, we consider the task of making a copy of an object, rather than an alias. This is necessary in applications when we want to subsequently modify either the original or the copy in an independent manner. Consider a scenario in which we manage various lists of colors, with each color represented by an instance of a presumed color class. We let identifier warmtones denote an existing list of such colors (e.g., oranges, browns). In this application, we wish to create a new list named palette, which is a copy of the warmtones list. However, we want to subsequently be able to add additional colors to palette, or to modify or remove some of the existing colors, without affecting the contents of warmtones. If we were to execute the command palette = warmtones this creates an alias, as shown in Figure 2.9. No new list is created; instead, the new identifier palette references the original list. red green blue color 52 163 169 list warmtones red green blue color 43 124 249 palette Figure 2.9: Two aliases for the same list of colors. Unfortunately, this does not meet our desired criteria, because if we subsequently add or remove colors from “palette,” we modify the list identified as warmtones. We can instead create a new instance of the list class by using the syntax: palette = list(warmtones) In this case, we explicitly call the list constructor, sending the first list as a param- eter. This causes a new list to be created, as shown in Figure 2.10; however, it is what is known as a shallow copy. The new list is initialized so that its contents are precisely the same as the original sequence. However, Python’s lists are referential (see page 9 of Section 1.2.3), and so the new list represents a sequence of references to the same elements as in the first.
- 124. 102 Chapter 2.Object-Oriented Programming red green blue color 52 163 169 list list warmtones palette red green blue color 43 124 249 Figure 2.10: A shallow copy of a list of colors. This is a better situation than our first attempt, as we can legitimately add or remove elements from palette without affecting warmtones. However, if we edit a color instance from the palette list, we effectively change the contents of warmtones. Although palette and warmtones are distinct lists, there remains indi- rect aliasing, for example, with palette[0] and warmtones[0] as aliases for the same color instance. We prefer that palette be what is known as a deep copy of warmtones. In a deep copy, the new copy references its own copies of those objects referenced by the original version. (See Figure 2.11.) blue color 52 163 169 list red green blue color 43 124 249 red green blue color 52 163 169 list red green blue color 43 124 249 warmtones palette red green Figure 2.11: A deep copy of a list of colors. Python’s copy Module To create a deep copy, we could populate our list by explicitly making copies of the original color instances, but this requires that we know how to make copies of colors (rather than aliasing). Python provides a very convenient module, named copy, that can produce both shallow copies and deep copies of arbitrary objects. This module supports two functions: the copy function creates a shallow copy of its argument, and the deepcopy function creates a deep copy of its argument. After importing the module, we may create a deep copy for our example, as shown in Figure 2.11, using the command: palette = copy.deepcopy(warmtones)
- 125. 2.7. Exercises 103 2.7Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-2.1 Give three examples of life-critical software applications. R-2.2 Give an example of a software application in which adaptability can mean the difference between a prolonged lifetime of sales and bankruptcy. R-2.3 Describe a component from a text-editor GUI and the methods that it en- capsulates. R-2.4 Write a Python class, Flower, that has three instance variables of type str, int, and float, that respectively represent the name of the flower, its num- ber of petals, and its price. Your class must include a constructor method that initializes each variable to an appropriate value, and your class should include methods for setting the value of each type, and retrieving the value of each type. R-2.5 Use the techniques of Section 1.7 to revise the charge and make payment methods of the CreditCard class to ensure that the caller sends a number as a parameter. R-2.6 If the parameter to the make payment method of the CreditCard class were a negative number, that would have the effect of raising the balance on the account. Revise the implementation so that it raises a ValueError if a negative value is sent. R-2.7 The CreditCard class of Section 2.3 initializes the balance of a new ac- count to zero. Modify that class so that a new account can be given a nonzero balance using an optional fifth parameter to the constructor. The four-parameter constructor syntax should continue to produce an account with zero balance. R-2.8 Modify the declaration of the first for loop in the CreditCard tests, from Code Fragment 2.3, so that it will eventually cause exactly one of the three credit cards to go over its credit limit. Which credit card is it? R-2.9 Implement the sub method for the Vector class of Section 2.3.3, so that the expression u−v returns a new vector instance representing the difference between two vectors. R-2.10 Implement the neg method for the Vector class of Section 2.3.3, so that the expression −v returns a new vector instance whose coordinates are all the negated values of the respective coordinates of v.
- 126. 104 Chapter 2.Object-Oriented Programming R-2.11 In Section 2.3.3, we note that our Vector class supports a syntax such as v = u + [5, 3, 10, −2, 1], in which the sum of a vector and list returns a new vector. However, the syntax v = [5, 3, 10, −2, 1] + u is illegal. Explain how the Vector class definition can be revised so that this syntax generates a new vector. R-2.12 Implement the mul method for the Vector class of Section 2.3.3, so that the expression v 3 returns a new vector with coordinates that are 3 times the respective coordinates of v. R-2.13 Exercise R-2.12 asks for an implementation of mul , for the Vector class of Section 2.3.3, to provide support for the syntax v 3. Implement the rmul method, to provide additional support for syntax 3 v. R-2.14 Implement the mul method for the Vector class of Section 2.3.3, so that the expression u v returns a scalar that represents the dot product of the vectors, that is, ∑d i=1 ui ·vi. R-2.15 The Vector class of Section 2.3.3 provides a constructor that takes an in- teger d, and produces a d-dimensional vector with all coordinates equal to 0. Another convenient form for creating a new vector would be to send the constructor a parameter that is some iterable type representing a sequence of numbers, and to create a vector with dimension equal to the length of that sequence and coordinates equal to the sequence values. For example, Vector([4, 7, 5]) would produce a three-dimensional vector with coordi- nates 4, 7, 5. Modify the constructor so that either of these forms is acceptable; that is, if a single integer is sent, it produces a vector of that dimension with all zeros, but if a sequence of numbers is provided, it pro- duces a vector with coordinates based on that sequence. R-2.16 Our Range class, from Section 2.3.5, relies on the formula max(0, (stop − start + step − 1) // step) to compute the number of elements in the range. It is not immediately ev- ident why this formula provides the correct calculation, even if assuming a positive step size. Justify this formula, in your own words. R-2.17 Draw a class inheritance diagram for the following set of classes: • Class Goat extends object and adds an instance variable tail and methods milk() and jump(). • Class Pig extends object and adds an instance variable nose and methods eat(food) and wallow(). • Class Horse extends object and adds instance variables height and color, and methods run() and jump(). • Class Racer extends Horse and adds a method race(). • Class Equestrian extends Horse, adding an instance variable weight and methods trot() and is trained().
- 127. 2.7. Exercises 105 R-2.18Give a short fragment of Python code that uses the progression classes from Section 2.4.2 to find the 8th value of a Fibonacci progression that starts with 2 and 2 as its first two values. R-2.19 When using the ArithmeticProgression class of Section 2.4.2 with an in- crement of 128 and a start of 0, how many calls to next can we make before we reach an integer of 263 or larger? R-2.20 What are some potential efficiency disadvantages of having very deep in- heritance trees, that is, a large set of classes, A, B, C, and so on, such that B extends A, C extends B, D extends C, etc.? R-2.21 What are some potential efficiency disadvantages of having very shallow inheritance trees, that is, a large set of classes, A, B, C, and so on, such that all of these classes extend a single class, Z? R-2.22 The collections.Sequence abstract base class does not provide support for comparing two sequences to each other. Modify our Sequence class from Code Fragment 2.14 to include a definition for the eq method, so that expression seq1 == seq2 will return True precisely when the two sequences are element by element equivalent. R-2.23 In similar spirit to the previous problem, augment the Sequence class with method lt , to support lexicographic comparison seq1 seq2. Creativity C-2.24 Suppose you are on the design team for a new e-book reader. What are the primary classes and methods that the Python software for your reader will need? You should include an inheritance diagram for this code, but you do not need to write any actual code. Your software architecture should at least include ways for customers to buy new books, view their list of purchased books, and read their purchased books. C-2.25 Exercise R-2.12 uses the mul method to support multiplying a Vector by a number, while Exercise R-2.14 uses the mul method to support computing a dot product of two vectors. Give a single implementation of Vector. mul that uses run-time type checking to support both syntaxes u v and u k, where u and v designate vector instances and k represents a number. C-2.26 The SequenceIterator class of Section 2.3.4 provides what is known as a forward iterator. Implement a class named ReversedSequenceIterator that serves as a reverse iterator for any Python sequence type. The first call to next should return the last element of the sequence, the second call to next should return the second-to-last element, and so forth.
- 128. 106 Chapter 2.Object-Oriented Programming C-2.27 In Section 2.3.5, we note that our version of the Range class has im- plicit support for iteration, due to its explicit support of both len and getitem . The class also receives implicit support of the Boolean test, “k in r” for Range r. This test is evaluated based on a forward itera- tion through the range, as evidenced by the relative quickness of the test 2 in Range(10000000) versus 9999999 in Range(10000000). Provide a more efficient implementation of the contains method to determine whether a particular value lies within a given range. The running time of your method should be independent of the length of the range. C-2.28 The PredatoryCreditCard class of Section 2.4.1 provides a process month method that models the completion of a monthly cycle. Modify the class so that once a customer has made ten calls to charge in the current month, each additional call to that function results in an additional $1 surcharge. C-2.29 Modify the PredatoryCreditCard class from Section 2.4.1 so that a cus- tomer is assigned a minimum monthly payment, as a percentage of the balance, and so that a late fee is assessed if the customer does not subse- quently pay that minimum amount before the next monthly cycle. C-2.30 At the close of Section 2.4.1, we suggest a model in which the CreditCard class supports a nonpublic method, set balance(b), that could be used by subclasses to affect a change to the balance, without directly accessing the balance data member. Implement such a model, revising both the CreditCard and PredatoryCreditCard classes accordingly. C-2.31 Write a Python class that extends the Progression class so that each value in the progression is the absolute value of the difference between the pre- vious two values. You should include a constructor that accepts a pair of numbers as the first two values, using 2 and 200 as the defaults. C-2.32 Write a Python class that extends the Progression class so that each value in the progression is the square root of the previous value. (Note that you can no longer represent each value with an integer.) Your construc- tor should accept an optional parameter specifying the start value, using 65,536 as a default. Projects P-2.33 Write a Python program that inputs a polynomial in standard algebraic notation and outputs the first derivative of that polynomial. P-2.34 Write a Python program that inputs a document and then outputs a bar- chart plot of the frequencies of each alphabet character that appears in that document.
- 129. 2.7. Exercises 107 P-2.35Write a set of Python classes that can simulate an Internet application in which one party, Alice, is periodically creating a set of packets that she wants to send to Bob. An Internet process is continually checking if Alice has any packets to send, and if so, it delivers them to Bob’s computer, and Bob is periodically checking if his computer has a packet from Alice, and, if so, he reads and deletes it. P-2.36 Write a Python program to simulate an ecosystem containing two types of creatures, bears and fish. The ecosystem consists of a river, which is modeled as a relatively large list. Each element of the list should be a Bear object, a Fish object, or None. In each time step, based on a random process, each animal either attempts to move into an adjacent list location or stay where it is. If two animals of the same type are about to collide in the same cell, then they stay where they are, but they create a new instance of that type of animal, which is placed in a random empty (i.e., previously None) location in the list. If a bear and a fish collide, however, then the fish dies (i.e., it disappears). P-2.37 Write a simulator, as in the previous project, but add a Boolean gender field and a floating-point strength field to each animal, using an Animal class as a base class. If two animals of the same type try to collide, then they only create a new instance of that type of animal if they are of differ- ent genders. Otherwise, if two animals of the same type and gender try to collide, then only the one of larger strength survives. P-2.38 Write a Python program that simulates a system that supports the func- tions of an e-book reader. You should include methods for users of your system to “buy” new books, view their list of purchased books, and read their purchased books. Your system should use actual books, which have expired copyrights and are available on the Internet, to populate your set of available books for users of your system to “purchase” and read. P-2.39 Develop an inheritance hierarchy based upon a Polygon class that has abstract methods area() and perimeter(). Implement classes Triangle, Quadrilateral, Pentagon, Hexagon, and Octagon that extend this base class, with the obvious meanings for the area() and perimeter() methods. Also implement classes, IsoscelesTriangle, EquilateralTriangle, Rectan- gle, and Square, that have the appropriate inheritance relationships. Fi- nally, write a simple program that allows users to create polygons of the various types and input their geometric dimensions, and the program then outputs their area and perimeter. For extra effort, allow users to input polygons by specifying their vertex coordinates and be able to test if two such polygons are similar.
- 130. 108 Chapter 2.Object-Oriented Programming Chapter Notes For a broad overview of developments in computer science and engineering, we refer the reader to The Computer Science and Engineering Handbook [96]. For more information about the Therac-25 incident, please see the paper by Leveson and Turner [69]. The reader interested in studying object-oriented programming further, is referred to the books by Booch [17], Budd [20], and Liskov and Guttag [71]. Liskov and Guttag also provide a nice discussion of abstract data types, as does the survey paper by Cardelli and Wegner [23] and the book chapter by Demurjian [33] in the The Computer Science and Engineering Handbook [96]. Design patterns are described in the book by Gamma et al. [41]. Books with specific focus on object-oriented programming in Python include those by Goldwasser and Letscher [43] at the introductory level, and by Phillips [83] at a more advanced level,
- 131. Chapter 3 Algorithm Analysis Contents 3.1Experimental Studies . . . . . . . . . . . . . . . . . . . . . 111 3.1.1 Moving Beyond Experimental Analysis . . . . . . . . . . . 113 3.2 The Seven Functions Used in This Book . . . . . . . . . . 115 3.2.1 Comparing Growth Rates . . . . . . . . . . . . . . . . . . 122 3.3 Asymptotic Analysis . . . . . . . . . . . . . . . . . . . . . . 123 3.3.1 The “Big-Oh” Notation . . . . . . . . . . . . . . . . . . . 123 3.3.2 Comparative Analysis . . . . . . . . . . . . . . . . . . . . 128 3.3.3 Examples of Algorithm Analysis . . . . . . . . . . . . . . 130 3.4 Simple Justification Techniques . . . . . . . . . . . . . . . 137 3.4.1 By Example . . . . . . . . . . . . . . . . . . . . . . . . . 137 3.4.2 The “Contra” Attack . . . . . . . . . . . . . . . . . . . . 137 3.4.3 Induction and Loop Invariants . . . . . . . . . . . . . . . 138 3.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
- 132. 110 Chapter 3.Algorithm Analysis In a classic story, the famous mathematician Archimedes was asked to deter- mine if a golden crown commissioned by the king was indeed pure gold, and not part silver, as an informant had claimed. Archimedes discovered a way to perform this analysis while stepping into a bath. He noted that water spilled out of the bath in proportion to the amount of him that went in. Realizing the implications of this fact, he immediately got out of the bath and ran naked through the city shouting, “Eureka, eureka!” for he had discovered an analysis tool (displacement), which, when combined with a simple scale, could determine if the king’s new crown was good or not. That is, Archimedes could dip the crown and an equal-weight amount of gold into a bowl of water to see if they both displaced the same amount. This discovery was unfortunate for the goldsmith, however, for when Archimedes did his analysis, the crown displaced more water than an equal-weight lump of pure gold, indicating that the crown was not, in fact, pure gold. In this book, we are interested in the design of “good” data structures and algo- rithms. Simply put, a data structure is a systematic way of organizing and access- ing data, and an algorithm is a step-by-step procedure for performing some task in a finite amount of time. These concepts are central to computing, but to be able to classify some data structures and algorithms as “good,” we must have precise ways of analyzing them. The primary analysis tool we will use in this book involves characterizing the running times of algorithms and data structure operations, with space usage also being of interest. Running time is a natural measure of “goodness,” since time is a precious resource—computer solutions should run as fast as possible. In general, the running time of an algorithm or data structure operation increases with the input size, although it may also vary for different inputs of the same size. Also, the run- ning time is affected by the hardware environment (e.g., the processor, clock rate, memory, disk) and software environment (e.g., the operating system, programming language) in which the algorithm is implemented and executed. All other factors being equal, the running time of the same algorithm on the same input data will be smaller if the computer has, say, a much faster processor or if the implementation is done in a program compiled into native machine code instead of an interpreted implementation. We begin this chapter by discussing tools for performing exper- imental studies, yet also limitations to the use of experiments as a primary means for evaluating algorithm efficiency. Focusing on running time as a primary measure of goodness requires that we be able to use a few mathematical tools. In spite of the possible variations that come from different environmental factors, we would like to focus on the relationship between the running time of an algorithm and the size of its input. We are interested in characterizing an algorithm’s running time as a function of the input size. But what is the proper way of measuring it? In this chapter, we “roll up our sleeves” and develop a mathematical way of analyzing algorithms.
- 133. 3.1. Experimental Studies111 3.1 Experimental Studies If an algorithm has been implemented, we can study its running time by executing it on various test inputs and recording the time spent during each execution. A simple approach for doing this in Python is by using the time function of the time module. This function reports the number of seconds, or fractions thereof, that have elapsed since a benchmark time known as the epoch. The choice of the epoch is not significant to our goal, as we can determine the elapsed time by recording the time just before the algorithm and the time just after the algorithm, and computing their difference, as follows: from time import time start time = time( ) # record the starting time run algorithm end time = time( ) # record the ending time elapsed = end time − start time # compute the elapsed time We will demonstrate use of this approach, in Chapter 5, to gather experimental data on the efficiency of Python’s list class. An elapsed time measured in this fashion is a decent reflection of the algorithm efficiency, but it is by no means perfect. The time function measures relative to what is known as the “wall clock.” Because many processes share use of a computer’s central processing unit (or CPU), the elapsed time will depend on what other processes are running on the computer when the test is performed. A fairer metric is the number of CPU cycles that are used by the algorithm. This can be determined using the clock function of the time module, but even this measure might not be consistent if repeating the identical algorithm on the identical input, and its granularity will depend upon the computer system. Python includes a more advanced module, named timeit, to help automate such evaluations with repetition to account for such variance among trials. Because we are interested in the general dependence of running time on the size and structure of the input, we should perform independent experiments on many different test inputs of various sizes. We can then visualize the results by plotting the performance of each run of the algorithm as a point with x-coordinate equal to the input size, n, and y-coordinate equal to the running time, t. Figure 3.1 displays such hypothetical data. This visualization may provide some intuition regarding the relationship between problem size and execution time for the algorithm. This may lead to a statistical analysis that seeks to fit the best function of the input size to the experimental data. To be meaningful, this analysis requires that we choose good sample inputs and test enough of them to be able to make sound statistical claims about the algorithm’s running time.
- 134. 112 Chapter 3.Algorithm Analysis Running Time (ms) 100 300 400 500 10000 0 0 5000 15000 200 Input Size Figure 3.1: Results of an experimental study on the running time of an algorithm. A dot with coordinates (n,t) indicates that on an input of size n, the running time of the algorithm was measured as t milliseconds (ms). Challenges of Experimental Analysis While experimental studies of running times are valuable, especially when fine- tuning production-quality code, there are three major limitations to their use for algorithm analysis: • Experimental running times of two algorithms are difficult to directly com- pare unless the experiments are performed in the same hardware and software environments. • Experiments can be done only on a limited set of test inputs; hence, they leave out the running times of inputs not included in the experiment (and these inputs may be important). • An algorithm must be fully implemented in order to execute it to study its running time experimentally. This last requirement is the most serious drawback to the use of experimental stud- ies. At early stages of design, when considering a choice of data structures or algorithms, it would be foolish to spend a significant amount of time implementing an approach that could easily be deemed inferior by a higher-level analysis.
- 135. 3.1. Experimental Studies113 3.1.1 Moving Beyond Experimental Analysis Our goal is to develop an approach to analyzing the efficiency of algorithms that: 1. Allows us to evaluate the relative efficiency of any two algorithms in a way that is independent of the hardware and software environment. 2. Is performed by studying a high-level description of the algorithm without need for implementation. 3. Takes into account all possible inputs. Counting Primitive Operations To analyze the running time of an algorithm without performing experiments, we perform an analysis directly on a high-level description of the algorithm (either in the form of an actual code fragment, or language-independent pseudo-code). We define a set of primitive operations such as the following: • Assigning an identifier to an object • Determining the object associated with an identifier • Performing an arithmetic operation (for example, adding two numbers) • Comparing two numbers • Accessing a single element of a Python list by index • Calling a function (excluding operations executed within the function) • Returning from a function. Formally, a primitive operation corresponds to a low-level instruction with an exe- cution time that is constant. Ideally, this might be the type of basic operation that is executed by the hardware, although many of our primitive operations may be trans- lated to a small number of instructions. Instead of trying to determine the specific execution time of each primitive operation, we will simply count how many prim- itive operations are executed, and use this number t as a measure of the running time of the algorithm. This operation count will correlate to an actual running time in a specific com- puter, for each primitive operation corresponds to a constant number of instructions, and there are only a fixed number of primitive operations. The implicit assumption in this approach is that the running times of different primitive operations will be fairly similar. Thus, the number, t, of primitive operations an algorithm performs will be proportional to the actual running time of that algorithm. Measuring Operations as a Function of Input Size To capture the order of growth of an algorithm’s running time, we will associate, with each algorithm, a function f(n) that characterizes the number of primitive operations that are performed as a function of the input size n. Section 3.2 will in- troduce the seven most common functions that arise, and Section 3.3 will introduce a mathematical framework for comparing functions to each other.
- 136. 114 Chapter 3.Algorithm Analysis Focusing on the Worst-Case Input An algorithm may run faster on some inputs than it does on others of the same size. Thus, we may wish to express the running time of an algorithm as the function of the input size obtained by taking the average over all possible inputs of the same size. Unfortunately, such an average-case analysis is typically quite challenging. It requires us to define a probability distribution on the set of inputs, which is often a difficult task. Figure 3.2 schematically shows how, depending on the input distri- bution, the running time of an algorithm can be anywhere between the worst-case time and the best-case time. For example, what if inputs are really only of types “A” or “D”? An average-case analysis usually requires that we calculate expected running times based on a given input distribution, which usually involves sophisticated probability theory. Therefore, for the remainder of this book, unless we specify otherwise, we will characterize running times in terms of the worst case, as a func- tion of the input size, n, of the algorithm. Worst-case analysis is much easier than average-case analysis, as it requires only the ability to identify the worst-case input, which is often simple. Also, this approach typically leads to better algorithms. Making the standard of success for an algorithm to perform well in the worst case necessarily requires that it will do well on every input. That is, designing for the worst case leads to stronger algorithmic “muscles,” much like a track star who always practices by running up an incline. best-case time B C D E F G average-case time? A Input Instance 1 ms 2 ms 3 ms 4 ms 5 ms Running Time (ms) worst-case time Figure 3.2: The difference between best-case and worst-case time. Each bar repre- sents the running time of some algorithm on a different possible input.
- 137. 3.2. The SevenFunctions Used in This Book 115 3.2 The Seven Functions Used in This Book In this section, we briefly discuss the seven most important functions used in the analysis of algorithms. We will use only these seven simple functions for almost all the analysis we do in this book. In fact, a section that uses a function other than one of these seven will be marked with a star () to indicate that it is optional. In addition to these seven fundamental functions, Appendix B contains a list of other useful mathematical facts that apply in the analysis of data structures and algorithms. The Constant Function The simplest function we can think of is the constant function. This is the function, f(n) = c, for some fixed constant c, such as c = 5, c = 27, or c = 210. That is, for any argument n, the constant function f(n) assigns the value c. In other words, it does not matter what the value of n is; f(n) will always be equal to the constant value c. Because we are most interested in integer functions, the most fundamental con- stant function is g(n) = 1, and this is the typical constant function we use in this book. Note that any other constant function, f(n) = c, can be written as a constant c times g(n). That is, f(n) = cg(n) in this case. As simple as it is, the constant function is useful in algorithm analysis, because it characterizes the number of steps needed to do a basic operation on a computer, like adding two numbers, assigning a value to some variable, or comparing two numbers. The Logarithm Function One of the interesting and sometimes even surprising aspects of the analysis of data structures and algorithms is the ubiquitous presence of the logarithm function, f(n) = logb n, for some constant b 1. This function is defined as follows: x = logb n if and only if bx = n. By definition, logb 1 = 0. The value b is known as the base of the logarithm. The most common base for the logarithm function in computer science is 2, as computers store integers in binary, and because a common operation in many algorithms is to repeatedly divide an input in half. In fact, this base is so common that we will typically omit it from the notation when it is 2. That is, for us, logn = log2 n.
- 138. 116 Chapter 3.Algorithm Analysis We note that most handheld calculators have a button marked LOG, but this is typically for calculating the logarithm base-10, not base-two. Computing the logarithm function exactly for any integer n involves the use of calculus, but we can use an approximation that is good enough for our pur- poses without calculus. In particular, we can easily compute the smallest integer greater than or equal to logb n (its so-called ceiling, logb n). For positive integer, n, this value is equal to the number of times we can divide n by b before we get a number less than or equal to 1. For example, the evaluation of log3 27 is 3, because ((27/3)/3)/3 = 1. Likewise, log4 64 is 3, because ((64/4)/4)/4 = 1, and log2 12 is 4, because (((12/2)/2)/2)/2 = 0.75 ≤ 1. The following proposition describes several important identities that involve logarithms for any base greater than 1. Proposition 3.1 (Logarithm Rules): Given real numbers a 0, b 1, c 0 and d 1, we have: 1. logb(ac) = logb a+logb c 2. logb(a/c) = logb a−logb c 3. logb(ac) = clogb a 4. logb a = logd a/logd b 5. blogd a = alogd b By convention, the unparenthesized notation lognc denotes the value log(nc). We use a notational shorthand, logc n, to denote the quantity, (logn)c, in which the result of the logarithm is raised to a power. The above identities can be derived from converse rules for exponentiation that we will present on page 121. We illustrate these identities with a few examples. Example 3.2: We demonstrate below some interesting applications of the loga- rithm rules from Proposition 3.1 (using the usual convention that the base of a logarithm is 2 if it is omitted). • log(2n) = log2+logn = 1+logn, by rule 1 • log(n/2) = logn−log2 = logn−1, by rule 2 • logn3 = 3logn, by rule 3 • log2n = nlog2 = n·1 = n, by rule 3 • log4 n = (logn)/log4 = (logn)/2, by rule 4 • 2logn = nlog2 = n1 = n, by rule 5. As a practical matter, we note that rule 4 gives us a way to compute the base-two logarithm on a calculator that has a base-10 logarithm button, LOG, for log2 n = LOG n/LOG 2.
- 139. 3.2. The SevenFunctions Used in This Book 117 The Linear Function Another simple yet important function is the linear function, f(n) = n. That is, given an input value n, the linear function f assigns the value n itself. This function arises in algorithm analysis any time we have to do a single basic operation for each of n elements. For example, comparing a number x to each element of a sequence of size n will require n comparisons. The linear function also represents the best running time we can hope to achieve for any algorithm that processes each of n objects that are not already in the computer’s memory, because reading in the n objects already requires n operations. The N-Log-N Function The next function we discuss in this section is the n-log-n function, f(n) = nlogn, that is, the function that assigns to an input n the value of n times the logarithm base-two of n. This function grows a little more rapidly than the linear function and a lot less rapidly than the quadratic function; therefore, we would greatly prefer an algorithm with a running time that is proportional to nlogn, than one with quadratic running time. We will see several important algorithms that exhibit a running time proportional to the n-log-n function. For example, the fastest possible algorithms for sorting n arbitrary values require time proportional to nlogn. The Quadratic Function Another function that appears often in algorithm analysis is the quadratic function, f(n) = n2 . That is, given an input value n, the function f assigns the product of n with itself (in other words, “n squared”). The main reason why the quadratic function appears in the analysis of algo- rithms is that there are many algorithms that have nested loops, where the inner loop performs a linear number of operations and the outer loop is performed a linear number of times. Thus, in such cases, the algorithm performs n · n = n2 operations.
- 140. 118 Chapter 3.Algorithm Analysis Nested Loops and the Quadratic Function The quadratic function can also arise in the context of nested loops where the first iteration of a loop uses one operation, the second uses two operations, the third uses three operations, and so on. That is, the number of operations is 1+2+3+···+(n−2)+(n−1)+n. In other words, this is the total number of operations that will be performed by the nested loop if the number of operations performed inside the loop increases by one with each iteration of the outer loop. This quantity also has an interesting history. In 1787, a German schoolteacher decided to keep his 9- and 10-year-old pupils occupied by adding up the integers from 1 to 100. But almost immediately one of the children claimed to have the answer! The teacher was suspicious, for the student had only the answer on his slate. But the answer, 5050, was correct and the student, Carl Gauss, grew up to be one of the greatest mathematicians of his time. We presume that young Gauss used the following identity. Proposition 3.3: For any integer n ≥ 1, we have: 1+2+3+···+(n−2)+(n−1)+n = n(n+1) 2 . We give two “visual” justifications of Proposition 3.3 in Figure 3.3. 1 2 n 0 1 2 n 3 3 ... 1 n/2 0 1 2 n 3 2 n+1 ... (a) (b) Figure 3.3: Visual justifications of Proposition 3.3. Both illustrations visualize the identity in terms of the total area covered by n unit-width rectangles with heights 1,2,...,n. In (a), the rectangles are shown to cover a big triangle of area n2/2 (base n and height n) plus n small triangles of area 1/2 each (base 1 and height 1). In (b), which applies only when n is even, the rectangles are shown to cover a big rectangle of base n/2 and height n+1.
- 141. 3.2. The SevenFunctions Used in This Book 119 The lesson to be learned from Proposition 3.3 is that if we perform an algorithm with nested loops such that the operations in the inner loop increase by one each time, then the total number of operations is quadratic in the number of times, n, we perform the outer loop. To be fair, the number of operations is n2/2 + n/2, and so this is just over half the number of operations than an algorithm that uses n operations each time the inner loop is performed. But the order of growth is still quadratic in n. The Cubic Function and Other Polynomials Continuing our discussion of functions that are powers of the input, we consider the cubic function, f(n) = n3 , which assigns to an input value n the product of n with itself three times. This func- tion appears less frequently in the context of algorithm analysis than the constant, linear, and quadratic functions previously mentioned, but it does appear from time to time. Polynomials Most of the functions we have listed so far can each be viewed as being part of a larger class of functions, the polynomials. A polynomial function has the form, f(n) = a0 +a1n+a2n2 +a3n3 +···+adnd , where a0,a1,...,ad are constants, called the coefficients of the polynomial, and ad = 0. Integer d, which indicates the highest power in the polynomial, is called the degree of the polynomial. For example, the following functions are all polynomials: • f(n) = 2+5n+n2 • f(n) = 1+n3 • f(n) = 1 • f(n) = n • f(n) = n2 Therefore, we could argue that this book presents just four important functions used in algorithm analysis, but we will stick to saying that there are seven, since the con- stant, linear, and quadratic functions are too important to be lumped in with other polynomials. Running times that are polynomials with small degree are generally better than polynomial running times with larger degree.
- 142. 120 Chapter 3.Algorithm Analysis Summations A notation that appears again and again in the analysis of data structures and algo- rithms is the summation, which is defined as follows: b ∑ i=a f(i) = f(a)+ f(a+1)+ f(a+2)+···+ f(b), where a and b are integers and a ≤ b. Summations arise in data structure and algo- rithm analysis because the running times of loops naturally give rise to summations. Using a summation, we can rewrite the formula of Proposition 3.3 as n ∑ i=1 i = n(n+1) 2 . Likewise, we can write a polynomial f(n) of degree d with coefficients a0,...,ad as f(n) = d ∑ i=0 aini . Thus, the summation notation gives us a shorthand way of expressing sums of in- creasing terms that have a regular structure. The Exponential Function Another function used in the analysis of algorithms is the exponential function, f(n) = bn , where b is a positive constant, called the base, and the argument n is the exponent. That is, function f(n) assigns to the input argument n the value obtained by mul- tiplying the base b by itself n times. As was the case with the logarithm function, the most common base for the exponential function in algorithm analysis is b = 2. For example, an integer word containing n bits can represent all the nonnegative integers less than 2n. If we have a loop that starts by performing one operation and then doubles the number of operations performed with each iteration, then the number of operations performed in the nth iteration is 2n. We sometimes have other exponents besides n, however; hence, it is useful for us to know a few handy rules for working with exponents. In particular, the following exponent rules are quite helpful.
- 143. 3.2. The SevenFunctions Used in This Book 121 Proposition 3.4 (Exponent Rules): Given positive integers a, b, and c, we have 1. (ba)c = bac 2. babc = ba+c 3. ba/bc = ba−c For example, we have the following: • 256 = 162 = (24)2 = 24·2 = 28 = 256 (Exponent Rule 1) • 243 = 35 = 32+3 = 3233 = 9·27 = 243 (Exponent Rule 2) • 16 = 1024/64 = 210/26 = 210−6 = 24 = 16 (Exponent Rule 3) We can extend the exponential function to exponents that are fractions or real numbers and to negative exponents, as follows. Given a positive integer k, we de- fine b1/k to be kth root of b, that is, the number r such that rk = b. For example, 251/2 = 5, since 52 = 25. Likewise, 271/3 = 3 and 161/4 = 2. This approach al- lows us to define any power whose exponent can be expressed as a fraction, for ba/c = (ba)1/c, by Exponent Rule 1. For example, 93/2 = (93)1/2 = 7291/2 = 27. Thus, ba/c is really just the cth root of the integral exponent ba. We can further extend the exponential function to define bx for any real number x, by computing a series of numbers of the form ba/c for fractions a/c that get pro- gressively closer and closer to x. Any real number x can be approximated arbitrarily closely by a fraction a/c; hence, we can use the fraction a/c as the exponent of b to get arbitrarily close to bx. For example, the number 2π is well defined. Finally, given a negative exponent d, we define bd = 1/b−d, which corresponds to applying Exponent Rule 3 with a = 0 and c = −d. For example, 2−3 = 1/23 = 1/8. Geometric Sums Suppose we have a loop for which each iteration takes a multiplicative factor longer than the previous one. This loop can be analyzed using the following proposition. Proposition 3.5: For any integer n ≥ 0 and any real number a such that a 0 and a = 1, consider the summation n ∑ i=0 ai = 1+a+a2 +···+an (remembering that a0 = 1 if a 0). This summation is equal to an+1 −1 a−1 . Summations as shown in Proposition 3.5 are called geometric summations, be- cause each term is geometrically larger than the previous one if a 1. For example, everyone working in computing should know that 1+2+4+8+···+2n−1 = 2n −1, for this is the largest integer that can be represented in binary notation using n bits.
- 144. 122 Chapter 3.Algorithm Analysis 3.2.1 Comparing Growth Rates To sum up, Table 3.1 shows, in order, each of the seven common functions used in algorithm analysis. constant logarithm linear n-log-n quadratic cubic exponential 1 logn n nlogn n2 n3 an Table 3.1: Classes of functions. Here we assume that a 1 is a constant. Ideally, we would like data structure operations to run in times proportional to the constant or logarithm function, and we would like our algorithms to run in linear or n-log-n time. Algorithms with quadratic or cubic running times are less practical, and algorithms with exponential running times are infeasible for all but the smallest sized inputs. Plots of the seven functions are shown in Figure 3.4. f (n) 107 106 n 105 104 103 102 Linear Exponential Constant Logarithmic N-Log-N Quadratic Cubic 1015 1014 1013 1012 1011 1010 109 108 101 100 104 108 1012 1016 1020 1028 1032 1036 1040 1044 100 1024 Figure 3.4: Growth rates for the seven fundamental functions used in algorithm analysis. We use base a = 2 for the exponential function. The functions are plotted on a log-log chart, to compare the growth rates primarily as slopes. Even so, the exponential function grows too fast to display all its values on the chart. The Ceiling and Floor Functions One additional comment concerning the functions above is in order. When dis- cussing logarithms, we noted that the value is generally not an integer, yet the running time of an algorithm is usually expressed by means of an integer quantity, such as the number of operations performed. Thus, the analysis of an algorithm may sometimes involve the use of the floor function and ceiling function, which are defined respectively as follows: • x = the largest integer less than or equal to x. • x = the smallest integer greater than or equal to x.
- 145. 3.3. Asymptotic Analysis123 3.3 Asymptotic Analysis In algorithm analysis, we focus on the growth rate of the running time as a function of the input size n, taking a “big-picture” approach. For example, it is often enough just to know that the running time of an algorithm grows proportionally to n. We analyze algorithms using a mathematical notation for functions that disre- gards constant factors. Namely, we characterize the running times of algorithms by using functions that map the size of the input, n, to values that correspond to the main factor that determines the growth rate in terms of n. This approach re- flects that each basic step in a pseudo-code description or a high-level language implementation may correspond to a small number of primitive operations. Thus, we can perform an analysis of an algorithm by estimating the number of primitive operations executed up to a constant factor, rather than getting bogged down in language-specific or hardware-specific analysis of the exact number of operations that execute on the computer. As a tangible example, we revisit the goal of finding the largest element of a Python list; we first used this example when introducing for loops on page 21 of Section 1.4.2. Code Fragment 3.1 presents a function named find max for this task. 1 def find max(data): 2 ”””Return the maximum element from a nonempty Python list.””” 3 biggest = data[0] # The initial value to beat 4 for val in data: # For each value: 5 if val biggest # if it is greater than the best so far, 6 biggest = val # we have found a new best (so far) 7 return biggest # When loop ends, biggest is the max Code Fragment 3.1: A function that returns the maximum value of a Python list. This is a classic example of an algorithm with a running time that grows pro- portional to n, as the loop executes once for each data element, with some fixed number of primitive operations executing for each pass. In the remainder of this section, we provide a framework to formalize this claim. 3.3.1 The “Big-Oh” Notation Let f(n) and g(n) be functions mapping positive integers to positive real numbers. We say that f(n) is O(g(n)) if there is a real constant c 0 and an integer constant n0 ≥ 1 such that f(n) ≤ cg(n), for n ≥ n0. This definition is often referred to as the “big-Oh” notation, for it is sometimes pro- nounced as “f(n) is big-Oh of g(n).” Figure 3.5 illustrates the general definition.
- 146. 124 Chapter 3.Algorithm Analysis Input Size Running Time cg(n) f(n) n0 Figure 3.5: Illustrating the “big-Oh” notation. The function f(n) is O(g(n)), since f(n) ≤ c·g(n) when n ≥ n0. Example 3.6: The function 8n+5 is O(n). Justification: By the big-Oh definition, we need to find a real constant c 0 and an integer constant n0 ≥ 1 such that 8n+5 ≤ cn for every integer n ≥ n0. It is easy to see that a possible choice is c = 9 and n0 = 5. Indeed, this is one of infinitely many choices available because there is a trade-off between c and n0. For example, we could rely on constants c = 13 and n0 = 1. The big-Oh notation allows us to say that a function f(n) is “less than or equal to” another function g(n) up to a constant factor and in the asymptotic sense as n grows toward infinity. This ability comes from the fact that the definition uses “≤” to compare f(n) to a g(n) times a constant, c, for the asymptotic cases when n ≥ n0. However, it is considered poor taste to say “f(n) ≤ O(g(n)),” since the big-Oh already denotes the “less-than-or-equal-to” concept. Likewise, although common, it is not fully correct to say “f(n) = O(g(n)),” with the usual understanding of the “=” relation, because there is no way to make sense of the symmetric statement, “O(g(n)) = f(n).” It is best to say, “ f(n) is O(g(n)).” Alternatively, we can say “f(n) is order of g(n).” For the more mathematically inclined, it is also correct to say, “f(n) ∈ O(g(n)),” for the big-Oh notation, techni- cally speaking, denotes a whole collection of functions. In this book, we will stick to presenting big-Oh statements as “f(n) is O(g(n)).” Even with this interpretation, there is considerable freedom in how we can use arithmetic operations with the big- Oh notation, and with this freedom comes a certain amount of responsibility.
- 147. 3.3. Asymptotic Analysis125 Characterizing Running Times Using the Big-Oh Notation The big-Oh notation is used widely to characterize running times and space bounds in terms of some parameter n, which varies from problem to problem, but is always defined as a chosen measure of the “size” of the problem. For example, if we are interested in finding the largest element in a sequence, as with the find max algorithm, we should let n denote the number of elements in that collection. Using the big-Oh notation, we can write the following mathematically precise statement on the running time of algorithm find max (Code Fragment 3.1) for any computer. Proposition 3.7: The algorithm, find max, for computing the maximum element of a list of n numbers, runs in O(n) time. Justification: The initialization before the loop begins requires only a constant number of primitive operations. Each iteration of the loop also requires only a con- stant number of primitive operations, and the loop executes n times. Therefore, we account for the number of primitive operations being c +c ·n for appropriate constants c and c that reflect, respectively, the work performed during initializa- tion and the loop body. Because each primitive operation runs in constant time, we have that the running time of algorithm find max on an input of size n is at most a constant times n; that is, we conclude that the running time of algorithm find max is O(n). Some Properties of the Big-Oh Notation The big-Oh notation allows us to ignore constant factors and lower-order terms and focus on the main components of a function that affect its growth. Example 3.8: 5n4 +3n3 +2n2 +4n+1 is O(n4). Justification: Note that 5n4 +3n3 +2n2 +4n+1 ≤ (5+3+2+4+1)n4 = cn4, for c = 15, when n ≥ n0 = 1. In fact, we can characterize the growth rate of any polynomial function. Proposition 3.9: If f(n) is a polynomial of degree d, that is, f(n) = a0 +a1n+···+adnd , and ad 0, then f(n) is O(nd). Justification: Note that, for n ≥ 1, we have 1 ≤ n ≤ n2 ≤ ··· ≤ nd; hence, a0 +a1n+a2n2 +···+adnd ≤ (|a0|+|a1|+|a2|+···+|ad|)nd . We show that f(n) is O(nd) by defining c = |a0|+|a1|+···+|ad| and n0 = 1.
- 148. 126 Chapter 3.Algorithm Analysis Thus, the highest-degree term in a polynomial is the term that determines the asymptotic growth rate of that polynomial. We consider some additional properties of the big-Oh notation in the exercises. Let us consider some further examples here, focusing on combinations of the seven fundamental functions used in algorithm design. We rely on the mathematical fact that logn ≤ n for n ≥ 1. Example 3.10: 5n2 +3nlogn+2n+5 is O(n2). Justification: 5n2 +3nlogn+2n+5 ≤ (5+3+2+5)n2 = cn2, for c = 15, when n ≥ n0 = 1. Example 3.11: 20n3 +10nlogn+5 is O(n3). Justification: 20n3 +10nlogn+5 ≤ 35n3, for n ≥ 1. Example 3.12: 3logn+2 is O(logn). Justification: 3logn+ 2 ≤ 5logn, for n ≥ 2. Note that logn is zero for n = 1. That is why we use n ≥ n0 = 2 in this case. Example 3.13: 2n+2 is O(2n). Justification: 2n+2 = 2n ·22 = 4·2n; hence, we can take c = 4 and n0 = 1 in this case. Example 3.14: 2n+100logn is O(n). Justification: 2n+100logn ≤ 102n, for n ≥ n0 = 1; hence, we can take c = 102 in this case. Characterizing Functions in Simplest Terms In general, we should use the big-Oh notation to characterize a function as closely as possible. While it is true that the function f(n) = 4n3 + 3n2 is O(n5) or even O(n4), it is more accurate to say that f(n) is O(n3). Consider, by way of analogy, a scenario where a hungry traveler driving along a long country road happens upon a local farmer walking home from a market. If the traveler asks the farmer how much longer he must drive before he can find some food, it may be truthful for the farmer to say, “certainly no longer than 12 hours,” but it is much more accurate (and helpful) for him to say, “you can find a market just a few minutes drive up this road.” Thus, even with the big-Oh notation, we should strive as much as possible to tell the whole truth. It is also considered poor taste to include constant factors and lower-order terms in the big-Oh notation. For example, it is not fashionable to say that the function 2n2 is O(4n2 + 6nlogn), although this is completely correct. We should strive instead to describe the function in the big-Oh in simplest terms.
- 149. 3.3. Asymptotic Analysis127 The seven functions listed in Section 3.2 are the most common functions used in conjunction with the big-Oh notation to characterize the running times and space usage of algorithms. Indeed, we typically use the names of these functions to refer to the running times of the algorithms they characterize. So, for example, we would say that an algorithm that runs in worst-case time 4n2 +nlogn is a quadratic-time algorithm, since it runs in O(n2) time. Likewise, an algorithm running in time at most 5n+20logn+4 would be called a linear-time algorithm. Big-Omega Just as the big-Oh notation provides an asymptotic way of saying that a function is “less than or equal to” another function, the following notations provide an asymp- totic way of saying that a function grows at a rate that is “greater than or equal to” that of another. Let f(n) and g(n) be functions mapping positive integers to positive real num- bers. We say that f(n) is Ω(g(n)), pronounced “f(n) is big-Omega of g(n),” if g(n) is O(f(n)), that is, there is a real constant c 0 and an integer constant n0 ≥ 1 such that f(n) ≥ cg(n), for n ≥ n0. This definition allows us to say asymptotically that one function is greater than or equal to another, up to a constant factor. Example 3.15: 3nlogn−2n is Ω(nlogn). Justification: 3nlogn− 2n = nlogn+ 2n(logn− 1) ≥ nlogn for n ≥ 2; hence, we can take c = 1 and n0 = 2 in this case. Big-Theta In addition, there is a notation that allows us to say that two functions grow at the same rate, up to constant factors. We say that f(n) is Θ(g(n)), pronounced “f(n) is big-Theta of g(n),” if f(n) is O(g(n)) and f(n) is Ω(g(n)) , that is, there are real constants c 0 and c 0, and an integer constant n0 ≥ 1 such that c g(n) ≤ f(n) ≤ c g(n), for n ≥ n0. Example 3.16: 3nlogn+4n+5logn is Θ(nlogn). Justification: 3nlogn ≤ 3nlogn+4n+5logn ≤ (3+4+5)nlogn for n ≥ 2.
- 150. 128 Chapter 3.Algorithm Analysis 3.3.2 Comparative Analysis Suppose two algorithms solving the same problem are available: an algorithm A, which has a running time of O(n), and an algorithm B, which has a running time of O(n2). Which algorithm is better? We know that n is O(n2), which implies that algorithm A is asymptotically better than algorithm B, although for a small value of n, B may have a lower running time than A. We can use the big-Oh notation to order classes of functions by asymptotic growth rate. Our seven functions are ordered by increasing growth rate in the fol- lowing sequence, that is, if a function f(n) precedes a function g(n) in the sequence, then f(n) is O(g(n)): 1, logn, n, nlogn, n2 , n3 , 2n . We illustrate the growth rates of the seven functions in Table 3.2. (See also Figure 3.4 from Section 3.2.1.) n logn n nlogn n2 n3 2n 8 3 8 24 64 512 256 16 4 16 64 256 4,096 65,536 32 5 32 160 1,024 32,768 4,294,967,296 64 6 64 384 4,096 262,144 1.84× 1019 128 7 128 896 16,384 2,097,152 3.40× 1038 256 8 256 2,048 65,536 16,777,216 1.15× 1077 512 9 512 4,608 262,144 134,217,728 1.34× 10154 Table 3.2: Selected values of fundamental functions in algorithm analysis. We further illustrate the importance of the asymptotic viewpoint in Table 3.3. This table explores the maximum size allowed for an input instance that is pro- cessed by an algorithm in 1 second, 1 minute, and 1 hour. It shows the importance of good algorithm design, because an asymptotically slow algorithm is beaten in the long run by an asymptotically faster algorithm, even if the constant factor for the asymptotically faster algorithm is worse. Running Maximum Problem Size (n) Time (μs) 1 second 1 minute 1 hour 400n 2,500 150,000 9,000,000 2n2 707 5,477 42,426 2n 19 25 31 Table 3.3: Maximum size of a problem that can be solved in 1 second, 1 minute, and 1 hour, for various running times measured in microseconds.
- 151. 3.3. Asymptotic Analysis129 The importance of good algorithm design goes beyond just what can be solved effectively on a given computer, however. As shown in Table 3.4, even if we achieve a dramatic speedup in hardware, we still cannot overcome the handicap of an asymptotically slow algorithm. This table shows the new maximum problem size achievable for any fixed amount of time, assuming algorithms with the given running times are now run on a computer 256 times faster than the previous one. Running Time New Maximum Problem Size 400n 256m 2n2 16m 2n m+8 Table 3.4: Increase in the maximum size of a problem that can be solved in a fixed amount of time, by using a computer that is 256 times faster than the previous one. Each entry is a function of m, the previous maximum problem size. Some Words of Caution A few words of caution about asymptotic notation are in order at this point. First, note that the use of the big-Oh and related notations can be somewhat misleading should the constant factors they “hide” be very large. For example, while it is true that the function 10100n is O(n), if this is the running time of an algorithm being compared to one whose running time is 10nlogn, we should prefer the O(nlogn)- time algorithm, even though the linear-time algorithm is asymptotically faster. This preference is because the constant factor, 10100, which is called “one googol,” is believed by many astronomers to be an upper bound on the number of atoms in the observable universe. So we are unlikely to ever have a real-world problem that has this number as its input size. Thus, even when using the big-Oh notation, we should at least be somewhat mindful of the constant factors and lower-order terms we are “hiding.” The observation above raises the issue of what constitutes a “fast” algorithm. Generally speaking, any algorithm running in O(nlogn) time (with a reasonable constant factor) should be considered efficient. Even an O(n2)-time function may be fast enough in some contexts, that is, when n is small. But an algorithm running in O(2n) time should almost never be considered efficient. Exponential Running Times There is a famous story about the inventor of the game of chess. He asked only that his king pay him 1 grain of rice for the first square on the board, 2 grains for the second, 4 grains for the third, 8 for the fourth, and so on. It is an interesting test of programming skills to write a program to compute exactly the number of grains of rice the king would have to pay.
- 152. 130 Chapter 3.Algorithm Analysis If we must draw a line between efficient and inefficient algorithms, therefore, it is natural to make this distinction be that between those algorithms running in polynomial time and those running in exponential time. That is, make the distinc- tion between algorithms with a running time that is O(nc), for some constant c 1, and those with a running time that is O(bn), for some constant b 1. Like so many notions we have discussed in this section, this too should be taken with a “grain of salt,” for an algorithm running in O(n100) time should probably not be considered “efficient.” Even so, the distinction between polynomial-time and exponential-time algorithms is considered a robust measure of tractability. 3.3.3 Examples of Algorithm Analysis Now that we have the big-Oh notation for doing algorithm analysis, let us give some examples by characterizing the running time of some simple algorithms using this notation. Moreover, in keeping with our earlier promise, we illustrate below how each of the seven functions given earlier in this chapter can be used to characterize the running time of an example algorithm. Rather than use pseudo-code in this section, we give complete Python imple- mentations for our examples. We use Python’s list class as the natural representa- tion for an “array” of values. In Chapter 5, we will fully explore the underpinnings of Python’s list class, and the efficiency of the various behaviors that it supports. In this section, we rely on just a few of its behaviors, discussing their efficiencies as introduced. Constant-Time Operations Given an instance, named data, of the Python list class, a call to the function, len(data), is evaluated in constant time. This is a very simple algorithm because the list class maintains, for each list, an instance variable that records the current length of the list. This allows it to immediately report that length, rather than take time to iteratively count each of the elements in the list. Using asymptotic notation, we say that this function runs in O(1) time; that is, the running time of this function is independent of the length, n, of the list. Another central behavior of Python’s list class is that it allows access to an arbi- trary element of the list using syntax, data[j], for integer index j. Because Python’s lists are implemented as array-based sequences, references to a list’s elements are stored in a consecutive block of memory. The jth element of the list can be found, not by iterating through the list one element at a time, but by validating the index, and using it as an offset into the underlying array. In turn, computer hardware sup- ports constant-time access to an element based on its memory address. Therefore, we say that the expression data[j] is evaluated in O(1) time for a Python list.
- 153. 3.3. Asymptotic Analysis131 Revisiting the Problem of Finding the Maximum of a Sequence For our next example, we revisit the find max algorithm, given in Code Frag- ment 3.1 on page 123, for finding the largest value in a sequence. Proposition 3.7 on page 125 claimed an O(n) run-time for the find max algorithm. Consistent with our earlier analysis of syntax data[0], the initialization uses O(1) time. The loop executes n times, and within each iteration, it performs one comparison and possi- bly one assignment statement (as well as maintenance of the loop variable). Finally, we note that the mechanism for enacting a return statement in Python uses O(1) time. Combining these steps, we have that the find max function runs in O(n) time. Further Analysis of the Maximum-Finding Algorithm A more interesting question about find max is how many times we might update the current “biggest” value. In the worst case, if the data is given to us in increasing order, the biggest value is reassigned n − 1 times. But what if the input is given to us in random order, with all orders equally likely; what would be the expected number of times we update the biggest value in this case? To answer this question, note that we update the current biggest in an iteration of the loop only if the current element is bigger than all the elements that precede it. If the sequence is given to us in random order, the probability that the jth element is the largest of the first j elements is 1/j (assuming uniqueness). Hence, the expected number of times we update the biggest (including initialization) is Hn = ∑n j=1 1/j, which is known as the nth Harmonic number. It turns out (see Proposition B.16) that Hn is O(logn). Therefore, the expected number of times the biggest value is updated by find max on a randomly ordered sequence is O(logn). Prefix Averages The next problem we consider is computing what are known as prefix averages of a sequence of numbers. Namely, given a sequence S consisting of n num- bers, we want to compute a sequence A such that A[ j] is the average of elements S[0],...,S[ j], for j = 0,...,n−1, that is, A[ j] = ∑ j i=0 S[i] j +1 . Computing prefix averages has many applications in economics and statistics. For example, given the year-by-year returns of a mutual fund, ordered from recent to past, an investor will typically want to see the fund’s average annual returns for the most recent year, the most recent three years, the most recent five years, and so on. Likewise, given a stream of daily Web usage logs, a Web site manager may wish to track average usage trends over various time periods. We analyze three different implementations that solve this problem but with rather different running times.
- 154. 132 Chapter 3.Algorithm Analysis A Quadratic-Time Algorithm Our first algorithm for computing prefix averages, named prefix average1, is shown in Code Fragment 3.2. It computes every element of A separately, using an inner loop to compute the partial sum. 1 def prefix average1(S): 2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].””” 3 n = len(S) 4 A = [0] n # create new list of n zeros 5 for j in range(n): 6 total = 0 # begin computing S[0] + ... + S[j] 7 for i in range(j + 1): 8 total += S[i] 9 A[j] = total / (j+1) # record the average 10 return A Code Fragment 3.2: Algorithm prefix average1. In order to analyze the prefix average1 algorithm, we consider the various steps that are executed. • The statement, n = len(S), executes in constant time, as described at the beginning of Section 3.3.3. • The statement, A = [0] n, causes the creation and initialization of a Python list with length n, and with all entries equal to zero. This uses a constant number of primitive operations per element, and thus runs in O(n) time. • There are two nested for loops, which are controlled, respectively, by coun- ters j and i. The body of the outer loop, controlled by counter j, is ex- ecuted n times, for j = 0,...,n − 1. Therefore, statements total = 0 and A[j] = total / (j+1) are executed n times each. This implies that these two statements, plus the management of counter j in the range, contribute a num- ber of primitive operations proportional to n, that is, O(n) time. • The body of the inner loop, which is controlled by counter i, is executed j+1 times, depending on the current value of the outer loop counter j. Thus, state- ment total += S[i], in the inner loop, is executed 1+ 2+ 3+ ··· + n times. By recalling Proposition 3.3, we know that 1+2+3+···+n = n(n+1)/2, which implies that the statement in the inner loop contributes O(n2) time. A similar argument can be done for the primitive operations associated with maintaining counter i, which also take O(n2) time. The running time of implementation prefix average1 is given by the sum of three terms. The first and the second terms are O(n), and the third term is O(n2). By a simple application of Proposition 3.9, the running time of prefix average1 is O(n2).
- 155. 3.3. Asymptotic Analysis133 Our second implementation for computing prefix averages, prefix average2, is presented in Code Fragment 3.3. 1 def prefix average2(S): 2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].””” 3 n = len(S) 4 A = [0] n # create new list of n zeros 5 for j in range(n): 6 A[j] = sum(S[0:j+1]) / (j+1) # record the average 7 return A Code Fragment 3.3: Algorithm prefix average2. This approach is essentially the same high-level algorithm as in prefix average1, but we have replaced the inner loop by using the single expression sum(S[0:j+1]) to compute the partial sum, S[0]+··· +S[ j]. While the use of that function greatly simplifies the presentation of the algorithm, it is worth asking how it affects the efficiency. Asymptotically, this implementation is no better. Even though the ex- pression, sum(S[0:j+1]), seems like a single command, it is a function call and an evaluation of that function takes O(j + 1) time in this context. Technically, the computation of the slice, S[0:j+1], also uses O(j + 1) time, as it constructs a new list instance for storage. So the running time of prefix average2 is still dominated by a series of steps that take time proportional to 1+2+3+···+n, and thus O(n2). A Linear-Time Algorithm Our final algorithm, prefix averages3, is given in Code Fragment 3.4. Just as with our first two algorithms, we are interested in computing, for each j, the prefix sum S[0] + S[1] + ··· + S[ j], denoted as total in our code, so that we can then compute the prefix average A[j] =total / (j + 1). However, there is a key difference that results in much greater efficiency. 1 def prefix average3(S): 2 ”””Return list such that, for all j, A[j] equals average of S[0], ..., S[j].””” 3 n = len(S) 4 A = [0] n # create new list of n zeros 5 total = 0 # compute prefix sum as S[0] + S[1] + ... 6 for j in range(n): 7 total += S[j] # update prefix sum to include S[j] 8 A[j] = total / (j+1) # compute average based on current sum 9 return A Code Fragment 3.4: Algorithm prefix average3.
- 156. 134 Chapter 3.Algorithm Analysis In our first two algorithms, the prefix sum is computed anew for each value of j. That contributed O(j) time for each j, leading to the quadratic behavior. In algo- rithm prefix average3, we maintain the current prefix sum dynamically, effectively computing S[0]+S[1]+··· +S[ j] as total + S[j], where value total is equal to the sum S[0] + S[1] + ··· + S[ j − 1] computed by the previous pass of the loop over j. The analysis of the running time of algorithm prefix average3 follows: • Initializing variables n and total uses O(1) time. • Initializing the list A uses O(n) time. • There is a single for loop, which is controlled by counter j. The maintenance of that counter by the range iterator contributes a total of O(n) time. • The body of the loop is executed n times, for j = 0,...,n − 1. Thus, state- ments total += S[j] and A[j] = total / (j+1) are executed n times each. Since each of these statements uses O(1) time per iteration, their overall contribution is O(n) time. The running time of algorithm prefix average3 is given by the sum of the four terms. The first is O(1) and the remaining three are O(n). By a simple application of Proposition 3.9, the running time of prefix average3 is O(n), which is much better than the quadratic time of algorithms prefix average1 and prefix average2. Three-Way Set Disjointness Suppose we are given three sequences of numbers, A, B, and C. We will assume that no individual sequence contains duplicate values, but that there may be some numbers that are in two or three of the sequences. The three-way set disjointness problem is to determine if the intersection of the three sequences is empty, namely, that there is no element x such that x ∈ A, x ∈ B, and x ∈ C. A simple Python function to determine this property is given in Code Fragment 3.5. 1 def disjoint1(A, B, C): 2 ”””Return True if there is no element common to all three lists.””” 3 for a in A: 4 for b in B: 5 for c in C: 6 if a == b == c: 7 return False # we found a common value 8 return True # if we reach this, sets are disjoint Code Fragment 3.5: Algorithm disjoint1 for testing three-way set disjointness. This simple algorithm loops through each possible triple of values from the three sets to see if those values are equivalent. If each of the original sets has size n, then the worst-case running time of this function is O(n3).
- 157. 3.3. Asymptotic Analysis135 We can improve upon the asymptotic performance with a simple observation. Once inside the body of the loop over B, if selected elements a and b do not match each other, it is a waste of time to iterate through all values of C looking for a matching triple. An improved solution to this problem, taking advantage of this observation, is presented in Code Fragment 3.6. 1 def disjoint2(A, B, C): 2 ”””Return True if there is no element common to all three lists.””” 3 for a in A: 4 for b in B: 5 if a == b: # only check C if we found match from A and B 6 for c in C: 7 if a == c # (and thus a == b == c) 8 return False # we found a common value 9 return True # if we reach this, sets are disjoint Code Fragment 3.6: Algorithm disjoint2 for testing three-way set disjointness. In the improved version, it is not simply that we save time if we get lucky. We claim that the worst-case running time for disjoint2 is O(n2). There are quadrat- ically many pairs (a,b) to consider. However, if A and B are each sets of distinct elements, there can be at most O(n) such pairs with a equal to b. Therefore, the innermost loop, over C, executes at most n times. To account for the overall running time, we examine the time spent executing each line of code. The management of the for loop over A requires O(n) time. The management of the for loop over B accounts for a total of O(n2) time, since that loop is executed n different times. The test a == b is evaluated O(n2) times. The rest of the time spent depends upon how many matching (a,b) pairs exist. As we have noted, there are at most n such pairs, and so the management of the loop over C, and the commands within the body of that loop, use at most O(n2) time. By our standard application of Proposition 3.9, the total time spent is O(n2). Element Uniqueness A problem that is closely related to the three-way set disjointness problem is the element uniqueness problem. In the former, we are given three collections and we presumed that there were no duplicates within a single collection. In the element uniqueness problem, we are given a single sequence S with n elements and asked whether all elements of that collection are distinct from each other. Our first solution to this problem uses a straightforward iterative algorithm. The unique1 function, given in Code Fragment 3.7, solves the element uniqueness problem by looping through all distinct pairs of indices j k, checking if any of
- 158. 136 Chapter 3.Algorithm Analysis 1 def unique1(S): 2 ”””Return True if there are no duplicate elements in sequence S.””” 3 for j in range(len(S)): 4 for k in range(j+1, len(S)): 5 if S[j] == S[k]: 6 return False # found duplicate pair 7 return True # if we reach this, elements were unique Code Fragment 3.7: Algorithm unique1 for testing element uniqueness. those pairs refer to elements that are equivalent to each other. It does this using two nested for loops, such that the first iteration of the outer loop causes n−1 iterations of the inner loop, the second iteration of the outer loop causes n − 2 iterations of the inner loop, and so on. Thus, the worst-case running time of this function is proportional to (n−1)+(n−2)+···+2+1, which we recognize as the familiar O(n2) summation from Proposition 3.3. Using Sorting as a Problem-Solving Tool An even better algorithm for the element uniqueness problem is based on using sorting as a problem-solving tool. In this case, by sorting the sequence of elements, we are guaranteed that any duplicate elements will be placed next to each other. Thus, to determine if there are any duplicates, all we need to do is perform a sin- gle pass over the sorted sequence, looking for consecutive duplicates. A Python implementation of this algorithm is as follows: 1 def unique2(S): 2 ”””Return True if there are no duplicate elements in sequence S.””” 3 temp = sorted(S) # create a sorted copy of S 4 for j in range(1, len(temp)): 5 if S[j−1] == S[j]: 6 return False # found duplicate pair 7 return True # if we reach this, elements were unique Code Fragment 3.8: Algorithm unique2 for testing element uniqueness. The built-in function, sorted, as described in Section 1.5.2, produces a copy of the original list with elements in sorted order. It guarantees a worst-case running time of O(nlogn); see Chapter 12 for a discussion of common sorting algorithms. Once the data is sorted, the subsequent loop runs in O(n) time, and so the entire unique2 algorithm runs in O(nlogn) time.
- 159. 3.4. Simple JustificationTechniques 137 3.4 Simple Justification Techniques Sometimes, we will want to make claims about an algorithm, such as showing that it is correct or that it runs fast. In order to rigorously make such claims, we must use mathematical language, and in order to back up such claims, we must justify or prove our statements. Fortunately, there are several simple ways to do this. 3.4.1 By Example Some claims are of the generic form, “There is an element x in a set S that has property P.” To justify such a claim, we only need to produce a particular x in S that has property P. Likewise, some hard-to-believe claims are of the generic form, “Every element x in a set S has property P.” To justify that such a claim is false, we only need to produce a particular x from S that does not have property P. Such an instance is called a counterexample. Example 3.17: Professor Amongus claims that every number of the form 2i − 1 is a prime, when i is an integer greater than 1. Professor Amongus is wrong. Justification: To prove Professor Amongus is wrong, we find a counterexample. Fortunately, we need not look too far, for 24 −1 = 15 = 3·5. 3.4.2 The “Contra” Attack Another set of justification techniques involves the use of the negative. The two primary such methods are the use of the contrapositive and the contradiction. The use of the contrapositive method is like looking through a negative mirror. To justify the statement “if p is true, then q is true,” we establish that “if q is not true, then p is not true” instead. Logically, these two statements are the same, but the latter, which is called the contrapositive of the first, may be easier to think about. Example 3.18: Let a and b be integers. If ab is even, then a is even or b is even. Justification: To justify this claim, consider the contrapositive, “If a is odd and b is odd, then ab is odd.” So, suppose a = 2j +1 and b = 2k+1, for some integers j and k. Then ab = 4jk+2j +2k+1 = 2(2jk+ j +k)+1; hence, ab is odd. Besides showing a use of the contrapositive justification technique, the previous example also contains an application of DeMorgan’s Law. This law helps us deal with negations, for it states that the negation of a statement of the form “p or q” is “not p and not q.” Likewise, it states that the negation of a statement of the form “p and q” is “not p or not q.”
- 160. 138 Chapter 3.Algorithm Analysis Contradiction Another negative justification technique is justification by contradiction, which also often involves using DeMorgan’s Law. In applying the justification by con- tradiction technique, we establish that a statement q is true by first supposing that q is false and then showing that this assumption leads to a contradiction (such as 2 = 2 or 1 3). By reaching such a contradiction, we show that no consistent sit- uation exists with q being false, so q must be true. Of course, in order to reach this conclusion, we must be sure our situation is consistent before we assume q is false. Example 3.19: Let a and b be integers. If ab is odd, then a is odd and b is odd. Justification: Let ab be odd. We wish to show that a is odd and b is odd. So, with the hope of leading to a contradiction, let us assume the opposite, namely, suppose a is even or b is even. In fact, without loss of generality, we can assume that a is even (since the case for b is symmetric). Then a = 2j for some integer j. Hence, ab = (2j)b = 2(jb), that is, ab is even. But this is a contradiction: ab cannot simultaneously be odd and even. Therefore, a is odd and b is odd. 3.4.3 Induction and Loop Invariants Most of the claims we make about a running time or a space bound involve an inte- ger parameter n (usually denoting an intuitive notion of the “size” of the problem). Moreover, most of these claims are equivalent to saying some statement q(n) is true “for all n ≥ 1.” Since this is making a claim about an infinite set of numbers, we cannot justify this exhaustively in a direct fashion. Induction We can often justify claims such as those above as true, however, by using the technique of induction. This technique amounts to showing that, for any particular n ≥ 1, there is a finite sequence of implications that starts with something known to be true and ultimately leads to showing that q(n) is true. Specifically, we begin a justification by induction by showing that q(n) is true for n = 1 (and possibly some other values n = 2,3,...,k, for some constant k). Then we justify that the inductive “step” is true for n k, namely, we show “if q(j) is true for all j n, then q(n) is true.” The combination of these two pieces completes the justification by induction.
- 161. 3.4. Simple JustificationTechniques 139 Proposition 3.20: Consider the Fibonacci function F(n), which is defined such that F(1) = 1, F(2) = 2, and F(n) = F(n − 2) + F(n − 1) for n 2. (See Sec- tion 1.8.) We claim that F(n) 2n. Justification: We will show our claim is correct by induction. Base cases: (n ≤ 2). F(1) = 1 2 = 21 and F(2) = 2 4 = 22. Induction step: (n 2). Suppose our claim is true for all n n. Consider F(n). Since n 2, F(n) = F(n−2)+F(n−1). Moreover, since both n−2 and n−1 are less than n, we can apply the inductive assumption (sometimes called the “inductive hypothesis”) to imply that F(n) 2n−2 +2n−1, since 2n−2 +2n−1 2n−1 +2n−1 = 2·2n−1 = 2n . Let us do another inductive argument, this time for a fact we have seen before. Proposition 3.21: (which is the same as Proposition 3.3) n ∑ i=1 i = n(n+1) 2 . Justification: We will justify this equality by induction. Base case: n = 1. Trivial, for 1 = n(n+1)/2, if n = 1. Induction step: n ≥ 2. Assume the claim is true for n n. Consider n. n ∑ i=1 i = n+ n−1 ∑ i=1 i. By the induction hypothesis, then n ∑ i=1 i = n+ (n−1)n 2 , which we can simplify as n+ (n−1)n 2 = 2n+n2 −n 2 = n2 +n 2 = n(n+1) 2 . We may sometimes feel overwhelmed by the task of justifying something true for all n ≥ 1. We should remember, however, the concreteness of the inductive tech- nique. It shows that, for any particular n, there is a finite step-by-step sequence of implications that starts with something true and leads to the truth about n. In short, the inductive argument is a template for building a sequence of direct justifications.
- 162. 140 Chapter 3.Algorithm Analysis Loop Invariants The final justification technique we discuss in this section is the loop invariant. To prove some statement L about a loop is correct, define L in terms of a series of smaller statements L0,L1,...,Lk, where: 1. The initial claim, L0, is true before the loop begins. 2. If Lj−1 is true before iteration j, then Lj will be true after iteration j. 3. The final statement, Lk, implies the desired statement L to be true. Let us give a simple example of using a loop-invariant argument to justify the correctness of an algorithm. In particular, we use a loop invariant to justify that the function, find (see Code Fragment 3.9), finds the smallest index at which ele- ment val occurs in sequence S. 1 def find(S, val): 2 ”””Return index j such that S[j] == val, or -1 if no such element.””” 3 n = len(S) 4 j = 0 5 while j n: 6 if S[j] == val: 7 return j # a match was found at index j 8 j += 1 9 return −1 Code Fragment 3.9: Algorithm for finding the first index at which a given element occurs in a Python list. To show that find is correct, we inductively define a series of statements, Lj, that lead to the correctness of our algorithm. Specifically, we claim the following is true at the beginning of iteration j of the while loop: Lj: val is not equal to any of the first j elements of S. This claim is true at the beginning of the first iteration of the loop, because j is 0 and there are no elements among the first 0 in S (this kind of a trivially true claim is said to hold vacuously). In iteration j, we compare element val to element S[ j] and return the index j if these two elements are equivalent, which is clearly correct and completes the algorithm in this case. If the two elements val and S[ j] are not equal, then we have found one more element not equal to val and we increment the index j. Thus, the claim Lj will be true for this new value of j; hence, it is true at the beginning of the next iteration. If the while loop terminates without ever returning an index in S, then we have j = n. That is, Ln is true—there are no elements of S equal to val. Therefore, the algorithm correctly returns −1 to indicate that val is not in S.
- 163. 3.5. Exercises 141 3.5Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-3.1 Graph the functions 8n, 4nlogn, 2n2, n3, and 2n using a logarithmic scale for the x- and y-axes; that is, if the function value f(n) is y, plot this as a point with x-coordinate at logn and y-coordinate at logy. R-3.2 The number of operations executed by algorithms A and B is 8nlogn and 2n2, respectively. Determine n0 such that A is better than B for n ≥ n0. R-3.3 The number of operations executed by algorithms A and B is 40n2 and 2n3, respectively. Determine n0 such that A is better than B for n ≥ n0. R-3.4 Give an example of a function that is plotted the same on a log-log scale as it is on a standard scale. R-3.5 Explain why the plot of the function nc is a straight line with slope c on a log-log scale. R-3.6 What is the sum of all the even numbers from 0 to 2n, for any positive integer n? R-3.7 Show that the following two statements are equivalent: (a) The running time of algorithm A is always O(f(n)). (b) In the worst case, the running time of algorithm A is O(f(n)). R-3.8 Order the following functions by asymptotic growth rate. 4nlogn+2n 210 2logn 3n+100logn 4n 2n n2 +10n n3 nlogn R-3.9 Show that if d(n) is O(f(n)), then ad(n) is O(f(n)), for any constant a 0. R-3.10 Show that if d(n) is O(f(n)) and e(n) is O(g(n)), then the product d(n)e(n) is O(f(n)g(n)). R-3.11 Show that if d(n) is O(f(n)) and e(n) is O(g(n)), then d(n) + e(n) is O(f(n)+g(n)). R-3.12 Show that if d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)−e(n) is not necessarily O(f(n)−g(n)). R-3.13 Show that if d(n) is O(f(n)) and f(n) is O(g(n)), then d(n) is O(g(n)). R-3.14 Show that O(max{f(n),g(n)}) = O(f(n)+g(n)).
- 164. 142 Chapter 3.Algorithm Analysis R-3.15 Show that f(n) is O(g(n)) if and only if g(n) is Ω(f(n)). R-3.16 Show that if p(n) is a polynomial in n, then log p(n) is O(logn). R-3.17 Show that (n+1)5 is O(n5). R-3.18 Show that 2n+1 is O(2n). R-3.19 Show that n is O(nlogn). R-3.20 Show that n2 is Ω(nlogn). R-3.21 Show that nlogn is Ω(n). R-3.22 Show that f(n) is O(f(n)), if f(n) is a positive nondecreasing function that is always greater than 1. R-3.23 Give a big-Oh characterization, in terms of n, of the running time of the example1 function shown in Code Fragment 3.10. R-3.24 Give a big-Oh characterization, in terms of n, of the running time of the example2 function shown in Code Fragment 3.10. R-3.25 Give a big-Oh characterization, in terms of n, of the running time of the example3 function shown in Code Fragment 3.10. R-3.26 Give a big-Oh characterization, in terms of n, of the running time of the example4 function shown in Code Fragment 3.10. R-3.27 Give a big-Oh characterization, in terms of n, of the running time of the example5 function shown in Code Fragment 3.10. R-3.28 For each function f(n) and time t in the following table, determine the largest size n of a problem P that can be solved in time t if the algorithm for solving P takes f(n) microseconds (one entry is already completed). 1 Second 1 Hour 1 Month 1 Century logn ≈ 10300000 n nlogn n2 2n R-3.29 Algorithm A executes an O(logn)-time computation for each entry of an n-element sequence. What is its worst-case running time? R-3.30 Given an n-element sequence S, Algorithm B chooses logn elements in S at random and executes an O(n)-time calculation for each. What is the worst-case running time of Algorithm B? R-3.31 Given an n-element sequence S of integers, Algorithm C executes an O(n)-time computation for each even number in S, and an O(logn)-time computation for each odd number in S. What are the best-case and worst- case running times of Algorithm C?
- 165. 3.5. Exercises 143 1def example1(S): 2 ”””Return the sum of the elements in sequence S.””” 3 n = len(S) 4 total = 0 5 for j in range(n): # loop from 0 to n-1 6 total += S[j] 7 return total 8 9 def example2(S): 10 ”””Return the sum of the elements with even index in sequence S.””” 11 n = len(S) 12 total = 0 13 for j in range(0, n, 2): # note the increment of 2 14 total += S[j] 15 return total 16 17 def example3(S): 18 ”””Return the sum of the prefix sums of sequence S.””” 19 n = len(S) 20 total = 0 21 for j in range(n): # loop from 0 to n-1 22 for k in range(1+j): # loop from 0 to j 23 total += S[k] 24 return total 25 26 def example4(S): 27 ”””Return the sum of the prefix sums of sequence S.””” 28 n = len(S) 29 prefix = 0 30 total = 0 31 for j in range(n): 32 prefix += S[j] 33 total += prefix 34 return total 35 36 def example5(A, B): # assume that A and B have equal length 37 ”””Return the number of elements in B equal to the sum of prefix sums in A.””” 38 n = len(A) 39 count = 0 40 for i in range(n): # loop from 0 to n-1 41 total = 0 42 for j in range(n): # loop from 0 to n-1 43 for k in range(1+j): # loop from 0 to j 44 total += A[k] 45 if B[i] == total: 46 count += 1 47 return count Code Fragment 3.10: Some sample algorithms for analysis.
- 166. 144 Chapter 3.Algorithm Analysis R-3.32 Given an n-element sequence S, Algorithm D calls Algorithm E on each element S[i]. Algorithm E runs in O(i) time when it is called on element S[i]. What is the worst-case running time of Algorithm D? R-3.33 Al and Bob are arguing about their algorithms. Al claims his O(nlogn)- time method is always faster than Bob’s O(n2)-time method. To settle the issue, they perform a set of experiments. To Al’s dismay, they find that if n 100, the O(n2)-time algorithm runs faster, and only when n ≥ 100 is the O(nlogn)-time one better. Explain how this is possible. R-3.34 There is a well-known city (which will go nameless here) whose inhabi- tants have the reputation of enjoying a meal only if that meal is the best they have ever experienced in their life. Otherwise, they hate it. Assum- ing meal quality is distributed uniformly across a person’s life, describe the expected number of times inhabitants of this city are happy with their meals? Creativity C-3.35 Assuming it is possible to sort n numbers in O(nlogn) time, show that it is possible to solve the three-way set disjointness problem in O(nlogn) time. C-3.36 Describe an efficient algorithm for finding the ten largest elements in a sequence of size n. What is the running time of your algorithm? C-3.37 Give an example of a positive function f(n) such that f(n) is neither O(n) nor Ω(n). C-3.38 Show that ∑n i=1 i2 is O(n3). C-3.39 Show that ∑n i=1 i/2i 2. (Hint: Try to bound this sum term by term with a geometric progression.) C-3.40 Show that logb f(n) is Θ(log f(n)) if b 1 is a constant. C-3.41 Describe an algorithm for finding both the minimum and maximum of n numbers using fewer than 3n/2 comparisons. (Hint: First, construct a group of candidate minimums and a group of candidate maximums.) C-3.42 Bob built a Web site and gave the URL only to his n friends, which he numbered from 1 to n. He told friend number i that he/she can visit the Web site at most i times. Now Bob has a counter, C, keeping track of the total number of visits to the site (but not the identities of who visits). What is the minimum value for C such that Bob can know that one of his friends has visited his/her maximum allowed number of times? C-3.43 Draw a visual justification of Proposition 3.3 analogous to that of Fig- ure 3.3(b) for the case when n is odd.
- 167. 3.5. Exercises 145 C-3.44Communication security is extremely important in computer networks, and one way many network protocols achieve security is to encrypt mes- sages. Typical cryptographic schemes for the secure transmission of mes- sages over such networks are based on the fact that no efficient algorithms are known for factoring large integers. Hence, if we can represent a secret message by a large prime number p, we can transmit, over the network, the number r = p·q, where q p is another large prime number that acts as the encryption key. An eavesdropper who obtains the transmitted num- ber r on the network would have to factor r in order to figure out the secret message p. Using factoring to figure out a message is very difficult without knowing the encryption key q. To understand why, consider the following naive factoring algorithm: for p in range(2,r): if r % p == 0: # if p divides r return The secret message is p! a. Suppose that the eavesdropper uses the above algorithm and has a computer that can carry out in 1 microsecond (1 millionth of a sec- ond) a division between two integers of up to 100 bits each. Give an estimate of the time that it will take in the worst case to decipher the secret message p if the transmitted message r has 100 bits. b. What is the worst-case time complexity of the above algorithm? Since the input to the algorithm is just one large number r, assume that the input size n is the number of bytes needed to store r, that is, n = (log2 r)/8+1, and that each division takes time O(n). C-3.45 A sequence S contains n − 1 unique integers in the range [0,n − 1], that is, there is one number from this range that is not in S. Design an O(n)- time algorithm for finding that number. You are only allowed to use O(1) additional space besides the sequence S itself. C-3.46 Al says he can prove that all sheep in a flock are the same color: Base case: One sheep. It is clearly the same color as itself. Induction step: A flock of n sheep. Take a sheep, a, out. The remaining n − 1 are all the same color by induction. Now put sheep a back in and take out a different sheep, b. By induction, the n− 1 sheep (now with a) are all the same color. Therefore, all the sheep in the flock are the same color. What is wrong with Al’s “justification”? C-3.47 Let S be a set of n lines in the plane such that no two are parallel and no three meet in the same point. Show, by induction, that the lines in S determine Θ(n2) intersection points.
- 168. 146 Chapter 3.Algorithm Analysis C-3.48 Consider the following “justification” that the Fibonacci function, F(n) (see Proposition 3.20) is O(n): Base case (n ≤ 2): F(1) = 1 and F(2) = 2. Induction step (n 2): Assume claim true for n n. Consider n. F(n) = F(n−2)+F(n−1). By induction, F(n−2) is O(n−2) and F(n−1) is O(n−1). Then, F(n) is O((n−2)+(n−1)), by the identity presented in Exercise R-3.11. Therefore, F(n) is O(n). What is wrong with this “justification”? C-3.49 Consider the Fibonacci function, F(n) (see Proposition 3.20). Show by induction that F(n) is Ω((3/2)n). C-3.50 Let p(x) be a polynomial of degree n, that is, p(x) = ∑n i=0 aixi. (a) Describe a simple O(n2)-time algorithm for computing p(x). (b) Describe an O(nlogn)-time algorithm for computing p(x), based upon a more efficient calculation of xi. (c) Now consider a rewriting of p(x) as p(x) = a0 +x(a1 +x(a2 +x(a3 +···+x(an−1 +xan)···))), which is known as Horner’s method. Using the big-Oh notation, charac- terize the number of arithmetic operations this method executes. C-3.51 Show that the summation ∑n i=1 logi is O(nlogn). C-3.52 Show that the summation ∑n i=1 logi is Ω(nlogn). C-3.53 An evil king has n bottles of wine, and a spy has just poisoned one of them. Unfortunately, they do not know which one it is. The poison is very deadly; just one drop diluted even a billion to one will still kill. Even so, it takes a full month for the poison to take effect. Design a scheme for determining exactly which one of the wine bottles was poisoned in just one month’s time while expending O(logn) taste testers. C-3.54 A sequence S contains n integers taken from the interval [0,4n], with repe- titions allowed. Describe an efficient algorithm for determining an integer value k that occurs the most often in S. What is the running time of your algorithm? Projects P-3.55 Perform an experimental analysis of the three algorithms prefix average1, prefix average2, and prefix average3, from Section 3.3.3. Visualize their running times as a function of the input size with a log-log chart. P-3.56 Perform an experimental analysis that compares the relative running times of the functions shown in Code Fragment 3.10.
- 169. Chapter Notes 147 P-3.57Perform experimental analysis to test the hypothesis that Python’s sorted method runs in O(nlogn) time on average. P-3.58 For each of the three algorithms, unique1, unique2, and unique3, which solve the element uniqueness problem, perform an experimental analysis to determine the largest value of n such that the given algorithm runs in one minute or less. Chapter Notes The big-Oh notation has prompted several comments about its proper use [19, 49, 63]. Knuth [64, 63] defines it using the notation f(n) = O(g(n)), but says this “equality” is only “one way.” We have chosen to take a more standard view of equality and view the big-Oh notation as a set, following Brassard [19]. The reader interested in studying average-case analysis is referred to the book chapter by Vitter and Flajolet [101]. For some additional mathematical tools, please refer to Appendix B.
- 170. Chapter 4 Recursion Contents 4.1 IllustrativeExamples . . . . . . . . . . . . . . . . . . . . . . 150 4.1.1 The Factorial Function . . . . . . . . . . . . . . . . . . . 150 4.1.2 Drawing an English Ruler . . . . . . . . . . . . . . . . . . 152 4.1.3 Binary Search . . . . . . . . . . . . . . . . . . . . . . . . 155 4.1.4 File Systems . . . . . . . . . . . . . . . . . . . . . . . . . 157 4.2 Analyzing Recursive Algorithms . . . . . . . . . . . . . . . 161 4.3 Recursion Run Amok . . . . . . . . . . . . . . . . . . . . . 165 4.3.1 Maximum Recursive Depth in Python . . . . . . . . . . . 168 4.4 Further Examples of Recursion . . . . . . . . . . . . . . . . 169 4.4.1 Linear Recursion . . . . . . . . . . . . . . . . . . . . . . . 169 4.4.2 Binary Recursion . . . . . . . . . . . . . . . . . . . . . . 174 4.4.3 Multiple Recursion . . . . . . . . . . . . . . . . . . . . . 175 4.5 Designing Recursive Algorithms . . . . . . . . . . . . . . . 177 4.6 Eliminating Tail Recursion . . . . . . . . . . . . . . . . . . 178 4.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
- 171. 149 One way todescribe repetition within a computer program is the use of loops, such as Python’s while-loop and for-loop constructs described in Section 1.4.2. An entirely different way to achieve repetition is through a process known as recursion. Recursion is a technique by which a function makes one or more calls to itself during execution, or by which a data structure relies upon smaller instances of the very same type of structure in its representation. There are many examples of recursion in art and nature. For example, fractal patterns are naturally recursive. A physical example of recursion used in art is in the Russian Matryoshka dolls. Each doll is either made of solid wood, or is hollow and contains another Matryoshka doll inside it. In computing, recursion provides an elegant and powerful alternative for per- forming repetitive tasks. In fact, a few programming languages (e.g., Scheme, Smalltalk) do not explicitly support looping constructs and instead rely directly on recursion to express repetition. Most modern programming languages support functional recursion using the identical mechanism that is used to support tradi- tional forms of function calls. When one invocation of the function make a recur- sive call, that invocation is suspended until the recursive call completes. Recursion is an important technique in the study of data structures and algo- rithms. We will use it prominently in several later chapters of this book (most notably, Chapters 8 and 12). In this chapter, we begin with the following four il- lustrative examples of the use of recursion, providing a Python implementation for each. • The factorial function (commonly denoted as n!) is a classic mathematical function that has a natural recursive definition. • An English ruler has a recursive pattern that is a simple example of a fractal structure. • Binary search is among the most important computer algorithms. It allows us to efficiently locate a desired value in a data set with upwards of billions of entries. • The file system for a computer has a recursive structure in which directories can be nested arbitrarily deeply within other directories. Recursive algo- rithms are widely used to explore and manage these file systems. We then describe how to perform a formal analysis of the running time of a recursive algorithm and we discuss some potential pitfalls when defining recur- sions. In the balance of the chapter, we provide many more examples of recursive algorithms, organized to highlight some common forms of design.
- 172. 150 Chapter 4.Recursion 4.1 Illustrative Examples 4.1.1 The Factorial Function To demonstrate the mechanics of recursion, we begin with a simple mathematical example of computing the value of the factorial function. The factorial of a posi- tive integer n, denoted n!, is defined as the product of the integers from 1 to n. If n = 0, then n! is defined as 1 by convention. More formally, for any integer n ≥ 0, n! = 1 if n = 0 n·(n−1)·(n−2)···3·2·1 if n ≥ 1. For example, 5! = 5·4·3·2·1 = 120. The factorial function is important because it is known to equal the number of ways in which n distinct items can be arranged into a sequence, that is, the number of permutations of n items. For example, the three characters a, b, and c can be arranged in 3! = 3 · 2 · 1 = 6 ways: abc, acb, bac, bca, cab, and cba. There is a natural recursive definition for the factorial function. To see this, observe that 5! = 5· (4· 3· 2· 1) = 5· 4!. More generally, for a positive integer n, we can define n! to be n·(n−1)!. This recursive definition can be formalized as n! = 1 if n = 0 n·(n−1)! if n ≥ 1. This definition is typical of many recursive definitions. First, it contains one or more base cases, which are defined nonrecursively in terms of fixed quantities. In this case, n = 0 is the base case. It also contains one or more recursive cases, which are defined by appealing to the definition of the function being defined. A Recursive Implementation of the Factorial Function Recursion is not just a mathematical notation; we can use recursion to design a Python implementation of a factorial function, as shown in Code Fragment 4.1. 1 def factorial(n): 2 if n == 0: 3 return 1 4 else: 5 return n factorial(n−1) Code Fragment 4.1: A recursive implementation of the factorial function.
- 173. 4.1. Illustrative Examples151 This function does not use any explicit loops. Repetition is provided by the repeated recursive invocations of the function. There is no circularity in this defini- tion, because each time the function is invoked, its argument is smaller by one, and when a base case is reached, no further recursive calls are made. We illustrate the execution of a recursive function using a recursion trace. Each entry of the trace corresponds to a recursive call. Each new recursive function call is indicated by a downward arrow to a new invocation. When the function returns, an arrow showing this return is drawn and the return value may be indicated alongside this arrow. An example of such a trace for the factorial function is shown in Figure 4.1. return 4 6 = 24 factorial(4) factorial(1) factorial(0) factorial(3) factorial(2) return 1 return 1 1 = 1 return 2 1 = 2 return 3 2 = 6 Figure 4.1: A recursion trace for the call factorial(5). A recursion trace closely mirrors the programming language’s execution of the recursion. In Python, each time a function (recursive or otherwise) is called, a struc- ture known as an activation record or frame is created to store information about the progress of that invocation of the function. This activation record includes a namespace for storing the function call’s parameters and local variables (see Sec- tion 1.10 for a discussion of namespaces), and information about which command in the body of the function is currently executing. When the execution of a function leads to a nested function call, the execu- tion of the former call is suspended and its activation record stores the place in the source code at which the flow of control should continue upon return of the nested call. This process is used both in the standard case of one function calling a dif- ferent function, or in the recursive case in which a function invokes itself. The key point is that there is a different activation record for each active call.
- 174. 152 Chapter 4.Recursion 4.1.2 Drawing an English Ruler In the case of computing a factorial, there is no compelling reason for preferring recursion over a direct iteration with a loop. As a more complex example of the use of recursion, consider how to draw the markings of a typical English ruler. For each inch, we place a tick with a numeric label. We denote the length of the tick designating a whole inch as the major tick length. Between the marks for whole inches, the ruler contains a series of minor ticks, placed at intervals of 1/2 inch, 1/4 inch, and so on. As the size of the interval decreases by half, the tick length decreases by one. Figure 4.2 demonstrates several such rulers with varying major tick lengths (although not drawn to scale). ---- 0 ----- 0 --- 0 - - - -- -- -- - - - --- --- --- 1 - - - -- -- -- - - - ---- 1 ---- --- 2 - - - -- -- -- - - - --- --- --- 3 - - -- -- - - ---- 2 ----- 1 (a) (b) (c) Figure 4.2: Three sample outputs of an English ruler drawing: (a) a 2-inch ruler with major tick length 4; (b) a 1-inch ruler with major tick length 5; (c) a 3-inch ruler with major tick length 3. A Recursive Approach to Ruler Drawing The English ruler pattern is a simple example of a fractal, that is, a shape that has a self-recursive structure at various levels of magnification. Consider the rule with major tick length 5 shown in Figure 4.2(b). Ignoring the lines containing 0 and 1, let us consider how to draw the sequence of ticks lying between these lines. The central tick (at 1/2 inch) has length 4. Observe that the two patterns of ticks above and below this central tick are identical, and each has a central tick of length 3.
- 175. 4.1. Illustrative Examples153 In general, an interval with a central tick length L ≥ 1 is composed of: • An interval with a central tick length L−1 • A single tick of length L • An interval with a central tick length L−1 Although it is possible to draw such a ruler using an iterative process (see Ex- ercise P-4.25), the task is considerably easier to accomplish with recursion. Our implementation consists of three functions, as shown in Code Fragment 4.2. The main function, draw ruler, manages the construction of the entire ruler. Its argu- ments specify the total number of inches in the ruler and the major tick length. The utility function, draw line, draws a single tick with a specified number of dashes (and an optional string label, that is printed after the tick). The interesting work is done by the recursive draw interval function. This function draws the sequence of minor ticks within some interval, based upon the length of the interval’s central tick. We rely on the intuition shown at the top of this page, and with a base case when L = 0 that draws nothing. For L ≥ 1, the first and last steps are performed by recursively calling draw interval(L − 1). The middle step is performed by calling the function draw line(L). 1 def draw line(tick length, tick label= ): 2 ”””Draw one line with given tick length (followed by optional label).””” 3 line = - tick length 4 if tick label: 5 line += + tick label 6 print(line) 7 8 def draw interval(center length): 9 ”””Draw tick interval based upon a central tick length.””” 10 if center length 0: # stop when length drops to 0 11 draw interval(center length − 1) # recursively draw top ticks 12 draw line(center length) # draw center tick 13 draw interval(center length − 1) # recursively draw bottom ticks 14 15 def draw ruler(num inches, major length): 16 ”””Draw English ruler with given number of inches, major tick length.””” 17 draw line(major length, 0 ) # draw inch 0 line 18 for j in range(1, 1 + num inches): 19 draw interval(major length − 1) # draw interior ticks for inch 20 draw line(major length, str(j)) # draw inch j line and label Code Fragment 4.2: A recursive implementation of a function that draws a ruler.
- 176. 154 Chapter 4.Recursion Illustrating Ruler Drawing Using a Recursion Trace The execution of the recursive draw interval function can be visualized using a re- cursion trace. The trace for draw interval is more complicated than in the factorial example, however, because each instance makes two recursive calls. To illustrate this, we will show the recursion trace in a form that is reminiscent of an outline for a document. See Figure 4.3. (previous pattern repeats) draw interval(3) draw interval(2) draw interval(1) draw interval(1) draw interval(0) draw line(1) draw interval(0) draw interval(0) draw line(1) draw interval(0) draw line(3) draw interval(2) draw line(2) Output Figure 4.3: A partial recursion trace for the call draw interval(3). The second pattern of calls for draw interval(2) is not shown, but it is identical to the first.
- 177. 4.1. Illustrative Examples155 4.1.3 Binary Search In this section, we describe a classic recursive algorithm, binary search, that is used to efficiently locate a target value within a sorted sequence of n elements. This is among the most important of computer algorithms, and it is the reason that we so often store data in sorted order (as in Figure 4.4). 37 5 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 9 2 4 5 7 8 12 14 17 19 22 25 27 28 33 Figure 4.4: Values stored in sorted order within an indexable sequence, such as a Python list. The numbers at top are the indices. When the sequence is unsorted, the standard approach to search for a target value is to use a loop to examine every element, until either finding the target or exhausting the data set. This is known as the sequential search algorithm. This algorithm runs in O(n) time (i.e., linear time) since every element is inspected in the worst case. When the sequence is sorted and indexable, there is a much more efficient algorithm. (For intuition, think about how you would accomplish this task by hand!) For any index j, we know that all the values stored at indices 0,..., j − 1 are less than or equal to the value at index j, and all the values stored at indices j +1,...,n−1 are greater than or equal to that at index j. This observation allows us to quickly “home in” on a search target using a variant of the children’s game “high-low.” We call an element of the sequence a candidate if, at the current stage of the search, we cannot rule out that this item matches the target. The algorithm maintains two parameters, low and high, such that all the candidate entries have index at least low and at most high. Initially, low = 0 and high = n− 1. We then compare the target value to the median candidate, that is, the item data[mid] with index mid = (low +high)/2. We consider three cases: • If the target equals data[mid], then we have found the item we are looking for, and the search terminates successfully. • If target data[mid], then we recur on the first half of the sequence, that is, on the interval of indices from low to mid−1. • If target data[mid], then we recur on the second half of the sequence, that is, on the interval of indices from mid+1 to high. An unsuccessful search occurs if low high, as the interval [low,high] is empty.
- 178. 156 Chapter 4.Recursion This algorithm is known as binary search. We give a Python implementation in Code Fragment 4.3, and an illustration of the execution of the algorithm in Fig- ure 4.5. Whereas sequential search runs in O(n) time, the more efficient binary search runs in O(logn) time. This is a significant improvement, given that if n is one billion, logn is only 30. (We defer our formal analysis of binary search’s running time to Proposition 4.2 in Section 4.2.) 1 def binary search(data, target, low, high): 2 ”””Return True if target is found in indicated portion of a Python list. 3 4 The search only considers the portion from data[low] to data[high] inclusive. 5 ””” 6 if low high: 7 return False # interval is empty; no match 8 else: 9 mid = (low + high) // 2 10 if target == data[mid]: # found a match 11 return True 12 elif target data[mid]: 13 # recur on the portion left of the middle 14 return binary search(data, target, low, mid − 1) 15 else: 16 # recur on the portion right of the middle 17 return binary search(data, target, mid + 1, high) Code Fragment 4.3: An implementation of the binary search algorithm. mid high high low low mid low mid low=mid=high high 14 19 22 25 27 28 33 37 6 7 8 9 10 11 12 13 14 15 7 5 4 2 9 8 9 2 4 5 7 8 12 14 17 37 33 28 27 25 22 19 9 2 4 5 7 8 12 14 17 19 22 25 27 28 33 37 19 22 25 27 28 33 37 5 0 1 2 3 4 17 14 12 9 2 4 5 7 8 12 17 Figure 4.5: Example of a binary search for target value 22.
- 179. 4.1. Illustrative Examples157 4.1.4 File Systems Modern operating systems define file-system directories (which are also sometimes called “folders”) in a recursive way. Namely, a file system consists of a top-level directory, and the contents of this directory consists of files and other directories, which in turn can contain files and other directories, and so on. The operating system allows directories to be nested arbitrarily deep (as long as there is enough space in memory), although there must necessarily be some base directories that contain only files, not further subdirectories. A representation of a portion of such a file system is given in Figure 4.6. /user/rt/courses/ cs016/ cs252/ programs/ homeworks/ projects/ papers/ demos/ hw1 hw2 hw3 pr1 pr2 pr3 grades market buylow sellhigh grades Figure 4.6: A portion of a file system demonstrating a nested organization. Given the recursive nature of the file-system representation, it should not come as a surprise that many common behaviors of an operating system, such as copying a directory or deleting a directory, are implemented with recursive algorithms. In this section, we consider one such algorithm: computing the total disk usage for all files and directories nested within a particular directory. For illustration, Figure 4.7 portrays the disk space being used by all entries in our sample file system. We differentiate between the immediate disk space used by each entry and the cumulative disk space used by that entry and all nested features. For example, the cs016 directory uses only 2K of immediate space, but a total of 249K of cumulative space.
- 180. 158 Chapter 4.Recursion /user/rt/courses/ cs016/ cs252/ programs/ homeworks/ projects/ papers/ demos/ hw1 3K hw2 2K hw3 4K pr1 57K pr2 97K pr3 74K grades 8K market 4786K buylow 26K sellhigh 55K grades 3K 2K 1K 1K 1K 1K 1K 1K 1K 10K 229K 4870K 82K 4787K 5124K 249K 4874K Figure 4.7: The same portion of a file system given in Figure 4.6, but with additional annotations to describe the amount of disk space that is used. Within the icon for each file or directory is the amount of space directly used by that artifact. Above the icon for each directory is an indication of the cumulative disk space used by that directory and all its (recursive) contents. The cumulative disk space for an entry can be computed with a simple recursive algorithm. It is equal to the immediate disk space used by the entry plus the sum of the cumulative disk space usage of any entries that are stored directly within the entry. For example, the cumulative disk space for cs016 is 249K because it uses 2K itself, 8K cumulatively in grades, 10K cumulatively in homeworks, and 229K cumulatively in programs. Pseudo-code for this algorithm is given in Code Fragment 4.4. Algorithm DiskUsage(path): Input: A string designating a path to a file-system entry Output: The cumulative disk space used by that entry and any nested entries total = size(path) {immediate disk space used by the entry} if path represents a directory then for each child entry stored within directory path do total = total + DiskUsage(child) {recursive call} return total Code Fragment 4.4: An algorithm for computing the cumulative disk space usage nested at a file-system entry. Function size returns the immediate disk space of an entry.
- 181. 4.1. Illustrative Examples159 Python’s os Module To provide a Python implementation of a recursive algorithm for computing disk usage, we rely on Python’s os module, which provides robust tools for interacting with the operating system during the execution of a program. This is an extensive library, but we will only need the following four functions: • os.path.getsize(path) Return the immediate disk usage (measured in bytes) for the file or directory that is identified by the string path (e.g., /user/rt/courses). • os.path.isdir(path) Return True if entry designated by string path is a directory; False otherwise. • os.listdir(path) Return a list of strings that are the names of all entries within a directory designated by string path. In our sample file system, if the parameter is /user/rt/courses, this returns the list [ cs016 , cs252 ]. • os.path.join(path, filename) Compose the path string and filename string using an appropriate operating system separator between the two (e.g., the / character for a Unix/Linux system, and the character for Windows). Return the string that represents the full path to the file. Python Implementation With use of the os module, we now convert the algorithm from Code Fragment 4.4 into the Python implementation of Code Fragment 4.5. 1 import os 2 3 def disk usage(path): 4 ”””Return the number of bytes used by a file/folder and any descendents.””” 5 total = os.path.getsize(path) # account for direct usage 6 if os.path.isdir(path): # if this is a directory, 7 for filename in os.listdir(path): # then for each child: 8 childpath = os.path.join(path, filename) # compose full path to child 9 total += disk usage(childpath) # add child’s usage to total 10 11 print ( {0:7} .format(total), path) # descriptive output (optional) 12 return total # return the grand total Code Fragment 4.5: A recursive function for reporting disk usage of a file system.
- 182. 160 Chapter 4.Recursion Recursion Trace To produce a different form of a recursion trace, we have included an extraneous print statement within our Python implementation (line 11 of Code Fragment 4.5). The precise format of that output intentionally mirrors output that is produced by a classic Unix/Linux utility named du (for “disk usage”). It reports the amount of disk space used by a directory and all contents nested within, and can produce a verbose report, as given in Figure 4.8. Our implementation of the disk usage function produces an identical result, when executed on the sample file system portrayed in Figure 4.7. During the ex- ecution of the algorithm, exactly one recursive call is made for each entry in the portion of the file system that is considered. Because the print statement is made just before returning from a recursive call, the output shown in Figure 4.8 reflects the order in which the recursive calls are completed. In particular, we begin and end a recursive call for each entry that is nested below another entry, computing the nested cumulative disk space before we can compute and report the cumulative disk space for the containing entry. For example, we do not know the cumulative total for entry /user/rt/courses/cs016 until after the recursive calls regarding contained entries grades, homeworks, and programs complete. 8 /user/rt/courses/cs016/grades 3 /user/rt/courses/cs016/homeworks/hw1 2 /user/rt/courses/cs016/homeworks/hw2 4 /user/rt/courses/cs016/homeworks/hw3 10 /user/rt/courses/cs016/homeworks 57 /user/rt/courses/cs016/programs/pr1 97 /user/rt/courses/cs016/programs/pr2 74 /user/rt/courses/cs016/programs/pr3 229 /user/rt/courses/cs016/programs 249 /user/rt/courses/cs016 26 /user/rt/courses/cs252/projects/papers/buylow 55 /user/rt/courses/cs252/projects/papers/sellhigh 82 /user/rt/courses/cs252/projects/papers 4786 /user/rt/courses/cs252/projects/demos/market 4787 /user/rt/courses/cs252/projects/demos 4870 /user/rt/courses/cs252/projects 3 /user/rt/courses/cs252/grades 4874 /user/rt/courses/cs252 5124 /user/rt/courses/ Figure 4.8: A report of the disk usage for the file system shown in Figure 4.7, as generated by the Unix/Linux utility du (with command-line options -ak), or equivalently by our disk usage function from Code Fragment 4.5.
- 183. 4.2. Analyzing RecursiveAlgorithms 161 4.2 Analyzing Recursive Algorithms In Chapter 3, we introduced mathematical techniques for analyzing the efficiency of an algorithm, based upon an estimate of the number of primitive operations that are executed by the algorithm. We use notations such as big-Oh to summarize the relationship between the number of operations and the input size for a problem. In this section, we demonstrate how to perform this type of running-time analysis to recursive algorithms. With a recursive algorithm, we will account for each operation that is performed based upon the particular activation of the function that manages the flow of control at the time it is executed. Stated another way, for each invocation of the function, we only account for the number of operations that are performed within the body of that activation. We can then account for the overall number of operations that are executed as part of the recursive algorithm by taking the sum, over all activations, of the number of operations that take place during each individual activation. (As an aside, this is also the way we analyze a nonrecursive function that calls other functions from within its body.) To demonstrate this style of analysis, we revisit the four recursive algorithms presented in Sections 4.1.1 through 4.1.4: factorial computation, drawing an En- glish ruler, binary search, and computation of the cumulative size of a file system. In general, we may rely on the intuition afforded by a recursion trace in recogniz- ing how many recursive activations occur, and how the parameterization of each activation can be used to estimate the number of primitive operations that occur within the body of that activation. However, each of these recursive algorithms has a unique structure and form. Computing Factorials It is relatively easy to analyze the efficiency of our function for computing fac- torials, as described in Section 4.1.1. A sample recursion trace for our factorial function was given in Figure 4.1. To compute factorial(n), we see that there are a total of n+1 activations, as the parameter decreases from n in the first call, to n−1 in the second call, and so on, until reaching the base case with parameter 0. It is also clear, given an examination of the function body in Code Fragment 4.1, that each individual activation of factorial executes a constant number of opera- tions. Therefore, we conclude that the overall number of operations for computing factorial(n) is O(n), as there are n+1 activations, each of which accounts for O(1) operations.
- 184. 162 Chapter 4.Recursion Drawing an English Ruler In analyzing the English ruler application from Section 4.1.2, we consider the fun- damental question of how many total lines of output are generated by an initial call to draw interval(c), where c denotes the center length. This is a reasonable bench- mark for the overall efficiency of the algorithm as each line of output is based upon a call to the draw line utility, and each recursive call to draw interval with nonzero parameter makes exactly one direct call to draw line. Some intuition may be gained by examining the source code and the recur- sion trace. We know that a call to draw interval(c) for c 0 spawns two calls to draw interval(c−1) and a single call to draw line. We will rely on this intuition to prove the following claim. Proposition 4.1: For c ≥ 0, a call to draw interval(c) results in precisely 2c − 1 lines of output. Justification: We provide a formal proof of this claim by induction (see Sec- tion 3.4.3). In fact, induction is a natural mathematical technique for proving the correctness and efficiency of a recursive process. In the case of the ruler, we note that an application of draw interval(0) generates no output, and that 20 −1 = 1−1 = 0. This serves as a base case for our claim. More generally, the number of lines printed by draw interval(c) is one more than twice the number generated by a call to draw interval(c−1), as one center line is printed between two such recursive calls. By induction, we have that the number of lines is thus 1+2·(2c−1 −1) = 1+2c −2 = 2c −1. This proof is indicative of a more mathematically rigorous tool, known as a recurrence equation that can be used to analyze the running time of a recursive algorithm. That technique is discussed in Section 12.2.4, in the context of recursive sorting algorithms. Performing a Binary Search Considering the running time of the binary search algorithm, as presented in Sec- tion 4.1.3, we observe that a constant number of primitive operations are executed at each recursive call of method of a binary search. Hence, the running time is proportional to the number of recursive calls performed. We will show that at most logn+1 recursive calls are made during a binary search of a sequence having n elements, leading to the following claim. Proposition 4.2: The binary search algorithm runs in O(logn) time for a sorted sequence with n elements.
- 185. 4.2. Analyzing RecursiveAlgorithms 163 Justification: To prove this claim, a crucial fact is that with each recursive call the number of candidate entries still to be searched is given by the value high−low +1. Moreover, the number of remaining candidates is reduced by at least one half with each recursive call. Specifically, from the definition of mid, the number of remain- ing candidates is either (mid−1)−low +1 = low +high 2 −low ≤ high−low +1 2 or high−(mid+1)+1 = high− low +high 2 ≤ high−low +1 2 . Initially, the number of candidates is n; after the first call in a binary search, it is at most n/2; after the second call, it is at most n/4; and so on. In general, after the jth call in a binary search, the number of candidate entries remaining is at most n/2j. In the worst case (an unsuccessful search), the recursive calls stop when there are no more candidate entries. Hence, the maximum number of recursive calls performed, is the smallest integer r such that n 2r 1. In other words (recalling that we omit a logarithm’s base when it is 2), r logn. Thus, we have r = logn+1, which implies that binary search runs in O(logn) time. Computing Disk Space Usage Our final recursive algorithm from Section 4.1 was that for computing the overall disk space usage in a specified portion of a file system. To characterize the “prob- lem size” for our analysis, we let n denote the number of file-system entries in the portion of the file system that is considered. (For example, the file system portrayed in Figure 4.6 has n = 19 entries.) To characterize the cumulative time spent for an initial call to the disk usage function, we must analyze the total number of recursive invocations that are made, as well as the number of operations that are executed within those invocations. We begin by showing that there are precisely n recursive invocations of the function, in particular, one for each entry in the relevant portion of the file system. Intuitively, this is because a call to disk usage for a particular entry e of the file system is only made from within the for loop of Code Fragment 4.5 when process- ing the entry for the unique directory that contains e, and that entry will only be explored once.
- 186. 164 Chapter 4.Recursion To formalize this argument, we can define the nesting level of each entry such that the entry on which we begin has nesting level 0, entries stored directly within it have nesting level 1, entries stored within those entries have nesting level 2, and so on. We can prove by induction that there is exactly one recursive invocation of disk usage upon each entry at nesting level k. As a base case, when k = 0, the only recursive invocation made is the initial one. As the inductive step, once we know there is exactly one recursive invocation for each entry at nesting level k, we can claim that there is exactly one invocation for each entry e at nesting level k, made within the for loop for the entry at level k that contains e. Having established that there is one recursive call for each entry of the file system, we return to the question of the overall computation time for the algorithm. It would be great if we could argue that we spend O(1) time in any single invocation of the function, but that is not the case. While there are a constant number of steps reflect in the call to os.path.getsize to compute the disk usage directly at that entry, when the entry is a directory, the body of the disk usage function includes a for loop that iterates over all entries that are contained within that directory. In the worst case, it is possible that one entry includes n−1 others. Based on this reasoning, we could conclude that there are O(n) recursive calls, each of which runs in O(n) time, leading to an overall running time that is O(n2). While this upper bound is technically true, it is not a tight upper bound. Remark- ably, we can prove the stronger bound that the recursive algorithm for disk usage completes in O(n) time! The weaker bound was pessimistic because it assumed a worst-case number of entries for each directory. While it is possible that some directories contain a number of entries proportional to n, they cannot all contain that many. To prove the stronger claim, we choose to consider the overall number of iterations of the for loop across all recursive calls. We claim there are precisely n − 1 such iteration of that loop overall. We base this claim on the fact that each iteration of that loop makes a recursive call to disk usage, and yet we have already concluded that there are a total of n calls to disk usage (including the original call). We therefore conclude that there are O(n) recursive calls, each of which uses O(1) time outside the loop, and that the overall number of operations due to the loop is O(n). Summing all of these bounds, the overall number of operations is O(n). The argument we have made is more advanced than with the earlier examples of recursion. The idea that we can sometimes get a tighter bound on a series of operations by considering the cumulative effect, rather than assuming that each achieves a worst case is a technique called amortization; we will see a further example of such analysis in Section 5.3. Furthermore, a file system is an implicit example of a data structure known as a tree, and our disk usage algorithm is really a manifestation of a more general algorithm known as a tree traversal. Trees will be the focus of Chapter 8, and our argument about the O(n) running time of the disk usage algorithm will be generalized for tree traversals in Section 8.4.
- 187. 4.3. Recursion RunAmok 165 4.3 Recursion Run Amok Although recursion is a very powerful tool, it can easily be misused in various ways. In this section, we examine several problems in which a poorly implemented recur- sion causes drastic inefficiency, and we discuss some strategies for recognizing and avoid such pitfalls. We begin by revisiting the element uniqueness problem, defined on page 135 of Section 3.3.3. We can use the following recursive formulation to determine if all n elements of a sequence are unique. As a base case, when n = 1, the elements are trivially unique. For n ≥ 2, the elements are unique if and only if the first n−1 elements are unique, the last n−1 items are unique, and the first and last elements are different (as that is the only pair that was not already checked as a subcase). A recursive implementation based on this idea is given in Code Fragment 4.6, named unique3 (to differentiate it from unique1 and unique2 from Chapter 3). 1 def unique3(S, start, stop): 2 ”””Return True if there are no duplicate elements in slice S[start:stop].””” 3 if stop − start = 1: return True # at most one item 4 elif not unique(S, start, stop−1): return False # first part has duplicate 5 elif not unique(S, start+1, stop): return False # second part has duplicate 6 else: return S[start] != S[stop−1] # do first and last differ? Code Fragment 4.6: Recursive unique3 for testing element uniqueness. Unfortunately, this is a terribly inefficient use of recursion. The nonrecursive part of each call uses O(1) time, so the overall running time will be proportional to the total number of recursive invocations. To analyze the problem, we let n denote the number of entries under consideration, that is, let n= stop − start. If n = 1, then the running time of unique3 is O(1), since there are no recursive calls for this case. In the general case, the important observation is that a single call to unique3 for a problem of size n may result in two recursive calls on problems of size n − 1. Those two calls with size n − 1 could in turn result in four calls (two each) with a range of size n − 2, and thus eight calls with size n − 3 and so on. Thus, in the worst case, the total number of function calls is given by the geometric summation 1+2+4+···+2n−1 , which is equal to 2n − 1 by Proposition 3.5. Thus, the running time of function unique3 is O(2n). This is an incredibly inefficient function for solving the ele- ment uniqueness problem. Its inefficiency comes not from the fact that it uses recursion—it comes from the fact that it uses recursion poorly, which is something we address in Exercise C-4.11.
- 188. 166 Chapter 4.Recursion An Inefficient Recursion for Computing Fibonacci Numbers In Section 1.8, we introduced a process for generating the Fibonacci numbers, which can be defined recursively as follows: F0 = 0 F1 = 1 Fn = Fn−2 +Fn−1 for n 1. Ironically, a direct implementation based on this definition results in the function bad fibonacci shown in Code Fragment 4.7, which computes the sequence of Fi- bonacci numbers by making two recursive calls in each non-base case. 1 def bad fibonacci(n): 2 ”””Return the nth Fibonacci number.””” 3 if n = 1: 4 return n 5 else: 6 return bad fibonacci(n−2) + bad fibonacci(n−1) Code Fragment 4.7: Computing the nth Fibonacci number using binary recursion. Unfortunately, such a direct implementation of the Fibonacci formula results in a terribly inefficient function. Computing the nth Fibonacci number in this way requires an exponential number of calls to the function. Specifically, let cn denote the number of calls performed in the execution of bad fibonacci(n). Then, we have the following values for the cn’s: c0 = 1 c1 = 1 c2 = 1+c0 +c1 = 1+1+1 = 3 c3 = 1+c1 +c2 = 1+1+3 = 5 c4 = 1+c2 +c3 = 1+3+5 = 9 c5 = 1+c3 +c4 = 1+5+9 = 15 c6 = 1+c4 +c5 = 1+9+15 = 25 c7 = 1+c5 +c6 = 1+15+25 = 41 c8 = 1+c6 +c7 = 1+25+41 = 67 If we follow the pattern forward, we see that the number of calls more than doubles for each two consecutive indices. That is, c4 is more than twice c2, c5 is more than twice c3, c6 is more than twice c4, and so on. Thus, cn 2n/2, which means that bad fibonacci(n) makes a number of calls that is exponential in n.
- 189. 4.3. Recursion RunAmok 167 An Efficient Recursion for Computing Fibonacci Numbers We were tempted into using the bad recursion formulation because of the way the nth Fibonacci number, Fn, depends on the two previous values, Fn−2 and Fn−1. But notice that after computing Fn−2, the call to compute Fn−1 requires its own recursive call to compute Fn−2, as it does not have knowledge of the value of Fn−2 that was computed at the earlier level of recursion. That is duplicative work. Worse yet, both of those calls will need to (re)compute the value of Fn−3, as will the computation of Fn−1. This snowballing effect is what leads to the exponential running time of bad recursion. We can compute Fn much more efficiently using a recursion in which each invo- cation makes only one recursive call. To do so, we need to redefine the expectations of the function. Rather than having the function return a single value, which is the nth Fibonacci number, we define a recursive function that returns a pair of con- secutive Fibonacci numbers (Fn,Fn−1), using the convention F−1 = 0. Although it seems to be a greater burden to report two consecutive Fibonacci numbers in- stead of one, passing this extra information from one level of the recursion to the next makes it much easier to continue the process. (It allows us to avoid having to recompute the second value that was already known within the recursion.) An implementation based on this strategy is given in Code Fragment 4.8. 1 def good fibonacci(n): 2 ”””Return pair of Fibonacci numbers, F(n) and F(n-1).””” 3 if n = 1: 4 return (n,0) 5 else: 6 (a, b) = good fibonacci(n−1) 7 return (a+b, a) Code Fragment 4.8: Computing the nth Fibonacci number using linear recursion. In terms of efficiency, the difference between the bad recursion and the good recursion for this problem is like night and day. The bad fibonacci function uses exponential time. We claim that the execution of function good fibonacci(n) takes O(n) time. Each recursive call to good fibonacci decreases the argument n by 1; therefore, a recursion trace includes a series of n function calls. Because the nonre- cursive work for each call uses constant time, the overall computation executes in O(n) time.
- 190. 168 Chapter 4.Recursion 4.3.1 Maximum Recursive Depth in Python Another danger in the misuse of recursion is known as infinite recursion. If each recursive call makes another recursive call, without ever reaching a base case, then we have an infinite series of such calls. This is a fatal error. An infinite recursion can quickly swamp computing resources, not only due to rapid use of the CPU, but because each successive call creates an activation record requiring additional memory. A blatant example of an ill-formed recursion is the following: def fib(n): return fib(n) # fib(n) equals fib(n) However, there are far more subtle errors that can lead to an infinite recursion. Revisiting our implementation of binary search in Code Fragment 4.3, in the final case (line 17) we make a recursive call on the right portion of the sequence, in particular going from index mid+1 to high. Had that line instead been written as return binary search(data, target, mid, high) # note the use of mid this could result in an infinite recursion. In particular, when searching a range of two elements, it becomes possible to make a recursive call on the identical range. A programmer should ensure that each recursive call is in some way progress- ing toward a base case (for example, by having a parameter value that decreases with each call). However, to combat against infinite recursions, the designers of Python made an intentional decision to limit the overall number of function acti- vations that can be simultaneously active. The precise value of this limit depends upon the Python distribution, but a typical default value is 1000. If this limit is reached, the Python interpreter raises a RuntimeError with a message, maximum recursion depth exceeded. For many legitimate applications of recursion, a limit of 1000 nested function calls suffices. For example, our binary search function (Section 4.1.3) has O(logn) recursive depth, and so for the default recursive limit to be reached, there would need to be 21000 elements (far, far more than the estimated number of atoms in the universe). However, in the next section we discuss several algorithms that have recursive depth proportional to n. Python’s artificial limit on the recursive depth could disrupt such otherwise legitimate computations. Fortunately, the Python interpreter can be dynamically reconfigured to change the default recursive limit. This is done through use of a module named sys, which supports a getrecursionlimit function and a setrecursionlimit. Sample usage of those functions is demonstrated as follows: import sys old = sys.getrecursionlimit( ) # perhaps 1000 is typical sys.setrecursionlimit(1000000) # change to allow 1 million nested calls
- 191. 4.4. Further Examplesof Recursion 169 4.4 Further Examples of Recursion In the remainder of this chapter, we provide additional examples of the use of re- cursion. We organize our presentation by considering the maximum number of recursive calls that may be started from within the body of a single activation. • If a recursive call starts at most one other, we call this a linear recursion. • If a recursive call may start two others, we call this a binary recursion. • If a recursive call may start three or more others, this is multiple recursion. 4.4.1 Linear Recursion If a recursive function is designed so that each invocation of the body makes at most one new recursive call, this is know as linear recursion. Of the recursions we have seen so far, the implementation of the factorial function (Section 4.1.1) and the good fibonacci function (Section 4.3) are clear examples of linear recursion. More interestingly, the binary search algorithm (Section 4.1.3) is also an example of linear recursion, despite the “binary” terminology in the name. The code for binary search (Code Fragment 4.3) includes a case analysis with two branches that lead to recursive calls, but only one of those calls can be reached during a particular execution of the body. A consequence of the definition of linear recursion is that any recursion trace will appear as a single sequence of calls, as we originally portrayed for the factorial function in Figure 4.1 of Section 4.1.1. Note that the linear recursion terminol- ogy reflects the structure of the recursion trace, not the asymptotic analysis of the running time; for example, we have seen that binary search runs in O(logn) time. Summing the Elements of a Sequence Recursively Linear recursion can be a useful tool for processing a data sequence, such as a Python list. Suppose, for example, that we want to compute the sum of a sequence, S, of n integers. We can solve this summation problem using linear recursion by observing that the sum of all n integers in S is trivially 0, if n = 0, and otherwise that it is the sum of the first n − 1 integers in S plus the last element in S. (See Figure 4.9.) 4 3 6 2 8 9 3 2 8 5 1 7 2 8 3 5 7 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 Figure 4.9: Computing the sum of a sequence recursively, by adding the last number to the sum of the first n−1.
- 192. 170 Chapter 4.Recursion A recursive algorithm for computing the sum of a sequence of numbers based on this intuition is implemented in Code Fragment 4.9. 1 def linear sum(S, n): 2 ”””Return the sum of the first n numbers of sequence S.””” 3 if n == 0: 4 return 0 5 else: 6 return linear sum(S, n−1) + S[n−1] Code Fragment 4.9: Summing the elements of a sequence using linear recursion. A recursion trace of the linear sum function for a small example is given in Figure 4.10. For an input of size n, the linear sum algorithm makes n+1 function calls. Hence, it will take O(n) time, because it spends a constant amount of time performing the nonrecursive part of each call. Moreover, we can also see that the memory space used by the algorithm (in addition to the sequence S) is also O(n), as we use a constant amount of memory space for each of the n+1 activation records in the trace at the time we make the final recursive call (with n = 0). return 15 + S[4] = 15 + 8 = 23 linear sum(S, 5) linear sum(S, 4) linear sum(S, 3) linear sum(S, 2) linear sum(S, 1) linear sum(S, 0) return 0 return 0 + S[0] = 0 + 4 = 4 return 4 + S[1] = 4 + 3 = 7 return 7 + S[2] = 7 + 6 = 13 return 13 + S[3] = 13 + 2 = 15 Figure 4.10: Recursion trace for an execution of linear sum(S, 5) with input pa- rameter S = [4, 3, 6, 2, 8].
- 193. 4.4. Further Examplesof Recursion 171 Reversing a Sequence with Recursion Next, let us consider the problem of reversing the n elements of a sequence, S, so that the first element becomes the last, the second element becomes second to the last, and so on. We can solve this problem using linear recursion, by observing that the reversal of a sequence can be achieved by swapping the first and last elements and then recursively reversing the remaining elements. We present an implemen- tation of this algorithm in Code Fragment 4.10, using the convention that the first time we call this algorithm we do so as reverse(S, 0, len(S)). 1 def reverse(S, start, stop): 2 ”””Reverse elements in implicit slice S[start:stop].””” 3 if start stop − 1: # if at least 2 elements: 4 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last 5 reverse(S, start+1, stop−1) # recur on rest Code Fragment 4.10: Reversing the elements of a sequence using linear recursion. Note that there are two implicit base case scenarios: When start == stop, the implicit range is empty, and when start == stop−1, the implicit range has only one element. In either of these cases, there is no need for action, as a sequence with zero elements or one element is trivially equal to its reversal. When otherwise invoking recursion, we are guaranteed to make progress towards a base case, as the difference, stop−start, decreases by two with each call (see Figure 4.11). If n is even, we will eventually reach the start == stop case, and if n is odd, we will eventually reach the start == stop − 1 case. The above argument implies that the recursive algorithm of Code Fragment 4.10 is guaranteed to terminate after a total of 1+ n 2 recursive calls. Since each call involves a constant amount of work, the entire process runs in O(n) time. 6 3 6 2 8 9 5 5 9 8 2 6 3 4 5 3 6 2 8 9 4 5 9 8 2 6 3 4 5 9 6 2 8 3 4 5 0 1 2 3 4 4 Figure 4.11: A trace of the recursion for reversing a sequence. The shaded portion has yet to be reversed.
- 194. 172 Chapter 4.Recursion Recursive Algorithms for Computing Powers As another interesting example of the use of linear recursion, we consider the prob- lem of raising a number x to an arbitrary nonnegative integer, n. That is, we wish to compute the power function, defined as power(x,n) = xn. (We use the name “power” for this discussion, to differentiate from the built-in function pow that pro- vides such functionality.) We will consider two different recursive formulations for the problem that lead to algorithms with very different performance. A trivial recursive definition follows from the fact that xn = x·xn−1 for n 0. power(x,n) = 1 if n = 0 x· power(x,n−1) otherwise. This definition leads to a recursive algorithm shown in Code Fragment 4.11. 1 def power(x, n): 2 ”””Compute the value x n for integer n.””” 3 if n == 0: 4 return 1 5 else: 6 return x power(x, n−1) Code Fragment 4.11: Computing the power function using trivial recursion. A recursive call to this version of power(x,n) runs in O(n) time. Its recursion trace has structure very similar to that of the factorial function from Figure 4.1, with the parameter decreasing by one with each call, and constant work performed at each of n+1 levels. However, there is a much faster way to compute the power function using an alternative definition that employs a squaring technique. Let k = n 2 denote the floor of the division (expressed as n // 2 in Python). We consider the expression xk 2 . When n is even, n 2 = n 2 and therefore xk 2 = x n 2 2 = xn. When n is odd, n 2 = n−1 2 and xk 2 = xn−1, and therefore xn = x· xk 2 , just as 213 = 2 · 26 · 26. This analysis leads to the following recursive definition: power(x,n) = ⎧ ⎪ ⎨ ⎪ ⎩ 1 if n = 0 x· power x, n 2 2 if n 0 is odd power x, n 2 2 if n 0 is even If we were to implement this recursion making two recursive calls to compute power(x, n 2 ) · power(x, n 2 ), a trace of the recursion would demonstrate O(n) calls. We can perform significantly fewer operations by computing power(x, n 2 ) as a partial result, and then multiplying it by itself. An implementation based on this recursive definition is given in Code Fragment 4.12.
- 195. 4.4. Further Examplesof Recursion 173 1 def power(x, n): 2 ”””Compute the value x n for integer n.””” 3 if n == 0: 4 return 1 5 else: 6 partial = power(x, n // 2) # rely on truncated division 7 result = partial partial 8 if n % 2 == 1: # if n odd, include extra factor of x 9 result = x 10 return result Code Fragment 4.12: Computing the power function using repeated squaring. To illustrate the execution of our improved algorithm, Figure 4.12 provides a recursion trace of the computation power(2, 13). return 64 64 2 = 8192 power(2, 13) power(2, 6) power(2, 3) power(2, 1) power(2, 0) return 1 return 1 1 2 = 2 return 2 2 2 = 8 return 8 8 = 64 Figure 4.12: Recursion trace for an execution of power(2, 13). To analyze the running time of the revised algorithm, we observe that the expo- nent in each recursive call of function power(x,n) is at most half of the preceding exponent. As we saw with the analysis of binary search, the number of times that we can divide n in half before getting to one or less is O(logn). Therefore, our new formulation of the power function results in O(logn) recursive calls. Each individ- ual activation of the function uses O(1) operations (excluding the recursive calls), and so the total number of operations for computing power(x,n) is O(logn). This is a significant improvement over the original O(n)-time algorithm. The improved version also provides significant saving in reducing the memory usage. The first version has a recursive depth of O(n), and therefore O(n) activation records are simultaneous stored in memory. Because the recursive depth of the improved version is O(logn), its memory usages is O(logn) as well.
- 196. 174 Chapter 4.Recursion 4.4.2 Binary Recursion When a function makes two recursive calls, we say that it uses binary recursion. We have already seen several examples of binary recursion, most notably when drawing the English ruler (Section 4.1.2), or in the bad fibonacci function of Sec- tion 4.3. As another application of binary recursion, let us revisit the problem of summing the n elements of a sequence, S, of numbers. Computing the sum of one or zero elements is trivial. With two or more elements, we can recursively com- pute the sum of the first half, and the sum of the second half, and add these sums together. Our implementation of such an algorithm, in Code Fragment 4.13, is initially invoked as binary sum(A, 0, len(A)). 1 def binary sum(S, start, stop): 2 ”””Return the sum of the numbers in implicit slice S[start:stop].””” 3 if start = stop: # zero elements in slice 4 return 0 5 elif start == stop−1: # one element in slice 6 return S[start] 7 else: # two or more elements in slice 8 mid = (start + stop) // 2 9 return binary sum(S, start, mid) + binary sum(S, mid, stop) Code Fragment 4.13: Summing the elements of a sequence using binary recursion. To analyze algorithm binary sum, we consider, for simplicity, the case where n is a power of two. Figure 4.13 shows the recursion trace of an execution of binary sum(0, 8). We label each box with the values of parameters start:stop for that call. The size of the range is divided in half at each recursive call, and so the depth of the recursion is 1 + log2 n. Therefore, binary sum uses O(logn) amount of additional space, which is a big improvement over the O(n) space used by the linear sum function of Code Fragment 4.9. However, the running time of binary sum is O(n), as there are 2n−1 function calls, each requiring constant time. 0:1 1:2 2:3 4:5 6:7 7:8 3:4 5:6 0:2 4:6 6:8 2:4 0:4 4:8 0:8 Figure 4.13: Recursion trace for the execution of binary sum(0, 8).
- 197. 4.4. Further Examplesof Recursion 175 4.4.3 Multiple Recursion Generalizing from binary recursion, we define multiple recursion as a process in which a function may make more than two recursive calls. Our recursion for an- alyzing the disk space usage of a file system (see Section 4.1.4) is an example of multiple recursion, because the number of recursive calls made during one invoca- tion was equal to the number of entries within a given directory of the file system. Another common application of multiple recursion is when we want to enumer- ate various configurations in order to solve a combinatorial puzzle. For example, the following are all instances of what are known as summation puzzles: pot + pan = bib dog + cat = pig boy+ girl = baby To solve such a puzzle, we need to assign a unique digit (that is, 0,1,...,9) to each letter in the equation, in order to make the equation true. Typically, we solve such a puzzle by using our human observations of the particular puzzle we are trying to solve to eliminate configurations (that is, possible partial assignments of digits to letters) until we can work though the feasible configurations left, testing for the correctness of each one. If the number of possible configurations is not too large, however, we can use a computer to simply enumerate all the possibilities and test each one, without employing any human observations. In addition, such an algorithm can use multiple recursion to work through the configurations in a systematic way. We show pseudo- code for such an algorithm in Code Fragment 4.14. To keep the description general enough to be used with other puzzles, the algorithm enumerates and tests all k- length sequences without repetitions of the elements of a given universe U. We build the sequences of k elements by the following steps: 1. Recursively generating the sequences of k−1 elements 2. Appending to each such sequence an element not already contained in it. Throughout the execution of the algorithm, we use a set U to keep track of the elements not contained in the current sequence, so that an element e has not been used yet if and only if e is in U. Another way to look at the algorithm of Code Fragment 4.14 is that it enumer- ates every possible size-k ordered subset of U, and tests each subset for being a possible solution to our puzzle. For summation puzzles, U = {0,1,2,3,4,5,6,7,8,9} and each position in the sequence corresponds to a given letter. For example, the first position could stand for b, the second for o, the third for y, and so on.
- 198. 176 Chapter 4.Recursion Algorithm PuzzleSolve(k,S,U): Input: An integer k, sequence S, and set U Output: An enumeration of all k-length extensions to S using elements in U without repetitions for each e in U do Add e to the end of S Remove e from U {e is now being used} if k == 1 then Test whether S is a configuration that solves the puzzle if S solves the puzzle then return “Solution found: ” S else PuzzleSolve(k−1,S,U) {a recursive call} Remove e from the end of S Add e back to U {e is now considered as unused} Code Fragment 4.14: Solving a combinatorial puzzle by enumerating and testing all possible configurations. In Figure 4.14, we show a recursion trace of a call to PuzzleSolve(3,S,U), where S is empty and U = {a,b,c}. During the execution, all the permutations of the three characters are generated and tested. Note that the initial call makes three recursive calls, each of which in turn makes two more. If we had executed PuzzleSolve(3,S,U) on a set U consisting of four elements, the initial call would have made four recursive calls, each of which would have a trace looking like the one in Figure 4.14. initial call PuzzleSolve(3, (), {a,b,c}) PuzzleSolve(2, b, {a,c}) PuzzleSolve(2, c, {a,b}) PuzzleSolve(1, ca, {b}) PuzzleSolve(2, a, {b,c}) PuzzleSolve(1, ab, {c}) PuzzleSolve(1, ba, {c}) PuzzleSolve(1, bc, {a}) PuzzleSolve(1, ac, {b}) PuzzleSolve(1, cb, {a}) acb abc bac cab bca cba Figure 4.14: Recursion trace for an execution of PuzzleSolve(3,S,U), where S is empty and U = {a,b,c}. This execution generates and tests all permutations of a, b, and c. We show the permutations generated directly below their respective boxes.
- 199. 4.5. Designing RecursiveAlgorithms 177 4.5 Designing Recursive Algorithms In general, an algorithm that uses recursion typically has the following form: • Test for base cases. We begin by testing for a set of base cases (there should be at least one). These base cases should be defined so that every possible chain of recursive calls will eventually reach a base case, and the handling of each base case should not use recursion. • Recur. If not a base case, we perform one or more recursive calls. This recur- sive step may involve a test that decides which of several possible recursive calls to make. We should define each possible recursive call so that it makes progress towards a base case. Parameterizing a Recursion To design a recursive algorithm for a given problem, it is useful to think of the different ways we might define subproblems that have the same general structure as the original problem. If one has difficulty finding the repetitive structure needed to design a recursive algorithm, it is sometimes useful to work out the problem on a few concrete examples to see how the subproblems should be defined. A successful recursive design sometimes requires that we redefine the original problem to facilitate similar-looking subproblems. Often, this involved reparam- eterizing the signature of the function. For example, when performing a binary search in a sequence, a natural function signature for a caller would appear as binary search(data, target). However, in Section 4.1.3, we defined our function with calling signature binary search(data, target, low, high), using the additional parameters to demarcate sublists as the recursion proceeds. This change in param- eterization is critical for binary search. If we had insisted on the cleaner signature, binary search(data, target), the only way to invoke a search on half the list would have been to make a new list instance with only those elements to send as the first parameter. However, making a copy of half the list would already take O(n) time, negating the whole benefit of the binary search algorithm. If we wished to provide a cleaner public interface to an algorithm like bi- nary search, without bothering a user with the extra parameters, a standard tech- nique is to make one function for public use with the cleaner interface, such as binary search(data, target), and then having its body invoke a nonpublic utility function having the desired recursive parameters. You will see that we similarly reparameterized the recursion in several other ex- amples of this chapter (e.g., reverse, linear sum, binary sum). We saw a different approach to redefining a recursion in our good fibonacci implementation, by in- tentionally strengthening the expectation of what is returned (in that case, returning a pair of numbers rather than a single number).
- 200. 178 Chapter 4.Recursion 4.6 Eliminating Tail Recursion The main benefit of a recursive approach to algorithm design is that it allows us to succinctly take advantage of a repetitive structure present in many problems. By making our algorithm description exploit the repetitive structure in a recursive way, we can often avoid complex case analyses and nested loops. This approach can lead to more readable algorithm descriptions, while still being quite efficient. However, the usefulness of recursion comes at a modest cost. In particular, the Python interpreter must maintain activation records that keep track of the state of each nested call. When computer memory is at a premium, it is useful in some cases to be able to derive nonrecursive algorithms from recursive ones. In general, we can use the stack data structure, which we will introduce in Section 6.1, to convert a recursive algorithm into a nonrecursive algorithm by man- aging the nesting of the recursive structure ourselves, rather than relying on the interpreter to do so. Although this only shifts the memory usage from the inter- preter to our stack, we may be able to reduce the memory usage by storing only the minimal information necessary. Even better, some forms of recursion can be eliminated without any use of axillary memory. A notable such form is known as tail recursion. A recursion is a tail recursion if any recursive call that is made from one context is the very last operation in that context, with the return value of the recursive call (if any) immediately returned by the enclosing recursion. By necessity, a tail recursion must be a linear recursion (since there is no way to make a second recursive call if you must immediately return the result of the first). Of the recursive functions demonstrated in this chapter, the binary search func- tion of Code Fragment 4.3 and the reverse function of Code Fragment 4.10 are examples of tail recursion. Several others of our linear recursions are almost like tail recursion, but not technically so. For example, our factorial function of Code Fragment 4.1 is not a tail recursion. It concludes with the command: return n factorial(n−1) This is not a tail recursion because an additional multiplication is performed after the recursive call is completed. For similar reasons, the linear sum function of Code Fragment 4.9 and the good fibonacci function of Code Fragment 4.7 fail to be tail recursions. Any tail recursion can be reimplemented nonrecursively by enclosing the body in a loop for repetition, and replacing a recursive call with new parameters by a reassignment of the existing parameters to those values. As a tangible example, our binary search function can be reimplemented as shown in Code Fragment 4.15. We initialize variables low and high, just prior to our while loop, to represent the full extent of the sequence. Then, during each pass of the loop, we either find
- 201. 4.6. Eliminating TailRecursion 179 1 def binary search iterative(data, target): 2 ”””Return True if target is found in the given Python list.””” 3 low = 0 4 high = len(data)−1 5 while low = high: 6 mid = (low + high) // 2 7 if target == data[mid]: # found a match 8 return True 9 elif target data[mid]: 10 high = mid − 1 # only consider values left of mid 11 else: 12 low = mid + 1 # only consider values right of mid 13 return False # loop ended without success Code Fragment 4.15: A nonrecursive implementation of binary search. the target, or we narrow the range of the candidate subsequence. Where we made the recursive call binary search(data, target, low, mid −1) in the original version, we simply replace high = mid − 1 in our new version and then continue to the next iteration of the loop. Our original base case condition of low high has simply been replaced by the opposite loop condition while low = high. In our new implementation, we return False to designate a failed search if the while loop ends (that is, without having ever returned True from within). We can similarly develop a nonrecursive implementation (Code Fragment 4.16) of the original recursive reverse method of Code Fragment 4.10. 1 def reverse iterative(S): 2 ”””Reverse elements in sequence S.””” 3 start, stop = 0, len(S) 4 while start stop − 1: 5 S[start], S[stop−1] = S[stop−1], S[start] # swap first and last 6 start, stop = start + 1, stop − 1 # narrow the range Code Fragment 4.16: Reversing the elements of a sequence using iteration. In this new version, we update the values start and stop during each pass of the loop, exiting once we reach the case of having one or less elements in that range. Many other linear recursions can be expressed quite efficiently with iteration, even if they were not formally tail recursions. For example, there are trivial non- recursive implementations for computing factorials, summing elements of a se- quence, or computing Fibonacci numbers efficiently. In fact, our implementation of a Fibonacci generator, from Section 1.8, produces each subsequent value in O(1) time, and thus takes O(n) time to generate the nth entry in the series.
- 202. 180 Chapter 4.Recursion 4.7 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-4.1 Describe a recursive algorithm for finding the maximum element in a se- quence, S, of n elements. What is your running time and space usage? R-4.2 Draw the recursion trace for the computation of power(2,5), using the traditional function implemented in Code Fragment 4.11. R-4.3 Draw the recursion trace for the computation of power(2,18), using the repeated squaring algorithm, as implemented in Code Fragment 4.12. R-4.4 Draw the recursion trace for the execution of function reverse(S, 0, 5) (Code Fragment 4.10) on S = [4, 3, 6, 2, 6]. R-4.5 Draw the recursion trace for the execution of function PuzzleSolve(3,S,U) (Code Fragment 4.14), where S is empty and U = {a,b,c,d}. R-4.6 Describe a recursive function for computing the nth Harmonic number, Hn = ∑n i=1 1/i. R-4.7 Describe a recursive function for converting a string of digits into the in- teger it represents. For example, 13531 represents the integer 13,531. R-4.8 Isabel has an interesting way of summing up the values in a sequence A of n integers, where n is a power of two. She creates a new sequence B of half the size of A and sets B[i] = A[2i]+A[2i+1], for i = 0,1,...,(n/2)−1. If B has size 1, then she outputs B[0]. Otherwise, she replaces A with B, and repeats the process. What is the running time of her algorithm? Creativity C-4.9 Write a short recursive Python function that finds the minimum and max- imum values in a sequence without using any loops. C-4.10 Describe a recursive algorithm to compute the integer part of the base-two logarithm of n using only addition and integer division. C-4.11 Describe an efficient recursive function for solving the element unique- ness problem, which runs in time that is at most O(n2) in the worst case without using sorting. C-4.12 Give a recursive algorithm to compute the product of two positive integers, m and n, using only addition and subtraction.
- 203. 4.7. Exercises 181 C-4.13In Section 4.2 we prove by induction that the number of lines printed by a call to draw interval(c) is 2c − 1. Another interesting question is how many dashes are printed during that process. Prove by induction that the number of dashes printed by draw interval(c) is 2c+1 −c−2. C-4.14 In the Towers of Hanoi puzzle, we are given a platform with three pegs, a, b, and c, sticking out of it. On peg a is a stack of n disks, each larger than the next, so that the smallest is on the top and the largest is on the bottom. The puzzle is to move all the disks from peg a to peg c, moving one disk at a time, so that we never place a larger disk on top of a smaller one. See Figure 4.15 for an example of the case n = 4. Describe a recursive algorithm for solving the Towers of Hanoi puzzle for arbitrary n. (Hint: Consider first the subproblem of moving all but the nth disk from peg a to another peg using the third as “temporary storage.”) Figure 4.15: An illustration of the Towers of Hanoi puzzle. C-4.15 Write a recursive function that will output all the subsets of a set of n elements (without repeating any subsets). C-4.16 Write a short recursive Python function that takes a character string s and outputs its reverse. For example, the reverse of potspans would be snapstop . C-4.17 Write a short recursive Python function that determines if a string s is a palindrome, that is, it is equal to its reverse. For example, racecar and gohangasalamiimalasagnahog are palindromes. C-4.18 Use recursion to write a Python function for determining if a string s has more vowels than consonants. C-4.19 Write a short recursive Python function that rearranges a sequence of in- teger values so that all the even values appear before all the odd values. C-4.20 Given an unsorted sequence, S, of integers and an integer k, describe a recursive algorithm for rearranging the elements in S so that all elements less than or equal to k come before any elements larger than k. What is the running time of your algorithm on a sequence of n values?
- 204. 182 Chapter 4.Recursion C-4.21 Suppose you are given an n-element sequence, S, containing distinct in- tegers that are listed in increasing order. Given a number k, describe a recursive algorithm to find two integers in S that sum to k, if such a pair exists. What is the running time of your algorithm? C-4.22 Develop a nonrecursive implementation of the version of power from Code Fragment 4.12 that uses repeated squaring. Projects P-4.23 Implement a recursive function with signature find(path, filename) that reports all entries of the file system rooted at the given path having the given file name. P-4.24 Write a program for solving summation puzzles by enumerating and test- ing all possible configurations. Using your program, solve the three puz- zles given in Section 4.4.3. P-4.25 Provide a nonrecursive implementation of the draw interval function for the English ruler project of Section 4.1.2. There should be precisely 2c −1 lines of output if c represents the length of the center tick. If incrementing a counter from 0 to 2c −2, the number of dashes for each tick line should be exactly one more than the number of consecutive 1’s at the end of the binary representation of the counter. P-4.26 Write a program that can solve instances of the Tower of Hanoi problem (from Exercise C-4.14). P-4.27 Python’s os module provides a function with signature walk(path) that is a generator yielding the tuple (dirpath, dirnames, filenames) for each subdirectory of the directory identified by string path, such that string dirpath is the full path to the subdirectory, dirnames is a list of the names of the subdirectories within dirpath, and filenames is a list of the names of non-directory entries of dirpath. For example, when visiting the cs016 subdirectory of the file system shown in Figure 4.6, the walk would yield ( /user/rt/courses/cs016 , [ homeworks , programs ], [ grades ]). Give your own implementation of such a walk function. Chapter Notes The use of recursion in programs belongs to the folkore of computer science (for example, see the article of Dijkstra [36]). It is also at the heart of functional programming languages (for example, see the classic book by Abelson, Sussman, and Sussman [1]). Interestingly, binary search was first published in 1946, but was not published in a fully correct form until 1962. For further discussions on lessons learned, please see papers by Bentley [14] and Lesuisse [68].
- 205. Chapter 5 Array-Based Sequences Contents 5.1Python’s Sequence Types . . . . . . . . . . . . . . . . . . . 184 5.2 Low-Level Arrays . . . . . . . . . . . . . . . . . . . . . . . . 185 5.2.1 Referential Arrays . . . . . . . . . . . . . . . . . . . . . . 187 5.2.2 Compact Arrays in Python . . . . . . . . . . . . . . . . . 190 5.3 Dynamic Arrays and Amortization . . . . . . . . . . . . . . 192 5.3.1 Implementing a Dynamic Array . . . . . . . . . . . . . . . 195 5.3.2 Amortized Analysis of Dynamic Arrays . . . . . . . . . . . 197 5.3.3 Python’s List Class . . . . . . . . . . . . . . . . . . . . . 201 5.4 Efficiency of Python’s Sequence Types . . . . . . . . . . . 202 5.4.1 Python’s List and Tuple Classes . . . . . . . . . . . . . . 202 5.4.2 Python’s String Class . . . . . . . . . . . . . . . . . . . . 208 5.5 Using Array-Based Sequences . . . . . . . . . . . . . . . . 210 5.5.1 Storing High Scores for a Game . . . . . . . . . . . . . . 210 5.5.2 Sorting a Sequence . . . . . . . . . . . . . . . . . . . . . 214 5.5.3 Simple Cryptography . . . . . . . . . . . . . . . . . . . . 216 5.6 Multidimensional Data Sets . . . . . . . . . . . . . . . . . . 219 5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224
- 206. 184 Chapter 5.Array-Based Sequences 5.1 Python’s Sequence Types In this chapter, we explore Python’s various “sequence” classes, namely the built- in list, tuple, and str classes. There is significant commonality between these classes, most notably: each supports indexing to access an individual element of a sequence, using a syntax such as seq[k], and each uses a low-level concept known as an array to represent the sequence. However, there are significant differences in the abstractions that these classes represent, and in the way that instances of these classes are represented internally by Python. Because these classes are used so widely in Python programs, and because they will become building blocks upon which we will develop more complex data structures, it is imperative that we estab- lish a clear understanding of both the public behavior and inner workings of these classes. Public Behaviors A proper understanding of the outward semantics for a class is a necessity for a good programmer. While the basic usage of lists, strings, and tuples may seem straightforward, there are several important subtleties regarding the behaviors as- sociated with these classes (such as what it means to make a copy of a sequence, or to take a slice of a sequence). Having a misunderstanding of a behavior can easily lead to inadvertent bugs in a program. Therefore, we establish an accurate men- tal model for each of these classes. These images will help when exploring more advanced usage, such as representing a multidimensional data set as a list of lists. Implementation Details A focus on the internal implementations of these classes seems to go against our stated principles of object-oriented programming. In Section 2.1.2, we emphasized the principle of encapsulation, noting that the user of a class need not know about the internal details of the implementation. While it is true that one only needs to understand the syntax and semantics of a class’s public interface in order to be able to write legal and correct code that uses instances of the class, the efficiency of a program depends greatly on the efficiency of the components upon which it relies. Asymptotic and Experimental Analyses In describing the efficiency of various operations for Python’s sequence classes, we will rely on the formal asymptotic analysis notations established in Chapter 3. We will also perform experimental analyses of the primary operations to provide empirical evidence that is consistent with the more theoretical asymptotic analyses.
- 207. 5.2. Low-Level Arrays185 5.2 Low-Level Arrays To accurately describe the way in which Python represents the sequence types, we must first discuss aspects of the low-level computer architecture. The primary memory of a computer is composed of bits of information, and those bits are typ- ically grouped into larger units that depend upon the precise system architecture. Such a typical unit is a byte, which is equivalent to 8 bits. A computer system will have a huge number of bytes of memory, and to keep track of what information is stored in what byte, the computer uses an abstraction known as a memory address. In effect, each byte of memory is associated with a unique number that serves as its address (more formally, the binary representation of the number serves as the address). In this way, the computer system can refer to the data in “byte #2150” versus the data in “byte #2157,” for example. Memory addresses are typically coordinated with the physical layout of the memory system, and so we often portray the numbers in sequential fashion. Figure 5.1 provides such a diagram, with the designated memory address for each byte. 2 1 6 0 2 1 4 5 2 1 4 6 2 1 4 7 2 1 4 8 2 1 4 9 2 1 5 0 2 1 5 1 2 1 5 2 2 1 5 3 2 1 5 4 2 1 5 5 2 1 5 6 2 1 5 7 2 1 5 8 2 1 4 4 2 1 5 9 Figure 5.1: A representation of a portion of a computer’s memory, with individual bytes labeled with consecutive memory addresses. Despite the sequential nature of the numbering system, computer hardware is designed, in theory, so that any byte of the main memory can be efficiently accessed based upon its memory address. In this sense, we say that a computer’s main mem- ory performs as random access memory (RAM). That is, it is just as easy to retrieve byte #8675309 as it is to retrieve byte #309. (In practice, there are complicating factors including the use of caches and external memory; we address some of those issues in Chapter 15.) Using the notation for asymptotic analysis, we say that any individual byte of memory can be stored or retrieved in O(1) time. In general, a programming language keeps track of the association between an identifier and the memory address in which the associated value is stored. For example, identifier x might be associated with one value stored in memory, while y is associated with another value stored in memory. A common programming task is to keep track of a sequence of related objects. For example, we may want a video game to keep track of the top ten scores for that game. Rather than use ten different variables for this task, we would prefer to use a single name for the group and use index numbers to refer to the high scores in that group.
- 208. 186 Chapter 5.Array-Based Sequences A group of related variables can be stored one after another in a contiguous portion of the computer’s memory. We will denote such a representation as an array. As a tangible example, a text string is stored as an ordered sequence of individual characters. In Python, each character is represented using the Unicode character set, and on most computing systems, Python internally represents each Unicode character with 16 bits (i.e., 2 bytes). Therefore, a six-character string, such as SAMPLE , would be stored in 12 consecutive bytes of memory, as diagrammed in Figure 5.2. 0 M P L E A S 2 1 6 0 2 1 5 0 2 1 5 1 2 1 5 2 2 1 5 3 2 1 5 4 2 1 5 5 2 1 5 6 2 1 5 7 2 1 5 8 2 1 4 4 2 1 5 9 5 4 3 2 1 2 1 4 5 2 1 4 6 2 1 4 7 2 1 4 8 2 1 4 9 Figure 5.2: A Python string embedded as an array of characters in the computer’s memory. We assume that each Unicode character of the string requires two bytes of memory. The numbers below the entries are indices into the string. We describe this as an array of six characters, even though it requires 12 bytes of memory. We will refer to each location within an array as a cell, and will use an integer index to describe its location within the array, with cells numbered starting with 0, 1, 2, and so on. For example, in Figure 5.2, the cell of the array with index 4 has contents L and is stored in bytes 2154 and 2155 of memory. Each cell of an array must use the same number of bytes. This requirement is what allows an arbitrary cell of the array to be accessed in constant time based on its index. In particular, if one knows the memory address at which an array starts (e.g., 2146 in Figure 5.2), the number of bytes per element (e.g., 2 for a Unicode character), and a desired index within the array, the appropriate memory address can be computed using the calculation, start + cellsize index. By this formula, the cell at index 0 begins precisely at the start of the array, the cell at index 1 begins precisely cellsize bytes beyond the start of the array, and so on. As an example, cell 4 of Figure 5.2 begins at memory location 2146+2·4 = 2146+8 = 2154. Of course, the arithmetic for calculating memory addresses within an array can be handled automatically. Therefore, a programmer can envision a more typical high-level abstraction of an array of characters as diagrammed in Figure 5.3. 0 A S M P L E 3 4 5 1 2 Figure 5.3: A higher-level abstraction for the string portrayed in Figure 5.2.
- 209. 5.2. Low-Level Arrays187 5.2.1 Referential Arrays As another motivating example, assume that we want a medical information system to keep track of the patients currently assigned to beds in a certain hospital. If we assume that the hospital has 200 beds, and conveniently that those beds are num- bered from 0 to 199, we might consider using an array-based structure to maintain the names of the patients currently assigned to those beds. For example, in Python we might use a list of names, such as: [ Rene , Joseph , Janet , Jonas , Helen , Virginia , ... ] To represent such a list with an array, Python must adhere to the requirement that each cell of the array use the same number of bytes. Yet the elements are strings, and strings naturally have different lengths. Python could attempt to reserve enough space for each cell to hold the maximum length string (not just of currently stored strings, but of any string we might ever want to store), but that would be wasteful. Instead, Python represents a list or tuple instance using an internal storage mechanism of an array of object references. At the lowest level, what is stored is a consecutive sequence of memory addresses at which the elements of the se- quence reside. A high-level diagram of such a list is shown in Figure 5.4. 0 3 1 2 5 4 Rene Virginia Joseph Helen Jonas Janet Figure 5.4: An array storing references to strings. Although the relative size of the individual elements may vary, the number of bits used to store the memory address of each element is fixed (e.g., 64-bits per address). In this way, Python can support constant-time access to a list or tuple element based on its index. In Figure 5.4, we characterize a list of strings that are the names of the patients in a hospital. It is more likely that a medical information system would manage more comprehensive information on each patient, perhaps represented as an in- stance of a Patient class. From the perspective of the list implementation, the same principle applies: The list will simply keep a sequence of references to those ob- jects. Note as well that a reference to the None object can be used as an element of the list to represent an empty bed in the hospital.
- 210. 188 Chapter 5.Array-Based Sequences The fact that lists and tuples are referential structures is significant to the se- mantics of these classes. A single list instance may include multiple references to the same object as elements of the list, and it is possible for a single object to be an element of two or more lists, as those lists simply store references back to that object. As an example, when you compute a slice of a list, the result is a new list instance, but that new list has references to the same elements that are in the original list, as portrayed in Figure 5.5. 3 4 5 6 7 0 1 2 0 1 2 primes: temp: 3 11 5 2 19 17 13 7 Figure 5.5: The result of the command temp = primes[3:6]. When the elements of the list are immutable objects, as with the integer in- stances in Figure 5.5, the fact that the two lists share elements is not that signifi- cant, as neither of the lists can cause a change to the shared object. If, for example, the command temp[2] = 15 were executed from this configuration, that does not change the existing integer object; it changes the reference in cell 2 of the temp list to reference a different object. The resulting configuration is shown in Figure 5.6. 3 4 5 6 7 0 1 2 0 1 2 primes: temp: 13 11 5 2 15 3 19 17 7 Figure 5.6: The result of the command temp[2] = 15 upon the configuration por- trayed in Figure 5.5. The same semantics is demonstrated when making a new list as a copy of an existing one, with a syntax such as backup = list(primes). This produces a new list that is a shallow copy (see Section 2.6), in that it references the same elements as in the first list. With immutable elements, this point is moot. If the contents of the list were of a mutable type, a deep copy, meaning a new list with new elements, can be produced by using the deepcopy function from the copy module.
- 211. 5.2. Low-Level Arrays189 As a more striking example, it is a common practice in Python to initialize an array of integers using a syntax such as counters = [0] 8. This syntax produces a list of length eight, with all eight elements being the value zero. Technically, all eight cells of the list reference the same object, as portrayed in Figure 5.7. 4 5 6 7 0 1 2 3 counters: 0 Figure 5.7: The result of the command data = [0] 8. At first glance, the extreme level of aliasing in this configuration may seem alarming. However, we rely on the fact that the referenced integer is immutable. Even a command such as counters[2] += 1 does not technically change the value of the existing integer instance. This computes a new integer, with value 0+1, and sets cell 2 to reference the newly computed value. The resulting configuration is shown in Figure 5.8. 4 5 6 7 0 1 2 3 counters: 0 1 Figure 5.8: The result of command data[2] += 1 upon the list from Figure 5.7. As a final manifestation of the referential nature of lists, we note that the extend command is used to add all elements from one list to the end of another list. The extended list does not receive copies of those elements, it receives references to those elements. Figure 5.9 portrays the effect of a call to extend. 3 4 5 6 7 8 10 9 2 1 0 0 1 2 primes: extras: 29 31 7 19 3 17 13 11 2 5 23 Figure 5.9: The effect of command primes.extend(extras), shown in light gray.
- 212. 190 Chapter 5.Array-Based Sequences 5.2.2 Compact Arrays in Python In the introduction to this section, we emphasized that strings are represented using an array of characters (not an array of references). We will refer to this more direct representation as a compact array because the array is storing the bits that represent the primary data (characters, in the case of strings). 0 A S M P L E 3 4 5 1 2 Compact arrays have several advantages over referential structures in terms of computing performance. Most significantly, the overall memory usage will be much lower for a compact structure because there is no overhead devoted to the explicit storage of the sequence of memory references (in addition to the primary data). That is, a referential structure will typically use 64-bits for the memory address stored in the array, on top of whatever number of bits are used to represent the object that is considered the element. Also, each Unicode character stored in a compact array within a string typically requires 2 bytes. If each character were stored independently as a one-character string, there would be significantly more bytes used. As another case study, suppose we wish to store a sequence of one million, 64-bit integers. In theory, we might hope to use only 64 million bits. However, we estimate that a Python list will use four to five times as much memory. Each element of the list will result in a 64-bit memory address being stored in the primary array, and an int instance being stored elsewhere in memory. Python allows you to query the actual number of bytes being used for the primary storage of any object. This is done using the getsizeof function of the sys module. On our system, the size of a typical int object requires 14 bytes of memory (well beyond the 4 bytes needed for representing the actual 64-bit number). In all, the list will be using 18 bytes per entry, rather than the 4 bytes that a compact list of integers would require. Another important advantage to a compact structure for high-performance com- puting is that the primary data are stored consecutively in memory. Note well that this is not the case for a referential structure. That is, even though a list maintains careful ordering of the sequence of memory addresses, where those elements reside in memory is not determined by the list. Because of the workings of the cache and memory hierarchies of computers, it is often advantageous to have data stored in memory near other data that might be used in the same computations. Despite the apparent inefficiencies of referential structures, we will generally be content with the convenience of Python’s lists and tuples in this book. The only place in which we consider alternatives will be in Chapter 15, which focuses on the impact of memory usage on data structures and algorithms. Python provides several means for creating compact arrays of various types.
- 213. 5.2. Low-Level Arrays191 Primary support for compact arrays is in a module named array. That module defines a class, also named array, providing compact storage for arrays of primitive data types. A portrayal of such an array of integers is shown in Figure 5.10. 3 4 5 6 7 0 1 2 17 5 2 3 7 11 13 19 Figure 5.10: Integers stored compactly as elements of a Python array. The public interface for the array class conforms mostly to that of a Python list. However, the constructor for the array class requires a type code as a first parameter, which is a character that designates the type of data that will be stored in the array. As a tangible example, the type code, i , designates an array of (signed) integers, typically represented using at least 16-bits each. We can declare the array shown in Figure 5.10 as, primes = array( i , [2, 3, 5, 7, 11, 13, 17, 19]) The type code allows the interpreter to determine precisely how many bits are needed per element of the array. The type codes supported by the array module, as shown in Table 5.1, are formally based upon the native data types used by the C programming language (the language in which the the most widely used distri- bution of Python is implemented). The precise number of bits for the C data types is system-dependent, but typical ranges are shown in the table. Code C Data Type Typical Number of Bytes b signed char 1 B unsigned char 1 u Unicode char 2 or 4 h signed short int 2 H unsigned short int 2 i signed int 2 or 4 I unsigned int 2 or 4 l signed long int 4 L unsigned long int 4 f float 4 d float 8 Table 5.1: Type codes supported by the array module. The array module does not provide support for making compact arrays of user- defined data types. Compact arrays of such structures can be created with the lower- level support of a module named ctypes. (See Section 5.3.1 for more discussion of the ctypes module.)
- 214. 192 Chapter 5.Array-Based Sequences 5.3 Dynamic Arrays and Amortization When creating a low-level array in a computer system, the precise size of that array must be explicitly declared in order for the system to properly allocate a consecutive piece of memory for its storage. For example, Figure 5.11 displays an array of 12 bytes that might be stored in memory locations 2146 through 2157. 2 1 6 0 2 1 4 5 2 1 4 6 2 1 4 7 2 1 4 8 2 1 4 9 2 1 5 0 2 1 5 1 2 1 5 2 2 1 5 3 2 1 5 4 2 1 5 5 2 1 5 6 2 1 5 7 2 1 5 8 2 1 4 4 2 1 5 9 Figure 5.11: An array of 12 bytes allocated in memory locations 2146 through 2157. Because the system might dedicate neighboring memory locations to store other data, the capacity of an array cannot trivially be increased by expanding into sub- sequent cells. In the context of representing a Python tuple or str instance, this constraint is no problem. Instances of those classes are immutable, so the correct size for an underlying array can be fixed when the object is instantiated. Python’s list class presents a more interesting abstraction. Although a list has a particular length when constructed, the class allows us to add elements to the list, with no apparent limit on the overall capacity of the list. To provide this abstraction, Python relies on an algorithmic sleight of hand known as a dynamic array. The first key to providing the semantics of a dynamic array is that a list instance maintains an underlying array that often has greater capacity than the current length of the list. For example, while a user may have created a list with five elements, the system may have reserved an underlying array capable of storing eight object references (rather than only five). This extra capacity makes it easy to append a new element to the list by using the next available cell of the array. If a user continues to append elements to a list, any reserved capacity will eventually be exhausted. In that case, the class requests a new, larger array from the system, and initializes the new array so that its prefix matches that of the existing smaller array. At that point in time, the old array is no longer needed, so it is reclaimed by the system. Intuitively, this strategy is much like that of the hermit crab, which moves into a larger shell when it outgrows its previous one. We give empirical evidence that Python’s list class is based upon such a strat- egy. The source code for our experiment is displayed in Code Fragment 5.1, and a sample output of that program is given in Code Fragment 5.2. We rely on a func- tion named getsizeof that is available from the sys module. This function reports the number of bytes that are being used to store an object in Python. For a list, it reports the number of bytes devoted to the array and other instance variables of the list, but not any space devoted to elements referenced by the list.
- 215. 5.3. Dynamic Arraysand Amortization 193 1 import sys # provides getsizeof function 2 data = [ ] 3 for k in range(n): # NOTE: must fix choice of n 4 a = len(data) # number of elements 5 b = sys.getsizeof(data) # actual size in bytes 6 print( Length: {0:3d}; Size in bytes: {1:4d} .format(a, b)) 7 data.append(None) # increase length by one Code Fragment 5.1: An experiment to explore the relationship between a list’s length and its underlying size in Python. Length: 0; Size in bytes: 72 Length: 1; Size in bytes: 104 Length: 2; Size in bytes: 104 Length: 3; Size in bytes: 104 Length: 4; Size in bytes: 104 Length: 5; Size in bytes: 136 Length: 6; Size in bytes: 136 Length: 7; Size in bytes: 136 Length: 8; Size in bytes: 136 Length: 9; Size in bytes: 200 Length: 10; Size in bytes: 200 Length: 11; Size in bytes: 200 Length: 12; Size in bytes: 200 Length: 13; Size in bytes: 200 Length: 14; Size in bytes: 200 Length: 15; Size in bytes: 200 Length: 16; Size in bytes: 200 Length: 17; Size in bytes: 272 Length: 18; Size in bytes: 272 Length: 19; Size in bytes: 272 Length: 20; Size in bytes: 272 Length: 21; Size in bytes: 272 Length: 22; Size in bytes: 272 Length: 23; Size in bytes: 272 Length: 24; Size in bytes: 272 Length: 25; Size in bytes: 272 Length: 26; Size in bytes: 352 Code Fragment 5.2: Sample output from the experiment of Code Fragment 5.1.
- 216. 194 Chapter 5.Array-Based Sequences In evaluating the results of the experiment, we draw attention to the first line of output from Code Fragment 5.2. We see that an empty list instance already requires a certain number of bytes of memory (72 on our system). In fact, each object in Python maintains some state, for example, a reference to denote the class to which it belongs. Although we cannot directly access private instance variables for a list, we can speculate that in some form it maintains state information akin to: n The number of actual elements currently stored in the list. capacity The maximum number of elements that could be stored in the currently allocated array. A The reference to the currently allocated array (initially None). As soon as the first element is inserted into the list, we detect a change in the underlying size of the structure. In particular, we see the number of bytes jump from 72 to 104, an increase of exactly 32 bytes. Our experiment was run on a 64-bit machine architecture, meaning that each memory address is a 64-bit number (i.e., 8 bytes). We speculate that the increase of 32 bytes reflects the allocation of an underlying array capable of storing four object references. This hypothesis is consistent with the fact that we do not see any underlying change in the memory usage after inserting the second, third, or fourth element into the list. After the fifth element has been added to the list, we see the memory usage jump from 104 bytes to 136 bytes. If we assume the original base usage of 72 bytes for the list, the total of 136 suggests an additional 64= 8×8 bytes that provide capacity for up to eight object references. Again, this is consistent with the experiment, as the memory usage does not increase again until the ninth insertion. At that point, the 200 bytes can be viewed as the original 72 plus an additional 128-byte array to store 16 object references. The 17th insertion pushes the overall memory usage to 272 = 72+200 = 72+25×8, hence enough to store up to 25 element references. Because a list is a referential structure, the result of getsizeof for a list instance only includes the size for representing its primary structure; it does not account for memory used by the objects that are elements of the list. In our experiment, we repeatedly append None to the list, because we do not care about the contents, but we could append any type of object without affecting the number of bytes reported by getsizeof(data). If we were to continue such an experiment for further iterations, we might try to discern the pattern for how large of an array Python creates each time the ca- pacity of the previous array is exhausted (see Exercises R-5.2 and C-5.13). Before exploring the precise sequence of capacities used by Python, we continue in this section by describing a general approach for implementing dynamic arrays and for performing an asymptotic analysis of their performance.
- 217. 5.3. Dynamic Arraysand Amortization 195 5.3.1 Implementing a Dynamic Array Although the Python list class provides a highly optimized implementation of dy- namic arrays, upon which we rely for the remainder of this book, it is instructive to see how such a class might be implemented. The key is to provide means to grow the array A that stores the elements of a list. Of course, we cannot actually grow that array, as its capacity is fixed. If an element is appended to a list at a time when the underlying array is full, we perform the following steps: 1. Allocate a new array B with larger capacity. 2. Set B[i] = A[i], for i = 0,...,n−1, where n denotes current number of items. 3. Set A = B, that is, we henceforth use B as the array supporting the list. 4. Insert the new element in the new array. An illustration of this process is shown in Figure 5.12. B A A B A (a) (b) (c) Figure 5.12: An illustration of the three steps for “growing” a dynamic array: (a) create new array B; (b) store elements of A in B; (c) reassign reference A to the new array. Not shown is the future garbage collection of the old array, or the insertion of the new element. The remaining issue to consider is how large of a new array to create. A com- monly used rule is for the new array to have twice the capacity of the existing array that has been filled. In Section 5.3.2, we will provide a mathematical analysis to justify such a choice. In Code Fragment 5.3, we offer a concrete implementation of dynamic arrays in Python. Our DynamicArray class is designed using ideas described in this sec- tion. While consistent with the interface of a Python list class, we provide only limited functionality in the form of an append method, and accessors len and getitem . Support for creating low-level arrays is provided by a module named ctypes. Because we will not typically use such a low-level structure in the remain- der of this book, we omit a detailed explanation of the ctypes module. Instead, we wrap the necessary command for declaring the raw array within a private util- ity method make array. The hallmark expansion procedure is performed in our nonpublic resize method.
- 218. 196 Chapter 5.Array-Based Sequences 1 import ctypes # provides low-level arrays 2 3 class DynamicArray: 4 ”””A dynamic array class akin to a simplified Python list.””” 5 6 def init (self): 7 ”””Create an empty array.””” 8 self. n = 0 # count actual elements 9 self. capacity = 1 # default array capacity 10 self. A = self. make array(self. capacity) # low-level array 11 12 def len (self): 13 ”””Return number of elements stored in the array.””” 14 return self. n 15 16 def getitem (self, k): 17 ”””Return element at index k.””” 18 if not 0 = k self. n: 19 raise IndexError( invalid index ) 20 return self. A[k] # retrieve from array 21 22 def append(self, obj): 23 ”””Add object to end of the array.””” 24 if self. n == self. capacity: # not enough room 25 self. resize(2 self. capacity) # so double capacity 26 self. A[self. n] = obj 27 self. n += 1 28 29 def resize(self, c): # nonpublic utitity 30 ”””Resize internal array to capacity c.””” 31 B = self. make array(c) # new (bigger) array 32 for k in range(self. n): # for each existing value 33 B[k] = self. A[k] 34 self. A = B # use the bigger array 35 self. capacity = c 36 37 def make array(self, c): # nonpublic utitity 38 ”””Return new array with capacity c.””” 39 return (c ctypes.py object)( ) # see ctypes documentation Code Fragment 5.3: An implementation of a DynamicArray class, using a raw array from the ctypes module as storage.
- 219. 5.3. Dynamic Arraysand Amortization 197 5.3.2 Amortized Analysis of Dynamic Arrays In this section, we perform a detailed analysis of the running time of operations on dynamic arrays. We use the big-Omega notation introduced in Section 3.3.1 to give an asymptotic lower bound on the running time of an algorithm or step within it. The strategy of replacing an array with a new, larger array might at first seem slow, because a single append operation may require Ω(n) time to perform, where n is the current number of elements in the array. However, notice that by doubling the capacity during an array replacement, our new array allows us to add n new elements before the array must be replaced again. In this way, there are many simple append operations for each expensive one (see Figure 5.13). This fact allows us to show that performing a series of operations on an initially empty dynamic array is efficient in terms of its total running time. Using an algorithmic design pattern called amortization, we can show that per- forming a sequence of such append operations on a dynamic array is actually quite efficient. To perform an amortized analysis, we use an accounting technique where we view the computer as a coin-operated appliance that requires the payment of one cyber-dollar for a constant amount of computing time. When an operation is executed, we should have enough cyber-dollars available in our current “bank account” to pay for that operation’s running time. Thus, the total amount of cyber- dollars spent for any computation will be proportional to the total time spent on that computation. The beauty of using this analysis method is that we can overcharge some operations in order to save up cyber-dollars to pay for others. primitive operations for an append current number of elements 13 10 12 5 6 7 8 11 14 15 16 1 2 3 4 9 Figure 5.13: Running times of a series of append operations on a dynamic array.
- 220. 198 Chapter 5.Array-Based Sequences Proposition 5.1: Let S be a sequence implemented by means of a dynamic array with initial capacity one, using the strategy of doubling the array size when full. The total time to perform a series of n append operations in S, starting from S being empty, is O(n). Justification: Let us assume that one cyber-dollar is enough to pay for the execu- tion of each append operation in S, excluding the time spent for growing the array. Also, let us assume that growing the array from size k to size 2k requires k cyber- dollars for the time spent initializing the new array. We shall charge each append operation three cyber-dollars. Thus, we overcharge each append operation that does not cause an overflow by two cyber-dollars. Think of the two cyber-dollars profited in an insertion that does not grow the array as being “stored” with the cell in which the element was inserted. An overflow occurs when the array S has 2i elements, for some integer i ≥ 0, and the size of the array used by the array representing S is 2i. Thus, doubling the size of the array will require 2i cyber-dollars. Fortunately, these cyber-dollars can be found stored in cells 2i−1 through 2i − 1. (See Figure 5.14.) Note that the previous overflow occurred when the number of elements became larger than 2i−1 for the first time, and thus the cyber-dollars stored in cells 2i−1 through 2i − 1 have not yet been spent. Therefore, we have a valid amortization scheme in which each operation is charged three cyber-dollars and all the comput- ing time is paid for. That is, we can pay for the execution of n append operations using 3n cyber-dollars. In other words, the amortized running time of each append operation is O(1); hence, the total running time of n append operations is O(n). (a) 0 2 4 5 6 7 3 1 $ $ $ $ $ $ $ $ (b) 0 2 4 5 6 7 8 9 11 3 10 12 13 14 15 1 $ $ Figure 5.14: Illustration of a series of append operations on a dynamic array: (a) an 8-cell array is full, with two cyber-dollars “stored” at cells 4 through 7; (b) an append operation causes an overflow and a doubling of capacity. Copying the eight old elements to the new array is paid for by the cyber-dollars already stored in the table. Inserting the new element is paid for by one of the cyber-dollars charged to the current append operation, and the two cyber-dollars profited are stored at cell 8.
- 221. 5.3. Dynamic Arraysand Amortization 199 Geometric Increase in Capacity Although the proof of Proposition 5.1 relies on the array being doubled each time we expand, the O(1) amortized bound per operation can be proven for any geo- metrically increasing progression of array sizes (see Section 2.4.2 for discussion of geometric progressions). When choosing the geometric base, there exists a trade- off between run-time efficiency and memory usage. With a base of 2 (i.e., doubling the array), if the last insertion causes a resize event, the array essentially ends up twice as large as it needs to be. If we instead increase the array by only 25% of its current size (i.e., a geometric base of 1.25), we do not risk wasting as much memory in the end, but there will be more intermediate resize events along the way. Still it is possible to prove an O(1) amortized bound, using a constant factor greater than the 3 cyber-dollars per operation used in the proof of Proposition 5.1 (see Exercise C-5.15). The key to the performance is that the amount of additional space is proportional to the current size of the array. Beware of Arithmetic Progression To avoid reserving too much space at once, it might be tempting to implement a dynamic array with a strategy in which a constant number of additional cells are reserved each time an array is resized. Unfortunately, the overall performance of such a strategy is significantly worse. At an extreme, an increase of only one cell causes each append operation to resize the array, leading to a familiar 1+ 2+ 3+ ··· + n summation and Ω(n2) overall cost. Using increases of 2 or 3 at a time is slightly better, as portrayed in Figure 5.13, but the overall cost remains quadratic. primitive operations for an append current number of elements 13 10 12 5 6 7 8 11 14 15 16 1 2 3 4 9 primitive operations for an append current number of elements 13 10 12 5 6 7 8 11 14 15 16 1 2 3 4 9 (a) (b) Figure 5.15: Running times of a series of append operations on a dynamic array using arithmetic progression of sizes. (a) Assumes increase of 2 in size of the array, while (b) assumes increase of 3.
- 222. 200 Chapter 5.Array-Based Sequences Using a fixed increment for each resize, and thus an arithmetic progression of intermediate array sizes, results in an overall time that is quadratic in the number of operations, as shown in the following proposition. Intuitively, even an increase in 1000 cells per resize will become insignificant for large data sets. Proposition 5.2: Performing a series of n append operations on an initially empty dynamic array using a fixed increment with each resize takes Ω(n2) time. Justification: Let c 0 represent the fixed increment in capacity that is used for each resize event. During the series of n append operations, time will have been spent initializing arrays of size c,2c,3c,...,mc for m = n/c, and therefore, the overall time would be proportional to c+ 2c+ 3c+ ··· + mc. By Proposition 3.3, this sum is m ∑ i=1 ci = c· m ∑ i=1 i = c m(m+1) 2 ≥ c n c (n c +1) 2 ≥ n2 2c . Therefore, performing the n append operations takes Ω(n2) time. A lesson to be learned from Propositions 5.1 and 5.2 is that a subtle difference in an algorithm design can produce drastic differences in the asymptotic performance, and that a careful analysis can provide important insights into the design of a data structure. Memory Usage and Shrinking an Array Another consequence of the rule of a geometric increase in capacity when append- ing to a dynamic array is that the final array size is guaranteed to be proportional to the overall number of elements. That is, the data structure uses O(n) memory. This is a very desirable property for a data structure. If a container, such as a Python list, provides operations that cause the removal of one or more elements, greater care must be taken to ensure that a dynamic array guarantees O(n) memory usage. The risk is that repeated insertions may cause the underlying array to grow arbitrarily large, and that there will no longer be a propor- tional relationship between the actual number of elements and the array capacity after many elements are removed. A robust implementation of such a data structure will shrink the underlying array, on occasion, while maintaining the O(1) amortized bound on individual op- erations. However, care must be taken to ensure that the structure cannot rapidly oscillate between growing and shrinking the underlying array, in which case the amortized bound would not be achieved. In Exercise C-5.16, we explore a strategy in which the array capacity is halved whenever the number of actual element falls below one fourth of that capacity, thereby guaranteeing that the array capacity is at most four times the number of elements; we explore the amortized analysis of such a strategy in Exercises C-5.17 and C-5.18.
- 223. 5.3. Dynamic Arraysand Amortization 201 5.3.3 Python’s List Class The experiments of Code Fragment 5.1 and 5.2, at the beginning of Section 5.3, provide empirical evidence that Python’s list class is using a form of dynamic arrays for its storage. Yet, a careful examination of the intermediate array capacities (see Exercises R-5.2 and C-5.13) suggests that Python is not using a pure geometric progression, nor is it using an arithmetic progression. With that said, it is clear that Python’s implementation of the append method exhibits amortized constant-time behavior. We can demonstrate this fact experi- mentally. A single append operation typically executes so quickly that it would be difficult for us to accurately measure the time elapsed at that granularity, although we should notice some of the more expensive operations in which a resize is per- formed. We can get a more accurate measure of the amortized cost per operation by performing a series of n append operations on an initially empty list and deter- mining the average cost of each. A function to perform that experiment is given in Code Fragment 5.4. 1 from time import time # import time function from time module 2 def compute average(n): 3 ”””Perform n appends to an empty list and return average time elapsed.””” 4 data = [ ] 5 start = time( ) # record the start time (in seconds) 6 for k in range(n): 7 data.append(None) 8 end = time( ) # record the end time (in seconds) 9 return (end − start) / n # compute average per operation Code Fragment 5.4: Measuring the amortized cost of append for Python’s list class. Technically, the time elapsed between the start and end includes the time to manage the iteration of the for loop, in addition to the append calls. The empirical results of the experiment, for increasingly large values of n, are shown in Table 5.2. We see higher average cost for the smaller data sets, perhaps in part due to the over- head of the loop range. There is also natural variance in measuring the amortized cost in this way, because of the impact of the final resize event relative to n. Taken as a whole, there seems clear evidence that the amortized time for each append is independent of n. n 100 1,000 10,000 100,000 1,000,000 10,000,000 100,000,000 μs 0.219 0.158 0.164 0.151 0.147 0.147 0.149 Table 5.2: Average running time of append, measured in microseconds, as observed over a sequence of n calls, starting with an empty list.
- 224. 202 Chapter 5.Array-Based Sequences 5.4 Efficiency of Python’s Sequence Types In the previous section, we began to explore the underpinnings of Python’s list class, in terms of implementation strategies and efficiency. We continue in this section by examining the performance of all of Python’s sequence types. 5.4.1 Python’s List and Tuple Classes The nonmutating behaviors of the list class are precisely those that are supported by the tuple class. We note that tuples are typically more memory efficient than lists because they are immutable; therefore, there is no need for an underlying dynamic array with surplus capacity. We summarize the asymptotic efficiency of the nonmutating behaviors of the list and tuple classes in Table 5.3. An explanation of this analysis follows. Operation Running Time len(data) O(1) data[j] O(1) data.count(value) O(n) data.index(value) O(k+1) value in data O(k+1) data1 == data2 O(k+1) (similarly !=, , =, , =) data[j:k] O(k− j +1) data1 + data2 O(n1 +n2) c data O(cn) Table 5.3: Asymptotic performance of the nonmutating behaviors of the list and tuple classes. Identifiers data, data1, and data2 designate instances of the list or tuple class, and n, n1, and n2 their respective lengths. For the containment check and index method, k represents the index of the leftmost occurrence (with k = n if there is no occurrence). For comparisons between two sequences, we let k denote the leftmost index at which they disagree or else k = min(n1,n2). Constant-Time Operations The length of an instance is returned in constant time because an instance explicitly maintains such state information. The constant-time efficiency of syntax data[j] is assured by the underlying access into an array.
- 225. 5.4. Efficiency ofPython’s Sequence Types 203 Searching for Occurrences of a Value Each of the count, index, and contains methods proceed through iteration of the sequence from left to right. In fact, Code Fragment 2.14 of Section 2.4.3 demonstrates how those behaviors might be implemented. Notably, the loop for computing the count must proceed through the entire sequence, while the loops for checking containment of an element or determining the index of an element immediately exit once they find the leftmost occurrence of the desired value, if one exists. So while count always examines the n elements of the sequence, index and contains examine n elements in the worst case, but may be faster. Empirical evidence can be found by setting data = list(range(10000000)) and then comparing the relative efficiency of the test, 5 in data, relative to the test, 9999995 in data, or even the failed test, −5 in data. Lexicographic Comparisons Comparisons between two sequences are defined lexicographically. In the worst case, evaluating such a condition requires an iteration taking time proportional to the length of the shorter of the two sequences (because when one sequence ends, the lexicographic result can be determined). However, in some cases the result of the test can be evaluated more efficiently. For example, if evaluating [7, 3, ...] [7, 5, ...], it is clear that the result is True without examining the re- mainders of those lists, because the second element of the left operand is strictly less than the second element of the right operand. Creating New Instances The final three behaviors in Table 5.3 are those that construct a new instance based on one or more existing instances. In all cases, the running time depends on the construction and initialization of the new result, and therefore the asymptotic be- havior is proportional to the length of the result. Therefore, we find that slice data[6000000:6000008] can be constructed almost immediately because it has only eight elements, while slice data[6000000:7000000] has one million elements, and thus is more time-consuming to create. Mutating Behaviors The efficiency of the mutating behaviors of the list class are described in Table 5.3. The simplest of those behaviors has syntax data[j] = val, and is supported by the special setitem method. This operation has worst-case O(1) running time be- cause it simply replaces one element of a list with a new value. No other elements are affected and the size of the underlying array does not change. The more inter- esting behaviors to analyze are those that add or remove elements from the list.
- 226. 204 Chapter 5.Array-Based Sequences Operation Running Time data[j] = val O(1) data.append(value) O(1)∗ data.insert(k, value) O(n−k+1)∗ data.pop() O(1)∗ data.pop(k) O(n−k)∗ del data[k] data.remove(value) O(n)∗ data1.extend(data2) O(n2)∗ data1 += data2 data.reverse() O(n) data.sort() O(nlogn) ∗amortized Table 5.4: Asymptotic performance of the mutating behaviors of the list class. Iden- tifiers data, data1, and data2 designate instances of the list class, and n, n1, and n2 their respective lengths. Adding Elements to a List In Section 5.3 we fully explored the append method. In the worst case, it requires Ω(n) time because the underlying array is resized, but it uses O(1) time in the amor- tized sense. Lists also support a method, with signature insert(k, value), that inserts a given value into the list at index 0 ≤ k ≤ n while shifting all subsequent elements back one slot to make room. For the purpose of illustration, Code Fragment 5.5 pro- vides an implementation of that method, in the context of our DynamicArray class introduced in Code Fragment 5.3. There are two complicating factors in analyzing the efficiency of such an operation. First, we note that the addition of one element may require a resizing of the dynamic array. That portion of the work requires Ω(n) worst-case time but only O(1) amortized time, as per append. The other expense for insert is the shifting of elements to make room for the new item. The time for 1 def insert(self, k, value): 2 ”””Insert value at index k, shifting subsequent values rightward.””” 3 # (for simplicity, we assume 0 = k = n in this verion) 4 if self. n == self. capacity: # not enough room 5 self. resize(2 self. capacity) # so double capacity 6 for j in range(self. n, k, −1): # shift rightmost first 7 self. A[j] = self. A[j−1] 8 self. A[k] = value # store newest element 9 self. n += 1 Code Fragment 5.5: Implementation of insert for our DynamicArray class.
- 227. 5.4. Efficiency ofPython’s Sequence Types 205 k 2 1 0 n−1 Figure 5.16: Creating room to insert a new element at index k of a dynamic array. that process depends upon the index of the new element, and thus the number of other elements that must be shifted. That loop copies the reference that had been at index n− 1 to index n, then the reference that had been at index n− 2 to n− 1, continuing until copying the reference that had been at index k to k + 1, as illus- trated in Figure 5.16. Overall this leads to an amortized O(n−k +1) performance for inserting at index k. When exploring the efficiency of Python’s append method in Section 5.3.3, we performed an experiment that measured the average cost of repeated calls on varying sizes of lists (see Code Fragment 5.4 and Table 5.2). We have repeated that experiment with the insert method, trying three different access patterns: • In the first case, we repeatedly insert at the beginning of a list, for n in range(N): data.insert(0, None) • In a second case, we repeatedly insert near the middle of a list, for n in range(N): data.insert(n // 2, None) • In a third case, we repeatedly insert at the end of the list, for n in range(N): data.insert(n, None) The results of our experiment are given in Table 5.5, reporting the average time per operation (not the total time for the entire loop). As expected, we see that inserting at the beginning of a list is most expensive, requiring linear time per operation. Inserting at the middle requires about half the time as inserting at the beginning, yet is still Ω(n) time. Inserting at the end displays O(1) behavior, akin to append. N 100 1,000 10,000 100,000 1,000,000 k = 0 0.482 0.765 4.014 36.643 351.590 k = n // 2 0.451 0.577 2.191 17.873 175.383 k = n 0.420 0.422 0.395 0.389 0.397 Table 5.5: Average running time of insert(k, val), measured in microseconds, as observed over a sequence of N calls, starting with an empty list. We let n denote the size of the current list (as opposed to the final list).
- 228. 206 Chapter 5.Array-Based Sequences Removing Elements from a List Python’s list class offers several ways to remove an element from a list. A call to pop() removes the last element from a list. This is most efficient, because all other elements remain in their original location. This is effectively an O(1) operation, but the bound is amortized because Python will occasionally shrink the underlying dynamic array to conserve memory. The parameterized version, pop(k), removes the element that is at index k n of a list, shifting all subsequent elements leftward to fill the gap that results from the removal. The efficiency of this operation is O(n−k), as the amount of shifting depends upon the choice of index k, as illustrated in Figure 5.17. Note well that this implies that pop(0) is the most expensive call, using Ω(n) time. (see experiments in Exercise R-5.8.) k 2 1 0 n−1 Figure 5.17: Removing an element at index k of a dynamic array. The list class offers another method, named remove, that allows the caller to specify the value that should be removed (not the index at which it resides). For- mally, it removes only the first occurrence of such a value from a list, or raises a ValueError if no such value is found. An implementation of such behavior is given in Code Fragment 5.6, again using our DynamicArray class for illustration. Interestingly, there is no “efficient” case for remove; every call requires Ω(n) time. One part of the process searches from the beginning until finding the value at index k, while the rest iterates from k to the end in order to shift elements leftward. This linear behavior can be observed experimentally (see Exercise C-5.24). 1 def remove(self, value): 2 ”””Remove first occurrence of value (or raise ValueError).””” 3 # note: we do not consider shrinking the dynamic array in this version 4 for k in range(self. n): 5 if self. A[k] == value: # found a match! 6 for j in range(k, self. n − 1): # shift others to fill gap 7 self. A[j] = self. A[j+1] 8 self. A[self. n − 1] = None # help garbage collection 9 self. n −= 1 # we have one less item 10 return # exit immediately 11 raise ValueError( value not found ) # only reached if no match Code Fragment 5.6: Implementation of remove for our DynamicArray class.
- 229. 5.4. Efficiency ofPython’s Sequence Types 207 Extending a List Python provides a method named extend that is used to add all elements of one list to the end of a second list. In effect, a call to data.extend(other) produces the same outcome as the code, for element in other: data.append(element) In either case, the running time is proportional to the length of the other list, and amortized because the underlying array for the first list may be resized to accom- modate the additional elements. In practice, the extend method is preferable to repeated calls to append because the constant factors hidden in the asymptotic analysis are significantly smaller. The greater efficiency of extend is threefold. First, there is always some advantage to using an appropriate Python method, because those methods are often implemented natively in a compiled language (rather than as interpreted Python code). Second, there is less overhead to a single function call that accomplishes all the work, versus many individual function calls. Finally, increased efficiency of extend comes from the fact that the resulting size of the updated list can be calculated in advance. If the second data set is quite large, there is some risk that the underlying dynamic array might be resized multiple times when using repeated calls to append. With a single call to extend, at most one resize operation will be performed. Exercise C-5.22 explores the relative efficiency of these two approaches experimentally. Constructing New Lists There are several syntaxes for constructing new lists. In almost all cases, the asymp- totic efficiency of the behavior is linear in the length of the list that is created. How- ever, as with the case in the preceding discussion of extend, there are significant differences in the practical efficiency. Section 1.9.2 introduces the topic of list comprehension, using an example such as squares = [ k k for k in range(1, n+1) ] as a shorthand for squares = [ ] for k in range(1, n+1): squares.append(k k) Experiments should show that the list comprehension syntax is significantly faster than building the list by repeatedly appending (see Exercise C-5.23). Similarly, it is a common Python idiom to initialize a list of constant values using the multiplication operator, as in [0] n to produce a list of length n with all values equal to zero. Not only is this succinct for the programmer; it is more efficient than building such a list incrementally.
- 230. 208 Chapter 5.Array-Based Sequences 5.4.2 Python’s String Class Strings are very important in Python. We introduced their use in Chapter 1, with a discussion of various operator syntaxes in Section 1.3. A comprehensive sum- mary of the named methods of the class is given in Tables A.1 through A.4 of Appendix A. We will not formally analyze the efficiency of each of those behav- iors in this section, but we do wish to comment on some notable issues. In general, we let n denote the length of a string. For operations that rely on a second string as a pattern, we let m denote the length of that pattern string. The analysis for many behaviors is quite intuitive. For example, methods that produce a new string (e.g., capitalize, center, strip) require time that is linear in the length of the string that is produced. Many of the behaviors that test Boolean conditions of a string (e.g., islower) take O(n) time, examining all n characters in the worst case, but short circuiting as soon as the answer becomes evident (e.g., islower can immediately return False if the first character is uppercased). The comparison operators (e.g., ==, ) fall into this category as well. Pattern Matching Some of the most interesting behaviors, from an algorithmic point of view, are those that in some way depend upon finding a string pattern within a larger string; this goal is at the heart of methods such as contains , find, index, count, replace, and split. String algorithms will be the topic of Chapter 13, and this particular problem known as pattern matching will be the focus of Section 13.2. A naive im- plementation runs in O(mn) time case, because we consider the n−m+1 possible starting indices for the pattern, and we spend O(m) time at each starting position, checking if the pattern matches. However, in Section 13.2, we will develop an al- gorithm for finding a pattern of length m within a longer string of length n in O(n) time. Composing Strings Finally, we wish to comment on several approaches for composing large strings. As an academic exercise, assume that we have a large string named document, and our goal is to produce a new string, letters, that contains only the alphabetic characters of the original string (e.g., with spaces, numbers, and punctuation removed). It may be tempting to compose a result through repeated concatenation, as follows. # WARNING: do not do this letters = # start with empty string for c in document: if c.isalpha(): letters += c # concatenate alphabetic character
- 231. 5.4. Efficiency ofPython’s Sequence Types 209 While the preceding code fragment accomplishes the goal, it may be terribly inefficient. Because strings are immutable, the command, letters += c, would presumably compute the concatenation, letters + c, as a new string instance and then reassign the identifier, letters, to that result. Constructing that new string would require time proportional to its length. If the final result has n characters, the series of concatenations would take time proportional to the familiar sum 1+ 2+ 3+···+n, and therefore O(n2) time. Inefficient code of this type is widespread in Python, perhaps because of the somewhat natural appearance of the code, and mistaken presumptions about how the += operator is evaluated with strings. Some later implementations of the Python interpreter have developed an optimization to allow such code to complete in linear time, but this is not guaranteed for all Python implementations. The op- timization is as follows. The reason that a command, letters += c, causes a new string instance to be created is that the original string must be left unchanged if another variable in a program refers to that string. On the other hand, if Python knew that there were no other references to the string in question, it could imple- ment += more efficiently by directly mutating the string (as a dynamic array). As it happens, the Python interpreter already maintains what are known as reference counts for each object; this count is used in part to determine if an object can be garbage collected. (See Section 15.1.2.) But in this context, it provides a means to detect when no other references exist to a string, thereby allowing the optimization. A more standard Python idiom to guarantee linear time composition of a string is to use a temporary list to store individual pieces, and then to rely on the join method of the str class to compose the final result. Using this technique with our previous example would appear as follows: temp = [ ] # start with empty list for c in document: if c.isalpha(): temp.append(c) # append alphabetic character letters = .join(temp) # compose overall result This approach is guaranteed to run in O(n) time. First, we note that the series of up to n append calls will require a total of O(n) time, as per the definition of the amortized cost of that operation. The final call to join also guarantees that it takes time that is linear in the final length of the composed string. As we discussed at the end of the previous section, we can further improve the practical execution time by using a list comprehension syntax to build up the temporary list, rather than by repeated calls to append. That solution appears as, letters = .join([c for c in document if c.isalpha()]) Better yet, we can entirely avoid the temporary list with a generator comprehension: letters = .join(c for c in document if c.isalpha())
- 232. 210 Chapter 5.Array-Based Sequences 5.5 Using Array-Based Sequences 5.5.1 Storing High Scores for a Game The first application we study is storing a sequence of high score entries for a video game. This is representative of many applications in which a sequence of objects must be stored. We could just as easily have chosen to store records for patients in a hospital or the names of players on a football team. Nevertheless, let us focus on storing high score entries, which is a simple application that is already rich enough to present some important data-structuring concepts. To begin, we consider what information to include in an object representing a high score entry. Obviously, one component to include is an integer representing the score itself, which we identify as score. Another useful thing to include is the name of the person earning this score, which we identify as name. We could go on from here, adding fields representing the date the score was earned or game statistics that led to that score. However, we omit such details to keep our example simple. A Python class, GameEntry, representing a game entry, is given in Code Fragment 5.7. 1 class GameEntry: 2 ”””Represents one entry of a list of high scores.””” 3 4 def init (self, name, score): 5 self. name = name 6 self. score = score 7 8 def get name(self): 9 return self. name 10 11 def get score(self): 12 return self. score 13 14 def str (self): 15 return ({0}, {1}) .format(self. name, self. score) # e.g., (Bob, 98) Code Fragment 5.7: Python code for a simple GameEntry class. We include meth- ods for returning the name and score for a game entry object, as well as a method for returning a string representation of this entry.
- 233. 5.5. Using Array-BasedSequences 211 A Class for High Scores To maintain a sequence of high scores, we develop a class named Scoreboard. A scoreboard is limited to a certain number of high scores that can be saved; once that limit is reached, a new score only qualifies for the scoreboard if it is strictly higher than the lowest “high score” on the board. The length of the desired scoreboard may depend on the game, perhaps 10, 50, or 500. Since that limit may vary depending on the game, we allow it to be specified as a parameter to our Scoreboard constructor. Internally, we will use a Python list named board in order to manage the GameEntry instances that represent the high scores. Since we expect the score- board to eventually reach full capacity, we initialize the list to be large enough to hold the maximum number of scores, but we initially set all entries to None. By allocating the list with maximum capacity initially, it never needs to be resized. As entries are added, we will maintain them from highest to lowest score, starting at index 0 of the list. We illustrate a typical state of the data structure in Figure 5.18. 3 2 0 1 9 7 8 6 5 4 660 Mike 1105 Paul 720 Rose 590 Rob 750 Anna Jack 510 Figure 5.18: An illustration of an ordered list of length ten, storing references to six GameEntry objects in the cells from index 0 to 5, with the rest being None. A complete Python implementation of the Scoreboard class is given in Code Fragment 5.8. The constructor is rather simple. The command self. board = [None] capacity creates a list with the desired length, yet all entries equal to None. We maintain an additional instance variable, n, that represents the number of actual entries currently in our table. For convenience, our class supports the getitem method to retrieve an entry at a given index with a syntax board[i] (or None if no such entry exists), and we support a simple str method that returns a string representation of the entire scoreboard, with one entry per line.
- 234. 212 Chapter 5.Array-Based Sequences 1 class Scoreboard: 2 ”””Fixed-length sequence of high scores in nondecreasing order.””” 3 4 def init (self, capacity=10): 5 ”””Initialize scoreboard with given maximum capacity. 6 7 All entries are initially None. 8 ””” 9 self. board = [None] capacity # reserve space for future scores 10 self. n = 0 # number of actual entries 11 12 def getitem (self, k): 13 ”””Return entry at index k.””” 14 return self. board[k] 15 16 def str (self): 17 ”””Return string representation of the high score list.””” 18 return n .join(str(self. board[j]) for j in range(self. n)) 19 20 def add(self, entry): 21 ”””Consider adding entry to high scores.””” 22 score = entry.get score() 23 24 # Does new entry qualify as a high score? 25 # answer is yes if board not full or score is higher than last entry 26 good = self. n len(self. board) or score self. board[−1].get score() 27 28 if good: 29 if self. n len(self. board): # no score drops from list 30 self. n += 1 # so overall number increases 31 32 # shift lower scores rightward to make room for new entry 33 j = self. n − 1 34 while j 0 and self. board[j−1].get score( ) score: 35 self. board[j] = self. board[j−1] # shift entry from j-1 to j 36 j −= 1 # and decrement j 37 self. board[j] = entry # when done, add new entry Code Fragment 5.8: Python code for a Scoreboard class that maintains an ordered series of scores as GameEntry objects.
- 235. 5.5. Using Array-BasedSequences 213 Adding an Entry The most interesting method of the Scoreboard class is add, which is responsible for considering the addition of a new entry to the scoreboard. Keep in mind that every entry will not necessarily qualify as a high score. If the board is not yet full, any new entry will be retained. Once the board is full, a new entry is only retained if it is strictly better than one of the other scores, in particular, the last entry of the scoreboard, which is the lowest of the high scores. When a new score is considered, we begin by determining whether it qualifies as a high score. If so, we increase the count of active scores, n, unless the board is already at full capacity. In that case, adding a new high score causes some other entry to be dropped from the scoreboard, so the overall number of entries remains the same. To correctly place a new entry within the list, the final task is to shift any in- ferior scores one spot lower (with the least score being dropped entirely when the scoreboard is full). This process is quite similar to the implementation of the insert method of the list class, as described on pages 204–205. In the context of our score- board, there is no need to shift any None references that remain near the end of the array, so the process can proceed as diagrammed in Figure 5.19. 0 1 9 8 7 6 5 4 3 2 Mike 1105 Rob 750 Paul 720 Rose 590 Anna 660 Jack 510 740 Jill Figure 5.19: Adding a new GameEntry for Jill to the scoreboard. In order to make room for the new reference, we have to shift the references for game entries with smaller scores than the new one to the right by one cell. Then we can insert the new entry with index 2. To implement the final stage, we begin by considering index j = self. n − 1, which is the index at which the last GameEntry instance will reside, after complet- ing the operation. Either j is the correct index for the newest entry, or one or more immediately before it will have lesser scores. The while loop at line 34 checks the compound condition, shifting references rightward and decrementing j, as long as there is another entry at index j −1 with a score less than the new score.
- 236. 214 Chapter 5.Array-Based Sequences 5.5.2 Sorting a Sequence In the previous subsection, we considered an application for which we added an ob- ject to a sequence at a given position while shifting other elements so as to keep the previous order intact. In this section, we use a similar technique to solve the sorting problem, that is, starting with an unordered sequence of elements and rearranging them into nondecreasing order. The Insertion-Sort Algorithm We study several sorting algorithms in this book, most of which are described in Chapter 12. As a warm-up, in this section we describe a nice, simple sorting al- gorithm known as insertion-sort. The algorithm proceeds as follows for an array- based sequence. We start with the first element in the array. One element by itself is already sorted. Then we consider the next element in the array. If it is smaller than the first, we swap them. Next we consider the third element in the array. We swap it leftward until it is in its proper order with the first two elements. We then consider the fourth element, and swap it leftward until it is in the proper order with the first three. We continue in this manner with the fifth element, the sixth, and so on, until the whole array is sorted. We can express the insertion-sort algorithm in pseudo-code, as shown in Code Fragment 5.9. Algorithm InsertionSort(A): Input: An array A of n comparable elements Output: The array A with elements rearranged in nondecreasing order for k from 1 to n − 1 do Insert A[k] at its proper location within A[0], A[1], ..., A[k]. Code Fragment 5.9: High-level description of the insertion-sort algorithm. This is a simple, high-level description of insertion-sort. If we look back to Code Fragment 5.8 of Section 5.5.1, we see that the task of inserting a new en- try into the list of high scores is almost identical to the task of inserting a newly considered element in insertion-sort (except that game scores were ordered from high to low). We provide a Python implementation of insertion-sort in Code Frag- ment 5.10, using an outer loop to consider each element in turn, and an inner loop that moves a newly considered element to its proper location relative to the (sorted) subarray of elements that are to its left. We illustrate an example run of the insertion-sort algorithm in Figure 5.20. The nested loops of insertion-sort lead to an O(n2) running time in the worst case. The most work is done if the array is initially in reverse order. On the other hand, if the initial array is nearly sorted or perfectly sorted, insertion-sort runs in O(n) time because there are few or no iterations of the inner loop.
- 237. 5.5. Using Array-BasedSequences 215 1 def insertion sort(A): 2 ”””Sort list of comparable elements into nondecreasing order.””” 3 for k in range(1, len(A)): # from 1 to n-1 4 cur = A[k] # current element to be inserted 5 j = k # find correct index j for current 6 while j 0 and A[j−1] cur: # element A[j-1] must be after current 7 A[j] = A[j−1] 8 j −= 1 9 A[j] = cur # cur is now in the right place Code Fragment 5.10: Python code for performing insertion-sort on a list. insert insert insert 0 0 0 0 0 0 0 0 0 0 0 0 Done! 0 C A E H G F B C A E H G F D B E H G F C D A H G F B C D E A F B C D E H A G F B C D E H E H G F D C B B E H G F D C A F B C D E H B C D E H G B A A B C D E G H A B C F F G A G H E A D C D E H H G F E D G C A B D no move 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 1 2 3 4 5 6 7 cur 1 2 3 4 5 6 7 move move move no move no move no move no move move no move move move 1 Figure 5.20: Execution of the insertion-sort algorithm on an array of eight charac- ters. Each row corresponds to an iteration of the outer loop, and each copy of the sequence in a row corresponds to an iteration of the inner loop. The current element that is being inserted is highlighted in the array, and shown as the cur value.
- 238. 216 Chapter 5.Array-Based Sequences 5.5.3 Simple Cryptography An interesting application of strings and lists is cryptography, the science of secret messages and their applications. This field studies ways of performing encryp- tion, which takes a message, called the plaintext, and converts it into a scrambled message, called the ciphertext. Likewise, cryptography also studies corresponding ways of performing decryption, which takes a ciphertext and turns it back into its original plaintext. Arguably the earliest encryption scheme is the Caesar cipher, which is named after Julius Caesar, who used this scheme to protect important military messages. (All of Caesar’s messages were written in Latin, of course, which already makes them unreadable for most of us!) The Caesar cipher is a simple way to obscure a message written in a language that forms words with an alphabet. The Caesar cipher involves replacing each letter in a message with the letter that is a certain number of letters after it in the alphabet. So, in an English message, we might replace each A with D, each B with E, each C with F, and so on, if shifting by three characters. We continue this approach all the way up to W, which is replaced with Z. Then, we let the substitution pattern wrap around, so that we replace X with A, Y with B, and Z with C. Converting Between Strings and Character Lists Given that strings are immutable, we cannot directly edit an instance to encrypt it. Instead, our goal will be to generate a new string. A convenient technique for per- forming string transformations is to create an equivalent list of characters, edit the list, and then reassemble a (new) string based on the list. The first step can be per- formed by sending the string as a parameter to the constructor of the list class. For example, the expression list( bird ) produces the result [ b , i , r , d ]. Conversely, we can use a list of characters to build a string by invoking the join method on an empty string, with the list of characters as the parameter. For exam- ple, the call .join([ b , i , r , d ]) returns the string bird . Using Characters as Array Indices If we were to number our letters like array indices, so that A is 0, B is 1, C is 2, and so on, then we can write the Caesar cipher with a rotation of r as a simple formula: Replace each letter i with the letter (i + r) mod 26, where mod is the modulo operator, which returns the remainder after performing an integer division. This operator is denoted with % in Python, and it is exactly the operator we need to easily perform the wrap around at the end of the alphabet. For 26 mod 26 is 0, 27 mod 26 is 1, and 28 mod 26 is 2. The decryption algorithm for the Caesar cipher is just the opposite—we replace each letter with the one r places before it, with wrap around (that is, letter i is replaced by letter (i−r) mod 26).
- 239. 5.5. Using Array-BasedSequences 217 We can represent a replacement rule using another string to describe the trans- lation. As a concrete example, suppose we are using a Caesar cipher with a three- character rotation. We can precompute a string that represents the replacements that should be used for each character from A to Z. For example, A should be re- placed by D, B replaced by E, and so on. The 26 replacement characters in order are DEFGHIJKLMNOPQRSTUVWXYZABC . We can subsequently use this translation string as a guide to encrypt a message. The remaining challenge is how to quickly locate the replacement for each character of the original message. Fortunately, we can rely on the fact that characters are represented in Unicode by integer code points, and the code points for the uppercase letters of the Latin alphabet are consecutive (for simplicity, we restrict our encryption to uppercase letters). Python supports functions that convert between integer code points and one-character strings. Specifically, the function ord(c) takes a one-character string as a parameter and returns the integer code point for that character. Conversely, the function chr(j) takes an integer and returns its associated one-character string. In order to find a replacement for a character in our Caesar cipher, we need to map the characters A to Z to the respective numbers 0 to 25. The formula for doing that conversion is j = ord(c) − ord( A ). As a sanity check, if character c is A , we have that j = 0. When c is B , we will find that its ordinal value is pre- cisely one more than that for A , so their difference is 1. In general, the integer j that results from such a calculation can be used as an index into our precomputed translation string, as illustrated in Figure 5.21. 10 24 23 22 21 20 19 18 17 16 15 14 13 12 11 25 9 8 7 6 5 4 3 2 1 0 M O P Q R S T U V W X Y Z A B C D E F G H I J K L N Here is the replacement for T ord( T ) − ord( A ) 84 − 65 19 = = In Unicode Using T as an index encoder array Figure 5.21: Illustrating the use of uppercase characters as indices, in this case to perform the replacement rule for Caesar cipher encryption. In Code Fragment 5.11, we develop a Python class for performing the Caesar cipher with an arbitrary rotational shift, and demonstrate its use. When we run this program (to perform a simple test), we get the following output. Secret: WKH HDJOH LV LQ SODB; PHHW DW MRH’V. Message: THE EAGLE IS IN PLAY; MEET AT JOE’S. The constructor for the class builds the forward and backward translation strings for the given rotation. With those in hand, the encryption and decryption algorithms are essentially the same, and so we perform both by means of a nonpublic utility method named transform.
- 240. 218 Chapter 5.Array-Based Sequences 1 class CaesarCipher: 2 ”””Class for doing encryption and decryption using a Caesar cipher.””” 3 4 def init (self, shift): 5 ”””Construct Caesar cipher using given integer shift for rotation.””” 6 encoder = [None] 26 # temp array for encryption 7 decoder = [None] 26 # temp array for decryption 8 for k in range(26): 9 encoder[k] = chr((k + shift) % 26 + ord( A )) 10 decoder[k] = chr((k − shift) % 26 + ord( A )) 11 self. forward = .join(encoder) # will store as string 12 self. backward = .join(decoder) # since fixed 13 14 def encrypt(self, message): 15 ”””Return string representing encripted message.””” 16 return self. transform(message, self. forward) 17 18 def decrypt(self, secret): 19 ”””Return decrypted message given encrypted secret.””” 20 return self. transform(secret, self. backward) 21 22 def transform(self, original, code): 23 ”””Utility to perform transformation based on given code string.””” 24 msg = list(original) 25 for k in range(len(msg)): 26 if msg[k].isupper(): 27 j = ord(msg[k]) − ord( A ) # index from 0 to 25 28 msg[k] = code[j] # replace this character 29 return .join(msg) 30 31 if name == __main__ : 32 cipher = CaesarCipher(3) 33 message = THE EAGLE IS IN PLAY; MEET AT JOE S. 34 coded = cipher.encrypt(message) 35 print( Secret: , coded) 36 answer = cipher.decrypt(coded) 37 print( Message: , answer) Code Fragment 5.11: A complete Python class for the Caesar cipher.
- 241. 5.6. Multidimensional DataSets 219 5.6 Multidimensional Data Sets Lists, tuples, and strings in Python are one-dimensional. We use a single index to access each element of the sequence. Many computer applications involve mul- tidimensional data sets. For example, computer graphics are often modeled in either two or three dimensions. Geographic information may be naturally repre- sented in two dimensions, medical imaging may provide three-dimensional scans of a patient, and a company’s valuation is often based upon a high number of in- dependent financial measures that can be modeled as multidimensional data. A two-dimensional array is sometimes also called a matrix. We may use two indices, say i and j, to refer to the cells in the matrix. The first index usually refers to a row number and the second to a column number, and these are traditionally zero- indexed in computer science. Figure 5.22 illustrates a two-dimensional data set with integer values. This data might, for example, represent the number of stores in various regions of Manhattan. 22 18 709 5 33 10 4 56 82 440 45 32 830 120 750 660 13 77 20 105 4 880 45 66 61 28 650 7 510 67 940 12 36 3 20 100 306 590 0 500 50 65 42 49 88 25 70 126 83 288 398 233 5 83 59 232 49 8 365 90 33 58 632 87 94 5 59 204 120 829 62 394 3 4 102 140 183 390 16 26 8 0 1 2 3 4 5 6 7 0 1 2 3 4 5 6 7 9 Figure 5.22: Illustration of a two-dimensional integer data set, which has 8 rows and 10 columns. The rows and columns are zero-indexed. If this data set were named stores, the value of stores[3][5] is 100 and the value of stores[6][2] is 632. A common representation for a two-dimensional data set in Python is as a list of lists. In particular, we can represent a two-dimensional array as a list of rows, with each row itself being a list of values. For example, the two-dimensional data 22 18 709 5 33 45 32 830 120 750 4 880 45 66 61 might be stored in Python as follows. data = [ [22, 18, 709, 5, 33], [45, 32, 830, 120, 750], [4, 880, 45, 66, 61] ] An advantage of this representation is that we can naturally use a syntax such as data[1][3] to represent the value that has row index 1 and column index 3, as data[1], the second entry in the outer list, is itself a list, and thus indexable.
- 242. 220 Chapter 5.Array-Based Sequences Constructing a Multidimensional List To quickly initialize a one-dimensional list, we generally rely on a syntax such as data = [0] n to create a list of n zeros. On page 189, we emphasized that from a technical perspective, this creates a list of length n with all entries referencing the same integer instance, but that there was no meaningful consequence of such aliasing because of the immutability of the int class in Python. We have to be considerably more careful when creating a list of lists. If our goal were to create the equivalent of a two-dimensional list of integers, with r rows and c columns, and to initialize all values to zero, a flawed approach might be to try the command data = ([0] c) r # Warning: this is a mistake While([0] c) is indeed a list of c zeros, multiplying that list by r unfortunately cre- ates a single list with length r ·c, just as [2,4,6] 2 results in list [2, 4, 6, 2, 4, 6]. A better, yet still flawed attempt is to make a list that contains the list of c zeros as its only element, and then to multiply that list by r. That is, we could try the command data = [ [0] c ] r # Warning: still a mistake This is much closer, as we actually do have a structure that is formally a list of lists. The problem is that all r entries of the list known as data are references to the same instance of a list of c zeros. Figure 5.23 provides a portrayal of such aliasing. 0 0 0 0 0 0 0 0 1 2 3 4 5 1 2 data: Figure 5.23: A flawed representation of a 3×6 data set as a list of lists, created with the command data = [ [0] 6 ] 3. (For simplicity, we overlook the fact that the values in the secondary list are referential.) This is truly a problem. Setting an entry such as data[2][0] = 100 would change the first entry of the secondary list to reference a new value, 100. Yet that cell of the secondary list also represents the value data[0][0], because “row” data[0] and “row” data[2] refer to the same secondary list.
- 243. 5.6. Multidimensional DataSets 221 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 1 2 2 3 4 5 1 2 4 5 1 2 data: 3 4 5 1 Figure 5.24: A valid representation of a 3×6 data set as a list of lists. (For simplic- ity, we overlook the fact that the values in the secondary lists are referential.) To properly initialize a two-dimensional list, we must ensure that each cell of the primary list refers to an independent instance of a secondary list. This can be accomplished through the use of Python’s list comprehension syntax. data = [ [0] c for j in range(r) ] This command produces a valid configuration, similar to the one shown in Fig- ure 5.24. By using list comprehension, the expression [0] c is reevaluated for each pass of the embedded for loop. Therefore, we get r distinct secondary lists, as desired. (We note that the variable j in that command is irrelevant; we simply need a for loop that iterates r times.) Two-Dimensional Arrays and Positional Games Many computer games, be they strategy games, simulation games, or first-person conflict games, involve objects that reside in a two-dimensional space. Software for such positional games need a way of representing such a two-dimensional “board,” and in Python the list of lists is a natural choice. Tic-Tac-Toe As most school children know, Tic-Tac-Toe is a game played in a three-by-three board. Two players—X and O—alternate in placing their respective marks in the cells of this board, starting with player X. If either player succeeds in getting three of his or her marks in a row, column, or diagonal, then that player wins. This is admittedly not a sophisticated positional game, and it’s not even that much fun to play, since a good player O can always force a tie. Tic-Tac-Toe’s saving grace is that it is a nice, simple example showing how two-dimensional arrays can be used for positional games. Software for more sophisticated positional games, such as checkers, chess, or the popular simulation games, are all based on the same approach we illustrate here for using a two-dimensional array for Tic-Tac-Toe.
- 244. 222 Chapter 5.Array-Based Sequences Our representation of a 3 × 3 board will be a list of lists of characters, with X or O designating a player’s move, or designating an empty space. For example, the board configuration X O O O X X will be stored internally as [ [ O , X , O ], [ , X , ], [ , O , X ] ] We develop a complete Python class for maintaining a Tic-Tac-Toe board for two players. That class will keep track of the moves and report a winner, but it does not perform any strategy or allow someone to play Tic-Tac-Toe against the computer. The details of such a program are beyond the scope of this chapter, but it might nonetheless make a good course project (see Exercise P-8.68). Before presenting the implementation of the class, we demonstrate its public interface with a simple test in Code Fragment 5.12. 1 game = TicTacToe() 2 # X moves: # O moves: 3 game.mark(1, 1); game.mark(0, 2) 4 game.mark(2, 2); game.mark(0, 0) 5 game.mark(0, 1); game.mark(2, 1) 6 game.mark(1, 2); game.mark(1, 0) 7 game.mark(2, 0) 8 9 print(game) 10 winner = game.winner() 11 if winner is None: 12 print( Tie ) 13 else: 14 print(winner, wins ) Code Fragment 5.12: A simple test for our Tic-Tac-Toe class. The basic operations are that a new game instance represents an empty board, that the mark(i,j) method adds a mark at the given position for the current player (with the software managing the alternating of turns), and that the game board can be printed and the winner determined. The complete source code for the TicTacToe class is given in Code Fragment 5.13. Our mark method performs error checking to make sure that valid indices are sent, that the position is not already occupied, and that no further moves are made after someone wins the game.
- 245. 5.6. Multidimensional DataSets 223 1 class TicTacToe: 2 ”””Management of a Tic-Tac-Toe game (does not do strategy).””” 3 4 def init (self): 5 ”””Start a new game.””” 6 self. board = [ [ ] 3 for j in range(3) ] 7 self. player = X 8 9 def mark(self, i, j): 10 ”””Put an X or O mark at position (i,j) for next player s turn.””” 11 if not (0 = i = 2 and 0 = j = 2): 12 raise ValueError( Invalid board position ) 13 if self. board[i][j] != : 14 raise ValueError( Board position occupied ) 15 if self.winner( ) is not None: 16 raise ValueError( Game is already complete ) 17 self. board[i][j] = self. player 18 if self. player == X : 19 self. player = O 20 else: 21 self. player = X 22 23 def is win(self, mark): 24 ”””Check whether the board configuration is a win for the given player.””” 25 board = self. board # local variable for shorthand 26 return (mark == board[0][0] == board[0][1] == board[0][2] or # row 0 27 mark == board[1][0] == board[1][1] == board[1][2] or # row 1 28 mark == board[2][0] == board[2][1] == board[2][2] or # row 2 29 mark == board[0][0] == board[1][0] == board[2][0] or # column 0 30 mark == board[0][1] == board[1][1] == board[2][1] or # column 1 31 mark == board[0][2] == board[1][2] == board[2][2] or # column 2 32 mark == board[0][0] == board[1][1] == board[2][2] or # diagonal 33 mark == board[0][2] == board[1][1] == board[2][0]) # rev diag 34 35 def winner(self): 36 ”””Return mark of winning player, or None to indicate a tie.””” 37 for mark in XO : 38 if self. is win(mark): 39 return mark 40 return None 41 42 def str (self): 43 ”””Return string representation of current game board.””” 44 rows = [ | .join(self. board[r]) for r in range(3)] 45 return n-----n .join(rows) Code Fragment 5.13: A complete Python class for managing a Tic-Tac-Toe game.
- 246. 224 Chapter 5.Array-Based Sequences 5.7 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-5.1 Execute the experiment from Code Fragment 5.1 and compare the results on your system to those we report in Code Fragment 5.2. R-5.2 In Code Fragment 5.1, we perform an experiment to compare the length of a Python list to its underlying memory usage. Determining the sequence of array sizes requires a manual inspection of the output of that program. Redesign the experiment so that the program outputs only those values of k at which the existing capacity is exhausted. For example, on a system consistent with the results of Code Fragment 5.2, your program should output that the sequence of array capacities are 0, 4, 8, 16, 25, . . . . R-5.3 Modify the experiment from Code Fragment 5.1 in order to demonstrate that Python’s list class occasionally shrinks the size of its underlying array when elements are popped from a list. R-5.4 Our DynamicArray class, as given in Code Fragment 5.3, does not support use of negative indices with getitem . Update that method to better match the semantics of a Python list. R-5.5 Redo the justification of Proposition 5.1 assuming that the the cost of growing the array from size k to size 2k is 3k cyber-dollars. How much should each append operation be charged to make the amortization work? R-5.6 Our implementation of insert for the DynamicArray class, as given in Code Fragment 5.5, has the following inefficiency. In the case when a re- size occurs, the resize operation takes time to copy all the elements from an old array to a new array, and then the subsequent loop in the body of insert shifts many of those elements. Give an improved implementation of the insert method, so that, in the case of a resize, the elements are shifted into their final position during that operation, thereby avoiding the subsequent shifting. R-5.7 Let A be an array of size n ≥ 2 containing integers from 1 to n−1, inclu- sive, with exactly one repeated. Describe a fast algorithm for finding the integer in A that is repeated. R-5.8 Experimentally evaluate the efficiency of the pop method of Python’s list class when using varying indices as a parameter, as we did for insert on page 205. Report your results akin to Table 5.5.
- 247. 5.7. Exercises 225 R-5.9Explain the changes that would have to be made to the program of Code Fragment 5.11 so that it could perform the Caesar cipher for messages that are written in an alphabet-based language other than English, such as Greek, Russian, or Hebrew. R-5.10 The constructor for the CaesarCipher class in Code Fragment 5.11 can be implemented with a two-line body by building the forward and back- ward strings using a combination of the join method and an appropriate comprehension syntax. Give such an implementation. R-5.11 Use standard control structures to compute the sum of all numbers in an n×n data set, represented as a list of lists. R-5.12 Describe how the built-in sum function can be combined with Python’s comprehension syntax to compute the sum of all numbers in an n×n data set, represented as a list of lists. Creativity C-5.13 In the experiment of Code Fragment 5.1, we begin with an empty list. If data were initially constructed with nonempty length, does this affect the sequence of values at which the underlying array is expanded? Perform your own experiments, and comment on any relationship you see between the initial length and the expansion sequence. C-5.14 The shuffle method, supported by the random module, takes a Python list and rearranges it so that every possible ordering is equally likely. Implement your own version of such a function. You may rely on the randrange(n) function of the random module, which returns a random number between 0 and n−1 inclusive. C-5.15 Consider an implementation of a dynamic array, but instead of copying the elements into an array of double the size (that is, from N to 2N) when its capacity is reached, we copy the elements into an array with N/4 additional cells, going from capacity N to capacity N + N/4. Prove that performing a sequence of n append operations still runs in O(n) time in this case. C-5.16 Implement a pop method for the DynamicArray class, given in Code Frag- ment 5.3, that removes the last element of the array, and that shrinks the capacity, N, of the array by half any time the number of elements in the array goes below N/4. C-5.17 Prove that when using a dynamic array that grows and shrinks as in the previous exercise, the following series of 2n operations takes O(n) time: n append operations on an initially empty array, followed by n pop oper- ations.
- 248. 226 Chapter 5.Array-Based Sequences C-5.18 Give a formal proof that any sequence of n append or pop operations on an initially empty dynamic array takes O(n) time, if using the strategy described in Exercise C-5.16. C-5.19 Consider a variant of Exercise C-5.16, in which an array of capacity N is resized to capacity precisely that of the number of elements, any time the number of elements in the array goes strictly below N/4. Give a formal proof that any sequence of n append or pop operations on an initially empty dynamic array takes O(n) time. C-5.20 Consider a variant of Exercise C-5.16, in which an array of capacity N, is resized to capacity precisely that of the number of elements, any time the number of elements in the array goes strictly below N/2. Show that there exists a sequence of n operations that requires Ω(n2) time to execute. C-5.21 In Section 5.4.2, we described four different ways to compose a long string: (1) repeated concatenation, (2) appending to a temporary list and then joining, (3) using list comprehension with join, and (4) using genera- tor comprehension with join. Develop an experiment to test the efficiency of all four of these approaches and report your findings. C-5.22 Develop an experiment to compare the relative efficiency of the extend method of Python’s list class versus using repeated calls to append to accomplish the equivalent task. C-5.23 Based on the discussion of page 207, develop an experiment to compare the efficiency of Python’s list comprehension syntax versus the construc- tion of a list by means of repeated calls to append. C-5.24 Perform experiments to evaluate the efficiency of the remove method of Python’s list class, as we did for insert on page 205. Use known values so that all removals occur either at the beginning, middle, or end of the list. Report your results akin to Table 5.5. C-5.25 The syntax data.remove(value) for Python list data removes only the first occurrence of element value from the list. Give an implementation of a function, with signature remove all(data, value), that removes all occur- rences of value from the given list, such that the worst-case running time of the function is O(n) on a list with n elements. Not that it is not efficient enough in general to rely on repeated calls to remove. C-5.26 Let B be an array of size n ≥ 6 containing integers from 1 to n−5, inclu- sive, with exactly five repeated. Describe a good algorithm for finding the five integers in B that are repeated. C-5.27 Given a Python list L of n positive integers, each represented with k = logn+ 1 bits, describe an O(n)-time method for finding a k-bit integer not in L. C-5.28 Argue why any solution to the previous problem must run in Ω(n) time.
- 249. Chapter Notes 227 C-5.29A useful operation in databases is the natural join. If we view a database as a list of ordered pairs of objects, then the natural join of databases A and B is the list of all ordered triples (x,y,z) such that the pair (x,y) is in A and the pair (y,z) is in B. Describe and analyze an efficient algorithm for computing the natural join of a list A of n pairs and a list B of m pairs. C-5.30 When Bob wants to send Alice a message M on the Internet, he breaks M into n data packets, numbers the packets consecutively, and injects them into the network. When the packets arrive at Alice’s computer, they may be out of order, so Alice must assemble the sequence of n packets in order before she can be sure she has the entire message. Describe an efficient scheme for Alice to do this, assuming that she knows the value of n. What is the running time of this algorithm? C-5.31 Describe a way to use recursion to add all the numbers in an n × n data set, represented as a list of lists. Projects P-5.32 Write a Python function that takes two three-dimensional numeric data sets and adds them componentwise. P-5.33 Write a Python program for a matrix class that can add and multiply two- dimensional arrays of numbers, assuming the dimensions agree appropri- ately for the operation. P-5.34 Write a program that can perform the Caesar cipher for English messages that include both upper- and lowercase characters. P-5.35 Implement a class, SubstitutionCipher, with a constructor that takes a string with the 26 uppercase letters in an arbitrary order and uses that for the forward mapping for encryption (akin to the self. forward string in our CaesarCipher class of Code Fragment 5.11). You should derive the backward mapping from the forward version. P-5.36 Redesign the CaesarCipher class as a subclass of the SubstitutionCipher from the previous problem. P-5.37 Design a RandomCipher class as a subclass of the SubstitutionCipher from Exercise P-5.35, so that each instance of the class relies on a random permutation of letters for its mapping. Chapter Notes The fundamental data structures of arrays belong to the folklore of computer science. They were first chronicled in the computer science literature by Knuth in his seminal book on Fundamental Algorithms [64].
- 250. Chapter 6 Stacks, Queues,and Deques Contents 6.1 Stacks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229 6.1.1 The Stack Abstract Data Type . . . . . . . . . . . . . . . 230 6.1.2 Simple Array-Based Stack Implementation . . . . . . . . . 231 6.1.3 Reversing Data Using a Stack . . . . . . . . . . . . . . . 235 6.1.4 Matching Parentheses and HTML Tags . . . . . . . . . . 236 6.2 Queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 6.2.1 The Queue Abstract Data Type . . . . . . . . . . . . . . 240 6.2.2 Array-Based Queue Implementation . . . . . . . . . . . . 241 6.3 Double-Ended Queues . . . . . . . . . . . . . . . . . . . . . 247 6.3.1 The Deque Abstract Data Type . . . . . . . . . . . . . . 247 6.3.2 Implementing a Deque with a Circular Array . . . . . . . . 248 6.3.3 Deques in the Python Collections Module . . . . . . . . . 249 6.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 250
- 251. 6.1. Stacks 229 6.1Stacks A stack is a collection of objects that are inserted and removed according to the last-in, first-out (LIFO) principle. A user may insert objects into a stack at any time, but may only access or remove the most recently inserted object that remains (at the so-called “top” of the stack). The name “stack” is derived from the metaphor of a stack of plates in a spring-loaded, cafeteria plate dispenser. In this case, the fundamental operations involve the “pushing” and “popping” of plates on the stack. When we need a new plate from the dispenser, we “pop” the top plate off the stack, and when we add a plate, we “push” it down on the stack to become the new top plate. Perhaps an even more amusing example is a PEZ® candy dispenser, which stores mint candies in a spring-loaded container that “pops” out the topmost candy in the stack when the top of the dispenser is lifted (see Figure 6.1). Stacks are a fundamental data structure. They are used in many applications, including the following. Example 6.1: Internet Web browsers store the addresses of recently visited sites in a stack. Each time a user visits a new site, that site’s address is “pushed” onto the stack of addresses. The browser then allows the user to “pop” back to previously visited sites using the “back” button. Example 6.2: Text editors usually provide an “undo” mechanism that cancels re- cent editing operations and reverts to former states of a document. This undo oper- ation can be accomplished by keeping text changes in a stack. Figure 6.1: A schematic drawing of a PEZ® dispenser; a physical implementation of the stack ADT. (PEZ® is a registered trademark of PEZ Candy, Inc.)
- 252. 230 Chapter 6.Stacks, Queues, and Deques 6.1.1 The Stack Abstract Data Type Stacks are the simplest of all data structures, yet they are also among the most important. They are used in a host of different applications, and as a tool for many more sophisticated data structures and algorithms. Formally, a stack is an abstract data type (ADT) such that an instance S supports the following two methods: S.push(e): Add element e to the top of stack S. S.pop(): Remove and return the top element from the stack S; an error occurs if the stack is empty. Additionally, let us define the following accessor methods for convenience: S.top(): Return a reference to the top element of stack S, without removing it; an error occurs if the stack is empty. S.is empty(): Return True if stack S does not contain any elements. len(S): Return the number of elements in stack S; in Python, we implement this with the special method len . By convention, we assume that a newly created stack is empty, and that there is no a priori bound on the capacity of the stack. Elements added to the stack can have arbitrary type. Example 6.3: The following table shows a series of stack operations and their effects on an initially empty stack S of integers. Operation Return Value Stack Contents S.push(5) – [5] S.push(3) – [5, 3] len(S) 2 [5, 3] S.pop() 3 [5] S.is empty() False [5] S.pop() 5 [ ] S.is empty() True [ ] S.pop() “error” [ ] S.push(7) – [7] S.push(9) – [7, 9] S.top() 9 [7, 9] S.push(4) – [7, 9, 4] len(S) 3 [7, 9, 4] S.pop() 4 [7, 9] S.push(6) – [7, 9, 6] S.push(8) – [7, 9, 6, 8] S.pop() 8 [7, 9, 6]
- 253. 6.1. Stacks 231 6.1.2Simple Array-Based Stack Implementation We can implement a stack quite easily by storing its elements in a Python list. The list class already supports adding an element to the end with the append method, and removing the last element with the pop method, so it is natural to align the top of the stack at the end of the list, as shown in Figure 6.2. 0 M B C D E F G K L A 1 2 top Figure 6.2: Implementing a stack with a Python list, storing the top element in the rightmost cell. Although a programmer could directly use the list class in place of a formal stack class, lists also include behaviors (e.g., adding or removing elements from arbitrary positions) that would break the abstraction that the stack ADT represents. Also, the terminology used by the list class does not precisely align with traditional nomenclature for a stack ADT, in particular the distinction between append and push. Instead, we demonstrate how to use a list for internal storage while providing a public interface consistent with a stack. The Adapter Pattern The adapter design pattern applies to any context where we effectively want to modify an existing class so that its methods match those of a related, but different, class or interface. One general way to apply the adapter pattern is to define a new class in such a way that it contains an instance of the existing class as a hidden field, and then to implement each method of the new class using methods of this hidden instance variable. By applying the adapter pattern in this way, we have created a new class that performs some of the same functions as an existing class, but repackaged in a more convenient way. In the context of the stack ADT, we can adapt Python’s list class using the correspondences shown in Table 6.1. Stack Method Realization with Python list S.push(e) L.append(e) S.pop() L.pop() S.top() L[−1] S.is empty() len(L) == 0 len(S) len(L) Table 6.1: Realization of a stack S as an adaptation of a Python list L.
- 254. 232 Chapter 6.Stacks, Queues, and Deques Implementing a Stack Using a Python List We use the adapter design pattern to define an ArrayStack class that uses an un- derlying Python list for storage. (We choose the name ArrayStack to emphasize that the underlying storage is inherently array based.) One question that remains is what our code should do if a user calls pop or top when the stack is empty. Our ADT suggests that an error occurs, but we must decide what type of error. When pop is called on an empty Python list, it formally raises an IndexError, as lists are index-based sequences. That choice does not seem appropriate for a stack, since there is no assumption of indices. Instead, we can define a new exception class that is more appropriate. Code Fragment 6.1 defines such an Empty class as a trivial subclass of the Python Exception class. class Empty(Exception): ”””Error attempting to access an element from an empty container.””” pass Code Fragment 6.1: Definition for an Empty exception class. The formal definition for our ArrayStack class is given in Code Fragment 6.2. The constructor establishes the member self. data as an initially empty Python list, for internal storage. The rest of the public stack behaviors are implemented, using the corresponding adaptation that was outlined in Table 6.1. Example Usage Below, we present an example of the use of our ArrayStack class, mirroring the operations at the beginning of Example 6.3 on page 230. S = ArrayStack( ) # contents: [ ] S.push(5) # contents: [5] S.push(3) # contents: [5, 3] print(len(S)) # contents: [5, 3]; outputs 2 print(S.pop()) # contents: [5]; outputs 3 print(S.is empty()) # contents: [5]; outputs False print(S.pop()) # contents: [ ]; outputs 5 print(S.is empty()) # contents: [ ]; outputs True S.push(7) # contents: [7] S.push(9) # contents: [7, 9] print(S.top()) # contents: [7, 9]; outputs 9 S.push(4) # contents: [7, 9, 4] print(len(S)) # contents: [7, 9, 4]; outputs 3 print(S.pop()) # contents: [7, 9]; outputs 4 S.push(6) # contents: [7, 9, 6]
- 255. 6.1. Stacks 233 1class ArrayStack: 2 ”””LIFO Stack implementation using a Python list as underlying storage.””” 3 4 def init (self): 5 ”””Create an empty stack.””” 6 self. data = [ ] # nonpublic list instance 7 8 def len (self): 9 ”””Return the number of elements in the stack.””” 10 return len(self. data) 11 12 def is empty(self): 13 ”””Return True if the stack is empty.””” 14 return len(self. data) == 0 15 16 def push(self, e): 17 ”””Add element e to the top of the stack.””” 18 self. data.append(e) # new item stored at end of list 19 20 def top(self): 21 ”””Return (but do not remove) the element at the top of the stack. 22 23 Raise Empty exception if the stack is empty. 24 ””” 25 if self.is empty(): 26 raise Empty( Stack is empty ) 27 return self. data[−1] # the last item in the list 28 29 def pop(self): 30 ”””Remove and return the element from the top of the stack (i.e., LIFO). 31 32 Raise Empty exception if the stack is empty. 33 ””” 34 if self.is empty(): 35 raise Empty( Stack is empty ) 36 return self. data.pop( ) # remove last item from list Code Fragment 6.2: Implementing a stack using a Python list as storage.
- 256. 234 Chapter 6.Stacks, Queues, and Deques Analyzing the Array-Based Stack Implementation Table 6.2 shows the running times for our ArrayStack methods. The analysis di- rectly mirrors the analysis of the list class given in Section 5.3. The implementa- tions for top, is empty, and len use constant time in the worst case. The O(1) time for push and pop are amortized bounds (see Section 5.3.2); a typical call to either of these methods uses constant time, but there is occasionally an O(n)-time worst case, where n is the current number of elements in the stack, when an operation causes the list to resize its internal array. The space usage for a stack is O(n). Operation Running Time S.push(e) O(1)∗ S.pop() O(1)∗ S.top() O(1) S.is empty() O(1) len(S) O(1) ∗amortized Table 6.2: Performance of our array-based stack implementation. The bounds for push and pop are amortized due to similar bounds for the list class. The space usage is O(n), where n is the current number of elements in the stack. Avoiding Amortization by Reserving Capacity In some contexts, there may be additional knowledge that suggests a maximum size that a stack will reach. Our implementation of ArrayStack from Code Fragment 6.2 begins with an empty list and expands as needed. In the analysis of lists from Section 5.4.1, we emphasized that it is more efficient in practice to construct a list with initial length n than it is to start with an empty list and append n items (even though both approaches run in O(n) time). As an alternate model for a stack, we might wish for the constructor to accept a parameter specifying the maximum capacity of a stack and to initialize the data member to a list of that length. Implementing such a model requires significant changes relative to Code Fragment 6.2. The size of the stack would no longer be synonymous with the length of the list, and pushes and pops of the stack would not require changing the length of the list. Instead, we suggest maintaining a separate integer as an instance variable that denotes the current number of elements in the stack. Details of such an implementation are left as Exercise C-6.17.
- 257. 6.1. Stacks 235 6.1.3Reversing Data Using a Stack As a consequence of the LIFO protocol, a stack can be used as a general tool to reverse a data sequence. For example, if the values 1, 2, and 3 are pushed onto a stack in that order, they will be popped from the stack in the order 3, 2, and then 1. This idea can be applied in a variety of settings. For example, we might wish to print lines of a file in reverse order in order to display a data set in decreasing order rather than increasing order. This can be accomplished by reading each line and pushing it onto a stack, and then writing the lines in the order they are popped. An implementation of such a process is given in Code Fragment 6.3. 1 def reverse file(filename): 2 ”””Overwrite given file with its contents line-by-line reversed.””” 3 S = ArrayStack() 4 original = open(filename) 5 for line in original: 6 S.push(line.rstrip( n )) # we will re-insert newlines when writing 7 original.close() 8 9 # now we overwrite with contents in LIFO order 10 output = open(filename, w ) # reopening file overwrites original 11 while not S.is empty(): 12 output.write(S.pop( ) + n ) # re-insert newline characters 13 output.close() Code Fragment 6.3: A function that reverses the order of lines in a file. One technical detail worth noting is that we intentionally strip trailing newlines from lines as they are read, and then re-insert newlines after each line when writing the resulting file. Our reason for doing this is to handle a special case in which the original file does not have a trailing newline for the final line. If we exactly echoed the lines read from the file in reverse order, then the original last line would be fol- lowed (without newline) by the original second-to-last line. In our implementation, we ensure that there will be a separating newline in the result. The idea of using a stack to reverse a data set can be applied to other types of sequences. For example, Exercise R-6.5 explores the use of a stack to provide yet another solution for reversing the contents of a Python list (a recursive solution for this goal was discussed in Section 4.4.1). A more challenging task is to reverse the order in which elements are stored within a stack. If we were to move them from one stack to another, they would be reversed, but if we were to then replace them into the original stack, they would be reversed again, thereby reverting to their original order. Exercise C-6.18 explores a solution for this task.
- 258. 236 Chapter 6.Stacks, Queues, and Deques 6.1.4 Matching Parentheses and HTML Tags In this subsection, we explore two related applications of stacks, both of which involve testing for pairs of matching delimiters. In our first application, we consider arithmetic expressions that may contain various pairs of grouping symbols, such as • Parentheses: “(” and “)” • Braces: “{” and “}” • Brackets: “[” and “]” Each opening symbol must match its corresponding closing symbol. For example, a left bracket, “[,” must match a corresponding right bracket, “],” as in the expression [(5+x)-(y+z)]. The following examples further illustrate this concept: • Correct: ()(()){([()])} • Correct: ((()(()){([()])})) • Incorrect: )(()){([()])} • Incorrect: ({[])} • Incorrect: ( We leave the precise definition of a matching group of symbols to Exercise R-6.6. An Algorithm for Matching Delimiters An important task when processing arithmetic expressions is to make sure their delimiting symbols match up correctly. Code Fragment 6.4 presents a Python im- plementation of such an algorithm. A discussion of the code follows. 1 def is matched(expr): 2 ”””Return True if all delimiters are properly match; False otherwise.””” 3 lefty = ({[ # opening delimiters 4 righty = )}] # respective closing delims 5 S = ArrayStack() 6 for c in expr: 7 if c in lefty: 8 S.push(c) # push left delimiter on stack 9 elif c in righty: 10 if S.is empty(): 11 return False # nothing to match with 12 if righty.index(c) != lefty.index(S.pop()): 13 return False # mismatched 14 return S.is empty( ) # were all symbols matched? Code Fragment 6.4: Function for matching delimiters in an arithmetic expression.
- 259. 6.1. Stacks 237 Weassume the input is a sequence of characters, such as [(5+x)-(y+z)] . We perform a left-to-right scan of the original sequence, using a stack S to facilitate the matching of grouping symbols. Each time we encounter an opening symbol, we push that symbol onto S, and each time we encounter a closing symbol, we pop a symbol from the stack S (assuming S is not empty), and check that these two symbols form a valid pair. If we reach the end of the expression and the stack is empty, then the original expression was properly matched. Otherwise, there must be an opening delimiter on the stack without a matching symbol. If the length of the original expression is n, the algorithm will make at most n calls to push and n calls to pop. Those calls run in a total of O(n) time, even con- sidering the amortized nature of the O(1) time bound for those methods. Given that our selection of possible delimiters, ({[, has constant size, auxiliary tests such as c in lefty and righty.index(c) each run in O(1) time. Combining these operations, the matching algorithm on a sequence of length n runs in O(n) time. Matching Tags in a Markup Language Another application of matching delimiters is in the validation of markup languages such as HTML or XML. HTML is the standard format for hyperlinked documents on the Internet and XML is an extensible markup language used for a variety of structured data sets. We show a sample HTML document and a possible rendering in Figure 6.3. body center h1 The Little Boat /h1 /center p The storm tossed the little boat like a cheap sneaker in an old washing machine. The three drunken fishermen were used to such treatment, of course, but not the tree salesman, who even as a stowaway now felt that he had overpaid for the voyage. /p ol li Will the salesman die? /li li What color is the boat? /li li And what about Naomi? /li /ol /body The Little Boat The storm tossed the little boat like a cheap sneaker in an old washing machine. The three drunken fishermen were used to such treatment, of course, but not the tree salesman, who even as a stowaway now felt that he had overpaid for the voyage. 1. Will the salesman die? 2. What color is the boat? 3. And what about Naomi? (a) (b) Figure 6.3: Illustrating HTML tags. (a) An HTML document; (b) its rendering.
- 260. 238 Chapter 6.Stacks, Queues, and Deques In an HTML document, portions of text are delimited by HTML tags. A simple opening HTML tag has the form “name” and the corresponding closing tag has the form “/name”. For example, we see the body tag on the first line of Figure 6.3(a), and the matching /body tag at the close of that document. Other commonly used HTML tags that are used in this example include: • body: document body • h1: section header • center: center justify • p: paragraph • ol: numbered (ordered) list • li: list item Ideally, an HTML document should have matching tags, although most browsers tolerate a certain number of mismatching tags. In Code Fragment 6.5, we give a Python function that matches tags in a string representing an HTML document. We make a left-to-right pass through the raw string, using index j to track our progress and the find method of the str class to locate the and characters that define the tags. Opening tags are pushed onto the stack, and matched against closing tags as they are popped from the stack, just as we did when matching delimiters in Code Fragment 6.4. By similar analysis, this algorithm runs in O(n) time, where n is the number of characters in the raw HTML source. 1 def is matched html(raw): 2 ”””Return True if all HTML tags are properly match; False otherwise.””” 3 S = ArrayStack() 4 j = raw.find( ) # find first ’’ character (if any) 5 while j != −1: 6 k = raw.find( , j+1) # find next ’’ character 7 if k == −1: 8 return False # invalid tag 9 tag = raw[j+1:k] # strip away 10 if not tag.startswith( / ): # this is opening tag 11 S.push(tag) 12 else: # this is closing tag 13 if S.is empty(): 14 return False # nothing to match with 15 if tag[1:] != S.pop(): 16 return False # mismatched delimiter 17 j = raw.find( , k+1) # find next ’’ character (if any) 18 return S.is empty( ) # were all opening tags matched? Code Fragment 6.5: Function for testing if an HTML document has matching tags.
- 261. 6.2. Queues 239 6.2Queues Another fundamental data structure is the queue. It is a close “cousin” of the stack, as a queue is a collection of objects that are inserted and removed according to the first-in, first-out (FIFO) principle. That is, elements can be inserted at any time, but only the element that has been in the queue the longest can be next removed. We usually say that elements enter a queue at the back and are removed from the front. A metaphor for this terminology is a line of people waiting to get on an amusement park ride. People waiting for such a ride enter at the back of the line and get on the ride from the front of the line. There are many other applications of queues (see Figure 6.4). Stores, theaters, reservation centers, and other similar services typically process customer requests according to the FIFO principle. A queue would therefore be a logical choice for a data structure to handle calls to a customer service center, or a wait-list at a restaurant. FIFO queues are also used by many computing devices, such as a networked printer, or a Web server responding to requests. Tickets (a) Call Center Call Queue (b) Figure 6.4: Real-world examples of a first-in, first-out queue. (a) People waiting in line to purchase tickets; (b) phone calls being routed to a customer service center.
- 262. 240 Chapter 6.Stacks, Queues, and Deques 6.2.1 The Queue Abstract Data Type Formally, the queue abstract data type defines a collection that keeps objects in a sequence, where element access and deletion are restricted to the first element in the queue, and element insertion is restricted to the back of the sequence. This restriction enforces the rule that items are inserted and deleted in a queue accord- ing to the first-in, first-out (FIFO) principle. The queue abstract data type (ADT) supports the following two fundamental methods for a queue Q: Q.enqueue(e): Add element e to the back of queue Q. Q.dequeue(): Remove and return the first element from queue Q; an error occurs if the queue is empty. The queue ADT also includes the following supporting methods (with first being analogous to the stack’s top method): Q.first(): Return a reference to the element at the front of queue Q, without removing it; an error occurs if the queue is empty. Q.is empty(): Return True if queue Q does not contain any elements. len(Q): Return the number of elements in queue Q; in Python, we implement this with the special method len . By convention, we assume that a newly created queue is empty, and that there is no a priori bound on the capacity of the queue. Elements added to the queue can have arbitrary type. Example 6.4: The following table shows a series of queue operations and their effects on an initially empty queue Q of integers. Operation Return Value first ← Q ← last Q.enqueue(5) – [5] Q.enqueue(3) – [5, 3] len(Q) 2 [5, 3] Q.dequeue() 5 [3] Q.is empty() False [3] Q.dequeue() 3 [ ] Q.is empty() True [ ] Q.dequeue() “error” [ ] Q.enqueue(7) – [7] Q.enqueue(9) – [7, 9] Q.first() 7 [7, 9] Q.enqueue(4) – [7, 9, 4] len(Q) 3 [7, 9, 4] Q.dequeue() 7 [9, 4]
- 263. 6.2. Queues 241 6.2.2Array-Based Queue Implementation For the stack ADT, we created a very simple adapter class that used a Python list as the underlying storage. It may be very tempting to use a similar approach for supporting the queue ADT. We could enqueue element e by calling append(e) to add it to the end of the list. We could use the syntax pop(0), as opposed to pop(), to intentionally remove the first element from the list when dequeuing. As easy as this would be to implement, it is tragically inefficient. As we dis- cussed in Section 5.4.1, when pop is called on a list with a non-default index, a loop is executed to shift all elements beyond the specified index to the left, so as to fill the hole in the sequence caused by the pop. Therefore, a call to pop(0) always causes the worst-case behavior of Θ(n) time. We can improve on the above strategy by avoiding the call to pop(0) entirely. We can replace the dequeued entry in the array with a reference to None, and main- tain an explicit variable f to store the index of the element that is currently at the front of the queue. Such an algorithm for dequeue would run in O(1) time. After several dequeue operations, this approach might lead to the configuration portrayed in Figure 6.5. 0 E F G K L M 1 2 f Figure 6.5: Allowing the front of the queue to drift away from index 0. Unfortunately, there remains a drawback to the revised approach. In the case of a stack, the length of the list was precisely equal to the size of the stack (even if the underlying array for the list was slightly larger). With the queue design that we are considering, the situation is worse. We can build a queue that has relatively few elements, yet which are stored in an arbitrarily large list. This occurs, for example, if we repeatedly enqueue a new element and then dequeue another (allowing the front to drift rightward). Over time, the size of the underlying list would grow to O(m) where m is the total number of enqueue operations since the creation of the queue, rather than the current number of elements in the queue. This design would have detrimental consequences in applications in which queues have relatively modest size, but which are used for long periods of time. For example, the wait-list for a restaurant might never have more than 30 entries at one time, but over the course of a day (or a week), the overall number of entries would be significantly larger.
- 264. 242 Chapter 6.Stacks, Queues, and Deques Using an Array Circularly In developing a more robust queue implementation, we allow the front of the queue to drift rightward, and we allow the contents of the queue to “wrap around” the end of an underlying array. We assume that our underlying array has fixed length N that is greater that the actual number of elements in the queue. New elements are enqueued toward the “end” of the current queue, progressing from the front to index N − 1 and continuing at index 0, then 1. Figure 6.6 illustrates such a queue with first element E and last element M. 0 M F G H I J K L E 1 2 f N −1 Figure 6.6: Modeling a queue with a circular array that wraps around the end. Implementing this circular view is not difficult. When we dequeue an element and want to “advance” the front index, we use the arithmetic f = (f + 1) % N. Re- call that the % operator in Python denotes the modulo operator, which is computed by taking the remainder after an integral division. For example, 14 divided by 3 has a quotient of 4 with remainder 2, that is, 14 3 = 42 3 . So in Python, 14 // 3 evaluates to the quotient 4, while 14 % 3 evaluates to the remainder 2. The modulo operator is ideal for treating an array circularly. As a concrete example, if we have a list of length 10, and a front index 7, we can advance the front by formally computing (7+1) % 10, which is simply 8, as 8 divided by 10 is 0 with a remainder of 8. Similarly, advancing index 8 results in index 9. But when we advance from index 9 (the last one in the array), we compute (9+1) % 10, which evaluates to index 0 (as 10 divided by 10 has a remainder of zero). A Python Queue Implementation A complete implementation of a queue ADT using a Python list in circular fashion is presented in Code Fragments 6.6 and 6.7. Internally, the queue class maintains the following three instance variables: data: is a reference to a list instance with a fixed capacity. size: is an integer representing the current number of elements stored in the queue (as opposed to the length of the data list). front: is an integer that represents the index within data of the first element of the queue (assuming the queue is not empty). We initially reserve a list of moderate size for storing data, although the queue formally has size zero. As a technicality, we initialize the front index to zero. When front or dequeue are called with no elements in the queue, we raise an instance of the Empty exception, defined in Code Fragment 6.1 for our stack.
- 265. 6.2. Queues 243 1class ArrayQueue: 2 ”””FIFO queue implementation using a Python list as underlying storage.””” 3 DEFAULT CAPACITY = 10 # moderate capacity for all new queues 4 5 def init (self): 6 ”””Create an empty queue.””” 7 self. data = [None] ArrayQueue.DEFAULT CAPACITY 8 self. size = 0 9 self. front = 0 10 11 def len (self): 12 ”””Return the number of elements in the queue.””” 13 return self. size 14 15 def is empty(self): 16 ”””Return True if the queue is empty.””” 17 return self. size == 0 18 19 def first(self): 20 ”””Return (but do not remove) the element at the front of the queue. 21 22 Raise Empty exception if the queue is empty. 23 ””” 24 if self.is empty(): 25 raise Empty( Queue is empty ) 26 return self. data[self. front] 27 28 def dequeue(self): 29 ”””Remove and return the first element of the queue (i.e., FIFO). 30 31 Raise Empty exception if the queue is empty. 32 ””” 33 if self.is empty(): 34 raise Empty( Queue is empty ) 35 answer = self. data[self. front] 36 self. data[self. front] = None # help garbage collection 37 self. front = (self. front + 1) % len(self. data) 38 self. size −= 1 39 return answer Code Fragment 6.6: Array-based implementation of a queue (continued in Code Fragment 6.7).
- 266. 244 Chapter 6.Stacks, Queues, and Deques 40 def enqueue(self, e): 41 ”””Add an element to the back of queue.””” 42 if self. size == len(self. data): 43 self. resize(2 len(self.data)) # double the array size 44 avail = (self. front + self. size) % len(self. data) 45 self. data[avail] = e 46 self. size += 1 47 48 def resize(self, cap): # we assume cap = len(self) 49 ”””Resize to a new list of capacity = len(self).””” 50 old = self. data # keep track of existing list 51 self. data = [None] cap # allocate list with new capacity 52 walk = self. front 53 for k in range(self. size): # only consider existing elements 54 self. data[k] = old[walk] # intentionally shift indices 55 walk = (1 + walk) % len(old) # use old size as modulus 56 self. front = 0 # front has been realigned Code Fragment 6.7: Array-based implementation of a queue (continued from Code Fragment 6.6). The implementation of len and is empty are trivial, given knowledge of the size. The implementation of the front method is also simple, as the front index tells us precisely where the desired element is located within the data list, assuming that list is not empty. Adding and Removing Elements The goal of the enqueue method is to add a new element to the back of the queue. We need to determine the proper index at which to place the new element. Although we do not explicitly maintain an instance variable for the back of the queue, we compute the location of the next opening based on the formula: avail = (self. front + self. size) % len(self. data) Note that we are using the size of the queue as it exists prior to the addition of the new element. For example, consider a queue with capacity 10, current size 3, and first element at index 5. The three elements of such a queue are stored at indices 5, 6, and 7. The new element should be placed at index (front + size) = 8. In a case with wrap-around, the use of the modular arithmetic achieves the desired circular semantics. For example, if our hypothetical queue had 3 elements with the first at index 8, our computation of (8+3) % 10 evaluates to 1, which is perfect since the three existing elements occupy indices 8, 9, and 0.
- 267. 6.2. Queues 245 Whenthe dequeue method is called, the current value of self. front designates the index of the value that is to be removed and returned. We keep a local refer- ence to the element that will be returned, setting answer = self. data[self. front] just prior to removing the reference to that object from the list, with the assignment self. data[self. front] = None. Our reason for the assignment to None relates to Python’s mechanism for reclaiming unused space. Internally, Python maintains a count of the number of references that exist to each object. If that count reaches zero, the object is effectively inaccessible, thus the system may reclaim that mem- ory for future use. (For more details, see Section 15.1.2.) Since we are no longer responsible for storing a dequeued element, we remove the reference to it from our list so as to reduce that element’s reference count. The second significant responsibility of the dequeue method is to update the value of front to reflect the removal of the element, and the presumed promotion of the second element to become the new first. In most cases, we simply want to increment the index by one, but because of the possibility of a wrap-around configuration, we rely on modular arithmetic as originally described on page 242. Resizing the Queue When enqueue is called at a time when the size of the queue equals the size of the underlying list, we rely on a standard technique of doubling the storage capacity of the underlying list. In this way, our approach is similar to the one used when we implemented a DynamicArray in Section 5.3.1. However, more care is needed in the queue’s resize utility than was needed in the corresponding method of the DynamicArray class. After creating a temporary reference to the old list of values, we allocate a new list that is twice the size and copy references from the old list to the new list. While transferring the contents, we intentionally realign the front of the queue with index 0 in the new array, as shown in Figure 6.7. This realignment is not purely cosmetic. Since the modular arith- metic depends on the size of the array, our state would be flawed had we transferred each element to its same index in the new array. E G H I J K E F G H I J K F f 1 2 f = 0 Figure 6.7: Resizing the queue, while realigning the front element with index 0.
- 268. 246 Chapter 6.Stacks, Queues, and Deques Shrinking the Underlying Array A desirable property of a queue implementation is to have its space usage be Θ(n) where n is the current number of elements in the queue. Our ArrayQueue imple- mentation, as given in Code Fragments 6.6 and 6.7, does not have this property. It expands the underlying array when enqueue is called with the queue at full ca- pacity, but the dequeue implementation never shrinks the underlying array. As a consequence, the capacity of the underlying array is proportional to the maximum number of elements that have ever been stored in the queue, not the current number of elements. We discussed this very issue on page 200, in the context of dynamic arrays, and in subsequent Exercises C-5.16 through C-5.20 of that chapter. A robust approach is to reduce the array to half of its current size, whenever the number of elements stored in it falls below one fourth of its capacity. We can implement this strategy by adding the following two lines of code in our dequeue method, just after reducing self. size at line 38 of Code Fragment 6.6, to reflect the loss of an element. if 0 self. size len(self. data) // 4: self. resize(len(self. data) // 2) Analyzing the Array-Based Queue Implementation Table 6.3 describes the performance of our array-based implementation of the queue ADT, assuming the improvement described above for occasionally shrinking the size of the array. With the exception of the resize utility, all of the methods rely on a constant number of statements involving arithmetic operations, comparisons, and assignments. Therefore, each method runs in worst-case O(1) time, except for enqueue and dequeue, which have amortized bounds of O(1) time, for reasons similar to those given in Section 5.3. Operation Running Time Q.enqueue(e) O(1)∗ Q.dequeue() O(1)∗ Q.first() O(1) Q.is empty() O(1) len(Q) O(1) ∗amortized Table 6.3: Performance of an array-based implementation of a queue. The bounds for enqueue and dequeue are amortized due to the resizing of the array. The space usage is O(n), where n is the current number of elements in the queue.
- 269. 6.3. Double-Ended Queues247 6.3 Double-Ended Queues We next consider a queue-like data structure that supports insertion and deletion at both the front and the back of the queue. Such a structure is called a double- ended queue, or deque, which is usually pronounced “deck” to avoid confusion with the dequeue method of the regular queue ADT, which is pronounced like the abbreviation “D.Q.” The deque abstract data type is more general than both the stack and the queue ADTs. The extra generality can be useful in some applications. For example, we described a restaurant using a queue to maintain a waitlist. Occassionally, the first person might be removed from the queue only to find that a table was not available; typically, the restaurant will re-insert the person at the first position in the queue. It may also be that a customer at the end of the queue may grow impatient and leave the restaurant. (We will need an even more general data structure if we want to model customers leaving the queue from other positions.) 6.3.1 The Deque Abstract Data Type To provide a symmetrical abstraction, the deque ADT is defined so that deque D supports the following methods: D.add first(e): Add element e to the front of deque D. D.add last(e): Add element e to the back of deque D. D.delete first(): Remove and return the first element from deque D; an error occurs if the deque is empty. D.delete last(): Remove and return the last element from deque D; an error occurs if the deque is empty. Additionally, the deque ADT will include the following accessors: D.first(): Return (but do not remove) the first element of deque D; an error occurs if the deque is empty. D.last(): Return (but do not remove) the last element of deque D; an error occurs if the deque is empty. D.is empty(): Return True if deque D does not contain any elements. len(D): Return the number of elements in deque D; in Python, we implement this with the special method len .
- 270. 248 Chapter 6.Stacks, Queues, and Deques Example 6.5: The following table shows a series of operations and their effects on an initially empty deque D of integers. Operation Return Value Deque D.add last(5) – [5] D.add first(3) – [3, 5] D.add first(7) – [7, 3, 5] D.first() 7 [7, 3, 5] D.delete last() 5 [7, 3] len(D) 2 [7, 3] D.delete last() 3 [7] D.delete last() 7 [ ] D.add first(6) – [6] D.last() 6 [6] D.add first(8) – [8, 6] D.is empty() False [8, 6] D.last() 6 [8, 6] 6.3.2 Implementing a Deque with a Circular Array We can implement the deque ADT in much the same way as the ArrayQueue class provided in Code Fragments 6.6 and 6.7 of Section 6.2.2 (so much so that we leave the details of an ArrayDeque implementation to Exercise P-6.32). We recommend maintaining the same three instance variables: data, size, and front. Whenever we need to know the index of the back of the deque, or the first available slot beyond the back of the deque, we use modular arithmetic for the computation. For example, our implementation of the last() method uses the index back = (self. front + self. size − 1) % len(self. data) Our implementation of the ArrayDeque.add last method is essentially the same as that for ArrayQueue.enqueue, including the reliance on a resize utility. Like- wise, the implementation of the ArrayDeque.delete first method is the same as ArrayQueue.dequeue. Implementations of add first and delete last use similar techniques. One subtlety is that a call to add first may need to wrap around the beginning of the array, so we rely on modular arithmetic to circularly decrement the index, as self. front = (self. front − 1) % len(self. data) # cyclic shift The efficiency of an ArrayDeque is similar to that of an ArrayQueue, with all operations having O(1) running time, but with that bound being amortized for op- erations that may change the size of the underlying list.
- 271. 6.3. Double-Ended Queues249 6.3.3 Deques in the Python Collections Module An implementation of a deque class is available in Python’s standard collections module. A summary of the most commonly used behaviors of the collections.deque class is given in Table 6.4. It uses more asymmetric nomenclature than our ADT. Our Deque ADT collections.deque Description len(D) len(D) number of elements D.add first() D.appendleft() add to beginning D.add last() D.append() add to end D.delete first() D.popleft() remove from beginning D.delete last() D.pop() remove from end D.first() D[0] access first element D.last() D[−1] access last element D[j] access arbitrary entry by index D[j] = val modify arbitrary entry by index D.clear() clear all contents D.rotate(k) circularly shift rightward k steps D.remove(e) remove first matching element D.count(e) count number of matches for e Table 6.4: Comparison of our deque ADT and the collections.deque class. The collections.deque interface was chosen to be consistent with established naming conventions of Python’s list class, for which append and pop are presumed to act at the end of the list. Therefore, appendleft and popleft designate an opera- tion at the beginning of the list. The library deque also mimics a list in that it is an indexed sequence, allowing arbitrary access or modification using the D[j] syntax. The library deque constructor also supports an optional maxlen parameter to force a fixed-length deque. However, if a call to append at either end is invoked when the deque is full, it does not throw an error; instead, it causes one element to be dropped from the opposite side. That is, calling appendleft when the deque is full causes an implicit pop from the right side to make room for the new element. The current Python distribution implements collections.deque with a hybrid ap- proach that uses aspects of circular arrays, but organized into blocks that are them- selves organized in a doubly linked list (a data structure that we will introduce in the next chapter). The deque class is formally documented to guarantee O(1)-time operations at either end, but O(n)-time worst-case operations when using index notation near the middle of the deque.
- 272. 250 Chapter 6.Stacks, Queues, and Deques 6.4 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-6.1 What values are returned during the following series of stack operations, if executed upon an initially empty stack? push(5), push(3), pop(), push(2), push(8), pop(), pop(), push(9), push(1), pop(), push(7), push(6), pop(), pop(), push(4), pop(), pop(). R-6.2 Suppose an initially empty stack S has executed a total of 25 push opera- tions, 12 top operations, and 10 pop operations, 3 of which raised Empty errors that were caught and ignored. What is the current size of S? R-6.3 Implement a function with signature transfer(S, T) that transfers all ele- ments from stack S onto stack T, so that the element that starts at the top of S is the first to be inserted onto T, and the element at the bottom of S ends up at the top of T. R-6.4 Give a recursive method for removing all the elements from a stack. R-6.5 Implement a function that reverses a list of elements by pushing them onto a stack in one order, and writing them back to the list in reversed order. R-6.6 Give a precise and complete definition of the concept of matching for grouping symbols in an arithmetic expression. Your definition may be recursive. R-6.7 What values are returned during the following sequence of queue opera- tions, if executed on an initially empty queue? enqueue(5), enqueue(3), dequeue(), enqueue(2), enqueue(8), dequeue(), dequeue(), enqueue(9), enqueue(1), dequeue(), enqueue(7), enqueue(6), dequeue(), dequeue(), enqueue(4), dequeue(), dequeue(). R-6.8 Suppose an initially empty queue Q has executed a total of 32 enqueue operations, 10 first operations, and 15 dequeue operations, 5 of which raised Empty errors that were caught and ignored. What is the current size of Q? R-6.9 Had the queue of the previous problem been an instance of ArrayQueue that used an initial array of capacity 30, and had its size never been greater than 30, what would be the final value of the front instance variable? R-6.10 Consider what happens if the loop in the ArrayQueue. resize method at lines 53–55 of Code Fragment 6.7 had been implemented as: for k in range(self. size): self. data[k] = old[k] # rather than old[walk] Give a clear explanation of what could go wrong.
- 273. 6.4. Exercises 251 R-6.11Give a simple adapter that implements our queue ADT while using a collections.deque instance for storage. R-6.12 What values are returned during the following sequence of deque ADT op- erations, on initially empty deque? add first(4), add last(8), add last(9), add first(5), back(), delete first(), delete last(), add last(7), first(), last(), add last(6), delete first(), delete first(). R-6.13 Suppose you have a deque D containing the numbers (1,2,3,4,5,6,7,8), in this order. Suppose further that you have an initially empty queue Q. Give a code fragment that uses only D and Q (and no other variables) and results in D storing the elements in the order (1,2,3,5,4,6,7,8). R-6.14 Repeat the previous problem using the deque D and an initially empty stack S. Creativity C-6.15 Suppose Alice has picked three distinct integers and placed them into a stack S in random order. Write a short, straight-line piece of pseudo-code (with no loops or recursion) that uses only one comparison and only one variable x, yet that results in variable x storing the largest of Alice’s three integers with probability 2/3. Argue why your method is correct. C-6.16 Modify the ArrayStack implementation so that the stack’s capacity is lim- ited to maxlen elements, where maxlen is an optional parameter to the constructor (that defaults to None). If push is called when the stack is at full capacity, throw a Full exception (defined similarly to Empty). C-6.17 In the previous exercise, we assume that the underlying list is initially empty. Redo that exercise, this time preallocating an underlying list with length equal to the stack’s maximum capacity. C-6.18 Show how to use the transfer function, described in Exercise R-6.3, and two temporary stacks, to replace the contents of a given stack S with those same elements, but in reversed order. C-6.19 In Code Fragment 6.5 we assume that opening tags in HTML have form name, as with li. More generally, HTML allows optional attributes to be expressed as part of an opening tag. The general form used is name attribute1=value1 attribute2=value2; for example, a table can be given a border and additional padding by using an opening tag of table border=3 cellpadding=5. Modify Code Frag- ment 6.5 so that it can properly match tags, even when an opening tag may include one or more such attributes. C-6.20 Describe a nonrecursive algorithm for enumerating all permutations of the numbers {1,2,...,n} using an explicit stack.
- 274. 252 Chapter 6.Stacks, Queues, and Deques C-6.21 Show how to use a stack S and a queue Q to generate all possible subsets of an n-element set T nonrecursively. C-6.22 Postfix notation is an unambiguous way of writing an arithmetic expres- sion without parentheses. It is defined so that if “(exp1)op(exp2)” is a normal, fully parenthesized expression whose operation is op, the postfix version of this is “pexp1 pexp2 op”, where pexp1 is the postfix version of exp1 and pexp2 is the postfix version of exp2. The postfix version of a sin- gle number or variable is just that number or variable. For example, the postfix version of “((5+2)∗(8−3))/4” is “5 2 + 8 3 − ∗ 4 /”. Describe a nonrecursive way of evaluating an expression in postfix notation. C-6.23 Suppose you have three nonempty stacks R, S, and T. Describe a sequence of operations that results in S storing all elements originally in T below all of S’s original elements, with both sets of those elements in their original order. The final configuration for R should be the same as its original configuration. For example, if R = [1,2,3], S = [4,5], and T = [6,7,8,9], the final configuration should have R = [1,2,3] and S = [6,7,8,9,4,5]. C-6.24 Describe how to implement the stack ADT using a single queue as an instance variable, and only constant additional local memory within the method bodies. What is the running time of the push(), pop(), and top() methods for your design? C-6.25 Describe how to implement the queue ADT using two stacks as instance variables, such that all queue operations execute in amortized O(1) time. Give a formal proof of the amortized bound. C-6.26 Describe how to implement the double-ended queue ADT using two stacks as instance variables. What are the running times of the methods? C-6.27 Suppose you have a stack S containing n elements and a queue Q that is initially empty. Describe how you can use Q to scan S to see if it contains a certain element x, with the additional constraint that your algorithm must return the elements back to S in their original order. You may only use S, Q, and a constant number of other variables. C-6.28 Modify the ArrayQueue implementation so that the queue’s capacity is limited to maxlen elements, where maxlen is an optional parameter to the constructor (that defaults to None). If enqueue is called when the queue is at full capacity, throw a Full exception (defined similarly to Empty). C-6.29 In certain applications of the queue ADT, it is common to repeatedly dequeue an element, process it in some way, and then immediately en- queue the same element. Modify the ArrayQueue implementation to in- clude a rotate() method that has semantics identical to the combina- tion, Q.enqueue(Q.dequeue()). However, your implementation should be more efficient than making two separate calls (for example, because there is no need to modify size).
- 275. 6.4. Exercises 253 C-6.30Alice has two queues, Q and R, which can store integers. Bob gives Alice 50 odd integers and 50 even integers and insists that she store all 100 integers in Q and R. They then play a game where Bob picks Q or R at random and then applies the round-robin scheduler, described in the chapter, to the chosen queue a random number of times. If the last number to be processed at the end of this game was odd, Bob wins. Otherwise, Alice wins. How can Alice allocate integers to queues to optimize her chances of winning? What is her chance of winning? C-6.31 Suppose Bob has four cows that he wants to take across a bridge, but only one yoke, which can hold up to two cows, side by side, tied to the yoke. The yoke is too heavy for him to carry across the bridge, but he can tie (and untie) cows to it in no time at all. Of his four cows, Mazie can cross the bridge in 2 minutes, Daisy can cross it in 4 minutes, Crazy can cross it in 10 minutes, and Lazy can cross it in 20 minutes. Of course, when two cows are tied to the yoke, they must go at the speed of the slower cow. Describe how Bob can get all his cows across the bridge in 34 minutes. Projects P-6.32 Give a complete ArrayDeque implementation of the double-ended queue ADT as sketched in Section 6.3.2. P-6.33 Give an array-based implementation of a double-ended queue supporting all of the public behaviors shown in Table 6.4 for the collections.deque class, including use of the maxlen optional parameter. When a length- limited deque is full, provide semantics similar to the collections.deque class, whereby a call to insert an element on one end of a deque causes an element to be lost from the opposite side. P-6.34 Implement a program that can input an expression in postfix notation (see Exercise C-6.22) and output its value. P-6.35 The introduction of Section 6.1 notes that stacks are often used to provide “undo” support in applications like a Web browser or text editor. While support for undo can be implemented with an unbounded stack, many applications provide only limited support for such an undo history, with a fixed-capacity stack. When push is invoked with the stack at full capacity, rather than throwing a Full exception (as described in Exercise C-6.16), a more typical semantic is to accept the pushed element at the top while “leaking” the oldest element from the bottom of the stack to make room. Give an implementation of such a LeakyStack abstraction, using a circular array with appropriate storage capacity. This class should have a public interface similar to the bounded-capacity stack in Exercise C-6.16, but with the desired leaky semantics when full.
- 276. 254 Chapter 6.Stacks, Queues, and Deques P-6.36 When a share of common stock of some company is sold, the capital gain (or, sometimes, loss) is the difference between the share’s selling price and the price originally paid to buy it. This rule is easy to under- stand for a single share, but if we sell multiple shares of stock bought over a long period of time, then we must identify the shares actually be- ing sold. A standard accounting principle for identifying which shares of a stock were sold in such a case is to use a FIFO protocol—the shares sold are the ones that have been held the longest (indeed, this is the de- fault method built into several personal finance software packages). For example, suppose we buy 100 shares at $20 each on day 1, 20 shares at $24 on day 2, 200 shares at $36 on day 3, and then sell 150 shares on day 4 at $30 each. Then applying the FIFO protocol means that of the 150 shares sold, 100 were bought on day 1, 20 were bought on day 2, and 30 were bought on day 3. The capital gain in this case would therefore be 100·10+20·6+30·(−6), or $940. Write a program that takes as input a sequence of transactions of the form “buy x share(s) at y each” or “sell x share(s) at y each,” assuming that the transactions oc- cur on consecutive days and the values x and y are integers. Given this input sequence, the output should be the total capital gain (or loss) for the entire sequence, using the FIFO protocol to identify shares. P-6.37 Design an ADT for a two-color, double-stack ADT that consists of two stacks—one “red” and one “blue”—and has as its operations color-coded versions of the regular stack ADT operations. For example, this ADT should support both a red push operation and a blue push operation. Give an efficient implementation of this ADT using a single array whose ca- pacity is set at some value N that is assumed to always be larger than the sizes of the red and blue stacks combined. Chapter Notes We were introduced to the approach of defining data structures first in terms of their ADTs and then in terms of concrete implementations by the classic books by Aho, Hopcroft, and Ullman [5, 6]. Exercises C-6.30, and C-6.31 are similar to interview questions said to be from a well-known software company. For further study of abstract data types, see Liskov and Guttag [71], Cardelli and Wegner [23], or Demurjian [33].
- 277. Chapter 7 Linked Lists Contents 7.1Singly Linked Lists . . . . . . . . . . . . . . . . . . . . . . . 256 7.1.1 Implementing a Stack with a Singly Linked List . . . . . . 261 7.1.2 Implementing a Queue with a Singly Linked List . . . . . . 264 7.2 Circularly Linked Lists . . . . . . . . . . . . . . . . . . . . . 266 7.2.1 Round-Robin Schedulers . . . . . . . . . . . . . . . . . . 267 7.2.2 Implementing a Queue with a Circularly Linked List . . . . 268 7.3 Doubly Linked Lists . . . . . . . . . . . . . . . . . . . . . . 270 7.3.1 Basic Implementation of a Doubly Linked List . . . . . . . 273 7.3.2 Implementing a Deque with a Doubly Linked List . . . . . 275 7.4 The Positional List ADT . . . . . . . . . . . . . . . . . . . 277 7.4.1 The Positional List Abstract Data Type . . . . . . . . . . 279 7.4.2 Doubly Linked List Implementation . . . . . . . . . . . . . 281 7.5 Sorting a Positional List . . . . . . . . . . . . . . . . . . . . 285 7.6 Case Study: Maintaining Access Frequencies . . . . . . . . 286 7.6.1 Using a Sorted List . . . . . . . . . . . . . . . . . . . . . 286 7.6.2 Using a List with the Move-to-Front Heuristic . . . . . . . 289 7.7 Link-Based vs. Array-Based Sequences . . . . . . . . . . . 292 7.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294
- 278. 256 Chapter 7.Linked Lists In Chapter 5 we carefully examined Python’s array-based list class, and in Chapter 6 we demonstrated use of that class in implementing the classic stack, queue, and dequeue ADTs. Python’s list class is highly optimized, and often a great choice for storage. With that said, there are some notable disadvantages: 1. The length of a dynamic array might be longer than the actual number of elements that it stores. 2. Amortized bounds for operations may be unacceptable in real-time systems. 3. Insertions and deletions at interior positions of an array are expensive. In this chapter, we introduce a data structure known as a linked list, which provides an alternative to an array-based sequence (such as a Python list). Both array-based sequences and linked lists keep elements in a certain order, but us- ing a very different style. An array provides the more centralized representation, with one large chunk of memory capable of accommodating references to many elements. A linked list, in contrast, relies on a more distributed representation in which a lightweight object, known as a node, is allocated for each element. Each node maintains a reference to its element and one or more references to neighboring nodes in order to collectively represent the linear order of the sequence. We will demonstrate a trade-off of advantages and disadvantages when con- trasting array-based sequences and linked lists. Elements of a linked list cannot be efficiently accessed by a numeric index k, and we cannot tell just by examining a node if it is the second, fifth, or twentieth node in the list. However, linked lists avoid the three disadvantages noted above for array-based sequences. 7.1 Singly Linked Lists A singly linked list, in its simplest form, is a collection of nodes that collectively form a linear sequence. Each node stores a reference to an object that is an element of the sequence, as well as a reference to the next node of the list (see Figures 7.1 and 7.2). next MSP element Figure 7.1: Example of a node instance that forms part of a singly linked list. The node’s element member references an arbitrary object that is an element of the se- quence (the airport code MSP, in this example), while the next member references the subsequent node of the linked list (or None if there is no further node).
- 279. 7.1. Singly LinkedLists 257 tail MSP BOS ATL head LAX Figure 7.2: Example of a singly linked list whose elements are strings indicating airport codes. The list instance maintains a member named head that identifies the first node of the list, and in some applications another member named tail that identifies the last node of the list. The None object is denoted as Ø. The first and last node of a linked list are known as the head and tail of the list, respectively. By starting at the head, and moving from one node to another by following each node’s next reference, we can reach the tail of the list. We can identify the tail as the node having None as its next reference. This process is commonly known as traversing the linked list. Because the next reference of a node can be viewed as a link or pointer to another node, the process of traversing a list is also known as link hopping or pointer hopping. A linked list’s representation in memory relies on the collaboration of many objects. Each node is represented as a unique object, with that instance storing a reference to its element and a reference to the next node (or None). Another object represents the linked list as a whole. Minimally, the linked list instance must keep a reference to the head of the list. Without an explicit reference to the head, there would be no way to locate that node (or indirectly, any others). There is not an absolute need to store a direct reference to the tail of the list, as it could otherwise be located by starting at the head and traversing the rest of the list. However, storing an explicit reference to the tail node is a common convenience to avoid such a traversal. In similar regard, it is common for the linked list instance to keep a count of the total number of nodes that comprise the list (commonly described as the size of the list), to avoid the need to traverse the list to count the nodes. For the remainder of this chapter, we continue to illustrate nodes as objects, and each node’s “next” reference as a pointer. However, for the sake of simplicity, we illustrate a node’s element embedded directly within the node structure, even though the element is, in fact, an independent object. For example, Figure 7.3 is a more compact illustration of the linked list from Figure 7.2. LAX MSP BOS ATL head tail Figure 7.3: A compact illustration of a singly linked list, with elements embedded in the nodes (rather than more accurately drawn as references to external objects).
- 280. 258 Chapter 7.Linked Lists Inserting an Element at the Head of a Singly Linked List An important property of a linked list is that it does not have a predetermined fixed size; it uses space proportionally to its current number of elements. When using a singly linked list, we can easily insert an element at the head of the list, as shown in Figure 7.4, and described with pseudo-code in Code Fragment 7.1. The main idea is that we create a new node, set its element to the new element, set its next link to refer to the current head, and then set the list’s head to point to the new node. ATL BOS MSP head (a) BOS newest MSP ATL head LAX (b) LAX MSP ATL BOS head newest (c) Figure 7.4: Insertion of an element at the head of a singly linked list: (a) before the insertion; (b) after creation of a new node; (c) after reassignment of the head reference. Algorithm add first(L,e): newest = Node(e) {create new node instance storing reference to element e} newest.next = L.head {set new node’s next to reference the old head node} L.head = newest {set variable head to reference the new node} L.size = L.size+1 {increment the node count} Code Fragment 7.1: Inserting a new element at the beginning of a singly linked list L. Note that we set the next pointer of the new node before we reassign variable L.head to it. If the list were initially empty (i.e., L.head is None), then a natural consequence is that the new node has its next reference set to None.
- 281. 7.1. Singly LinkedLists 259 Inserting an Element at the Tail of a Singly Linked List We can also easily insert an element at the tail of the list, provided we keep a reference to the tail node, as shown in Figure 7.5. In this case, we create a new node, assign its next reference to None, set the next reference of the tail to point to this new node, and then update the tail reference itself to this new node. We give the details in Code Fragment 7.2. ATL BOS MSP tail (a) MIA ATL BOS MSP tail newest (b) MSP MIA tail newest ATL BOS (c) Figure 7.5: Insertion at the tail of a singly linked list: (a) before the insertion; (b) after creation of a new node; (c) after reassignment of the tail reference. Note that we must set the next link of the tail in (b) before we assign the tail variable to point to the new node in (c). Algorithm add last(L,e): newest = Node(e) {create new node instance storing reference to element e} newest.next = None {set new node’s next to reference the None object} L.tail.next = newest {make old tail node point to new node} L.tail = newest {set variable tail to reference the new node} L.size = L.size+1 {increment the node count} Code Fragment 7.2: Inserting a new node at the end of a singly linked list. Note that we set the next pointer for the old tail node before we make variable tail point to the new node. This code would need to be adjusted for inserting onto an empty list, since there would not be an existing tail node.
- 282. 260 Chapter 7.Linked Lists Removing an Element from a Singly Linked List Removing an element from the head of a singly linked list is essentially the reverse operation of inserting a new element at the head. This operation is illustrated in Figure 7.6 and given in detail in Code Fragment 7.3. head MSP ATL BOS LAX (a) BOS head MSP ATL LAX (b) ATL BOS MSP head (c) Figure 7.6: Removal of an element at the head of a singly linked list: (a) before the removal; (b) after “linking out” the old head; (c) final configuration. Algorithm remove first(L): if L.head is None then Indicate an error: the list is empty. L.head = L.head.next {make head point to next node (or None)} L.size = L.size−1 {decrement the node count} Code Fragment 7.3: Removing the node at the beginning of a singly linked list. Unfortunately, we cannot easily delete the last node of a singly linked list. Even if we maintain a tail reference directly to the last node of the list, we must be able to access the node before the last node in order to remove the last node. But we cannot reach the node before the tail by following next links from the tail. The only way to access this node is to start from the head of the list and search all the way through the list. But such a sequence of link-hopping operations could take a long time. If we want to support such an operation efficiently, we will need to make our list doubly linked (as we do in Section 7.3).
- 283. 7.1. Singly LinkedLists 261 7.1.1 Implementing a Stack with a Singly Linked List In this section, we demonstrate use of a singly linked list by providing a complete Python implementation of the stack ADT (see Section 6.1). In designing such an implementation, we need to decide whether to model the top of the stack at the head or at the tail of the list. There is clearly a best choice here; we can efficiently insert and delete elements in constant time only at the head. Since all stack operations affect the top, we orient the top of the stack at the head of our list. To represent individual nodes of the list, we develop a lightweight Node class. This class will never be directly exposed to the user of our stack class, so we will formally define it as a nonpublic, nested class of our eventual LinkedStack class (see Section 2.5.1 for discussion of nested classes). The definition of the Node class is shown in Code Fragment 7.4. class Node: ”””Lightweight, nonpublic class for storing a singly linked node.””” slots = _element , _next # streamline memory usage def init (self, element, next): # initialize node’s fields self. element = element # reference to user’s element self. next = next # reference to next node Code Fragment 7.4: A lightweight Node class for a singly linked list. A node has only two instance variables: element and next. We intentionally define slots to streamline the memory usage (see page 99 of Section 2.5.1 for discussion), because there may potentially be many node instances in a single list. The constructor of the Node class is designed for our convenience, allowing us to specify initial values for both fields of a newly created node. A complete implementation of our LinkedStack class is given in Code Frag- ments 7.5 and 7.6. Each stack instance maintains two variables. The head mem- ber is a reference to the node at the head of the list (or None, if the stack is empty). We keep track of the current number of elements with the size instance variable, for otherwise we would be forced to traverse the entire list to count the number of elements when reporting the size of the stack. The implementation of push essentially mirrors the pseudo-code for insertion at the head of a singly linked list as outlined in Code Fragment 7.1. When we push a new element e onto the stack, we accomplish the necessary changes to the linked structure by invoking the constructor of the Node class as follows: self. head = self. Node(e, self. head) # create and link a new node Note that the next field of the new node is set to the existing top node, and then self. head is reassigned to the new node.
- 284. 262 Chapter 7.Linked Lists 1 class LinkedStack: 2 ”””LIFO Stack implementation using a singly linked list for storage.””” 3 4 #-------------------------- nested Node class -------------------------- 5 class Node: 6 ”””Lightweight, nonpublic class for storing a singly linked node.””” 7 slots = _element , _next # streamline memory usage 8 9 def init (self, element, next): # initialize node’s fields 10 self. element = element # reference to user’s element 11 self. next = next # reference to next node 12 13 #------------------------------- stack methods ------------------------------- 14 def init (self): 15 ”””Create an empty stack.””” 16 self. head = None # reference to the head node 17 self. size = 0 # number of stack elements 18 19 def len (self): 20 ”””Return the number of elements in the stack.””” 21 return self. size 22 23 def is empty(self): 24 ”””Return True if the stack is empty.””” 25 return self. size == 0 26 27 def push(self, e): 28 ”””Add element e to the top of the stack.””” 29 self. head = self. Node(e, self. head) # create and link a new node 30 self. size += 1 31 32 def top(self): 33 ”””Return (but do not remove) the element at the top of the stack. 34 35 Raise Empty exception if the stack is empty. 36 ””” 37 if self.is empty(): 38 raise Empty( Stack is empty ) 39 return self. head. element # top of stack is at head of list Code Fragment 7.5: Implementation of a stack ADT using a singly linked list (con- tinued in Code Fragment 7.6).
- 285. 7.1. Singly LinkedLists 263 40 def pop(self): 41 ”””Remove and return the element from the top of the stack (i.e., LIFO). 42 43 Raise Empty exception if the stack is empty. 44 ””” 45 if self.is empty(): 46 raise Empty( Stack is empty ) 47 answer = self. head. element 48 self. head = self. head. next # bypass the former top node 49 self. size −= 1 50 return answer Code Fragment 7.6: Implementation of a stack ADT using a singly linked list (con- tinued from Code Fragment 7.5). When implementing the top method, the goal is to return the element that is at the top of the stack. When the stack is empty, we raise an Empty exception, as originally defined in Code Fragment 6.1 of Chapter 6. When the stack is nonempty, self. head is a reference to the first node of the linked list. The top element can be identified as self. head. element. Our implementation of pop essentially mirrors the pseudo-code given in Code Fragment 7.3, except that we maintain a local reference to the element that is stored at the node that is being removed, and we return that element to the caller of pop. The analysis of our LinkedStack operations is given in Table 7.1. We see that all of the methods complete in worst-case constant time. This is in contrast to the amortized bounds for the ArrayStack that were given in Table 6.2. Operation Running Time S.push(e) O(1) S.pop() O(1) S.top() O(1) len(S) O(1) S.is empty() O(1) Table 7.1: Performance of our LinkedStack implementation. All bounds are worst- case and our space usage is O(n), where n is the current number of elements in the stack.
- 286. 264 Chapter 7.Linked Lists 7.1.2 Implementing a Queue with a Singly Linked List As we did for the stack ADT, we can use a singly linked list to implement the queue ADT while supporting worst-case O(1)-time for all operations. Because we need to perform operations on both ends of the queue, we will explicitly maintain both a head reference and a tail reference as instance variables for each queue. The natural orientation for a queue is to align the front of the queue with the head of the list, and the back of the queue with the tail of the list, because we must be able to enqueue elements at the back, and dequeue them from the front. (Recall from the introduction of Section 7.1 that we are unable to efficiently remove elements from the tail of a singly linked list.) Our implementation of a LinkedQueue class is given in Code Fragments 7.7 and 7.8. 1 class LinkedQueue: 2 ”””FIFO queue implementation using a singly linked list for storage.””” 3 4 class Node: 5 ”””Lightweight, nonpublic class for storing a singly linked node.””” 6 (omitted here; identical to that of LinkedStack. Node) 7 8 def init (self): 9 ”””Create an empty queue.””” 10 self. head = None 11 self. tail = None 12 self. size = 0 # number of queue elements 13 14 def len (self): 15 ”””Return the number of elements in the queue.””” 16 return self. size 17 18 def is empty(self): 19 ”””Return True if the queue is empty.””” 20 return self. size == 0 21 22 def first(self): 23 ”””Return (but do not remove) the element at the front of the queue.””” 24 if self.is empty(): 25 raise Empty( Queue is empty ) 26 return self. head. element # front aligned with head of list Code Fragment 7.7: Implementation of a queue ADT using a singly linked list (continued in Code Fragment 7.8).
- 287. 7.1. Singly LinkedLists 265 27 def dequeue(self): 28 ”””Remove and return the first element of the queue (i.e., FIFO). 29 30 Raise Empty exception if the queue is empty. 31 ””” 32 if self.is empty(): 33 raise Empty( Queue is empty ) 34 answer = self. head. element 35 self. head = self. head. next 36 self. size −= 1 37 if self.is empty(): # special case as queue is empty 38 self. tail = None # removed head had been the tail 39 return answer 40 41 def enqueue(self, e): 42 ”””Add an element to the back of queue.””” 43 newest = self. Node(e, None) # node will be new tail node 44 if self.is empty(): 45 self. head = newest # special case: previously empty 46 else: 47 self. tail. next = newest 48 self. tail = newest # update reference to tail node 49 self. size += 1 Code Fragment 7.8: Implementation of a queue ADT using a singly linked list (continued from Code Fragment 7.7). Many aspects of our implementation are similar to that of the LinkedStack class, such as the definition of the nested Node class. Our implementation of dequeue for LinkedQueue is similar to that of pop for LinkedStack, as both remove the head of the linked list. However, there is a subtle difference because our queue must accurately maintain the tail reference (no such variable was maintained for our stack). In general, an operation at the head has no effect on the tail, but when dequeue is invoked on a queue with one element, we are simultaneously removing the tail of the list. We therefore set self. tail to None for consistency. There is a similar complication in our implementation of enqueue. The newest node always becomes the new tail. Yet a distinction is made depending on whether that new node is the only node in the list. In that case, it also becomes the new head; otherwise the new node must be linked immediately after the existing tail node. In terms of performance, the LinkedQueue is similar to the LinkedStack in that all operations run in worst-case constant time, and the space usage is linear in the current number of elements.
- 288. 266 Chapter 7.Linked Lists 7.2 Circularly Linked Lists In Section 6.2.2, we introduced the notion of a “circular” array and demonstrated its use in implementing the queue ADT. In reality, the notion of a circular array was artificial, in that there was nothing about the representation of the array itself that was circular in structure. It was our use of modular arithmetic when “advancing” an index from the last slot to the first slot that provided such an abstraction. In the case of linked lists, there is a more tangible notion of a circularly linked list, as we can have the tail of the list use its next reference to point back to the head of the list, as shown in Figure 7.7. We call such a structure a circularly linked list. BOS head tail LAX MSP ATL Figure 7.7: Example of a singly linked list with circular structure. A circularly linked list provides a more general model than a standard linked list for data sets that are cyclic, that is, which do not have any particular notion of a beginning and end. Figure 7.8 provides a more symmetric illustration of the same circular list structure as Figure 7.7. ATL BOS current MSP LAX Figure 7.8: Example of a circular linked list, with current denoting a reference to a select node. A circular view similar to Figure 7.8 could be used, for example, to describe the order of train stops in the Chicago loop, or the order in which players take turns during a game. Even though a circularly linked list has no beginning or end, per se, we must maintain a reference to a particular node in order to make use of the list. We use the identifier current to describe such a designated node. By setting current = current.next, we can effectively advance through the nodes of the list.
- 289. 7.2. Circularly LinkedLists 267 7.2.1 Round-Robin Schedulers To motivate the use of a circularly linked list, we consider a round-robin scheduler, which iterates through a collection of elements in a circular fashion and “services” each element by performing a given action on it. Such a scheduler is used, for example, to fairly allocate a resource that must be shared by a collection of clients. For instance, round-robin scheduling is often used to allocate slices of CPU time to various applications running concurrently on a computer. A round-robin scheduler could be implemented with the general queue ADT, by repeatedly performing the following steps on queue Q (see Figure 7.9): 1. e = Q.dequeue() 2. Service element e 3. Q.enqueue(e) The Queue Shared Service 1. Deque the next element 3. Enqueue the serviced element 2. Service the next element Figure 7.9: The three iterative steps for round-robin scheduling using a queue. If we use of the LinkedQueue class of Section 7.1.2 for such an application, there is unnecessary effort in the combination of a dequeue operation followed soon after by an enqueue of the same element. One node is removed from the list, with appropriate adjustments to the head of the list and the size decremented, and then a new node is created to reinsert at the tail of the list and the size is incremented. If using a circularly linked list, the effective transfer of an item from the “head” of the list to the “tail” of the list can be accomplished by advancing a reference that marks the boundary of the queue. We will next provide an implementation of a CircularQueue class that supports the entire queue ADT, together with an ad- ditional method, rotate(), that moves the first element of the queue to the back. (A similar method is supported by the deque class of Python’s collections module; see Table 6.4.) With this operation, a round-robin schedule can more efficiently be implemented by repeatedly performing the following steps: 1. Service element Q.front() 2. Q.rotate()
- 290. 268 Chapter 7.Linked Lists 7.2.2 Implementing a Queue with a Circularly Linked List To implement the queue ADT using a circularly linked list, we rely on the intuition of Figure 7.7, in which the queue has a head and a tail, but with the next reference of the tail linked to the head. Given such a model, there is no need for us to explicitly store references to both the head and the tail; as long as we keep a reference to the tail, we can always find the head by following the tail’s next reference. Code Fragments 7.9 and 7.10 provide an implementation of a CircularQueue class based on this model. The only two instance variables are tail, which is a reference to the tail node (or None when empty), and size, which is the current number of elements in the queue. When an operation involves the front of the queue, we recognize self. tail. next as the head of the queue. When enqueue is called, a new node is placed just after the tail but before the current head, and then the new node becomes the tail. In addition to the traditional queue operations, the CircularQueue class supports a rotate method that more efficiently enacts the combination of removing the front element and reinserting it at the back of the queue. With the circular representation, we simply set self. tail = self. tail. next to make the old head become the new tail (with the node after the old head becoming the new head). 1 class CircularQueue: 2 ”””Queue implementation using circularly linked list for storage.””” 3 4 class Node: 5 ”””Lightweight, nonpublic class for storing a singly linked node.””” 6 (omitted here; identical to that of LinkedStack. Node) 7 8 def init (self): 9 ”””Create an empty queue.””” 10 self. tail = None # will represent tail of queue 11 self. size = 0 # number of queue elements 12 13 def len (self): 14 ”””Return the number of elements in the queue.””” 15 return self. size 16 17 def is empty(self): 18 ”””Return True if the queue is empty.””” 19 return self. size == 0 Code Fragment 7.9: Implementation of a CircularQueue class, using a circularly linked list as storage (continued in Code Fragment 7.10).
- 291. 7.2. Circularly LinkedLists 269 20 def first(self): 21 ”””Return (but do not remove) the element at the front of the queue. 22 23 Raise Empty exception if the queue is empty. 24 ””” 25 if self.is empty(): 26 raise Empty( Queue is empty ) 27 head = self. tail. next 28 return head. element 29 30 def dequeue(self): 31 ”””Remove and return the first element of the queue (i.e., FIFO). 32 33 Raise Empty exception if the queue is empty. 34 ””” 35 if self.is empty(): 36 raise Empty( Queue is empty ) 37 oldhead = self. tail. next 38 if self. size == 1: # removing only element 39 self. tail = None # queue becomes empty 40 else: 41 self. tail. next = oldhead. next # bypass the old head 42 self. size −= 1 43 return oldhead. element 44 45 def enqueue(self, e): 46 ”””Add an element to the back of queue.””” 47 newest = self. Node(e, None) # node will be new tail node 48 if self.is empty(): 49 newest. next = newest # initialize circularly 50 else: 51 newest. next = self. tail. next # new node points to head 52 self. tail. next = newest # old tail points to new node 53 self. tail = newest # new node becomes the tail 54 self. size += 1 55 56 def rotate(self): 57 ”””Rotate front element to the back of the queue.””” 58 if self. size 0: 59 self. tail = self. tail. next # old head becomes new tail Code Fragment 7.10: Implementation of a CircularQueue class, using a circularly linked list as storage (continued from Code Fragment 7.9).
- 292. 270 Chapter 7.Linked Lists 7.3 Doubly Linked Lists In a singly linked list, each node maintains a reference to the node that is immedi- ately after it. We have demonstrated the usefulness of such a representation when managing a sequence of elements. However, there are limitations that stem from the asymmetry of a singly linked list. In the opening of Section 7.1, we empha- sized that we can efficiently insert a node at either end of a singly linked list, and can delete a node at the head of a list, but we are unable to efficiently delete a node at the tail of the list. More generally, we cannot efficiently delete an arbitrary node from an interior position of the list if only given a reference to that node, because we cannot determine the node that immediately precedes the node to be deleted (yet, that node needs to have its next reference updated). To provide greater symmetry, we define a linked list in which each node keeps an explicit reference to the node before it and a reference to the node after it. Such a structure is known as a doubly linked list. These lists allow a greater variety of O(1)-time update operations, including insertions and deletions at arbitrary posi- tions within the list. We continue to use the term “next” for the reference to the node that follows another, and we introduce the term “prev” for the reference to the node that precedes it. Header and Trailer Sentinels In order to avoid some special cases when operating near the boundaries of a doubly linked list, it helps to add special nodes at both ends of the list: a header node at the beginning of the list, and a trailer node at the end of the list. These “dummy” nodes are known as sentinels (or guards), and they do not store elements of the primary sequence. A doubly linked list with such sentinels is shown in Figure 7.10. SFO JFK PVD next next next prev prev prev prev header trailer next Figure 7.10: A doubly linked list representing the sequence { JFK, PVD, SFO }, using sentinels header and trailer to demarcate the ends of the list. When using sentinel nodes, an empty list is initialized so that the next field of the header points to the trailer, and the prev field of the trailer points to the header; the remaining fields of the sentinels are irrelevant (presumably None, in Python). For a nonempty list, the header’s next will refer to a node containing the first real element of a sequence, just as the trailer’s prev references the node containing the last element of a sequence.
- 293. 7.3. Doubly LinkedLists 271 Advantage of Using Sentinels Although we could implement a doubly linked list without sentinel nodes (as we did with our singly linked list in Section 7.1), the slight extra space devoted to the sentinels greatly simplifies the logic of our operations. Most notably, the header and trailer nodes never change—only the nodes between them change. Furthermore, we can treat all insertions in a unified manner, because a new node will always be placed between a pair of existing nodes. In similar fashion, every element that is to be deleted is guaranteed to be stored in a node that has neighbors on each side. For contrast, look back at our LinkedQueue implementation from Section 7.1.2. Its enqueue method, given in Code Fragment 7.8, adds a new node to the end of the list. However, its implementation required a conditional to manage the special case of inserting into an empty list. In the general case, the new node was linked after the existing tail. But when adding to an empty list, there is no existing tail; instead it is necessary to reassign self. head to reference the new node. The use of a sentinel node in that implementation would eliminate the special case, as there would always be an existing node (possibly the header) before a new node. Inserting and Deleting with a Doubly Linked List Every insertion into our doubly linked list representation will take place between a pair of existing nodes, as diagrammed in Figure 7.11. For example, when a new element is inserted at the front of the sequence, we will simply add the new node between the header and the node that is currently after the header. (See Figure 7.12.) JFK BWI SFO trailer header (a) BWI PVD SFO JFK trailer header (b) BWI PVD SFO JFK trailer header (c) Figure 7.11: Adding an element to a doubly linked list with header and trailer sen- tinels: (a) before the operation; (b) after creating the new node; (c) after linking the neighbors to the new node.
- 294. 272 Chapter 7.Linked Lists JFK BWI SFO trailer header (a) PVD BWI JFK SFO trailer header (b) PVD JFK SFO BWI trailer header (c) Figure 7.12: Adding an element to the front of a sequence represented by a dou- bly linked list with header and trailer sentinels: (a) before the operation; (b) after creating the new node; (c) after linking the neighbors to the new node. The deletion of a node, portrayed in Figure 7.13, proceeds in the opposite fash- ion of an insertion. The two neighbors of the node to be deleted are linked directly to each other, thereby bypassing the original node. As a result, that node will no longer be considered part of the list and it can be reclaimed by the system. Because of our use of sentinels, the same implementation can be used when deleting the first or the last element of a sequence, because even such an element will be stored at a node that lies between two others. BWI PVD SFO JFK trailer header (a) BWI PVD SFO JFK trailer header (b) JFK BWI SFO trailer header (c) Figure 7.13: Removing the element PVD from a doubly linked list: (a) before the removal; (b) after linking out the old node; (c) after the removal (and garbage collection).
- 295. 7.3. Doubly LinkedLists 273 7.3.1 Basic Implementation of a Doubly Linked List We begin by providing a preliminary implementation of a doubly linked list, in the form of a class named DoublyLinkedBase. We intentionally name the class with a leading underscore because we do not intend for it to provide a coherent public interface for general use. We will see that linked lists can support general insertions and deletions in O(1) worst-case time, but only if the location of an operation can be succinctly identified. With array-based sequences, an integer index was a convenient means for describing a position within a sequence. However, an index is not convenient for linked lists as there is no efficient way to find the jth element; it would seem to require a traversal of a portion of the list. When working with a linked list, the most direct way to describe the location of an operation is by identifying a relevant node of the list. However, we prefer to encapsulate the inner workings of our data structure to avoid having users di- rectly access nodes of a list. In the remainder of this chapter, we will develop two public classes that inherit from our DoublyLinkedBase class to provide more coherent abstractions. Specifically, in Section 7.3.2, we provide a LinkedDeque class that implements the double-ended queue ADT introduced in Section 6.3; that class only supports operations at the ends of the queue, so there is no need for a user to identify an interior position within the list. In Section 7.4, we introduce a new PositionalList abstraction that provides a public interface that allows arbitrary insertions and deletions from a list. Our low-level DoublyLinkedBase class relies on the use of a nonpublic Node class that is similar to that for a singly linked list, as given in Code Fragment 7.4, except that the doubly linked version includes a prev attribute, in addition to the next and element attributes, as shown in Code Fragment 7.11. class Node: ”””Lightweight, nonpublic class for storing a doubly linked node.””” slots = _element , _prev , _next # streamline memory def init (self, element, prev, next): # initialize node’s fields self. element = element # user’s element self. prev = prev # previous node reference self. next = next # next node reference Code Fragment 7.11: A Python Node class for use in a doubly linked list. The remainder of our DoublyLinkedBase class is given in Code Fragment 7.12. The constructor instantiates the two sentinel nodes and links them directly to each other. We maintain a size member and provide public support for len and is empty so that these behaviors can be directly inherited by the subclasses.
- 296. 274 Chapter 7.Linked Lists 1 class DoublyLinkedBase: 2 ”””A base class providing a doubly linked list representation.””” 3 4 class Node: 5 ”””Lightweight, nonpublic class for storing a doubly linked node.””” 6 (omitted here; see previous code fragment) 7 8 def init (self): 9 ”””Create an empty list.””” 10 self. header = self. Node(None, None, None) 11 self. trailer = self. Node(None, None, None) 12 self. header. next = self. trailer # trailer is after header 13 self. trailer. prev = self. header # header is before trailer 14 self. size = 0 # number of elements 15 16 def len (self): 17 ”””Return the number of elements in the list.””” 18 return self. size 19 20 def is empty(self): 21 ”””Return True if list is empty.””” 22 return self. size == 0 23 24 def insert between(self, e, predecessor, successor): 25 ”””Add element e between two existing nodes and return new node.””” 26 newest = self. Node(e, predecessor, successor) # linked to neighbors 27 predecessor. next = newest 28 successor. prev = newest 29 self. size += 1 30 return newest 31 32 def delete node(self, node): 33 ”””Delete nonsentinel node from the list and return its element.””” 34 predecessor = node. prev 35 successor = node. next 36 predecessor. next = successor 37 successor. prev = predecessor 38 self. size −= 1 39 element = node. element # record deleted element 40 node. prev = node. next = node. element = None # deprecate node 41 return element # return deleted element Code Fragment 7.12: A base class for managing a doubly linked list.
- 297. 7.3. Doubly LinkedLists 275 The other two methods of our class are the nonpublic utilities, insert between and delete node. These provide generic support for insertions and deletions, re- spectively, but require one or more node references as parameters. The implemen- tation of the insert between method is modeled upon the algorithm that was previ- ously portrayed in Figure 7.11. It creates a new node, with that node’s fields initial- ized to link to the specified neighboring nodes. Then the fields of the neighboring nodes are updated to include the newest node in the list. For later convenience, the method returns a reference to the newly created node. The implementation of the delete node method is modeled upon the algorithm portrayed in Figure 7.13. The neighbors of the node to be deleted are linked directly to each other, thereby bypassing the deleted node from the list. As a formality, we intentionally reset the prev, next, and element fields of the deleted node to None (after recording the element to be returned). Although the deleted node will be ignored by the rest of the list, setting its fields to None is advantageous as it may help Python’s garbage collection, since unnecessary links to the other nodes and the stored element are eliminated. We will also rely on this configuration to recognize a node as “deprecated” when it is no longer part of the list. 7.3.2 Implementing a Deque with a Doubly Linked List The double-ended queue (deque) ADT was introduced in Section 6.3. With an array-based implementation, we achieve all operations in amortized O(1) time, due to the occasional need to resize the array. With an implementation based upon a doubly linked list, we can achieve all deque operation in worst-case O(1) time. We provide an implementation of a LinkedDeque class (Code Fragment 7.13) that inherits from the DoublyLinkedBase class of the preceding section. We do not provide an explicit init method for the LinkedDeque class, as the inherited version of that method suffices to initialize a new instance. We also rely on the inherited methods len and is empty in meeting the deque ADT. With the use of sentinels, the key to our implementation is to remember that the header does not store the first element of the deque—it is the node just after the header that stores the first element (assuming the deque is nonempty). Similarly, the node just before the trailer stores the last element of the deque. We use the inherited insert between method to insert at either end of the deque. To insert an element at the front of the deque, we place it immediately between the header and the node just after the header. An insertion at the end of deque is placed immediately before the trailer node. Note that these operations succeed, even when the deque is empty; in such a situation, the new node is placed between the two sentinels. When deleting an element from a nonempty deque, we rely upon the inherited delete node method, knowing that the designated node is assured to have neighbors on each side.
- 298. 276 Chapter 7.Linked Lists 1 class LinkedDeque( DoublyLinkedBase): # note the use of inheritance 2 ”””Double-ended queue implementation based on a doubly linked list.””” 3 4 def first(self): 5 ”””Return (but do not remove) the element at the front of the deque.””” 6 if self.is empty(): 7 raise Empty(Deque is empty) 8 return self. header. next. element # real item just after header 9 10 def last(self): 11 ”””Return (but do not remove) the element at the back of the deque.””” 12 if self.is empty(): 13 raise Empty(Deque is empty) 14 return self. trailer. prev. element # real item just before trailer 15 16 def insert first(self, e): 17 ”””Add an element to the front of the deque.””” 18 self. insert between(e, self. header, self. header. next) # after header 19 20 def insert last(self, e): 21 ”””Add an element to the back of the deque.””” 22 self. insert between(e, self. trailer. prev, self. trailer) # before trailer 23 24 def delete first(self): 25 ”””Remove and return the element from the front of the deque. 26 27 Raise Empty exception if the deque is empty. 28 ””” 29 if self.is empty(): 30 raise Empty(Deque is empty) 31 return self. delete node(self. header. next) # use inherited method 32 33 def delete last(self): 34 ”””Remove and return the element from the back of the deque. 35 36 Raise Empty exception if the deque is empty. 37 ””” 38 if self.is empty(): 39 raise Empty(Deque is empty) 40 return self. delete node(self. trailer. prev) # use inherited method Code Fragment 7.13: Implementation of a LinkedDeque class that inherits from the DoublyLinkedBase class.
- 299. 7.4. The PositionalList ADT 277 7.4 The Positional List ADT The abstract data types that we have considered thus far, namely stacks, queues, and double-ended queues, only allow update operations that occur at one end of a sequence or the other. We wish to have a more general abstraction. For example, although we motivated the FIFO semantics of a queue as a model for customers who are waiting to speak with a customer service representative, or fans who are waiting in line to buy tickets to a show, the queue ADT is too limiting. What if a waiting customer decides to hang up before reaching the front of the customer service queue? Or what if someone who is waiting in line to buy tickets allows a friend to “cut” into line at that position? We would like to design an abstract data type that provides a user a way to refer to elements anywhere in a sequence, and to perform arbitrary insertions and deletions. When working with array-based sequences (such as a Python list), integer in- dices provide an excellent means for describing the location of an element, or the location at which an insertion or deletion should take place. However, numeric in- dices are not a good choice for describing positions within a linked list because we cannot efficiently access an entry knowing only its index; finding an element at a given index within a linked list requires traversing the list incrementally from its beginning or end, counting elements as we go. Furthermore, indices are not a good abstraction for describing a local position in some applications, because the index of an entry changes over time due to inser- tions or deletions that happen earlier in the sequence. For example, it may not be convenient to describe the location of a person waiting in line by knowing precisely how far away that person is from the front of the line. We prefer an abstraction, as characterized in Figure 7.14, in which there is some other means for describing a position. We then wish to model situations such as when an identified person leaves the line before reaching the front, or in which a new person is added to a line immediately behind another identified person. Tickets me Figure 7.14: We wish to be able to identify the position of an element in a sequence without the use of an integer index.
- 300. 278 Chapter 7.Linked Lists As another example, a text document can be viewed as a long sequence of characters. A word processor uses the abstraction of a cursor to describe a position within the document without explicit use of an integer index, allowing operations such as “delete the character at the cursor” or “insert a new character just after the cursor.” Furthermore, we may be able to refer to an inherent position within a doc- ument, such as the beginning of a particular section, without relying on a character index (or even a section number) that may change as the document evolves. A Node Reference as a Position? One of the great benefits of a linked list structure is that it is possible to perform O(1)-time insertions and deletions at arbitrary positions of the list, as long as we are given a reference to a relevant node of the list. It is therefore very tempting to develop an ADT in which a node reference serves as the mechanism for describing a position. In fact, our DoublyLinkedBase class of Section 7.3.1 has methods insert between and delete node that accept node references as parameters. However, such direct use of nodes would violate the object-oriented design principles of abstraction and encapsulation that were introduced in Chapter 2. There are several reasons to prefer that we encapsulate the nodes of a linked list, for both our sake and for the benefit of users of our abstraction. • It will be simpler for users of our data structure if they are not bothered with unnecessary details of our implementation, such as low-level manipulation of nodes, or our reliance on the use of sentinel nodes. Notice that to use the insert between method of our DoublyLinkedBase class to add a node at the beginning of a sequence, the header sentinel must be sent as a parameter. • We can provide a more robust data structure if we do not permit users to directly access or manipulate the nodes. In that way, we ensure that users cannot invalidate the consistency of a list by mismanaging the linking of nodes. A more subtle problem arises if a user were allowed to call the insert between or delete node method of our DoublyLinkedBase class, sending a node that does not belong to the given list as a parameter. (Go back and look at that code and see why it causes a problem!) • By better encapsulating the internal details of our implementation, we have greater flexibility to redesign the data structure and improve its performance. In fact, with a well-designed abstraction, we can provide a notion of a non- numeric position, even if using an array-based sequence. For these reasons, instead of relying directly on nodes, we introduce an inde- pendent position abstraction to denote the location of an element within a list, and then a complete positional list ADT that can encapsulate a doubly linked list (or even an array-based sequence; see Exercise P-7.46).
- 301. 7.4. The PositionalList ADT 279 7.4.1 The Positional List Abstract Data Type To provide for a general abstraction of a sequence of elements with the ability to identify the location of an element, we define a positional list ADT as well as a simpler position abstract data type to describe a location within a list. A position acts as a marker or token within the broader positional list. A position p is unaf- fected by changes elsewhere in a list; the only way in which a position becomes invalid is if an explicit command is issued to delete it. A position instance is a simple object, supporting only the following method: p.element(): Return the element stored at position p. In the context of the positional list ADT, positions serve as parameters to some methods and as return values from other methods. In describing the behaviors of a positional list, we being by presenting the accessor methods supported by a list L: L.first(): Return the position of the first element of L, or None if L is empty. L.last(): Return the position of the last element of L, or None if L is empty. L.before(p): Return the position of L immediately before position p, or None if p is the first position. L.after(p): Return the position of L immediately after position p, or None if p is the last position. L.is empty(): Return True if list L does not contain any elements. len(L): Return the number of elements in the list. iter(L): Return a forward iterator for the elements of the list. See Sec- tion 1.8 for discussion of iterators in Python. The positional list ADT also includes the following update methods: L.add first(e): Insert a new element e at the front of L, returning the position of the new element. L.add last(e): Insert a new element e at the back of L, returning the position of the new element. L.add before(p, e): Insert a new element e just before position p in L, returning the position of the new element. L.add after(p, e): Insert a new element e just after position p in L, returning the position of the new element. L.replace(p, e): Replace the element at position p with element e, returning the element formerly at position p. L.delete(p): Remove and return the element at position p in L, invalidat- ing the position. For those methods of the ADT that accept a position p as a parameter, an error occurs if p is not a valid position for list L.
- 302. 280 Chapter 7.Linked Lists Note well that the first() and last() methods of the positional list ADT return the associated positions, not the elements. (This is in contrast to the corresponding first and last methods of the deque ADT.) The first element of a positional list can be determined by subsequently invoking the element method on that position, as L.first().element(). The advantage of receiving a position as a return value is that we can use that position to navigate the list. For example, the following code fragment prints all elements of a positional list named data. cursor = data.first() while cursor is not None: print(cursor.element()) # print the element stored at the position cursor = data.after(cursor) # advance to the next position (if any) This code relies on the stated convention that the None object is returned when after is called upon the last position. That return value is clearly distinguishable from any legitimate position. The positional list ADT similarly indicates that the None value is returned when the before method is invoked at the front of the list, or when first or last methods are called upon an empty list. Therefore, the above code fragment works correctly even if the data list is empty. Because the ADT includes support for Python’s iter function, users may rely on the traditional for-loop syntax for such a forward traversal of a list named data. for e in data: print(e) More general navigational and update methods of the positional list ADT are shown in the following example. Example 7.1: The following table shows a series of operations on an initially empty positional list L. To identify position instances, we use variables such as p and q. For ease of exposition, when displaying the list contents, we use subscript notation to denote its positions. Operation Return Value L L.add last(8) p 8p L.first() p 8p L.add after(p, 5) q 8p, 5q L.before(q) p 8p, 5q L.add before(q, 3) r 8p, 3r, 5q r.element() 3 8p, 3r, 5q L.after(p) r 8p, 3r, 5q L.before(p) None 8p, 3r, 5q L.add first(9) s 9s, 8p, 3r, 5q L.delete(L.last()) 5 9s, 8p, 3r L.replace(p, 7) 8 9s, 7p, 3r
- 303. 7.4. The PositionalList ADT 281 7.4.2 Doubly Linked List Implementation In this section, we present a complete implementation of a PositionalList class using a doubly linked list that satisfies the following important proposition. Proposition 7.2: Each method of the positional list ADT runs in worst-case O(1) time when implemented with a doubly linked list. We rely on the DoublyLinkedBase class from Section 7.3.1 for our low-level representation; the primary responsibility of our new class is to provide a public interface in accordance with the positional list ADT. We begin our class definition in Code Fragment 7.14 with the definition of the public Position class, nested within our PositionalList class. Position instances will be used to represent the locations of elements within the list. Our various PositionalList methods may end up creating redundant Position instances that reference the same underlying node (for example, when first and last are the same). For that reason, our Position class defines the eq and ne special methods so that a test such as p == q evaluates to True when two positions refer to the same node. Validating Positions Each time a method of the PositionalList class accepts a position as a parameter, we want to verify that the position is valid, and if so, to determine the underlying node associated with the position. This functionality is implemented by a non- public method named validate. Internally, a position maintains a reference to the associated node of the linked list, and also a reference to the list instance that con- tains the specified node. With the container reference, we can robustly detect when a caller sends a position instance that does not belong to the indicated list. We are also able to detect a position instance that belongs to the list, but that refers to a node that is no longer part of that list. Recall that the delete node of the base class sets the previous and next references of a deleted node to None; we can recognize that condition to detect a deprecated node. Access and Update Methods The access methods of the PositionalList class are given in Code Fragment 7.15 and the update methods are given in Code Fragment 7.16. All of these methods trivially adapt the underlying doubly linked list implementation to support the pub- lic interface of the positional list ADT. Those methods rely on the validate utility to “unwrap” any position that is sent. They also rely on a make position utility to “wrap” nodes as Position instances to return to the user, making sure never to return a position referencing a sentinel. For convenience, we have overridden the inherited insert between utility method so that ours returns a position associated with the newly created node (whereas the inherited version returns the node itself).
- 304. 282 Chapter 7.Linked Lists 1 class PositionalList( DoublyLinkedBase): 2 ”””A sequential container of elements allowing positional access.””” 3 4 #-------------------------- nested Position class -------------------------- 5 class Position: 6 ”””An abstraction representing the location of a single element.””” 7 8 def init (self, container, node): 9 ”””Constructor should not be invoked by user.””” 10 self. container = container 11 self. node = node 12 13 def element(self): 14 ”””Return the element stored at this Position.””” 15 return self. node. element 16 17 def eq (self, other): 18 ”””Return True if other is a Position representing the same location.””” 19 return type(other) is type(self) and other. node is self. node 20 21 def ne (self, other): 22 ”””Return True if other does not represent the same location.””” 23 return not (self == other) # opposite of eq 24 25 #------------------------------- utility method ------------------------------- 26 def validate(self, p): 27 ”””Return position s node, or raise appropriate error if invalid.””” 28 if not isinstance(p, self.Position): 29 raise TypeError( p must be proper Position type ) 30 if p. container is not self: 31 raise ValueError( p does not belong to this container ) 32 if p. node. next is None: # convention for deprecated nodes 33 raise ValueError( p is no longer valid ) 34 return p. node Code Fragment 7.14: A PositionalList class based on a doubly linked list. (Contin- ues in Code Fragments 7.15 and 7.16.)
- 305. 7.4. The PositionalList ADT 283 35 #------------------------------- utility method ------------------------------- 36 def make position(self, node): 37 ”””Return Position instance for given node (or None if sentinel).””” 38 if node is self. header or node is self. trailer: 39 return None # boundary violation 40 else: 41 return self.Position(self, node) # legitimate position 42 43 #------------------------------- accessors ------------------------------- 44 def first(self): 45 ”””Return the first Position in the list (or None if list is empty).””” 46 return self. make position(self. header. next) 47 48 def last(self): 49 ”””Return the last Position in the list (or None if list is empty).””” 50 return self. make position(self. trailer. prev) 51 52 def before(self, p): 53 ”””Return the Position just before Position p (or None if p is first).””” 54 node = self. validate(p) 55 return self. make position(node. prev) 56 57 def after(self, p): 58 ”””Return the Position just after Position p (or None if p is last).””” 59 node = self. validate(p) 60 return self. make position(node. next) 61 62 def iter (self): 63 ”””Generate a forward iteration of the elements of the list.””” 64 cursor = self.first() 65 while cursor is not None: 66 yield cursor.element() 67 cursor = self.after(cursor) Code Fragment 7.15: A PositionalList class based on a doubly linked list. (Contin- ued from Code Fragment 7.14; continues in Code Fragment 7.16.)
- 306. 284 Chapter 7.Linked Lists 68 #------------------------------- mutators ------------------------------- 69 # override inherited version to return Position, rather than Node 70 def insert between(self, e, predecessor, successor): 71 ”””Add element between existing nodes and return new Position.””” 72 node = super(). insert between(e, predecessor, successor) 73 return self. make position(node) 74 75 def add first(self, e): 76 ”””Insert element e at the front of the list and return new Position.””” 77 return self. insert between(e, self. header, self. header. next) 78 79 def add last(self, e): 80 ”””Insert element e at the back of the list and return new Position.””” 81 return self. insert between(e, self. trailer. prev, self. trailer) 82 83 def add before(self, p, e): 84 ”””Insert element e into list before Position p and return new Position.””” 85 original = self. validate(p) 86 return self. insert between(e, original. prev, original) 87 88 def add after(self, p, e): 89 ”””Insert element e into list after Position p and return new Position.””” 90 original = self. validate(p) 91 return self. insert between(e, original, original. next) 92 93 def delete(self, p): 94 ”””Remove and return the element at Position p.””” 95 original = self. validate(p) 96 return self. delete node(original) # inherited method returns element 97 98 def replace(self, p, e): 99 ”””Replace the element at Position p with e. 100 101 Return the element formerly at Position p. 102 ””” 103 original = self. validate(p) 104 old value = original. element # temporarily store old element 105 original. element = e # replace with new element 106 return old value # return the old element value Code Fragment 7.16: A PositionalList class based on a doubly linked list. (Contin- ued from Code Fragments 7.14 and 7.15.)
- 307. 7.5. Sorting aPositional List 285 7.5 Sorting a Positional List In Section 5.5.2, we introduced the insertion-sort algorithm, in the context of an array-based sequence. In this section, we develop an implementation that operates on a PositionalList, relying on the same high-level algorithm in which each element is placed relative to a growing collection of previously sorted elements. We maintain a variable named marker that represents the rightmost position of the currently sorted portion of a list. During each pass, we consider the position just past the marker as the pivot and consider where the pivot’s element belongs relative to the sorted portion; we use another variable, named walk, to move leftward from the marker, as long as there remains a preceding element with value larger than the pivot’s. A typical configuration of these variables is diagrammed in Figure 7.15. A Python implementation of this strategy is given in Code 7.17. 15 22 25 29 36 23 53 11 42 marker pivot walk Figure 7.15: Overview of one step of our insertion-sort algorithm. The shaded elements, those up to and including marker, have already been sorted. In this step, the pivot’s element should be relocated immediately before the walk position. 1 def insertion sort(L): 2 ”””Sort PositionalList of comparable elements into nondecreasing order.””” 3 if len(L) 1: # otherwise, no need to sort it 4 marker = L.first() 5 while marker != L.last(): 6 pivot = L.after(marker) # next item to place 7 value = pivot.element() 8 if value marker.element(): # pivot is already sorted 9 marker = pivot # pivot becomes new marker 10 else: # must relocate pivot 11 walk = marker # find leftmost item greater than value 12 while walk != L.first( ) and L.before(walk).element( ) value: 13 walk = L.before(walk) 14 L.delete(pivot) 15 L.add before(walk, value) # reinsert value before walk Code Fragment 7.17: Python code for performing insertion-sort on a positional list.
- 308. 286 Chapter 7.Linked Lists 7.6 Case Study: Maintaining Access Frequencies The positional list ADT is useful in a number of settings. For example, a program that simulates a game of cards could model each person’s hand as a positional list (Exercise P-7.47). Since most people keep cards of the same suit together, inserting and removing cards from a person’s hand could be implemented using the methods of the positional list ADT, with the positions being determined by a natural order of the suits. Likewise, a simple text editor embeds the notion of positional insertion and deletion, since such editors typically perform all updates relative to a cursor, which represents the current position in the list of characters of text being edited. In this section, we consider maintaining a collection of elements while keeping track of the number of times each element is accessed. Keeping such access counts allows us to know which elements are among the most popular. Examples of such scenarios include a Web browser that keeps track of a user’s most accessed URLs, or a music collection that maintains a list of the most frequently played songs for a user. We model this with a new favorites list ADT that supports the len and is empty methods as well as the following: access(e): Access the element e, incrementing its access count, and adding it to the favorites list if it is not already present. remove(e): Remove element e from the favorites list, if present. top(k): Return an iteration of the k most accessed elements. 7.6.1 Using a Sorted List Our first approach for managing a list of favorites is to store elements in a linked list, keeping them in nonincreasing order of access counts. We access or remove an element by searching the list from the most frequently accessed to the least frequently accessed. Reporting the top k most accessed elements is easy, as they are the first k entries of the list. To maintain the invariant that elements are stored in nonincreasing order of access counts, we must consider how a single access operation may affect the order. The accessed element’s count increases by one, and so it may become larger than one or more of its preceding neighbors in the list, thereby violating the invariant. Fortunately, we can reestablish the sorted invariant using a technique similar to a single pass of the insertion-sort algorithm, introduced in the previous section. We can perform a backward traversal of the list, starting at the position of the element whose access count has increased, until we locate a valid position after which the element can be relocated.
- 309. 7.6. Case Study:Maintaining Access Frequencies 287 Using the Composition Pattern We wish to implement a favorites list by making use of a PositionalList for storage. If elements of the positional list were simply elements of the favorites list, we would be challenged to maintain access counts and to keep the proper count with the associated element as the contents of the list are reordered. We use a general object-oriented design pattern, the composition pattern, in which we define a single object that is composed of two or more other objects. Specifically, we define a nonpublic nested class, Item, that stores the element and its access count as a single instance. We then maintain our favorites list as a PositionalList of item instances, so that the access count for a user’s element is embedded alongside it in our representation. (An Item is never exposed to a user of a FavoritesList.) 1 class FavoritesList: 2 ”””List of elements ordered from most frequently accessed to least.””” 3 4 #------------------------------ nested Item class ------------------------------ 5 class Item: 6 slots = _value , _count # streamline memory usage 7 def init (self, e): 8 self. value = e # the user s element 9 self. count = 0 # access count initially zero 10 11 #------------------------------- nonpublic utilities ------------------------------- 12 def find position(self, e): 13 ”””Search for element e and return its Position (or None if not found).””” 14 walk = self. data.first() 15 while walk is not None and walk.element(). value != e: 16 walk = self. data.after(walk) 17 return walk 18 19 def move up(self, p): 20 ”””Move item at Position p earlier in the list based on access count.””” 21 if p != self. data.first(): # consider moving... 22 cnt = p.element(). count 23 walk = self. data.before(p) 24 if cnt walk.element(). count: # must shift forward 25 while (walk != self. data.first( ) and 26 cnt self. data.before(walk).element(). count): 27 walk = self. data.before(walk) 28 self. data.add before(walk, self. data.delete(p)) # delete/reinsert Code Fragment 7.18: Class FavoritesList. (Continues in Code Fragment 7.19.)
- 310. 288 Chapter 7.Linked Lists 29 #------------------------------- public methods ------------------------------- 30 def init (self): 31 ”””Create an empty list of favorites.””” 32 self. data = PositionalList( ) # will be list of Item instances 33 34 def len (self): 35 ”””Return number of entries on favorites list.””” 36 return len(self. data) 37 38 def is empty(self): 39 ”””Return True if list is empty.””” 40 return len(self. data) == 0 41 42 def access(self, e): 43 ”””Access element e, thereby increasing its access count.””” 44 p = self. find position(e) # try to locate existing element 45 if p is None: 46 p = self. data.add last(self. Item(e)) # if new, place at end 47 p.element(). count += 1 # always increment count 48 self. move up(p) # consider moving forward 49 50 def remove(self, e): 51 ”””Remove element e from the list of favorites.””” 52 p = self. find position(e) # try to locate existing element 53 if p is not None: 54 self. data.delete(p) # delete, if found 55 56 def top(self, k): 57 ”””Generate sequence of top k elements in terms of access count.””” 58 if not 1 = k = len(self): 59 raise ValueError( Illegal value for k ) 60 walk = self. data.first() 61 for j in range(k): 62 item = walk.element( ) # element of list is Item 63 yield item. value # report user’s element 64 walk = self. data.after(walk) Code Fragment 7.19: Class FavoritesList. (Continued from Code Fragment 7.18.)
- 311. 7.6. Case Study:Maintaining Access Frequencies 289 7.6.2 Using a List with the Move-to-Front Heuristic The previous implementation of a favorites list performs the access(e) method in time proportional to the index of e in the favorites list. That is, if e is the kth most popular element in the favorites list, then accessing it takes O(k) time. In many real-life access sequences (e.g., Web pages visited by a user), once an element is accessed it is more likely to be accessed again in the near future. Such scenarios are said to possess locality of reference. A heuristic, or rule of thumb, that attempts to take advantage of the locality of reference that is present in an access sequence is the move-to-front heuristic. To apply this heuristic, each time we access an element we move it all the way to the front of the list. Our hope, of course, is that this element will be accessed again in the near future. Consider, for example, a scenario in which we have n elements and the following series of n2 accesses: • element 1 is accessed n times • element 2 is accessed n times • ··· • element n is accessed n times. If we store the elements sorted by their access counts, inserting each element the first time it is accessed, then • each access to element 1 runs in O(1) time • each access to element 2 runs in O(2) time • ··· • each access to element n runs in O(n) time. Thus, the total time for performing the series of accesses is proportional to n+2n+3n+···+n·n = n(1+2+3+···+n) = n· n(n+1) 2 , which is O(n3). On the other hand, if we use the move-to-front heuristic, inserting each element the first time it is accessed, then • each subsequent access to element 1 takes O(1) time • each subsequent access to element 2 takes O(1) time • ··· • each subsequent access to element n runs in O(1) time. So the running time for performing all the accesses in this case is O(n2). Thus, the move-to-front implementation has faster access times for this scenario. Still, the move-to-front approach is just a heuristic, for there are access sequences where using the move-to-front approach is slower than simply keeping the favorites list ordered by access counts.
- 312. 290 Chapter 7.Linked Lists The Trade-Offs with the Move-to-Front Heuristic If we no longer maintain the elements of the favorites list ordered by their access counts, when we are asked to find the k most accessed elements, we need to search for them. We will implement the top(k) method as follows: 1. We copy all entries of our favorites list into another list, named temp. 2. We scan the temp list k times. In each scan, we find the entry with the largest access count, remove this entry from temp, and report it in the results. This implementation of method top takes O(kn) time. Thus, when k is a constant, method top runs in O(n) time. This occurs, for example, when we want to get the “top ten” list. However, if k is proportional to n, then top runs in O(n2) time. This occurs, for example, when we want a “top 25%” list. In Chapter 9 we will introduce a data structure that will allow us to implement top in O(n + klogn) time (see Exercise P-9.54), and more advanced techniques could be used to perform top in O(n+klogk) time. We could easily achieve O(nlogn) time if we use a standard sorting algorithm to reorder the temporary list before reporting the top k (see Chapter 12); this ap- proach would be preferred to the original in the case that k is Ω(logn). (Recall the big-Omega notation introduced in Section 3.3.1 to give an asymptotic lower bound on the running time of an algorithm.) There is a more specialized sorting algorithm (see Section 12.4.2) that can take advantage of the fact that access counts are integers in order to achieve O(n) time for top, for any value of k. Implementing the Move-to-Front Heuristic in Python We give an implementation of a favorites list using the move-to-front heuristic in Code Fragment 7.20. The new FavoritesListMTF class inherits most of its func- tionality from the original FavoritesList as a base class. By our original design, the access method of the original class relies on a non- public utility named move up to enact the potential shifting of an element forward in the list, after its access count had been incremented. Therefore, we implement the move-to-front heuristic by simply overriding the move up method so that each accessed element is moved directly to the front of the list (if not already there). This action is easily implemented by means of the positional list ADT. The more complex portion of our FavoritesListMTF class is the new definition for the top method. We rely on the first of the approaches outlined above, inserting copies of the items into a temporary list and then repeatedly finding, reporting, and removing an element that has the largest access count of those remaining.
- 313. 7.6. Case Study:Maintaining Access Frequencies 291 1 class FavoritesListMTF(FavoritesList): 2 ”””List of elements ordered with move-to-front heuristic.””” 3 4 # we override move up to provide move-to-front semantics 5 def move up(self, p): 6 ”””Move accessed item at Position p to front of list.””” 7 if p != self. data.first(): 8 self. data.add first(self. data.delete(p)) # delete/reinsert 9 10 # we override top because list is no longer sorted 11 def top(self, k): 12 ”””Generate sequence of top k elements in terms of access count.””” 13 if not 1 = k = len(self): 14 raise ValueError( Illegal value for k ) 15 16 # we begin by making a copy of the original list 17 temp = PositionalList() 18 for item in self. data: # positional lists support iteration 19 temp.add last(item) 20 21 # we repeatedly find, report, and remove element with largest count 22 for j in range(k): 23 # find and report next highest from temp 24 highPos = temp.first() 25 walk = temp.after(highPos) 26 while walk is not None: 27 if walk.element(). count highPos.element(). count: 28 highPos = walk 29 walk = temp.after(walk) 30 # we have found the element with highest count 31 yield highPos.element(). value # report element to user 32 temp.delete(highPos) # remove from temp list Code Fragment 7.20: Class FavoritesListMTF implementing the move-to-front heuristic. This class extends FavoritesList (Code Fragments 7.18 and 7.19) and overrides methods move up and top.
- 314. 292 Chapter 7.Linked Lists 7.7 Link-Based vs. Array-Based Sequences We close this chapter by reflecting on the relative pros and cons of array-based and link-based data structures that have been introduced thus far. The dichotomy between these approaches presents a common design decision when choosing an appropriate implementation of a data structure. There is not a one-size-fits-all so- lution, as each offers distinct advantages and disadvantages. Advantages of Array-Based Sequences • Arrays provide O(1)-time access to an element based on an integer index. The ability to access the kth element for any k in O(1) time is a hallmark advantage of arrays (see Section 5.2). In contrast, locating the kth element in a linked list requires O(k) time to traverse the list from the beginning, or possibly O(n− k) time, if traversing backward from the end of a doubly linked list. • Operations with equivalent asymptotic bounds typically run a constant factor more efficiently with an array-based structure versus a linked structure. As an example, consider the typical enqueue operation for a queue. Ignoring the issue of resizing an array, this operation for the ArrayQueue class (see Code Fragment 6.7) involves an arithmetic calculation of the new index, an increment of an integer, and storing a reference to the element in the array. In contrast, the process for a LinkedQueue (see Code Fragment 7.8) requires the instantiation of a node, appropriate linking of nodes, and an increment of an integer. While this operation completes in O(1) time in either model, the actual number of CPU operations will be more in the linked version, especially given the instantiation of the new node. • Array-based representations typically use proportionally less memory than linked structures. This advantage may seem counterintuitive, especially given that the length of a dynamic array may be longer than the number of elements that it stores. Both array-based lists and linked lists are referential structures, so the primary memory for storing the actual objects that are elements is the same for either structure. What differs is the auxiliary amounts of memory that are used by the two structures. For an array-based container of n ele- ments, a typical worst case may be that a recently resized dynamic array has allocated memory for 2n object references. With linked lists, memory must be devoted not only to store a reference to each contained object, but also explicit references that link the nodes. So a singly linked list of length n already requires 2n references (an element reference and next reference for each node). With a doubly linked list, there are 3n references.
- 315. 7.7. Link-Based vs.Array-Based Sequences 293 Advantages of Link-Based Sequences • Link-based structures provide worst-case time bounds for their operations. This is in contrast to the amortized bounds associated with the expansion or contraction of a dynamic array (see Section 5.3). When many individual operations are part of a larger computation, and we only care about the total time of that computation, an amortized bound is as good as a worst-case bound precisely because it gives a guarantee on the sum of the time spent on the individual operations. However, if data structure operations are used in a real-time system that is de- signed to provide more immediate responses (e.g., an operating system, Web server, air traffic control system), a long delay caused by a single (amortized) operation may have an adverse effect. • Link-based structures support O(1)-time insertions and deletions at arbi- trary positions. The ability to perform a constant-time insertion or deletion with the PositionalList class, by using a Position to efficiently describe the location of the operation, is perhaps the most significant advantage of the linked list. This is in stark contrast to an array-based sequence. Ignoring the issue of resizing an array, inserting or deleting an element from the end of an array- based list can be done in constant time. However, more general insertions and deletions are expensive. For example, with Python’s array-based list class, a call to insert or pop with index k uses O(n−k+1) time because of the loop to shift all subsequent elements (see Section 5.4). As an example application, consider a text editor that maintains a document as a sequence of characters. Although users often add characters to the end of the document, it is also possible to use the cursor to insert or delete one or more characters at an arbitrary position within the document. If the charac- ter sequence were stored in an array-based sequence (such as a Python list), each such edit operation may require linearly many characters to be shifted, leading to O(n) performance for each edit operation. With a linked-list rep- resentation, an arbitrary edit operation (insertion or deletion of a character at the cursor) can be performed in O(1) worst-case time, assuming we are given a position that represents the location of the cursor.
- 316. 294 Chapter 7.Linked Lists 7.8 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-7.1 Give an algorithm for finding the second-to-last node in a singly linked list in which the last node is indicated by a next reference of None. R-7.2 Describe a good algorithm for concatenating two singly linked lists L and M, given only references to the first node of each list, into a single list L that contains all the nodes of L followed by all the nodes of M. R-7.3 Describe a recursive algorithm that counts the number of nodes in a singly linked list. R-7.4 Describe in detail how to swap two nodes x and y (and not just their con- tents) in a singly linked list L given references only to x and y. Repeat this exercise for the case when L is a doubly linked list. Which algorithm takes more time? R-7.5 Implement a function that counts the number of nodes in a circularly linked list. R-7.6 Suppose that x and y are references to nodes of circularly linked lists, although not necessarily the same list. Describe a fast algorithm for telling if x and y belong to the same list. R-7.7 Our CircularQueue class of Section 7.2.2 provides a rotate() method that has semantics equivalent to Q.enqueue(Q.dequeue()), for a nonempty queue. Implement such a method for the LinkedQueue class of Sec- tion 7.1.2 without the creation of any new nodes. R-7.8 Describe a nonrecursive method for finding, by link hopping, the middle node of a doubly linked list with header and trailer sentinels. In the case of an even number of nodes, report the node slightly left of center as the “middle.” (Note: This method must only use link hopping; it cannot use a counter.) What is the running time of this method? R-7.9 Give a fast algorithm for concatenating two doubly linked lists L and M, with header and trailer sentinel nodes, into a single list L. R-7.10 There seems to be some redundancy in the repertoire of the positional list ADT, as the operation L.add first(e) could be enacted by the alter- native L.add before(L.first(), e). Likewise, L.add last(e) might be per- formed as L.add after(L.last(), e). Explain why the methods add first and add last are necessary.
- 317. 7.8. Exercises 295 R-7.11Implement a function, with calling syntax max(L), that returns the max- imum element from a PositionalList instance L containing comparable elements. R-7.12 Redo the previously problem with max as a method of the PositionalList class, so that calling syntax L.max() is supported. R-7.13 Update the PositionalList class to support an additional method find(e), which returns the position of the (first occurrence of) element e in the list (or None if not found). R-7.14 Repeat the previous process using recursion. Your method should not contain any loops. How much space does your method use in addition to the space used for L? R-7.15 Provide support for a reversed method of the PositionalList class that is similar to the given iter , but that iterates the elements in reversed order. R-7.16 Describe an implementation of the PositionalList methods add last and add before realized by using only methods in the set {is empty, first, last, prev, next, add after, and add first}. R-7.17 In the FavoritesListMTF class, we rely on public methods of the positional list ADT to move an element of a list at position p to become the first ele- ment of the list, while keeping the relative order of the remaining elements unchanged. Internally, that combination of operations causes one node to be removed and a new node to be inserted. Augment the PositionalList class to support a new method, move to front(p), that accomplishes this goal more directly, by relinking the existing node. R-7.18 Given the set of element {a,b,c,d,e, f} stored in a list, show the final state of the list, assuming we use the move-to-front heuristic and access the el- ements according to the following sequence: (a,b,c,d,e, f,a,c, f,b,d,e). R-7.19 Suppose that we have made kn total accesses to the elements in a list L of n elements, for some integer k ≥ 1. What are the minimum and maximum number of elements that have been accessed fewer than k times? R-7.20 Let L be a list of n items maintained according to the move-to-front heuris- tic. Describe a series of O(n) accesses that will reverse L. R-7.21 Suppose we have an n-element list L maintained according to the move- to-front heuristic. Describe a sequence of n2 accesses that is guaranteed to take Ω(n3) time to perform on L. R-7.22 Implement a clear() method for the FavoritesList class that returns the list to empty. R-7.23 Implement a reset counts() method for the FavoritesList class that resets all elements’ access counts to zero (while leaving the order of the list unchanged).
- 318. 296 Chapter 7.Linked Lists Creativity C-7.24 Give a complete implementation of the stack ADT using a singly linked list that includes a header sentinel. C-7.25 Give a complete implementation of the queue ADT using a singly linked list that includes a header sentinel. C-7.26 Implement a method, concatenate(Q2) for the LinkedQueue class that takes all elements of LinkedQueue Q2 and appends them to the end of the original queue. The operation should run in O(1) time and should result in Q2 being an empty queue. C-7.27 Give a recursive implementation of a singly linked list class, such that an instance of a nonempty list stores its first element and a reference to a list of remaining elements. C-7.28 Describe a fast recursive algorithm for reversing a singly linked list. C-7.29 Describe in detail an algorithm for reversing a singly linked list L using only a constant amount of additional space and not using any recursion. C-7.30 Exercise P-6.35 describes a LeakyStack abstraction. Implement that ADT using a singly linked list for storage. C-7.31 Design a forward list ADT that abstracts the operations on a singly linked list, much as the positional list ADT abstracts the use of a doubly linked list. Implement a ForwardList class that supports such an ADT. C-7.32 Design a circular positional list ADT that abstracts a circularly linked list in the same way that the positional list ADT abstracts a doubly linked list, with a notion of a designated “cursor” position within the list. C-7.33 Modify the DoublyLinkedBase class to include a reverse method that re- verses the order of the list, yet without creating or destroying any nodes. C-7.34 Modify the PositionalList class to support a method swap(p, q) that causes the underlying nodes referenced by positions p and q to be exchanged for each other. Relink the existing nodes; do not create any new nodes. C-7.35 To implement the iter method of the PositionalList class, we relied on the convenience of Python’s generator syntax and the yield statement. Give an alternative implementation of iter by designing a nested iterator class. (See Section 2.3.4 for discussion of iterators.) C-7.36 Give a complete implementation of the positional list ADT using a doubly linked list that does not include any sentinel nodes. C-7.37 Implement a function that accepts a PositionalList L of n integers sorted in nondecreasing order, and another value V, and determines in O(n) time if there are two elements of L that sum precisely toV. The function should return a pair of positions of such elements, if found, or None otherwise.
- 319. 7.8. Exercises 297 C-7.38There is a simple, but inefficient, algorithm, called bubble-sort, for sorting a list L of n comparable elements. This algorithm scans the list n−1 times, where, in each scan, the algorithm compares the current element with the next one and swaps them if they are out of order. Implement a bubble sort function that takes a positional list L as a parameter. What is the running time of this algorithm, assuming the positional list is implemented with a doubly linked list? C-7.39 To better model a FIFO queue in which entries may be deleted before reaching the front, design a PositionalQueue class that supports the com- plete queue ADT, yet with enqueue returning a position instance and sup- port for a new method, delete(p), that removes the element associated with position p from the queue. You may use the adapter design pattern (Section 6.1.2), using a PositionalList as your storage. C-7.40 Describe an efficient method for maintaining a favorites list L, with move- to-front heuristic, such that elements that have not been accessed in the most recent n accesses are automatically purged from the list. C-7.41 Exercise C-5.29 introduces the notion of a natural join of two databases. Describe and analyze an efficient algorithm for computing the natural join of a linked list A of n pairs and a linked list B of m pairs. C-7.42 Write a Scoreboard class that maintains the top 10 scores for a game ap- plication using a singly linked list, rather than the array that was used in Section 5.5.1. C-7.43 Describe a method for performing a card shuffle of a list of 2n elements, by converting it into two lists. A card shuffle is a permutation where a list L is cut into two lists, L1 and L2, where L1 is the first half of L and L2 is the second half of L, and then these two lists are merged into one by taking the first element in L1, then the first element in L2, followed by the second element in L1, the second element in L2, and so on. Projects P-7.44 Write a simple text editor that stores and displays a string of characters using the positional list ADT, together with a cursor object that highlights a position in this string. A simple interface is to print the string and then to use a second line of output to underline the position of the cursor. Your editor should support the following operations: • left: Move cursor left one character (do nothing if at beginning). • right: Move cursor right one character (do nothing if at end). • insert c: Insert the character c just after the cursor. • delete: Delete the character just after the cursor (do nothing at end).
- 320. 298 Chapter 7.Linked Lists P-7.45 An array A is sparse if most of its entries are empty (i.e., None). A list L can be used to implement such an array efficiently. In particular, for each nonempty cell A[i], we can store an entry (i,e) in L, where e is the element stored at A[i]. This approach allows us to represent A using O(m) storage, where m is the number of nonempty entries in A. Provide such a SparseArray class that minimally supports methods getitem (j) and setitem (j, e) to provide standard indexing operations. Analyze the efficiency of these methods. P-7.46 Although we have used a doubly linked list to implement the positional list ADT, it is possible to support the ADT with an array-based imple- mentation. The key is to use the composition pattern and store a sequence of position items, where each item stores an element as well as that ele- ment’s current index in the array. Whenever an element’s place in the array is changed, the recorded index in the position must be updated to match. Given a complete class providing such an array-based implementation of the positional list ADT. What is the efficiency of the various operations? P-7.47 Implement a CardHand class that supports a person arranging a group of cards in his or her hand. The simulator should represent the sequence of cards using a single positional list ADT so that cards of the same suit are kept together. Implement this strategy by means of four “fingers” into the hand, one for each of the suits of hearts, clubs, spades, and diamonds, so that adding a new card to the person’s hand or playing a correct card from the hand can be done in constant time. The class should support the following methods: • add card(r, s): Add a new card with rank r and suit s to the hand. • play(s): Remove and return a card of suit s from the player’s hand; if there is no card of suit s, then remove and return an arbitrary card from the hand. • iter (): Iterate through all cards currently in the hand. • all of suit(s): Iterate through all cards of suit s that are currently in the hand. Chapter Notes A view of data structures as collections (and other principles of object-oriented design) can be found in object-oriented design books by Booch [17], Budd [20], Goldberg and Robson [42], and Liskov and Guttag [71]. Our positional list ADT is derived from the “position” abstraction introduced by Aho, Hopcroft, and Ullman [6], and the list ADT of Wood [104]. Implementations of linked lists are discussed by Knuth [64].
- 321. Chapter 8 Trees Contents 8.1 GeneralTrees . . . . . . . . . . . . . . . . . . . . . . . . . 300 8.1.1 Tree Definitions and Properties . . . . . . . . . . . . . . . 301 8.1.2 The Tree Abstract Data Type . . . . . . . . . . . . . . . 305 8.1.3 Computing Depth and Height . . . . . . . . . . . . . . . . 308 8.2 Binary Trees . . . . . . . . . . . . . . . . . . . . . . . . . . 311 8.2.1 The Binary Tree Abstract Data Type . . . . . . . . . . . . 313 8.2.2 Properties of Binary Trees . . . . . . . . . . . . . . . . . 315 8.3 Implementing Trees . . . . . . . . . . . . . . . . . . . . . . 317 8.3.1 Linked Structure for Binary Trees . . . . . . . . . . . . . . 317 8.3.2 Array-Based Representation of a Binary Tree . . . . . . . 325 8.3.3 Linked Structure for General Trees . . . . . . . . . . . . . 327 8.4 Tree Traversal Algorithms . . . . . . . . . . . . . . . . . . . 328 8.4.1 Preorder and Postorder Traversals of General Trees . . . . 328 8.4.2 Breadth-First Tree Traversal . . . . . . . . . . . . . . . . 330 8.4.3 Inorder Traversal of a Binary Tree . . . . . . . . . . . . . 331 8.4.4 Implementing Tree Traversals in Python . . . . . . . . . . 333 8.4.5 Applications of Tree Traversals . . . . . . . . . . . . . . . 337 8.4.6 Euler Tours and the Template Method Pattern . . . . . 341 8.5 Case Study: An Expression Tree . . . . . . . . . . . . . . . 348 8.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
- 322. 300 Chapter 8.Trees 8.1 General Trees Productivity experts say that breakthroughs come by thinking “nonlinearly.” In this chapter, we discuss one of the most important nonlinear data structures in computing—trees. Tree structures are indeed a breakthrough in data organization, for they allow us to implement a host of algorithms much faster than when using linear data structures, such as array-based lists or linked lists. Trees also provide a natural organization for data, and consequently have become ubiquitous structures in file systems, graphical user interfaces, databases, Web sites, and other computer systems. It is not always clear what productivity experts mean by “nonlinear” thinking, but when we say that trees are “nonlinear,” we are referring to an organizational relationship that is richer than the simple “before” and “after” relationships be- tween objects in sequences. The relationships in a tree are hierarchical, with some objects being “above” and some “below” others. Actually, the main terminology for tree data structures comes from family trees, with the terms “parent,” “child,” “ancestor,” and “descendant” being the most common words used to describe rela- tionships. We show an example of a family tree in Figure 8.1. Eldaah Nebaioth Kedar Adbeel Mibsam Mishma Dumah Massa Hadad Tema Jetur Naphish Kedemah Ishmael Gad Naphtali Dan Judah Levi Simeon Asher Issachar Zebulun Dinah Joseph Benjamin Eliphaz Reuel Jeush Jalam Reuben Korah Jacob (Israel) Esau Isaac Zimran Jokshan Medan Midian Ishbak Shuah Abraham Sheba Dedan Ephah Epher Hanoch Abida Figure 8.1: A family tree showing some descendants of Abraham, as recorded in Genesis, chapters 25–36.
- 323. 8.1. General Trees301 8.1.1 Tree Definitions and Properties A tree is an abstract data type that stores elements hierarchically. With the excep- tion of the top element, each element in a tree has a parent element and zero or more children elements. A tree is usually visualized by placing elements inside ovals or rectangles, and by drawing the connections between parents and children with straight lines. (See Figure 8.2.) We typically call the top element the root of the tree, but it is drawn as the highest element, with the other elements being connected below (just the opposite of a botanical tree). Europe Asia Africa Australia Canada Overseas S. America Domestic International TV CD Tuner Sales Purchasing Manufacturing RD Electronics R’Us Figure 8.2: A tree with 17 nodes representing the organization of a fictitious cor- poration. The root stores Electronics R’Us. The children of the root store RD, Sales, Purchasing, and Manufacturing. The internal nodes store Sales, Interna- tional, Overseas, Electronics R’Us, and Manufacturing. Formal Tree Definition Formally, we define a tree T as a set of nodes storing elements such that the nodes have a parent-child relationship that satisfies the following properties: • If T is nonempty, it has a special node, called the root of T, that has no parent. • Each node v of T different from the root has a unique parent node w; every node with parent w is a child of w. Note that according to our definition, a tree can be empty, meaning that it does not have any nodes. This convention also allows us to define a tree recursively such that a tree T is either empty or consists of a node r, called the root of T, and a (possibly empty) set of subtrees whose roots are the children of r.
- 324. 302 Chapter 8.Trees Other Node Relationships Two nodes that are children of the same parent are siblings. A node v is external if v has no children. A node v is internal if it has one or more children. External nodes are also known as leaves. Example 8.1: In Section 4.1.4, we discussed the hierarchical relationship be- tween files and directories in a computer’s file system, although at the time we did not emphasize the nomenclature of a file system as a tree. In Figure 8.3, we revisit an earlier example. We see that the internal nodes of the tree are associ- ated with directories and the leaves are associated with regular files. In the UNIX and Linux operating systems, the root of the tree is appropriately called the “root directory,” and is represented by the symbol “/.” /user/rt/courses/ cs016/ cs252/ programs/ homeworks/ projects/ papers/ demos/ hw1 hw2 hw3 pr1 pr2 pr3 grades market buylow sellhigh grades Figure 8.3: Tree representing a portion of a file system. A node u is an ancestor of a node v if u = v or u is an ancestor of the parent of v. Conversely, we say that a node v is a descendant of a node u if u is an ancestor of v. For example, in Figure 8.3, cs252/ is an ancestor of papers/, and pr3 is a descendant of cs016/. The subtree of T rooted at a node v is the tree consisting of all the descendants of v in T (including v itself). In Figure 8.3, the subtree rooted at cs016/ consists of the nodes cs016/, grades, homeworks/, programs/, hw1, hw2, hw3, pr1, pr2, and pr3. Edges and Paths in Trees An edge of tree T is a pair of nodes (u,v) such that u is the parent of v, or vice versa. A path of T is a sequence of nodes such that any two consecutive nodes in the sequence form an edge. For example, the tree in Figure 8.3 contains the path (cs252/, projects/, demos/, market).
- 325. 8.1. General Trees303 Example 8.2: The inheritance relation between classes in a Python program forms a tree when single inheritance is used. For example, in Section 2.4 we provided a summary of the hierarchy for Python’s exception types, as portrayed in Figure 8.4 (originally Figure 2.5). The BaseException class is the root of that hierarchy, while all user-defined exception classes should conventionally be declared as descendants of the more specific Exception class. (See, for example, the Empty class we intro- duced in Code Fragment 6.1 of Chapter 6.) ValueError Exception KeyboardInterrupt SystemExit BaseException IndexError KeyError ZeroDivisionError LookupError ArithmeticError Figure 8.4: A portion of Python’s hierarchy of exception types. In Python, all classes are organized into a single hierarchy, as there exists a built-in class named object as the ultimate base class. It is a direct or indirect base class of all other types in Python (even if not declared as such when defining a new class). Therefore, the hierarchy pictured in Figure 8.4 is only a portion of Python’s complete class hierarchy. As a preview of the remainder of this chapter, Figure 8.5 portrays our own hierarchy of classes for representing various forms of a tree. ArrayBinaryTree LinkedBinaryTree Tree LinkedTree BinaryTree Figure 8.5: Our own inheritance hierarchy for modeling various abstractions and implementations of tree data structures. In the remainder of this chapter, we provide implementations of Tree, BinaryTree, and LinkedBinaryTree classes, and high- level sketches for how LinkedTree and ArrayBinaryTree might be designed.
- 326. 304 Chapter 8.Trees Ordered Trees A tree is ordered if there is a meaningful linear order among the children of each node; that is, we purposefully identify the children of a node as being the first, second, third, and so on. Such an order is usually visualized by arranging siblings left to right, according to their order. Example 8.3: The components of a structured document, such as a book, are hier- archically organized as a tree whose internal nodes are parts, chapters, and sections, and whose leaves are paragraphs, tables, figures, and so on. (See Figure 8.6.) The root of the tree corresponds to the book itself. We could, in fact, consider expanding the tree further to show paragraphs consisting of sentences, sentences consisting of words, and words consisting of characters. Such a tree is an example of an ordered tree, because there is a well-defined order among the children of each node. ... ... ¶ ¶ ... ¶ ¶ Book Part A Part B References Preface ... ... ... ... Ch. 1 Ch. 5 Ch. 6 Ch. 9 ¶ ¶ ¶ ¶ ... ... ... ... § 1.4 § 1.1 § 5.7 § 5.1 § 9.6 § 9.1 § 6.5 § 6.1 Figure 8.6: An ordered tree associated with a book. Let’s look back at the other examples of trees that we have described thus far, and consider whether the order of children is significant. A family tree that de- scribes generational relationships, as in Figure 8.1, is often modeled as an ordered tree, with siblings ordered according to their birth. In contrast, an organizational chart for a company, as in Figure 8.2, is typically considered an unordered tree. Likewise, when using a tree to describe an inher- itance hierarchy, as in Figure 8.4, there is no particular significance to the order among the subclasses of a parent class. Finally, we consider the use of a tree in modeling a computer’s file system, as in Figure 8.3. Although an operating system often displays entries of a directory in a particular order (e.g., alphabetical, chrono- logical), such an order is not typically inherent to the file system’s representation.
- 327. 8.1. General Trees305 8.1.2 The Tree Abstract Data Type As we did with positional lists in Section 7.4, we define a tree ADT using the concept of a position as an abstraction for a node of a tree. An element is stored at each position, and positions satisfy parent-child relationships that define the tree structure. A position object for a tree supports the method: p.element(): Return the element stored at position p. The tree ADT then supports the following accessor methods, allowing a user to navigate the various positions of a tree: T.root(): Return the position of the root of tree T, or None if T is empty. T.is root(p): Return True if position p is the root of Tree T. T.parent(p): Return the position of the parent of position p, or None if p is the root of T. T.num children(p): Return the number of children of position p. T.children(p): Generate an iteration of the children of position p. T.is leaf(p): Return True if position p does not have any children. len(T): Return the number of positions (and hence elements) that are contained in tree T. T.is empty(): Return True if tree T does not contain any positions. T.positions(): Generate an iteration of all positions of tree T. iter(T): Generate an iteration of all elements stored within tree T. Any of the above methods that accepts a position as an argument should generate a ValueError if that position is invalid for T. If a tree T is ordered, then T.children(p) reports the children of p in the natural order. If p is a leaf, then T.children(p) generates an empty iteration. In similar regard, if tree T is empty, then both T.positions() and iter(T) generate empty iter- ations. We will discuss general means for iterating through all positions of a tree in Sections 8.4. We do not define any methods for creating or modifying trees at this point. We prefer to describe different tree update methods in conjunction with specific implementations of the tree interface, and specific applications of trees.
- 328. 306 Chapter 8.Trees A Tree Abstract Base Class in Python In discussing the object-oriented design principle of abstraction in Section 2.1.2, we noted that a public interface for an abstract data type is often managed in Python via duck typing. For example, we defined the notion of the public interface for a queue ADT in Section 6.2, and have since presented several classes that implement the queue interface (e.g., ArrayQueue in Section 6.2.2, LinkedQueue in Section 7.1.2, CircularQueue in Section 7.2.2). However, we never gave any formal definition of the queue ADT in Python; all of the concrete implementations were self-contained classes that just happen to adhere to the same public interface. A more formal mechanism to designate the relationships between different implementations of the same abstraction is through the definition of one class that serves as an abstract base class, via inheritance, for one or more concrete classes. (See Section 2.4.3.) We choose to define a Tree class, in Code Fragment 8.1, that serves as an ab- stract base class corresponding to the tree ADT. Our reason for doing so is that there is quite a bit of useful code that we can provide, even at this level of abstraction, al- lowing greater code reuse in the concrete tree implementations we later define. The Tree class provides a definition of a nested Position class (which is also abstract), and declarations of many of the accessor methods included in the tree ADT. However, our Tree class does not define any internal representation for stor- ing a tree, and five of the methods given in that code fragment remain abstract (root, parent, num children, children, and len ); each of these methods raises a NotImplementedError. (A more formal approach for defining abstract base classes and abstract methods, using Python’s abc module, is described in Section 2.4.3.) The subclasses are responsible for overriding abstract methods, such as children, to provide a working implementation for each behavior, based on their chosen internal representation. Although the Tree class is an abstract base class, it includes several concrete methods with implementations that rely on calls to the abstract methods of the class. In defining the tree ADT in the previous section, we declare ten accessor methods. Five of those are the ones we left as abstract, in Code Fragment 8.1. The other five can be implemented based on the former. Code Fragment 8.2 provides concrete implementations for methods is root, is leaf, and is empty. In Section 8.4, we will explore general algorithms for traversing a tree that can be used to provide concrete implementations of the positions and iter methods within the Tree class. The beauty of this design is that the concrete methods defined within the Tree abstract base class will be inherited by all subclasses. This promotes greater code reuse, as there will be no need for those subclasses to reimplement such behaviors. We note that, with the Tree class being abstract, there is no reason to create a direct instance of it, nor would such an instance be useful. The class exists to serve as a base for inheritance, and users will create instances of concrete subclasses.
- 329. 8.1. General Trees307 1 class Tree: 2 ”””Abstract base class representing a tree structure.””” 3 4 #------------------------------- nested Position class ------------------------------- 5 class Position: 6 ”””An abstraction representing the location of a single element.””” 7 8 def element(self): 9 ”””Return the element stored at this Position.””” 10 raise NotImplementedError( must be implemented by subclass ) 11 12 def eq (self, other): 13 ”””Return True if other Position represents the same location.””” 14 raise NotImplementedError( must be implemented by subclass ) 15 16 def ne (self, other): 17 ”””Return True if other does not represent the same location.””” 18 return not (self == other) # opposite of eq 19 20 # ---------- abstract methods that concrete subclass must support ---------- 21 def root(self): 22 ”””Return Position representing the tree s root (or None if empty).””” 23 raise NotImplementedError( must be implemented by subclass ) 24 25 def parent(self, p): 26 ”””Return Position representing p s parent (or None if p is root).””” 27 raise NotImplementedError( must be implemented by subclass ) 28 29 def num children(self, p): 30 ”””Return the number of children that Position p has.””” 31 raise NotImplementedError( must be implemented by subclass ) 32 33 def children(self, p): 34 ”””Generate an iteration of Positions representing p s children.””” 35 raise NotImplementedError( must be implemented by subclass ) 36 37 def len (self): 38 ”””Return the total number of elements in the tree.””” 39 raise NotImplementedError( must be implemented by subclass ) Code Fragment 8.1: A portion of our Tree abstract base class (continued in Code Fragment 8.2).
- 330. 308 Chapter 8.Trees 40 # ---------- concrete methods implemented in this class ---------- 41 def is root(self, p): 42 ”””Return True if Position p represents the root of the tree.””” 43 return self.root( ) == p 44 45 def is leaf(self, p): 46 ”””Return True if Position p does not have any children.””” 47 return self.num children(p) == 0 48 49 def is empty(self): 50 ”””Return True if the tree is empty.””” 51 return len(self) == 0 Code Fragment 8.2: Some concrete methods of our Tree abstract base class. 8.1.3 Computing Depth and Height Let p be the position of a node of a tree T. The depth of p is the number of ancestors of p, excluding p itself. For example, in the tree of Figure 8.2, the node storing International has depth 2. Note that this definition implies that the depth of the root of T is 0. The depth of p can also be recursively defined as follows: • If p is the root, then the depth of p is 0. • Otherwise, the depth of p is one plus the depth of the parent of p. Based on this definition, we present a simple, recursive algorithm, depth, in Code Fragment 8.3, for computing the depth of a position p in Tree T. This method calls itself recursively on the parent of p, and adds 1 to the value returned. 52 def depth(self, p): 53 ”””Return the number of levels separating Position p from the root.””” 54 if self.is root(p): 55 return 0 56 else: 57 return 1 + self.depth(self.parent(p)) Code Fragment 8.3: Method depth of the Tree class. The running time of T.depth(p) for position p is O(dp +1), where dp denotes the depth of p in the tree T, because the algorithm performs a constant-time recur- sive step for each ancestor of p. Thus, algorithm T.depth(p) runs in O(n) worst- case time, where n is the total number of positions of T, because a position of T may have depth n− 1 if all nodes form a single branch. Although such a running time is a function of the input size, it is more informative to characterize the running time in terms of the parameter dp, as this parameter may be much smaller than n.
- 331. 8.1. General Trees309 Height The height of a position p in a tree T is also defined recursively: • If p is a leaf, then the height of p is 0. • Otherwise, the height of p is one more than the maximum of the heights of p’s children. The height of a nonempty tree T is the height of the root of T. For example, the tree of Figure 8.2 has height 4. In addition, height can also be viewed as follows. Proposition 8.4: The height of a nonempty tree T is equal to the maximum of the depths of its leaf positions. We leave the justification of this fact to an exercise (R-8.3). We present an algorithm, height1, implemented in Code Fragment 8.4 as a nonpublic method height1 of the Tree class. It computes the height of a nonempty tree T based on Proposition 8.4 and the algorithm depth from Code Fragment 8.3. 58 def height1(self): # works, but O(nˆ2) worst-case time 59 ”””Return the height of the tree.””” 60 return max(self.depth(p) for p in self.positions( ) if self.is leaf(p)) Code Fragment 8.4: Method height1 of the Tree class. Note that this method calls the depth method. Unfortunately, algorithm height1 is not very efficient. We have not yet defined the positions() method; we will see that it can be implemented to run in O(n) time, where n is the number of positions of T. Because height1 calls algorithm depth(p) on each leaf of T, its running time is O(n + ∑p∈L(dp + 1)), where L is the set of leaf positions of T. In the worst case, the sum ∑p∈L(dp + 1) is proportional to n2. (See Exercise C-8.33.) Thus, algorithm height1 runs in O(n2) worst-case time. We can compute the height of a tree more efficiently, in O(n) worst-case time, by relying instead on the original recursive definition. To do this, we will param- eterize a function based on a position within the tree, and calculate the height of the subtree rooted at that position. Algorithm height2, shown as nonpublic method height2 in Code Fragment 8.5, computes the height of tree T in this way. 61 def height2(self, p): # time is linear in size of subtree 62 ”””Return the height of the subtree rooted at Position p.””” 63 if self.is leaf(p): 64 return 0 65 else: 66 return 1 + max(self. height2(c) for c in self.children(p)) Code Fragment 8.5: Method height2 for computing the height of a subtree rooted at a position p of a Tree.
- 332. 310 Chapter 8.Trees It is important to understand why algorithm height2 is more efficient than height1. The algorithm is recursive, and it progresses in a top-down fashion. If the method is initially called on the root of T, it will eventually be called once for each position of T. This is because the root eventually invokes the recursion on each of its children, which in turn invokes the recursion on each of their children, and so on. We can determine the running time of the height2 algorithm by summing, over all the positions, the amount of time spent on the nonrecursive part of each call. (Review Section 4.2 for analyses of recursive processes.) In our implementation, there is a constant amount of work per position, plus the overhead of computing the maximum over the iteration of children. Although we do not yet have a concrete implementation of children(p), we assume that such an iteration is generated in O(cp +1) time, where cp denotes the number of children of p. Algorithm height2 spends O(cp +1) time at each position p to compute the maximum, and its overall running time is O(∑p(cp + 1)) = O(n+ ∑p cp). In order to complete the analysis, we make use of the following property. Proposition 8.5: Let T be a tree with n positions, and let cp denote the number of children of a position p of T. Then, summing over the positions of T, ∑p cp = n−1. Justification: Each position of T, with the exception of the root, is a child of another position, and thus contributes one unit to the above sum. By Proposition 8.5, the running time of algorithm height2, when called on the root of T, is O(n), where n is the number of positions of T. Revisiting the public interface for our Tree class, the ability to compute heights of subtrees is beneficial, but a user might expect to be able to compute the height of the entire tree without explicitly designating the tree root. We can wrap the non- public height2 in our implementation with a public height method that provides a default interpretation when invoked on tree T with syntax T.height(). Such an implementation is given in Code Fragment 8.6. 67 def height(self, p=None): 68 ”””Return the height of the subtree rooted at Position p. 69 70 If p is None, return the height of the entire tree. 71 ””” 72 if p is None: 73 p = self.root() 74 return self. height2(p) # start height2 recursion Code Fragment 8.6: Public method Tree.height that computes the height of the entire tree by default, or a subtree rooted at given position, if specified.
- 333. 8.2. Binary Trees311 8.2 Binary Trees A binary tree is an ordered tree with the following properties: 1. Every node has at most two children. 2. Each child node is labeled as being either a left child or a right child. 3. A left child precedes a right child in the order of children of a node. The subtree rooted at a left or right child of an internal node v is called a left subtree or right subtree, respectively, of v. A binary tree is proper if each node has either zero or two children. Some people also refer to such trees as being full binary trees. Thus, in a proper binary tree, every internal node has exactly two children. A binary tree that is not proper is improper. Example 8.6: An important class of binary trees arises in contexts where we wish to represent a number of different outcomes that can result from answering a series of yes-or-no questions. Each internal node is associated with a question. Starting at the root, we go to the left or right child of the current node, depending on whether the answer to the question is “Yes” or “No.” With each decision, we follow an edge from a parent to a child, eventually tracing a path in the tree from the root to a leaf. Such binary trees are known as decision trees, because a leaf position p in such a tree represents a decision of what to do if the questions associated with p’s ancestors are answered in a way that leads to p. A decision tree is a proper binary tree. Figure 8.7 illustrates a decision tree that provides recommendations to a prospective investor. Yes Yes Yes No No No Are you nervous? Will you need to access most of the money within the next 5 years? Are you willing to accept risks in exchange for higher expected returns? Money market fund. Stock portfolio. Savings account. Diversified portfolio with stocks, bonds, and short-term instruments. Figure 8.7: A decision tree providing investment advice.
- 334. 312 Chapter 8.Trees Example 8.7: An arithmetic expression can be represented by a binary tree whose leaves are associated with variables or constants, and whose internal nodes are associated with one of the operators +, −, ×, and /. (See Figure 8.8.) Each node in such a tree has a value associated with it. • If a node is leaf, then its value is that of its variable or constant. • If a node is internal, then its value is defined by applying its operation to the values of its children. An arithmetic expression tree is a proper binary tree, since each operator +, −, ×, and / takes exactly two operands. Of course, if we were to allow unary operators, like negation (−), as in “−x,” then we could have an improper binary tree. / 3 1 + × 3 2 + 9 5 − 3 × 4 7 − 6 + − Figure 8.8: A binary tree representing an arithmetic expression. This tree represents the expression ((((3 + 1) × 3)/((9 − 5) + 2)) − ((3 × (7 − 4)) + 6)). The value associated with the internal node labeled “/” is 2. A Recursive Binary Tree Definition Incidentally, we can also define a binary tree in a recursive way such that a binary tree is either empty or consists of: • A node r, called the root of T, that stores an element • A binary tree (possibly empty), called the left subtree of T • A binary tree (possibly empty), called the right subtree of T
- 335. 8.2. Binary Trees313 8.2.1 The Binary Tree Abstract Data Type As an abstract data type, a binary tree is a specialization of a tree that supports three additional accessor methods: T.left(p): Return the position that represents the left child of p, or None if p has no left child. T.right(p): Return the position that represents the right child of p, or None if p has no right child. T.sibling(p): Return the position that represents the sibling of p, or None if p has no sibling. Just as in Section 8.1.2 for the tree ADT, we do not define specialized update meth- ods for binary trees here. Instead, we will consider some possible update methods when we describe specific implementations and applications of binary trees. The BinaryTree Abstract Base Class in Python Just as Tree was defined as an abstract base class in Section 8.1.2, we define a new BinaryTree class associated with the binary tree ADT. We rely on inheritance to define the BinaryTree class based upon the existing Tree class. However, our BinaryTree class remains abstract, as we still do not provide complete specifica- tions for how such a structure will be represented internally, nor implementations for some necessary behaviors. Our Python implementation of the BinaryTree class is given in Code Frag- ment 8.7. By using inheritance, a binary tree supports all the functionality that was defined for general trees (e.g., parent, is leaf, root). Our new class also inherits the nested Position class that was originally defined within the Tree class definition. In addition, the new class provides declarations for new abstract methods left and right that should be supported by concrete subclasses of BinaryTree. Our new class also provides two concrete implementations of methods. The new sibling method is derived from the combination of left, right, and parent. Typ- ically, we identify the sibling of a position p as the “other” child of p’s parent. However, if p is the root, it has no parent, and thus no sibling. Also, p may be the only child of its parent, and thus does not have a sibling. Finally, Code Fragment 8.7 provides a concrete implementation of the children method; this method is abstract in the Tree class. Although we have still not speci- fied how the children of a node will be stored, we derive a generator for the ordered children based upon the implied behavior of abstract methods left and right.
- 336. 314 Chapter 8.Trees 1 class BinaryTree(Tree): 2 ”””Abstract base class representing a binary tree structure.””” 3 4 # --------------------- additional abstract methods --------------------- 5 def left(self, p): 6 ”””Return a Position representing p s left child. 7 8 Return None if p does not have a left child. 9 ””” 10 raise NotImplementedError( must be implemented by subclass ) 11 12 def right(self, p): 13 ”””Return a Position representing p s right child. 14 15 Return None if p does not have a right child. 16 ””” 17 raise NotImplementedError( must be implemented by subclass ) 18 19 # ---------- concrete methods implemented in this class ---------- 20 def sibling(self, p): 21 ”””Return a Position representing p s sibling (or None if no sibling).””” 22 parent = self.parent(p) 23 if parent is None: # p must be the root 24 return None # root has no sibling 25 else: 26 if p == self.left(parent): 27 return self.right(parent) # possibly None 28 else: 29 return self.left(parent) # possibly None 30 31 def children(self, p): 32 ”””Generate an iteration of Positions representing p s children.””” 33 if self.left(p) is not None: 34 yield self.left(p) 35 if self.right(p) is not None: 36 yield self.right(p) Code Fragment 8.7: A BinaryTree abstract base class that extends the existing Tree abstract base class from Code Fragments 8.1 and 8.2.
- 337. 8.2. Binary Trees315 8.2.2 Properties of Binary Trees Binary trees have several interesting properties dealing with relationships between their heights and number of nodes. We denote the set of all nodes of a tree T at the same depth d as level d of T. In a binary tree, level 0 has at most one node (the root), level 1 has at most two nodes (the children of the root), level 2 has at most four nodes, and so on. (See Figure 8.9.) In general, level d has at most 2d nodes. ... 0 ... ... 1 2 3 1 ... 2 4 8 Level Nodes Figure 8.9: Maximum number of nodes in the levels of a binary tree. We can see that the maximum number of nodes on the levels of a binary tree grows exponentially as we go down the tree. From this simple observation, we can derive the following properties relating the height of a binary tree T with its number of nodes. A detailed justification of these properties is left as Exercise R-8.8. Proposition 8.8: Let T be a nonempty binary tree, and let n, nE, nI and h denote the number of nodes, number of external nodes, number of internal nodes, and height of T, respectively. Then T has the following properties: 1. h+1 ≤ n ≤ 2h+1 −1 2. 1 ≤ nE ≤ 2h 3. h ≤ nI ≤ 2h −1 4. log(n+1)−1 ≤ h ≤ n−1 Also, if T is proper, then T has the following properties: 1. 2h+1 ≤ n ≤ 2h+1 −1 2. h+1 ≤ nE ≤ 2h 3. h ≤ nI ≤ 2h −1 4. log(n+1)−1 ≤ h ≤ (n−1)/2
- 338. 316 Chapter 8.Trees Relating Internal Nodes to External Nodes in a Proper Binary Tree In addition to the earlier binary tree properties, the following relationship exists between the number of internal nodes and external nodes in a proper binary tree. Proposition 8.9: In a nonempty proper binary tree T, with nE external nodes and nI internal nodes, we have nE = nI +1. Justification: We justify this proposition by removing nodes from T and divid- ing them up into two “piles,” an internal-node pile and an external-node pile, until T becomes empty. The piles are initially empty. By the end, we will show that the external-node pile has one more node than the internal-node pile. We consider two cases: Case 1: If T has only one node v, we remove v and place it on the external-node pile. Thus, the external-node pile has one node and the internal-node pile is empty. Case 2: Otherwise (T has more than one node), we remove from T an (arbitrary) external node w and its parent v, which is an internal node. We place w on the external-node pile and v on the internal-node pile. If v has a parent u, then we reconnect u with the former sibling z of w, as shown in Figure 8.10. This operation, removes one internal node and one external node, and leaves the tree being a proper binary tree. Repeating this operation, we eventually are left with a final tree consisting of a single node. Note that the same number of external and internal nodes have been removed and placed on their respective piles by the sequence of operations leading to this final tree. Now, we remove the node of the final tree and we place it on the external-node pile. Thus, the the external-node pile has one more node than the internal-node pile. v u w z u z u z (a) (b) (c) Figure 8.10: Operation that removes an external node and its parent node, used in the justification of Proposition 8.9. Note that the above relationship does not hold, in general, for improper binary trees and nonbinary trees, although there are other interesting relationships that do hold. (See Exercises C-8.32 through C-8.34.)
- 339. 8.3. Implementing Trees317 8.3 Implementing Trees The Tree and BinaryTree classes that we have defined thus far in this chapter are both formally abstract base classes. Although they provide a great deal of support, neither of them can be directly instantiated. We have not yet defined key imple- mentation details for how a tree will be represented internally, and how we can effectively navigate between parents and children. Specifically, a concrete imple- mentation of a tree must provide methods root, parent, num children, children, len , and in the case of BinaryTree, the additional accessors left and right. There are several choices for the internal representation of trees. We describe the most common representations in this section. We begin with the case of a binary tree, since its shape is more narrowly defined. 8.3.1 Linked Structure for Binary Trees A natural way to realize a binary tree T is to use a linked structure, with a node (see Figure 8.11a) that maintains references to the element stored at a position p and to the nodes associated with the children and parent of p. If p is the root of T, then the parent field of p is None. Likewise, if p does not have a left child (respectively, right child), the associated field is None. The tree itself maintains an instance variable storing a reference to the root node (if any), and a variable, called size, that represents the overall number of nodes of T. We show such a linked structure representation of a binary tree in Figure 8.11b. parent element right left root ∅ ∅ ∅ ∅ ∅ ∅ ∅ Baltimore Chicago New York Providence Seattle size 5 (a) (b) Figure 8.11: A linked structure for representing: (a) a single node; (b) a binary tree.
- 340. 318 Chapter 8.Trees Python Implementation of a Linked Binary Tree Structure In this section, we define a concrete LinkedBinaryTree class that implements the binary tree ADT by subclassing the BinaryTree class. Our general approach is very similar to what we used when developing the PositionalList in Section 7.4: We define a simple, nonpublic Node class to represent a node, and a public Position class that wraps a node. We provide a validate utility for robustly checking the validity of a given position instance when unwrapping it, and a make position utility for wrapping a node as a position to return to a caller. Those definitions are provided in Code Fragment 8.8. As a formality, the new Position class is declared to inherit immediately from BinaryTree.Position. Tech- nically, the BinaryTree class definition (see Code Fragment 8.7) does not formally declare such a nested class; it trivially inherits it from Tree.Position. A minor ben- efit from this design is that our position class inherits the ne special method so that syntax p != q is derived appropriately relative to eq . Our class definition continues, in Code Fragment 8.9, with a constructor and with concrete implementations for the methods that remain abstract in the Tree and BinaryTree classes. The constructor creates an empty tree by initializing root to None and size to zero. These accessor methods are implemented with careful use of the validate and make position utilities to safeguard against boundary cases. Operations for Updating a Linked Binary Tree Thus far, we have provided functionality for examining an existing binary tree. However, the constructor for our LinkedBinaryTree class results in an empty tree and we have not provided any means for changing the structure or content of a tree. We chose not to declare update methods as part of the Tree or BinaryTree ab- stract base classes for several reasons. First, although the principle of encapsula- tion suggests that the outward behaviors of a class need not depend on the internal representation, the efficiency of the operations depends greatly upon the representa- tion. We prefer to have each concrete implementation of a tree class offer the most suitable options for updating a tree. The second reason we do not provide update methods in the base class is that we may not want such update methods to be part of a public interface. There are many applications of trees, and some forms of update operations that are suitable for one application may be unacceptable in another. However, if we place an update method in a base class, any class that inherits from that base will inherit the update method. Consider, for example, the possibility of a method T.replace(p, e) that replaces the element stored at position p with another element e. Such a general method may be unacceptable in the context of an arithmetic expression tree (see Example 8.7 on page 312, and a later case study in Section 8.5), because we may want to enforce that internal nodes store only operators as elements.
- 341. 8.3. Implementing Trees319 For linked binary trees, a reasonable set of update methods to support for gen- eral usage are the following: T.add root(e): Create a root for an empty tree, storing e as the element, and return the position of that root; an error occurs if the tree is not empty. T.add left(p, e): Create a new node storing element e, link the node as the left child of position p, and return the resulting position; an error occurs if p already has a left child. T.add right(p, e): Create a new node storing element e, link the node as the right child of position p, and return the resulting position; an error occurs if p already has a right child. T.replace(p, e): Replace the element stored at position p with element e, and return the previously stored element. T.delete(p): Remove the node at position p, replacing it with its child, if any, and return the element that had been stored at p; an error occurs if p has two children. T.attach(p, T1, T2): Attach the internal structure of trees T1 and T2, respec- tively, as the left and right subtrees of leaf position p of T, and reset T1 and T2 to empty trees; an error condition occurs if p is not a leaf. We have specifically chosen this collection of operations because each can be implemented in O(1) worst-case time with our linked representation. The most complex of these are delete and attach, due to the case analyses involving the various parent-child relationships and boundary conditions, yet there remains only a constant number of operations to perform. (The implementation of both methods could be greatly simplified if we used a tree representation with a sentinel node, akin to our treatment of positional lists; see Exercise C-8.40). To avoid the problem of undesirable update methods being inherited by sub- classes of LinkedBinaryTree, we have chosen an implementation in which none of the above methods are publicly supported. Instead, we provide nonpublic ver- sions of each, for example, providing the underscored delete in lieu of a public delete. Our implementations of these six update methods are provided in Code Fragments 8.10 and 8.11. In particular applications, subclasses of LinkedBinaryTree can invoke the non- public methods internally, while preserving a public interface that is appropriate for the application. A subclass may also choose to wrap one or more of the non- public update methods with a public method to expose it to the user. We leave as an exercise (R-8.15), the task of defining a MutableLinkedBinaryTree subclass that provides public methods wrapping each of these six update methods.
- 342. 320 Chapter 8.Trees 1 class LinkedBinaryTree(BinaryTree): 2 ”””Linked representation of a binary tree structure.””” 3 4 class Node: # Lightweight, nonpublic class for storing a node. 5 slots = _element , _parent , _left , _right 6 def init (self, element, parent=None, left=None, right=None): 7 self. element = element 8 self. parent = parent 9 self. left = left 10 self. right = right 11 12 class Position(BinaryTree.Position): 13 ”””An abstraction representing the location of a single element.””” 14 15 def init (self, container, node): 16 ”””Constructor should not be invoked by user.””” 17 self. container = container 18 self. node = node 19 20 def element(self): 21 ”””Return the element stored at this Position.””” 22 return self. node. element 23 24 def eq (self, other): 25 ”””Return True if other is a Position representing the same location.””” 26 return type(other) is type(self) and other. node is self. node 27 28 def validate(self, p): 29 ”””Return associated node, if position is valid.””” 30 if not isinstance(p, self.Position): 31 raise TypeError( p must be proper Position type ) 32 if p. container is not self: 33 raise ValueError( p does not belong to this container ) 34 if p. node. parent is p. node: # convention for deprecated nodes 35 raise ValueError( p is no longer valid ) 36 return p. node 37 38 def make position(self, node): 39 ”””Return Position instance for given node (or None if no node).””” 40 return self.Position(self, node) if node is not None else None Code Fragment 8.8: The beginning of our LinkedBinaryTree class (continued in Code Fragments 8.9 through 8.11).
- 343. 8.3. Implementing Trees321 41 #-------------------------- binary tree constructor -------------------------- 42 def init (self): 43 ”””Create an initially empty binary tree.””” 44 self. root = None 45 self. size = 0 46 47 #-------------------------- public accessors -------------------------- 48 def len (self): 49 ”””Return the total number of elements in the tree.””” 50 return self. size 51 52 def root(self): 53 ”””Return the root Position of the tree (or None if tree is empty).””” 54 return self. make position(self. root) 55 56 def parent(self, p): 57 ”””Return the Position of p s parent (or None if p is root).””” 58 node = self. validate(p) 59 return self. make position(node. parent) 60 61 def left(self, p): 62 ”””Return the Position of p s left child (or None if no left child).””” 63 node = self. validate(p) 64 return self. make position(node. left) 65 66 def right(self, p): 67 ”””Return the Position of p s right child (or None if no right child).””” 68 node = self. validate(p) 69 return self. make position(node. right) 70 71 def num children(self, p): 72 ”””Return the number of children of Position p.””” 73 node = self. validate(p) 74 count = 0 75 if node. left is not None: # left child exists 76 count += 1 77 if node. right is not None: # right child exists 78 count += 1 79 return count Code Fragment 8.9: Public accessors for our LinkedBinaryTree class. The class begins in Code Fragment 8.8 and continues in Code Fragments 8.10 and 8.11.
- 344. 322 Chapter 8.Trees 80 def add root(self, e): 81 ”””Place element e at the root of an empty tree and return new Position. 82 83 Raise ValueError if tree nonempty. 84 ””” 85 if self. root is not None: raise ValueError( Root exists ) 86 self. size = 1 87 self. root = self. Node(e) 88 return self. make position(self. root) 89 90 def add left(self, p, e): 91 ”””Create a new left child for Position p, storing element e. 92 93 Return the Position of new node. 94 Raise ValueError if Position p is invalid or p already has a left child. 95 ””” 96 node = self. validate(p) 97 if node. left is not None: raise ValueError( Left child exists ) 98 self. size += 1 99 node. left = self. Node(e, node) # node is its parent 100 return self. make position(node. left) 101 102 def add right(self, p, e): 103 ”””Create a new right child for Position p, storing element e. 104 105 Return the Position of new node. 106 Raise ValueError if Position p is invalid or p already has a right child. 107 ””” 108 node = self. validate(p) 109 if node. right is not None: raise ValueError( Right child exists ) 110 self. size += 1 111 node. right = self. Node(e, node) # node is its parent 112 return self. make position(node. right) 113 114 def replace(self, p, e): 115 ”””Replace the element at position p with e, and return old element.””” 116 node = self. validate(p) 117 old = node. element 118 node. element = e 119 return old Code Fragment 8.10: Nonpublic update methods for the LinkedBinaryTree class (continued in Code Fragment 8.11).
- 345. 8.3. Implementing Trees323 120 def delete(self, p): 121 ”””Delete the node at Position p, and replace it with its child, if any. 122 123 Return the element that had been stored at Position p. 124 Raise ValueError if Position p is invalid or p has two children. 125 ””” 126 node = self. validate(p) 127 if self.num children(p) == 2: raise ValueError( p has two children ) 128 child = node. left if node. left else node. right # might be None 129 if child is not None: 130 child. parent = node. parent # child s grandparent becomes parent 131 if node is self. root: 132 self. root = child # child becomes root 133 else: 134 parent = node. parent 135 if node is parent. left: 136 parent. left = child 137 else: 138 parent. right = child 139 self. size −= 1 140 node. parent = node # convention for deprecated node 141 return node. element 142 143 def attach(self, p, t1, t2): 144 ”””Attach trees t1 and t2 as left and right subtrees of external p.””” 145 node = self. validate(p) 146 if not self.is leaf(p): raise ValueError( position must be leaf ) 147 if not type(self) is type(t1) is type(t2): # all 3 trees must be same type 148 raise TypeError( Tree types must match ) 149 self. size += len(t1) + len(t2) 150 if not t1.is empty(): # attached t1 as left subtree of node 151 t1. root. parent = node 152 node. left = t1. root 153 t1. root = None # set t1 instance to empty 154 t1. size = 0 155 if not t2.is empty(): # attached t2 as right subtree of node 156 t2. root. parent = node 157 node. right = t2. root 158 t2. root = None # set t2 instance to empty 159 t2. size = 0 Code Fragment 8.11: Nonpublic update methods for the LinkedBinaryTree class (continued from Code Fragment 8.10).
- 346. 324 Chapter 8.Trees Performance of the Linked Binary Tree Implementation To summarize the efficiencies of the linked structure representation, we analyze the running times of the LinkedBinaryTree methods, including derived methods that are inherited from the Tree and BinaryTree classes: • The len method, implemented in LinkedBinaryTree, uses an instance variable storing the number of nodes of T and takes O(1) time. Method is empty, inherited from Tree, relies on a single call to len and thus takes O(1) time. • The accessor methods root, left, right, parent, and num children are imple- mented directly in LinkedBinaryTree and take O(1) time. The sibling and children methods are derived in BinaryTree based on a constant number of calls to these other accessors, so they run in O(1) time as well. • The is root and is leaf methods, from the Tree class, both run in O(1) time, as is root calls root and then relies on equivalence testing of positions, while is leaf calls left and right and verifies that None is returned by both. • Methods depth and height were each analyzed in Section 8.1.3. The depth method at position p runs in O(dp +1) time where dp is its depth; the height method on the root of the tree runs in O(n) time. • The various update methods add root, add left, add right, replace, delete, and attach (that is, their nonpublic implementations) each run in O(1) time, as they involve relinking only a constant number of nodes per operation. Table 8.1 summarizes the performance of the linked structure implementation of a binary tree. Operation Running Time len, is empty O(1) root, parent, left, right, sibling, children, num children O(1) is root, is leaf O(1) depth(p) O(dp +1) height O(n) add root, add left, add right, replace, delete, attach O(1) Table 8.1: Running times for the methods of an n-node binary tree implemented with a linked structure. The space usage is O(n).
- 347. 8.3. Implementing Trees325 8.3.2 Array-Based Representation of a Binary Tree An alternative representation of a binary tree T is based on a way of numbering the positions of T. For every position p of T, let f(p) be the integer defined as follows. • If p is the root of T, then f(p) = 0. • If p is the left child of position q, then f(p) = 2f(q)+1. • If p is the right child of position q, then f(p) = 2f(q)+2. The numbering function f is known as a level numbering of the positions in a binary tree T, for it numbers the positions on each level of T in increasing order from left to right. (See Figure 8.12.) Note well that the level numbering is based on potential positions within the tree, not actual positions of a given tree, so they are not necessarily consecutive. For example, in Figure 8.12(b), there are no nodes with level numbering 13 or 14, because the node with level numbering 6 has no children. (a) ... ... 4 10 11 12 13 14 8 7 0 2 6 5 1 3 9 (b) 15 + − + × 3 9 5 + 2 − × 3 − 6 3 1 7 4 / 0 1 2 5 4 3 6 12 11 10 25 26 20 9 19 7 8 16 Figure 8.12: Binary tree level numbering: (a) general scheme; (b) an example.
- 348. 326 Chapter 8.Trees The level numbering function f suggests a representation of a binary tree T by means of an array-based structure A (such as a Python list), with the element at position p of T stored at index f(p) of the array. We show an example of an array-based representation of a binary tree in Figure 8.13. / 4 2 0 2 1 3 4 5 6 12 11 8 7 3 1 + × 9 5 − + 0 6 12 1 2 3 4 5 7 8 9 10 11 13 14 5 × + + 4 − 2 3 1 9 / Figure 8.13: Representation of a binary tree by means of an array. One advantage of an array-based representation of a binary tree is that a posi- tion p can be represented by the single integer f(p), and that position-based meth- ods such as root, parent, left, and right can be implemented using simple arithmetic operations on the number f(p). Based on our formula for the level numbering, the left child of p has index 2f(p) + 1, the right child of p has index 2f(p) + 2, and the parent of p has index (f(p)− 1)/2. We leave the details of a complete im- plementation as an exercise (R-8.18). The space usage of an array-based representation depends greatly on the shape of the tree. Let n be the number of nodes of T, and let fM be the maximum value of f(p) over all the nodes of T. The array A requires length N = 1 + fM, since elements range from A[0] to A[ fM]. Note that A may have a number of empty cells that do not refer to existing nodes of T. In fact, in the worst case, N = 2n − 1, the justification of which is left as an exercise (R-8.16). In Section 9.3, we will see a class of binary trees, called “heaps” for which N = n. Thus, in spite of the worst-case space usage, there are applications for which the array representation of a binary tree is space efficient. Still, for general binary trees, the exponential worst-case space requirement of this representation is prohibitive. Another drawback of an array representation is that some update operations for trees cannot be efficiently supported. For example, deleting a node and promoting its child takes O(n) time because it is not just the child that moves locations within the array, but all descendants of that child.
- 349. 8.3. Implementing Trees327 8.3.3 Linked Structure for General Trees When representing a binary tree with a linked structure, each node explicitly main- tains fields left and right as references to individual children. For a general tree, there is no a priori limit on the number of children that a node may have. A natural way to realize a general tree T as a linked structure is to have each node store a single container of references to its children. For example, a children field of a node can be a Python list of references to the children of the node (if any). Such a linked representation is schematically illustrated in Figure 8.14. element parent children Baltimore Chicago New York Providence Seattle (a) (b) Figure 8.14: The linked structure for a general tree: (a) the structure of a node; (b) a larger portion of the data structure associated with a node and its children. Table 8.2 summarizes the performance of the implementation of a general tree using a linked structure. The analysis is left as an exercise (R-8.14), but we note that, by using a collection to store the children of each position p, we can implement children(p) by simply iterating that collection. Operation Running Time len, is empty O(1) root, parent, is root, is leaf O(1) children(p) O(cp +1) depth(p) O(dp +1) height O(n) Table 8.2: Running times of the accessor methods of an n-node general tree im- plemented with a linked structure. We let cp denote the number of children of a position p. The space usage is O(n).
- 350. 328 Chapter 8.Trees 8.4 Tree Traversal Algorithms A traversal of a tree T is a systematic way of accessing, or “visiting,” all the posi- tions of T. The specific action associated with the “visit” of a position p depends on the application of this traversal, and could involve anything from increment- ing a counter to performing some complex computation for p. In this section, we describe several common traversal schemes for trees, implement them in the con- text of our various tree classes, and discuss several common applications of tree traversals. 8.4.1 Preorder and Postorder Traversals of General Trees In a preorder traversal of a tree T, the root of T is visited first and then the sub- trees rooted at its children are traversed recursively. If the tree is ordered, then the subtrees are traversed according to the order of the children. The pseudo-code for the preorder traversal of the subtree rooted at a position p is shown in Code Fragment 8.12. Algorithm preorder(T, p): perform the “visit” action for position p for each child c in T.children(p) do preorder(T, c) {recursively traverse the subtree rooted at c} Code Fragment 8.12: Algorithm preorder for performing the preorder traversal of a subtree rooted at position p of a tree T. Figure 8.15 portrays the order in which positions of a sample tree are visited during an application of the preorder traversal algorithm. Paper Title Abstract § 1 References § 2 § 3 § 1.1 § 1.2 § 2.1 § 2.2 § 2.3 § 3.1 § 3.2 Figure 8.15: Preorder traversal of an ordered tree, where the children of each posi- tion are ordered from left to right.
- 351. 8.4. Tree TraversalAlgorithms 329 Postorder Traversal Another important tree traversal algorithm is the postorder traversal. In some sense, this algorithm can be viewed as the opposite of the preorder traversal, be- cause it recursively traverses the subtrees rooted at the children of the root first, and then visits the root (hence, the name “postorder”). Pseudo-code for the postorder traversal is given in Code Fragment 8.13, and an example of a postorder traversal is portrayed in Figure 8.16. Algorithm postorder(T, p): for each child c in T.children(p) do postorder(T, c) {recursively traverse the subtree rooted at c} perform the “visit” action for position p Code Fragment 8.13: Algorithm postorder for performing the postorder traversal of a subtree rooted at position p of a tree T. Paper Title Abstract § 1 References § 2 § 3 § 1.1 § 1.2 § 2.1 § 2.2 § 2.3 § 3.1 § 3.2 Figure 8.16: Postorder traversal of the ordered tree of Figure 8.15. Running-Time Analysis Both preorder and postorder traversal algorithms are efficient ways to access all the positions of a tree. The analysis of either of these traversal algorithms is similar to that of algorithm height2, given in Code Fragment 8.5 of Section 8.1.3. At each position p, the nonrecursive part of the traversal algorithm requires time O(cp +1), where cp is the number of children of p, under the assumption that the “visit” itself takes O(1) time. By Proposition 8.5, the overall running time for the traversal of tree T is O(n), where n is the number of positions in the tree. This running time is asymptotically optimal since the traversal must visit all the n positions of the tree.
- 352. 330 Chapter 8.Trees 8.4.2 Breadth-First Tree Traversal Although the preorder and postorder traversals are common ways of visiting the positions of a tree, another common approach is to traverse a tree so that we visit all the positions at depth d before we visit the positions at depth d + 1. Such an algorithm is known as a breadth-first traversal. A breadth-first traversal is a common approach used in software for playing games. A game tree represents the possible choices of moves that might be made by a player (or computer) during a game, with the root of the tree being the initial configuration for the game. For example, Figure 8.17 displays a partial game tree for Tic-Tac-Toe. X X X O X X O X O X O X O X O O X X O X O X O O X X O 16 3 2 4 1 5 6 8 7 9 10 11 12 13 14 15 Figure 8.17: Partial game tree for Tic-Tac-Toe, with annotations displaying the or- der in which positions are visited in a breadth-first traversal. A breadth-first traversal of such a game tree is often performed because a computer may be unable to explore a complete game tree in a limited amount of time. So the computer will consider all moves, then responses to those moves, going as deep as computational time allows. Pseudo-code for a breadth-first traversal is given in Code Fragment 8.14. The process is not recursive, since we are not traversing entire subtrees at once. We use a queue to produce a FIFO (i.e., first-in first-out) semantics for the order in which we visit nodes. The overall running time is O(n), due to the n calls to enqueue and n calls to dequeue. Algorithm breadthfirst(T): Initialize queue Q to contain T.root() while Q not empty do p = Q.dequeue() {p is the oldest entry in the queue} perform the “visit” action for position p for each child c in T.children(p) do Q.enqueue(c) {add p’s children to the end of the queue for later visits} Code Fragment 8.14: Algorithm for performing a breadth-first traversal of a tree.
- 353. 8.4. Tree TraversalAlgorithms 331 8.4.3 Inorder Traversal of a Binary Tree The standard preorder, postorder, and breadth-first traversals that were introduced for general trees, can be directly applied to binary trees. In this section, we intro- duce another common traversal algorithm specifically for a binary tree. During an inorder traversal, we visit a position between the recursive traver- sals of its left and right subtrees. The inorder traversal of a binary tree T can be informally viewed as visiting the nodes of T “from left to right.” Indeed, for every position p, the inorder traversal visits p after all the positions in the left subtree of p and before all the positions in the right subtree of p. Pseudo-code for the inorder traversal algorithm is given in Code Fragment 8.15, and an example of an inorder traversal is portrayed in Figure 8.18. Algorithm inorder(p): if p has a left child lc then inorder(lc) {recursively traverse the left subtree of p} perform the “visit” action for position p if p has a right child rc then inorder(rc) {recursively traverse the right subtree of p} Code Fragment 8.15: Algorithm inorder for performing an inorder traversal of a subtree rooted at position p of a binary tree. 3 1 9 5 4 7 + 3 2 − 3 − × + × 6 / + − Figure 8.18: Inorder traversal of a binary tree. The inorder traversal algorithm has several important applications. When using a binary tree to represent an arithmetic expression, as in Figure 8.18, the inorder traversal visits positions in a consistent order with the standard representation of the expression, as in 3+1×3/9−5+2... (albeit without parentheses).
- 354. 332 Chapter 8.Trees Binary Search Trees An important application of the inorder traversal algorithm arises when we store an ordered sequence of elements in a binary tree, defining a structure we call a binary search tree. Let S be a set whose unique elements have an order relation. For example, S could be a set of integers. A binary search tree for S is a binary tree T such that, for each position p of T: • Position p stores an element of S, denoted as e(p). • Elements stored in the left subtree of p (if any) are less than e(p). • Elements stored in the right subtree of p (if any) are greater than e(p). An example of a binary search tree is shown in Figure 8.19. The above properties assure that an inorder traversal of a binary search tree T visits the elements in nondecreasing order. 36 25 31 42 12 62 75 58 90 Figure 8.19: A binary search tree storing integers. The solid path is traversed when searching (successfully) for 36. The dashed path is traversed when searching (un- successfully) for 70. We can use a binary search tree T for set S to find whether a given search value v is in S, by traversing a path down the tree T, starting at the root. At each internal position p encountered, we compare our search value v with the element e(p) stored at p. If v e(p), then the search continues in the left subtree of p. If v = e(p), then the search terminates successfully. If v e(p), then the search continues in the right subtree of p. Finally, if we reach an empty subtree, the search terminates unsuccessfully. In other words, a binary search tree can be viewed as a binary decision tree (recall Example 8.6), where the question asked at each internal node is whether the element at that node is less than, equal to, or larger than the element being searched for. We illustrate several examples of the search operation in Figure 8.19. Note that the running time of searching in a binary search tree T is proportional to the height of T. Recall from Proposition 8.8 that the height of a binary tree with n nodes can be as small as log(n+1)−1 or as large as n−1. Thus, binary search trees are most efficient when they have small height. Chapter 11 is devoted to the study of search trees.
- 355. 8.4. Tree TraversalAlgorithms 333 8.4.4 Implementing Tree Traversals in Python When first defining the tree ADT in Section 8.1.2, we stated that tree T should include support for the following methods: T.positions(): Generate an iteration of all positions of tree T. iter(T): Generate an iteration of all elements stored within tree T. At that time, we did not make any assumption about the order in which these iterations report their results. In this section, we demonstrate how any of the tree traversal algorithms we have introduced could be used to produce these iterations. To begin, we note that it is easy to produce an iteration of all elements of a tree, if we rely on a presumed iteration of all positions. Therefore, support for the iter(T) syntax can be formally provided by a concrete implementation of the special method iter within the abstract base class Tree. We rely on Python’s generator syntax as the mechanism for producing iterations. (See Section 1.8.) Our implementation of Tree. iter is given in Code Fragment 8.16. 75 def iter (self): 76 ”””Generate an iteration of the tree s elements.””” 77 for p in self.positions(): # use same order as positions() 78 yield p.element( ) # but yield each element Code Fragment 8.16: Iterating all elements of a Tree instance, based upon an iter- ation of the positions of the tree. This code should be included in the body of the Tree class. To implement the positions method, we have a choice of tree traversal algo- rithms. Given that there are advantages to each of those traversal orders, we will provide independent implementations of each strategy that can be called directly by a user of our class. We can then trivially adapt one of those as a default order for the positions method of the tree ADT. Preorder Traversal We begin by considering the preorder traversal algorithm. We will support a public method with calling signature T.preorder() for tree T, which generates a preorder iteration of all positions within the tree. However, the recursive algorithm for gen- erating a preorder traversal, as originally described in Code Fragment 8.12, must be parameterized by a specific position within the tree that serves as the root of a subtree to traverse. A standard solution for such a circumstance is to define a non- public utility method with the desired recursive parameterization, and then to have the public method preorder invoke the nonpublic method upon the root of the tree. Our implementation of such a design is given in Code Fragment 8.17.
- 356. 334 Chapter 8.Trees 79 def preorder(self): 80 ”””Generate a preorder iteration of positions in the tree.””” 81 if not self.is empty(): 82 for p in self. subtree preorder(self.root()): # start recursion 83 yield p 84 85 def subtree preorder(self, p): 86 ”””Generate a preorder iteration of positions in subtree rooted at p.””” 87 yield p # visit p before its subtrees 88 for c in self.children(p): # for each child c 89 for other in self. subtree preorder(c): # do preorder of c’s subtree 90 yield other # yielding each to our caller Code Fragment 8.17: Support for performing a preorder traversal of a tree. This code should be included in the body of the Tree class. Formally, both preorder and the utility subtree preorder are generators. Rather than perform a “visit” action from within this code, we yield each position to the caller and let the caller decide what action to perform at that position. The subtree preorder method is the recursive one. However, because we are relying on generators rather than traditional functions, the recursion has a slightly different form. In order to yield all positions within the subtree of child c, we loop over the positions yielded by the recursive call self. subtree preorder(c), and re- yield each position in the outer context. Note that if p is a leaf, the for loop over self.children(p) is trivial (this is the base case for our recursion). We rely on a similar technique in the public preorder method to re-yield all positions that are generated by the recursive process starting at the root of the tree; if the tree is empty, nothing is yielded. At this point, we have provided full support for the preorder generator. A user of the class can therefore write code such as for p in T.preorder(): # ”visit” position p The official tree ADT requires that all trees support a positions method as well. To use a preorder traversal as the default order of iteration, we include the definition shown in Code Fragment 8.18 within our Tree class. Rather than loop over the results returned by the preorder call, we return the entire iteration as an object. 91 def positions(self): 92 ”””Generate an iteration of the tree s positions.””” 93 return self.preorder( ) # return entire preorder iteration Code Fragment 8.18: An implementation of the positions method for the Tree class that relies on a preorder traversal to generate the results.
- 357. 8.4. Tree TraversalAlgorithms 335 Postorder Traversal We can implement a postorder traversal using very similar technique as with a preorder traversal. The only difference is that within the recursive utility for a post- order we wait to yield position p until after we have recursively yield the positions in its subtrees. An implementation is given in Code Fragment 8.19. 94 def postorder(self): 95 ”””Generate a postorder iteration of positions in the tree.””” 96 if not self.is empty(): 97 for p in self. subtree postorder(self.root()): # start recursion 98 yield p 99 100 def subtree postorder(self, p): 101 ”””Generate a postorder iteration of positions in subtree rooted at p.””” 102 for c in self.children(p): # for each child c 103 for other in self. subtree postorder(c): # do postorder of c’s subtree 104 yield other # yielding each to our caller 105 yield p # visit p after its subtrees Code Fragment 8.19: Support for performing a postorder traversal of a tree. This code should be included in the body of the Tree class. Breadth-First Traversal In Code Fragment 8.20, we provide an implementation of the breadth-first traversal algorithm in the context of our Tree class. Recall that the breadth-first traversal algorithm is not recursive; it relies on a queue of positions to manage the traver- sal process. Our implementation uses the LinkedQueue class from Section 7.1.2, although any implementation of the queue ADT would suffice. Inorder Traversal for Binary Trees The preorder, postorder, and breadth-first traversal algorithms are applicable to all trees, and so we include their implementations within the Tree abstract base class. Those methods are inherited by the abstract BinaryTree class, the concrete LinkedBinaryTree class, and any other dependent tree classes we might develop. The inorder traversal algorithm, because it explicitly relies on the notion of a left and right child of a node, only applies to binary trees. We therefore include its definition within the body of the BinaryTree class. We use a similar technique to implement an inorder traversal (Code Fragment 8.21) as we did with preorder and postorder traversals.
- 358. 336 Chapter 8.Trees 106 def breadthfirst(self): 107 ”””Generate a breadth-first iteration of the positions of the tree.””” 108 if not self.is empty(): 109 fringe = LinkedQueue( ) # known positions not yet yielded 110 fringe.enqueue(self.root()) # starting with the root 111 while not fringe.is empty(): 112 p = fringe.dequeue( ) # remove from front of the queue 113 yield p # report this position 114 for c in self.children(p): 115 fringe.enqueue(c) # add children to back of queue Code Fragment 8.20: An implementation of a breadth-first traversal of a tree. This code should be included in the body of the Tree class. 37 def inorder(self): 38 ”””Generate an inorder iteration of positions in the tree.””” 39 if not self.is empty(): 40 for p in self. subtree inorder(self.root()): 41 yield p 42 43 def subtree inorder(self, p): 44 ”””Generate an inorder iteration of positions in subtree rooted at p.””” 45 if self.left(p) is not None: # if left child exists, traverse its subtree 46 for other in self. subtree inorder(self.left(p)): 47 yield other 48 yield p # visit p between its subtrees 49 if self.right(p) is not None: # if right child exists, traverse its subtree 50 for other in self. subtree inorder(self.right(p)): 51 yield other Code Fragment 8.21: Support for performing an inorder traversal of a binary tree. This code should be included in the BinaryTree class (given in Code Fragment 8.7). For many applications of binary trees, an inorder traversal provides a natural iteration. We could make it the default for the BinaryTree class by overriding the positions method that was inherited from the Tree class (see Code Fragment 8.22). 52 # override inherited version to make inorder the default 53 def positions(self): 54 ”””Generate an iteration of the tree s positions.””” 55 return self.inorder( ) # make inorder the default Code Fragment 8.22: Defining the BinaryTree.position method so that positions are reported using inorder traversal.
- 359. 8.4. Tree TraversalAlgorithms 337 8.4.5 Applications of Tree Traversals In this section, we demonstrate several representative applications of tree traversals, including some customizations of the standard traversal algorithms. Table of Contents When using a tree to represent the hierarchical structure of a document, a preorder traversal of the tree can naturally be used to produce a table of contents for the doc- ument. For example, the table of contents associated with the tree from Figure 8.15 is displayed in Figure 8.20. Part (a) of that figure gives a simple presentation with one element per line; part (b) shows a more attractive presentation produced by indenting each element based on its depth within the tree. A similar presentation could be used to display the contents of a computer’s file system, based on its tree representation (as in Figure 8.3). Paper Paper Title Title Abstract Abstract 1 1 1.1 1.1 1.2 1.2 2 2 2.1 2.1 ... ... (a) (b) Figure 8.20: Table of contents for a document represented by the tree in Figure 8.15: (a) without indentation; (b) with indentation based on depth within the tree. The unindented version of the table of contents, given a tree T, can be produced with the following code: for p in T.preorder(): print(p.element()) To produce the presentation of Figure 8.20(b), we indent each element with a number of spaces equal to twice the element’s depth in the tree (hence, the root ele- ment was unindented). Although we could replace the body of the above loop with the statement print(2 T.depth(p) + str(p.element())), such an approach is unnecessarily inefficient. Although the work to produce the preorder traversal runs in O(n) time, based on the analysis of Section 8.4.1, the calls to depth incur a hid- den cost. Making a call to depth from every position of the tree results in O(n2) worst-case time, as noted when analyzing the algorithm height1 in Section 8.1.3.
- 360. 338 Chapter 8.Trees A preferred approach to producing an indented table of contents is to redesign a top-down recursion that includes the current depth as an additional parameter. Such an implementation is provided in Code Fragment 8.23. This implementation runs in worst-case O(n) time (except, technically, the time it takes to print strings of increasing lengths). 1 def preorder indent(T, p, d): 2 ”””Print preorder representation of subtree of T rooted at p at depth d.””” 3 print(2 d + str(p.element())) # use depth for indentation 4 for c in T.children(p): 5 preorder indent(T, c, d+1) # child depth is d+1 Code Fragment 8.23: Efficient recursion for printing indented version of a pre- order traversal. On a complete tree T, the recursion should be started with form preorder indent(T, T.root(), 0). In the example of Figure 8.20, we were fortunate in that the numbering was embedded within the elements of the tree. More generally, we might be interested in using a preorder traversal to display the structure of a tree, with indentation and also explicit numbering that was not present in the tree. For example, we might display the tree from Figure 8.2 beginning as: Electronics R’Us 1 RD 2 Sales 2.1 Domestic 2.2 International 2.2.1 Canada 2.2.2 S. America This is more challenging, because the numbers used as labels are implicit in the structure of the tree. A label depends on the index of each position, relative to its siblings, along the path from the root to the current position. To accomplish the task, we add a representation of that path as an additional parameter to the recursive signature. Specifically, we use a list of zero-indexed numbers, one for each position along the downward path, other than the root. (We convert those numbers to one- indexed form when printing.) At the implementation level, we wish to avoid the inefficiency of duplicating such lists when sending a new parameter from one level of the recursion to the next. A standard solution is to share the same list instance throughout the recursion. At one level of the recursion, a new entry is temporarily added to the end of the list before making further recursive calls. In order to “leave no trace,” that same block of code must remove the extraneous entry from the list before completing its task. An implementation based on this approach is given in Code Fragment 8.24.
- 361. 8.4. Tree TraversalAlgorithms 339 1 def preorder label(T, p, d, path): 2 ”””Print labeled representation of subtree of T rooted at p at depth d.””” 3 label = . .join(str(j+1) for j in path) # displayed labels are one-indexed 4 print(2 d + label, p.element()) 5 path.append(0) # path entries are zero-indexed 6 for c in T.children(p): 7 preorder label(T, c, d+1, path) # child depth is d+1 8 path[−1] += 1 9 path.pop() Code Fragment 8.24: Efficient recursion for printing an indented and labeled pre- sentation of a preorder traversal. Parenthetic Representations of a Tree It is not possible to reconstruct a general tree, given only the preorder sequence of elements, as in Figure 8.20(a). Some additional context is necessary for the structure of the tree to be well defined. The use of indentation or numbered labels provides such context, with a very human-friendly presentation. However, there are more concise string representations of trees that are computer-friendly. In this section, we explore one such representation. The parenthetic string representation P(T) of tree T is recursively defined as follows. If T consists of a single position p, then P(T) = str(p.element()). Otherwise, it is defined recursively as, P(T) = str(p.element())+ ( +P(T1)+ , + ··· + , +P(Tk)+ ) where p is the root of T and T1,T2,...,Tk are the subtrees rooted at the children of p, which are given in order if T is an ordered tree. We are using “+” here to denote string concatenation. As an example, the parenthetic representation of the tree of Figure 8.2 would appear as follows (line breaks are cosmetic): Electronics R’Us (RD, Sales (Domestic, International (Canada, S. America, Overseas (Africa, Europe, Asia, Australia))), Purchasing, Manufacturing (TV, CD, Tuner)) Although the parenthetic representation is essentially a preorder traversal, we cannot easily produce the additional punctuation using the formal implementation of preorder, as given in Code Fragment 8.17. The opening parenthesis must be produced just before the loop over a position’s children and the closing parenthesis must be produced just after that loop. Furthermore, the separating commas must be produced. The Python function parenthesize, shown in Code Fragment 8.25, is a custom traversal that prints such a parenthetic string representation of a tree T.
- 362. 340 Chapter 8.Trees 1 def parenthesize(T, p): 2 ”””Print parenthesized representation of subtree of T rooted at p.””” 3 print(p.element(), end= ) # use of end avoids trailing newline 4 if not T.is leaf(p): 5 first time = True 6 for c in T.children(p): 7 sep = ( if first time else , # determine proper separator 8 print(sep, end= ) 9 first time = False # any future passes will not be the first 10 parenthesize(T, c) # recur on child 11 print( ) , end= ) # include closing parenthesis Code Fragment 8.25: Function that prints parenthetic string representation of a tree. Computing Disk Space In Example 8.1, we considered the use of a tree as a model for a file-system struc- ture, with internal positions representing directories and leaves representing files. In fact, when introducing the use of recursion back in Chapter 4, we specifically examined the topic of file systems (see Section 4.1.4). Although we did not explic- itly model it as a tree at that time, we gave an implementation of an algorithm for computing the disk usage (Code Fragment 4.5). The recursive computation of disk space is emblematic of a postorder traversal, as we cannot effectively compute the total space used by a directory until after we know the space that is used by its children directories. Unfortunately, the formal implementation of postorder, as given in Code Fragment 8.19 does not suffice for this purpose. As it visits the position of a directory, there is no easy way to discern which of the previous positions represent children of that directory, nor how much recursive disk space was allocated. We would like to have a mechanism for children to return information to the parent as part of the traversal process. A custom solution to the disk space prob- lem, with each level of recursion providing a return value to the (parent) caller, is provided in Code Fragment 8.26. 1 def disk space(T, p): 2 ”””Return total disk space for subtree of T rooted at p.””” 3 subtotal = p.element().space( ) # space used at position p 4 for c in T.children(p): 5 subtotal += disk space(T, c) # add child’s space to subtotal 6 return subtotal Code Fragment 8.26: Recursive computation of disk space for a tree. We assume that a space() method of each tree element reports the local space used at that position.
- 363. 8.4. Tree TraversalAlgorithms 341 8.4.6 Euler Tours and the Template Method Pattern The various applications described in Section 8.4.5 demonstrate the great power of recursive tree traversals. Unfortunately, they also show that the specific imple- mentations of the preorder and postorder methods of our Tree class, or the inorder method of the BinaryTree class, are not general enough to capture the range of computations we desire. In some cases, we need more of a blending of the ap- proaches, with initial work performed before recurring on subtrees, additional work performed after those recursions, and in the case of a binary tree, work performed between the two possible recursions. Furthermore, in some contexts it was impor- tant to know the depth of a position, or the complete path from the root to that position, or to return information from one level of the recursion to another. For each of the previous applications, we were able to develop a custom implementa- tion to properly adapt the recursive ideas, but the great principles of object-oriented programming introduced in Section 2.1.1 include adaptability and reusability. In this section, we develop a more general framework for implementing tree traversals based on a concept known as an Euler tour traversal. The Euler tour traversal of a general tree T can be informally defined as a “walk” around T, where we start by going from the root toward its leftmost child, viewing the edges of T as being “walls” that we always keep to our left. (See Figure 8.21.) 3 1 9 5 4 7 + 3 2 − 3 − × + × 6 / + − Figure 8.21: Euler tour traversal of a tree. The complexity of the walk is O(n), because it progresses exactly two times along each of the n−1 edges of the tree—once going downward along the edge, and later going upward along the edge. To unify the concept of preorder and postorder traversals, we can think of there being two notable “visits” to each position p: • A “pre visit” occurs when first reaching the position, that is, when the walk passes immediately left of the node in our visualization. • A “post visit” occurs when the walk later proceeds upward from that position, that is, when the walk passes to the right of the node in our visualization.
- 364. 342 Chapter 8.Trees The process of an Euler tour can easily be viewed recursively. In between the “pre visit” and “post visit” of a given position will be a recursive tour of each of its subtrees. Looking at Figure 8.21 as an example, there is a contiguous portion of the entire tour that is itself an Euler tour of the subtree of the node with element “/”. That tour contains two contiguous subtours, one traversing that position’s left subtree and another traversing the right subtree. The pseudo-code for an Euler tour traversal of a subtree rooted at a position p is shown in Code Fragment 8.27. Algorithm eulertour(T, p): perform the “pre visit” action for position p for each child c in T.children(p) do eulertour(T, c) {recursively tour the subtree rooted at c} perform the “post visit” action for position p Code Fragment 8.27: Algorithm eulertour for performing an Euler tour traversal of a subtree rooted at position p of a tree. The Template Method Pattern To provide a framework that is reusable and adaptable, we rely on an interesting object-oriented software design pattern, the template method pattern. The template method pattern describes a generic computation mechanism that can be specialized for a particular application by redefining certain steps. To allow customization, the primary algorithm calls auxiliary functions known as hooks at designated steps of the process. In the context of an Euler tour traversal, we define two separate hooks, a pre- visit hook that is called before the subtrees are traversed, and a postvisit hook that is called after the completion of the subtree traversals. Our implementation will take the form of an EulerTour class that manages the process, and defines trivial defi- nitions for the hooks that do nothing. The traversal can be customized by defining a subclass of EulerTour and overriding one or both hooks to provide specialized behavior. Python Implementation Our implementation of an EulerTour class is provided in Code Fragment 8.28. The primary recursive process is defined in the nonpublic tour method. A tour instance is created by sending a reference to a specific tree to the constructor, and then by calling the public execute method, which beings the tour and returns a final result of the computation.
- 365. 8.4. Tree TraversalAlgorithms 343 1 class EulerTour: 2 ”””Abstract base class for performing Euler tour of a tree. 3 4 hook previsit and hook postvisit may be overridden by subclasses. 5 ””” 6 def init (self, tree): 7 ”””Prepare an Euler tour template for given tree.””” 8 self. tree = tree 9 10 def tree(self): 11 ”””Return reference to the tree being traversed.””” 12 return self. tree 13 14 def execute(self): 15 ”””Perform the tour and return any result from post visit of root.””” 16 if len(self. tree) 0: 17 return self. tour(self. tree.root(), 0, [ ]) # start the recursion 18 19 def tour(self, p, d, path): 20 ”””Perform tour of subtree rooted at Position p. 21 22 p Position of current node being visited 23 d depth of p in the tree 24 path list of indices of children on path from root to p 25 ””” 26 self. hook previsit(p, d, path) # ”pre visit” p 27 results = [ ] 28 path.append(0) # add new index to end of path before recursion 29 for c in self. tree.children(p): 30 results.append(self. tour(c, d+1, path)) # recur on child s subtree 31 path[−1] += 1 # increment index 32 path.pop( ) # remove extraneous index from end of path 33 answer = self. hook postvisit(p, d, path, results) # ”post visit” p 34 return answer 35 36 def hook previsit(self, p, d, path): # can be overridden 37 pass 38 39 def hook postvisit(self, p, d, path, results): # can be overridden 40 pass Code Fragment 8.28: An EulerTour base class providing a framework for perform- ing Euler tour traversals of a tree.
- 366. 344 Chapter 8.Trees Based on our experience of customizing traversals for sample applications Sec- tion 8.4.5, we build support into the primary EulerTour for maintaining the re- cursive depth and the representation of the recursive path through a tree, using the approach that we introduced in Code Fragment 8.24. We also provide a mechanism for one recursive level to return a value to another when post-processing. Formally, our framework relies on the following two hooks that can be specialized: • method hook previsit(p, d, path) This function is called once for each position, immediately before its subtrees (if any) are traversed. Parameter p is a position in the tree, d is the depth of that position, and path is a list of indices, using the convention described in the discussion of Code Fragment 8.24. No return value is expected from this function. • method hook postvisit(p, d, path, results) This function is called once for each position, immediately after its subtrees (if any) are traversed. The first three parameters use the same convention as did hook previsit. The final parameter is a list of objects that were provided as return values from the post visits of the respective subtrees of p. Any value returned by this call will be available to the parent of p during its postvisit. For more complex tasks, subclasses of EulerTour may also choose to initialize and maintain additional state in the form of instance variables that can be accessed within the bodies of the hooks. Using the Euler Tour Framework To demonstrate the flexibility of our Euler tour framework, we revisit the sample applications from Section 8.4.5. As a simple example, an indented preorder traver- sal, akin to that originally produced by Code Fragment 8.23, can be generated with the simple subclass given in Code Fragment 8.29. 1 class PreorderPrintIndentedTour(EulerTour): 2 def hook previsit(self, p, d, path): 3 print(2 d + str(p.element())) Code Fragment 8.29: A subclass of EulerTour that produces an indented preorder list of a tree’s elements. Such a tour would be started by creating an instance of the subclass for a given tree T, and invoking its execute method. This could be expressed as follows: tour = PreorderPrintIndentedTour(T) tour.execute()
- 367. 8.4. Tree TraversalAlgorithms 345 A labeled version of an indented, preorder presentation, akin to Code Frag- ment 8.24, could be generated by the new subclass of EulerTour shown in Code Fragment 8.30. 1 class PreorderPrintIndentedLabeledTour(EulerTour): 2 def hook previsit(self, p, d, path): 3 label = . .join(str(j+1) for j in path) # labels are one-indexed 4 print(2 d + label, p.element()) Code Fragment 8.30: A subclass of EulerTour that produces a labeled and indented, preorder list of a tree’s elements. To produce the parenthetic string representation, originally achieved with Code Fragment 8.25, we define a subclass that overrides both the previsit and postvisit hooks. Our new implementation is given in Code Fragment 8.31. 1 class ParenthesizeTour(EulerTour): 2 def hook previsit(self, p, d, path): 3 if path and path[−1] 0: # p follows a sibling 4 print( , , end= ) # so preface with comma 5 print(p.element(), end= ) # then print element 6 if not self.tree().is leaf(p): # if p has children 7 print( ( , end= ) # print opening parenthesis 8 9 def hook postvisit(self, p, d, path, results): 10 if not self.tree().is leaf(p): # if p has children 11 print( ) , end= ) # print closing parenthesis Code Fragment 8.31: A subclass of EulerTour that prints a parenthetic string repre- sentation of a tree. Notice that in this implementation, we need to invoke a method on the tree instance that is being traversed from within the hooks. The public tree() method of the EulerTour class serves as an accessor for that tree. Finally, the task of computing disk space, as originally implemented in Code Fragment 8.26, can be performed quite easily with the EulerTour subclass shown in Code Fragment 8.32. The postvisit result of the root will be returned by the call to execute(). 1 class DiskSpaceTour(EulerTour): 2 def hook postvisit(self, p, d, path, results): 3 # we simply add space associated with p to that of its subtrees 4 return p.element().space( ) + sum(results) Code Fragment 8.32: A subclass of EulerTour that computes disk space for a tree.
- 368. 346 Chapter 8.Trees The Euler Tour Traversal of a Binary Tree In Section 8.4.6, we introduced the concept of an Euler tour traversal of a general graph, using the template method pattern in designing the EulerTour class. That class provided methods hook previsit and hook postvisit that could be overrid- den to customize a tour. In Code Fragment 8.33 we provide a BinaryEulerTour specialization that includes an additional hook invisit that is called once for each position—after its left subtree is traversed, but before its right subtree is traversed. Our implementation of BinaryEulerTour replaces the original tour utility to specialize to the case in which a node has at most two children. If a node has only one child, a tour differentiates between whether that is a left child or a right child, with the “in visit” taking place after the visit of a sole left child, but before the visit of a sole right child. In the case of a leaf, the three hooks are called in succession. 1 class BinaryEulerTour(EulerTour): 2 ”””Abstract base class for performing Euler tour of a binary tree. 3 4 This version includes an additional hook invisit that is called after the tour 5 of the left subtree (if any), yet before the tour of the right subtree (if any). 6 7 Note: Right child is always assigned index 1 in path, even if no left sibling. 8 ””” 9 def tour(self, p, d, path): 10 results = [None, None] # will update with results of recursions 11 self. hook previsit(p, d, path) # ”pre visit” for p 12 if self. tree.left(p) is not None: # consider left child 13 path.append(0) 14 results[0] = self. tour(self. tree.left(p), d+1, path) 15 path.pop() 16 self. hook invisit(p, d, path) # ”in visit” for p 17 if self. tree.right(p) is not None: # consider right child 18 path.append(1) 19 results[1] = self. tour(self. tree.right(p), d+1, path) 20 path.pop() 21 answer = self. hook postvisit(p, d, path, results) # ”post visit” p 22 return answer 23 24 def hook invisit(self, p, d, path): pass # can be overridden Code Fragment 8.33: A BinaryEulerTour base class providing a specialized tour for binary trees. The original EulerTour base class was given in Code Fragment 8.28.
- 369. 8.4. Tree TraversalAlgorithms 347 3 2 1 0 0 1 2 3 4 5 6 7 8 9 10 11 12 4 Figure 8.22: An inorder drawing of a binary tree. To demonstrate use of the BinaryEulerTour framework, we develop a subclass that computes a graphical layout of a binary tree, as shown in Figure 8.22. The geometry is determined by an algorithm that assigns x- and y-coordinates to each position p of a binary tree T using the following two rules: • x(p) is the number of positions visited before p in an inorder traversal of T. • y(p) is the depth of p in T. In this application, we take the convention common in computer graphics that x- coordinates increase left to right and y-coordinates increase top to bottom. So the origin is in the upper left corner of the computer screen. Code Fragment 8.34 provides an implementation of a BinaryLayout subclass that implements the above algorithm for assigning (x,y) coordinates to the element stored at each position of a binary tree. We adapt the BinaryEulerTour framework by introducing additional state in the form of a count instance variable that repre- sents the number of “in visits” that we have performed. The x-coordinate for each position is set according to that counter. 1 class BinaryLayout(BinaryEulerTour): 2 ”””Class for computing (x,y) coordinates for each node of a binary tree.””” 3 def init (self, tree): 4 super(). init (tree) # must call the parent constructor 5 self. count = 0 # initialize count of processed nodes 6 7 def hook invisit(self, p, d, path): 8 p.element().setX(self. count) # x-coordinate serialized by count 9 p.element().setY(d) # y-coordinate is depth 10 self. count += 1 # advance count of processed nodes Code Fragment 8.34: A BinaryLayout class that computes coordinates at which to draw positions of a binary tree. We assume that the element type for the original tree supports setX and setY methods.
- 370. 348 Chapter 8.Trees 8.5 Case Study: An Expression Tree In Example 8.7, we introduced the use of a binary tree to represent the structure of an arithmetic expression. In this section, we define a new ExpressionTree class that provides support for constructing such trees, and for displaying and evaluating the arithmetic expression that such a tree represents. Our ExpressionTree class is de- fined as a subclass of LinkedBinaryTree, and we rely on the nonpublic mutators to construct such trees. Each internal node must store a string that defines a binary op- erator (e.g., + ), and each leaf must store a numeric value (or a string representing a numeric value). Our eventual goal is to build arbitrarily complex expression trees for compound arithmetic expressions such as (((3+ 1)× 4)/((9− 5) + 2)). However, it suffices for the ExpressionTree class to support two basic forms of initialization: ExpressionTree(value): Create a tree storing the given value at the root. ExpressionTree(op,E1,E2): Create a tree storing string op at the root (e.g., +), and with the structures of existing ExpressionTree instances E1 and E2 as the left and right subtrees of the root, respectively. Such a constructor for the ExpressionTree class is given in Code Fragment 8.35. The class formally inherits from LinkedBinaryTree, so it has access to all the non- public update methods that were defined in Section 8.3.1. We use add root to cre- ate an initial root of the tree storing the token provided as the first parameter. Then we perform run-time checking of the parameters to determine whether the caller invoked the one-parameter version of the constructor (in which case, we are done), or the three-parameter form. In that case, we use the inherited attach method to incorporate the structure of the existing trees as subtrees of the root. Composing a Parenthesized String Representation A string representation of an existing expression tree instance, for example, as (((3+1)x4)/((9-5)+2)) , can be produced by displaying tree elements us- ing an inorder traversal, but with opening and closing parentheses inserted with a preorder and postorder step, respectively. In the context of an ExpressionTree class, we support a special str method (see Section 2.3.2) that returns the appropriate string. Because it is more efficient to first build a sequence of individ- ual strings to be joined together (see discussion of “Composing Strings” in Sec- tion 5.4.2), the implementation of str relies on a nonpublic, recursive method named parenthesize recur that appends a series of strings to a list. These methods are included in Code 8.35.
- 371. 8.5. Case Study:An Expression Tree 349 1 class ExpressionTree(LinkedBinaryTree): 2 ”””An arithmetic expression tree.””” 3 4 def init (self, token, left=None, right=None): 5 ”””Create an expression tree. 6 7 In a single parameter form, token should be a leaf value (e.g., 42 ), 8 and the expression tree will have that value at an isolated node. 9 10 In a three-parameter version, token should be an operator, 11 and left and right should be existing ExpressionTree instances 12 that become the operands for the binary operator. 13 ””” 14 super(). init ( ) # LinkedBinaryTree initialization 15 if not isinstance(token, str): 16 raise TypeError( Token must be a string ) 17 self. add root(token) # use inherited, nonpublic method 18 if left is not None: # presumably three-parameter form 19 if token not in +-*x/ : 20 raise ValueError( token must be valid operator ) 21 self. attach(self.root(), left, right) # use inherited, nonpublic method 22 23 def str (self): 24 ”””Return string representation of the expression.””” 25 pieces = [ ] # sequence of piecewise strings to compose 26 self. parenthesize recur(self.root(), pieces) 27 return .join(pieces) 28 29 def parenthesize recur(self, p, result): 30 ”””Append piecewise representation of p s subtree to resulting list.””” 31 if self.is leaf(p): 32 result.append(str(p.element())) # leaf value as a string 33 else: 34 result.append( ( ) # opening parenthesis 35 self. parenthesize recur(self.left(p), result) # left subtree 36 result.append(p.element()) # operator 37 self. parenthesize recur(self.right(p), result) # right subtree 38 result.append( ) ) # closing parenthesis Code Fragment 8.35: The beginning of an ExpressionTree class.
- 372. 350 Chapter 8.Trees Expression Tree Evaluation The numeric evaluation of an expression tree can be accomplished with a simple application of a postorder traversal. If we know the values represented by the two subtrees of an internal position, we can calculate the result of the computation that position designates. Pseudo-code for the recursive evaluation of the value repre- sented by a subtree rooted at position p is given in Code Fragment 8.36. Algorithm evaluate recur(p): if p is a leaf then return the value stored at p else let ◦ be the operator stored at p x = evaluate recur(left(p)) y = evaluate recur(right(p)) return x ◦ y Code Fragment 8.36: Algorithm evaluate recur for evaluating the expression rep- resented by a subtree of an arithmetic expression tree rooted at position p. To implement this algorithm in the context of a Python ExpressionTree class, we provide a public evaluate method that is invoked on instance T as T.evaluate(). Code Fragment 8.37 provides such an implementation, relying on a nonpublic evaluate recur method that computes the value of a designated subtree. 39 def evaluate(self): 40 ”””Return the numeric result of the expression.””” 41 return self. evaluate recur(self.root()) 42 43 def evaluate recur(self, p): 44 ”””Return the numeric result of subtree rooted at p.””” 45 if self.is leaf(p): 46 return float(p.element()) # we assume element is numeric 47 else: 48 op = p.element() 49 left val = self. evaluate recur(self.left(p)) 50 right val = self. evaluate recur(self.right(p)) 51 if op == + : return left val + right val 52 elif op == - : return left val − right val 53 elif op == / : return left val / right val 54 else: return left val right val # treat x or as multiplication Code Fragment 8.37: Support for evaluating an ExpressionTree instance.
- 373. 8.5. Case Study:An Expression Tree 351 Building an Expression Tree The constructor for the ExpressionTree class, from Code Fragment 8.35, provides basic functionality for combining existing trees to build larger expression trees. However, the question still remains how to construct a tree that represents an ex- pression for a given string, such as (((3+1)x4)/((9-5)+2)) . To automate this process, we rely on a bottom-up construction algorithm, as- suming that a string can first be tokenized so that multidigit numbers are treated atomically (see Exercise R-8.30), and that the expression is fully parenthesized. The algorithm uses a stack S while scanning tokens of the input expression E to find values, operators, and right parentheses. (Left parentheses are ignored.) • When we see an operator ◦, we push that string on the stack. • When we see a literal value v, we create a single-node expression tree T storing v, and push T on the stack. • When we see a right parenthesis, ) , we pop the top three items from the stack S, which represent a subexpression (E1 ◦ E2). We then construct a tree T using trees for E1 and E2 as subtrees of the root storing ◦, and push the resulting tree T back on the stack. We repeat this until the expression E has been processed, at which time the top element on the stack is the expression tree for E. The total running time is O(n). An implementation of this algorithm is given in Code Fragment 8.38 in the form of a stand-alone function named build expression tree, which produces and returns an appropriate ExpressionTree instance, assuming the input has been tokenized. 1 def build expression tree(tokens): 2 ”””Returns an ExpressionTree based upon by a tokenized expression.””” 3 S = [ ] # we use Python list as stack 4 for t in tokens: 5 if t in +-x*/ : # t is an operator symbol 6 S.append(t) # push the operator symbol 7 elif t not in () : # consider t to be a literal 8 S.append(ExpressionTree(t)) # push trivial tree storing value 9 elif t == ) : # compose a new tree from three constituent parts 10 right = S.pop( ) # right subtree as per LIFO 11 op = S.pop( ) # operator symbol 12 left = S.pop( ) # left subtree 13 S.append(ExpressionTree(op, left, right)) # repush tree 14 # we ignore a left parenthesis 15 return S.pop() Code Fragment 8.38: Implementation of a build expression tree that produces an ExpressionTree from a sequence of tokens representing an arithmetic expression.
- 374. 352 Chapter 8.Trees 8.6 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-8.1 The following questions refer to the tree of Figure 8.3. a. Which node is the root? b. What are the internal nodes? c. How many descendants does node cs016/ have? d. How many ancestors does node cs016/ have? e. What are the siblings of node homeworks/? f. Which nodes are in the subtree rooted at node projects/? g. What is the depth of node papers/? h. What is the height of the tree? R-8.2 Show a tree achieving the worst-case running time for algorithm depth. R-8.3 Give a justification of Proposition 8.4. R-8.4 What is the running time of a call to T. height2(p) when called on a position p distinct from the root of T? (See Code Fragment 8.5.) R-8.5 Describe an algorithm, relying only on the BinaryTree operations, that counts the number of leaves in a binary tree that are the left child of their respective parent. R-8.6 Let T be an n-node binary tree that may be improper. Describe how to represent T by means of a proper binary tree T with O(n) nodes. R-8.7 What are the minimum and maximum number of internal and external nodes in an improper binary tree with n nodes? R-8.8 Answer the following questions so as to justify Proposition 8.8. a. What is the minimum number of external nodes for a proper binary tree with height h? Justify your answer. b. What is the maximum number of external nodes for a proper binary tree with height h? Justify your answer. c. Let T be a proper binary tree with height h and n nodes. Show that log(n+1)−1 ≤ h ≤ (n−1)/2. d. For which values of n and h can the above lower and upper bounds on h be attained with equality? R-8.9 Give a proof by induction of Proposition 8.9. R-8.10 Give a direct implementation of the num children method within the class BinaryTree.
- 375. 8.6. Exercises 353 R-8.11Find the value of the arithmetic expression associated with each subtree of the binary tree of Figure 8.8. R-8.12 Draw an arithmetic expression tree that has four external nodes, storing the numbers 1, 5, 6, and 7 (with each number stored in a distinct external node, but not necessarily in this order), and has three internal nodes, each storing an operator from the set {+,−,×,/}, so that the value of the root is 21. The operators may return and act on fractions, and an operator may be used more than once. R-8.13 Draw the binary tree representation of the following arithmetic expres- sion: “(((5+2)∗(2−1))/((2+9)+((7−2)−1))∗8)”. R-8.14 Justify Table 8.2, summarizing the running time of the methods of a tree represented with a linked structure, by providing, for each method, a de- scription of its implementation, and an analysis of its running time. R-8.15 The LinkedBinaryTree class provides only nonpublic versions of the up- date methods discussed on page 319. Implement a simple subclass named MutableLinkedBinaryTree that provides public wrapper functions for each of the inherited nonpublic update methods. R-8.16 Let T be a binary tree with n nodes, and let f() be the level numbering function of the positions of T, as given in Section 8.3.2. a. Show that, for every position p of T, f(p) ≤ 2n −2. b. Show an example of a binary tree with seven nodes that attains the above upper bound on f(p) for some position p. R-8.17 Show how to use the Euler tour traversal to compute the level number f(p), as defined in Section 8.3.2, of each position in a binary tree T. R-8.18 Let T be a binary tree with n positions that is realized with an array rep- resentation A, and let f() be the level numbering function of the positions of T, as given in Section 8.3.2. Give pseudo-code descriptions of each of the methods root, parent, left, right, is leaf, and is root. R-8.19 Our definition of the level numbering function f(p), as given in Sec- tion 8.3.2, began with the root having number 0. Some authors prefer to use a level numbering g(p) in which the root is assigned number 1, be- cause it simplifies the arithmetic for finding neighboring positions. Redo Exercise R-8.18, but assuming that we use a level numbering g(p) in which the root is assigned number 1. R-8.20 Draw a binary tree T that simultaneously satisfies the following: • Each internal node of T stores a single character. • A preorder traversal of T yields EXAMFUN. • An inorder traversal of T yields MAFXUEN. R-8.21 In what order are positions visited during a preorder traversal of the tree of Figure 8.8?
- 376. 354 Chapter 8.Trees R-8.22 In what order are positions visited during a postorder traversal of the tree of Figure 8.8? R-8.23 Let T be an ordered tree with more than one node. Is it possible that the preorder traversal of T visits the nodes in the same order as the postorder traversal of T? If so, give an example; otherwise, explain why this cannot occur. Likewise, is it possible that the preorder traversal of T visits the nodes in the reverse order of the postorder traversal of T? If so, give an example; otherwise, explain why this cannot occur. R-8.24 Answer the previous question for the case when T is a proper binary tree with more than one node. R-8.25 Consider the example of a breadth-first traversal given in Figure 8.17. Using the annotated numbers from that figure, describe the contents of the queue before each pass of the while loop in Code Fragment 8.14. To get started, the queue has contents {1} before the first pass, and contents {2,3,4} before the second pass. R-8.26 The collections.deque class supports an extend method that adds a col- lection of elements to the end of the queue at once. Reimplement the breadthfirst method of the Tree class to take advantage of this feature. R-8.27 Give the output of the function parenthesize(T, T.root()), as described in Code Fragment 8.25, when T is the tree of Figure 8.8. R-8.28 What is the running time of parenthesize(T, T.root()), as given in Code Fragment 8.25, for a tree T with n nodes? R-8.29 Describe, in pseudo-code, an algorithm for computing the number of de- scendants of each node of a binary tree. The algorithm should be based on the Euler tour traversal. R-8.30 The build expression tree method of the ExpressionTree class requires input that is an iterable of string tokens. We used a convenient exam- ple, (((3+1)x4)/((9-5)+2)) , in which each character is its own to- ken, so that the string itself sufficed as input to build expression tree. In general, a string, such as (35 + 14) , must be explicitly tokenized into list [ ( , 35 , + , 14 , ) ] so as to ignore whitespace and to recognize multidigit numbers as a single token. Write a utility method, tokenize(raw), that returns such a list of tokens for a raw string. Creativity C-8.31 Define the internal path length, I(T), of a tree T to be the sum of the depths of all the internal positions in T. Likewise, define the external path length, E(T), of a tree T to be the sum of the depths of all the external positions in T. Show that if T is a proper binary tree with n positions, then E(T) = I(T)+n−1.
- 377. 8.6. Exercises 355 C-8.32Let T be a (not necessarily proper) binary tree with n nodes, and let D be the sum of the depths of all the external nodes of T. Show that if T has the minimum number of external nodes possible, then D is O(n) and if T has the maximum number of external nodes possible, then D is O(nlogn). C-8.33 Let T be a (possibly improper) binary tree with n nodes, and let D be the sum of the depths of all the external nodes of T. Describe a configuration for T such that D is Ω(n2). Such a tree would be the worst case for the asymptotic running time of method height1 (Code Fragment 8.4). C-8.34 For a tree T, let nI denote the number of its internal nodes, and let nE denote the number of its external nodes. Show that if every internal node in T has exactly 3 children, then nE = 2nI +1. C-8.35 Two ordered trees T and T are said to be isomorphic if one of the fol- lowing holds: • Both T and T are empty. • The roots of T and T have the same number k ≥ 0 of subtrees, and the ith such subtree of T is isomorphic to the ith such subtree of T for i = 1,...,k. Design an algorithm that tests whether two given ordered trees are iso- morphic. What is the running time of your algorithm? C-8.36 Show that there are more than 2n improper binary trees with n internal nodes such that no pair are isomorphic (see Exercise C-8.35). C-8.37 If we exclude isomorphic trees (see Exercise C-8.35), exactly how many proper binary trees exist with exactly 4 leaves? C-8.38 Add support in LinkedBinaryTree for a method, delete subtree(p), that removes the entire subtree rooted at position p, making sure to maintain the count on the size of the tree. What is the running time of your imple- mentation? C-8.39 Add support in LinkedBinaryTree for a method, swap(p,q), that has the effect of restructuring the tree so that the node referenced by p takes the place of the node referenced by q, and vice versa. Make sure to properly handle the case when the nodes are adjacent. C-8.40 We can simplify parts of our LinkedBinaryTree implementation if we make use of of a single sentinel node, referenced as the sentinel member of the tree instance, such that the sentinel is the parent of the real root of the tree, and the root is referenced as the left child of the sentinel. Fur- thermore, the sentinel will take the place of None as the value of the left or right member for a node without such a child. Give a new imple- mentation of the update methods delete and attach, assuming such a representation.
- 378. 356 Chapter 8.Trees C-8.41 Describe how to clone a LinkedBinaryTree instance representing a proper binary tree, with use of the attach method. C-8.42 Describe how to clone a LinkedBinaryTree instance representing a (not necessarily proper) binary tree, with use of the add left and add right methods. C-8.43 We can define a binary tree representation T for an ordered general tree T as follows (see Figure 8.23): • For each position p of T, there is an associated position p of T. • If p is a leaf of T, then p in T does not have a left child; otherwise the left child of p is q, where q is the first child of p in T. • If p has a sibling q ordered immediately after it in T, then q is the right child of p in T; otherwise p does not have a right child. Given such a representation T of a general ordered tree T, answer each of the following questions: a. Is a preorder traversal of T equivalent to a preorder traversal of T? b. Is a postorder traversal of T equivalent to a postorder traversal of T? c. Is an inorder traversal of T equivalent to one of the standard traver- sals of T? If so, which one? D F G E C A B A D F G E C B (a) (b) Figure 8.23: Representation of a tree with a binary tree: (a) tree T; (b) binary tree T for T. The dashed edges connect nodes of T that are siblings in T. C-8.44 Give an efficient algorithm that computes and prints, for every position p of a tree T, the element of p followed by the height of p’s subtree. C-8.45 Give an O(n)-time algorithm for computing the depths of all positions of a tree T, where n is the number of nodes of T. C-8.46 The path length of a tree T is the sum of the depths of all positions in T. Describe a linear-time method for computing the path length of a tree T. C-8.47 The balance factor of an internal position p of a proper binary tree is the difference between the heights of the right and left subtrees of p. Show how to specialize the Euler tour traversal of Section 8.4.6 to print the balance factors of all the internal nodes of a proper binary tree.
- 379. 8.6. Exercises 357 C-8.48Given a proper binary tree T, define the reflection of T to be the binary tree T such that each node v in T is also in T, but the left child of v in T is v’s right child in T and the right child of v in T is v’s left child in T. Show that a preorder traversal of a proper binary tree T is the same as the postorder traversal of T’s reflection, but in reverse order. C-8.49 Let the rank of a position p during a traversal be defined such that the first element visited has rank 1, the second element visited has rank 2, and so on. For each position p in a tree T, let pre(p) be the rank of p in a preorder traversal of T, let post(p) be the rank of p in a postorder traversal of T, let depth(p) be the depth of p, and let desc(p) be the number of descendants of p, including p itself. Derive a formula defining post(p) in terms of desc(p), depth(p), and pre(p), for each node p in T. C-8.50 Design algorithms for the following operations for a binary tree T: • preorder next(p): Return the position visited after p in a preorder traversal of T (or None if p is the last node visited). • inorder next(p): Return the position visited after p in an inorder traversal of T (or None if p is the last node visited). • postorder next(p): Return the position visited after p in a postorder traversal of T (or None if p is the last node visited). What are the worst-case running times of your algorithms? C-8.51 To implement the preorder method of the LinkedBinaryTree class, we re- lied on the convenience of Python’s generator syntax and the yield state- ment. Give an alternative implementation of preorder that returns an ex- plicit instance of a nested iterator class. (See Section 2.3.4 for discussion of iterators.) C-8.52 Algorithm preorder draw draws a binary tree T by assigning x- and y- coordinates to each position p such that x(p) is the number of nodes pre- ceding p in the preorder traversal of T and y(p) is the depth of p in T. a. Show that the drawing of T produced by preorder draw has no pairs of crossing edges. b. Redraw the binary tree of Figure 8.22 using preorder draw. C-8.53 Redo the previous problem for the algorithm postorder draw that is simi- lar to preorder draw except that it assigns x(p) to be the number of nodes preceding position p in the postorder traversal. C-8.54 Design an algorithm for drawing general trees, using a style similar to the inorder traversal approach for drawing binary trees. C-8.55 Exercise P-4.27 described the walk function of the os module. This func- tion performs a traversal of the implicit tree represented by the file system. Read the formal documentation for the function, and in particular its use of an optional Boolean parameter named topdown. Describe how its be- havior relates to tree traversal algorithms described in this chapter.
- 380. 358 Chapter 8.Trees Europe Asia Africa Australia Canada Overseas S. America Domestic International Sales Sales ( Domestic International ( Canada S. America Overseas ( Africa Europe Asia Australia ) ) ) (a) (b) Figure 8.24: (a) Tree T; (b) indented parenthetic representation of T. C-8.56 The indented parenthetic representation of a tree T is a variation of the parenthetic representation of T (see Code Fragment 8.25) that uses inden- tation and line breaks as illustrated in Figure 8.24. Give an algorithm that prints this representation of a tree. C-8.57 Let T be a binary tree with n positions. Define a Roman position to be a position p in T, such that the number of descendants in p’s left subtree differ from the number of descendants in p’s right subtree by at most 5. Describe a linear-time method for finding each position p of T, such that p is not a Roman position, but all of p’s descendants are Roman. C-8.58 Let T be a tree with n positions. Define the lowest common ancestor (LCA) between two positions p and q as the lowest position in T that has both p and q as descendants (where we allow a position to be a descendant of itself). Given two positions p and q, describe an efficient algorithm for finding the LCA of p and q. What is the running time of your algorithm? C-8.59 Let T be a binary tree with n positions, and, for any position p in T, let dp denote the depth of p in T. The distance between two positions p and q in T is dp +dq −2da, where a is the lowest common ancestor (LCA) of p and q. The diameter of T is the maximum distance between two positions in T. Describe an efficient algorithm for finding the diameter of T. What is the running time of your algorithm? C-8.60 Suppose each position p of a binary tree T is labeled with its value f(p) in a level numbering of T. Design a fast method for determining f(a) for the lowest common ancestor (LCA), a, of two positions p and q in T, given f(p) and f(q). You do not need to find position a, just value f(a). C-8.61 Give an alternative implementation of the build expression tree method of the ExpressionTree class that relies on recursion to perform an implicit Euler tour of the tree that is being built.
- 381. 8.6. Exercises 359 C-8.62Note that the build expression tree function of the ExpressionTree class is written in such a way that a leaf token can be any string; for exam- ple, it parses the expression (a*(b+c)) . However, within the evaluate method, an error would occur when attempting to convert a leaf token to a number. Modify the evaluate method to accept an optional Python dic- tionary that can be used to map such string variables to numeric values, with a syntax such as T.evaluate({ a :3, b :1, c :5}). In this way, the same algebraic expression can be evaluated using different values. C-8.63 As mentioned in Exercise C-6.22, postfix notation is an unambiguous way of writing an arithmetic expression without parentheses. It is defined so that if “(exp1)op(exp2)” is a normal (infix) fully parenthesized expres- sion with operation op, then its postfix equivalent is “pexp1 pexp2 op”, where pexp1 is the postfix version of exp1 and pexp2 is the postfix ver- sion of exp2. The postfix version of a single number or variable is just that number or variable. So, for example, the postfix version of the infix expression “((5 + 2) ∗ (8 − 3))/4” is “5 2 + 8 3 − ∗ 4 /”. Implement a postfix method of the ExpressionTree class of Section 8.5 that produces the postfix notation for the given expression. Projects P-8.64 Implement the binary tree ADT using the array-based representation de- scribed in Section 8.3.2. P-8.65 Implement the tree ADT using a linked structure as described in Sec- tion 8.3.3. Provide a reasonable set of update methods for your tree. P-8.66 The memory usage for the LinkedBinaryTree class can be streamlined by removing the parent reference from each node, and instead having each Position instance keep a member, path, that is a list of nodes representing the entire path from the root to that position. (This generally saves mem- ory because there are typically relatively few stored position instances.) Reimplement the LinkedBinaryTree class using this strategy. P-8.67 A slicing floor plan divides a rectangle with horizontal and vertical sides using horizontal and vertical cuts. (See Figure 8.25a.) A slicing floor plan can be represented by a proper binary tree, called a slicing tree, whose internal nodes represent the cuts, and whose external nodes represent the basic rectangles into which the floor plan is decomposed by the cuts. (See Figure 8.25b.) The compaction problem for a slicing floor plan is defined as follows. Assume that each basic rectangle of a slicing floor plan is assigned a minimum width w and a minimum height h. The compaction problem is to find the smallest possible height and width for each rectangle of the slicing floor plan that is compatible with the minimum dimensions
- 382. 360 Chapter 8.Trees A B C D E F D E F B C A (a) (b) Figure 8.25: (a) Slicing floor plan; (b) slicing tree associated with the floor plan. of the basic rectangles. Namely, this problem requires the assignment of values h(p) and w(p) to each position p of the slicing tree such that: w(p) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ w if p is a leaf whose basic rectangle has minimum width w max(w(),w(r)) if p is an internal position, associated with a horizontal cut, with left child and right child r w()+w(r) if p is an internal position, associated with a vertical cut, with left child and right child r h(p) = ⎧ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎪ ⎩ h if p is a leaf node whose basic rectangle has minimum height h h()+h(r) if p is an internal position, associated with a horizontal cut, with left child and right child r max(h(),h(r)) if p is an internal position, associated with a vertical cut, with left child and right child r Design a data structure for slicing floor plans that supports the operations: • Create a floor plan consisting of a single basic rectangle. • Decompose a basic rectangle by means of a horizontal cut. • Decompose a basic rectangle by means of a vertical cut. • Assign minimum height and width to a basic rectangle. • Draw the slicing tree associated with the floor plan. • Compact and draw the floor plan.
- 383. Chapter Notes 361 P-8.68Write a program that can play Tic-Tac-Toe effectively. (See Section 5.6.) To do this, you will need to create a game tree T, which is a tree where each position corresponds to a game configuration, which, in this case, is a representation of the Tic-Tac-Toe board. (See Section 8.4.2.) The root corresponds to the initial configuration. For each internal position p in T, the children of p correspond to the game states we can reach from p’s game state in a single legal move for the appropriate player, A (the first player) or B (the second player). Positions at even depths correspond to moves for A and positions at odd depths correspond to moves for B. Leaves are either final game states or are at a depth beyond which we do not want to explore. We score each leaf with a value that indicates how good this state is for player A. In large games, like chess, we have to use a heuristic scoring function, but for small games, like Tic-Tac-Toe, we can construct the entire game tree and score leaves as +1, 0, −1, indicating whether player A has a win, draw, or lose in that configuration. A good algorithm for choosing moves is minimax. In this algorithm, we assign a score to each internal position p in T, such that if p represents A’s turn, we compute p’s score as the maximum of the scores of p’s children (which corresponds to A’s optimal play from p). If an internal node p represents B’s turn, then we compute p’s score as the minimum of the scores of p’s children (which corresponds to B’s optimal play from p). P-8.69 Implement the tree ADT using the binary tree representation described in Exercise C-8.43. You may adapt the LinkedBinaryTree implementation. P-8.70 Write a program that takes as input a general tree T and a position p of T and converts T to another tree with the same set of position adjacencies, but now with p as its root. Chapter Notes Discussions of the classic preorder, inorder, and postorder tree traversal methods can be found in Knuth’s Fundamental Algorithms book [64]. The Euler tour traversal technique comes from the parallel algorithms community; it is introduced by Tarjan and Vishkin [93] and is discussed by JáJá [54] and by Karp and Ramachandran [58]. The algorithm for drawing a tree is generally considered to be a part of the “folklore” of graph-drawing al- gorithms. The reader interested in graph drawing is referred to the book by Di Battista, Eades, Tamassia, and Tollis [34] and the survey by Tamassia and Liotta [92]. The puzzle in Exercise R-8.12 was communicated by Micha Sharir.
- 384. Chapter 9 Priority Queues Contents 9.1The Priority Queue Abstract Data Type . . . . . . . . . . . 363 9.1.1 Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . 363 9.1.2 The Priority Queue ADT . . . . . . . . . . . . . . . . . . 364 9.2 Implementing a Priority Queue . . . . . . . . . . . . . . . . 365 9.2.1 The Composition Design Pattern . . . . . . . . . . . . . . 365 9.2.2 Implementation with an Unsorted List . . . . . . . . . . . 366 9.2.3 Implementation with a Sorted List . . . . . . . . . . . . . 368 9.3 Heaps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 370 9.3.1 The Heap Data Structure . . . . . . . . . . . . . . . . . . 370 9.3.2 Implementing a Priority Queue with a Heap . . . . . . . . 372 9.3.3 Array-Based Representation of a Complete Binary Tree . . 376 9.3.4 Python Heap Implementation . . . . . . . . . . . . . . . . 376 9.3.5 Analysis of a Heap-Based Priority Queue . . . . . . . . . . 379 9.3.6 Bottom-Up Heap Construction . . . . . . . . . . . . . 380 9.3.7 Python’s heapq Module . . . . . . . . . . . . . . . . . . . 384 9.4 Sorting with a Priority Queue . . . . . . . . . . . . . . . . . 385 9.4.1 Selection-Sort and Insertion-Sort . . . . . . . . . . . . . . 386 9.4.2 Heap-Sort . . . . . . . . . . . . . . . . . . . . . . . . . . 388 9.5 Adaptable Priority Queues . . . . . . . . . . . . . . . . . . 390 9.5.1 Locators . . . . . . . . . . . . . . . . . . . . . . . . . . . 390 9.5.2 Implementing an Adaptable Priority Queue . . . . . . . . 391 9.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
- 385. 9.1. The PriorityQueue Abstract Data Type 363 9.1 The Priority Queue Abstract Data Type 9.1.1 Priorities In Chapter 6, we introduced the queue ADT as a collection of objects that are added and removed according to the first-in, first-out (FIFO) principle. A com- pany’s customer call center embodies such a model in which waiting customers are told “calls will be answered in the order that they were received.” In that setting, a new call is added to the back of the queue, and each time a customer service rep- resentative becomes available, he or she is connected with the call that is removed from the front of the call queue. In practice, there are many applications in which a queue-like structure is used to manage objects that must be processed in some way, but for which the first-in, first-out policy does not suffice. Consider, for example, an air-traffic control center that has to decide which flight to clear for landing from among many approaching the airport. This choice may be influenced by factors such as each plane’s distance from the runway, time spent waiting in a holding pattern, or amount of remaining fuel. It is unlikely that the landing decisions are based purely on a FIFO policy. There are other situations in which a “first come, first serve” policy might seem reasonable, yet for which other priorities come into play. To use another airline analogy, suppose a certain flight is fully booked an hour prior to departure. Be- cause of the possibility of cancellations, the airline maintains a queue of standby passengers hoping to get a seat. Although the priority of a standby passenger is influenced by the check-in time of that passenger, other considerations include the fare paid and frequent-flyer status. So it may be that an available seat is given to a passenger who has arrived later than another, if such a passenger is assigned a better priority by the airline agent. In this chapter, we introduce a new abstract data type known as a priority queue. This is a collection of prioritized elements that allows arbitrary element insertion, and allows the removal of the element that has first priority. When an element is added to a priority queue, the user designates its priority by providing an associated key. The element with the minimum key will be the next to be removed from the queue (thus, an element with key 1 will be given priority over an element with key 2). Although it is quite common for priorities to be expressed numerically, any Python object may be used as a key, as long as the object type supports a consistent meaning for the test a b, for any instances a and b, so as to define a natural order of the keys. With such generality, applications may develop their own notion of priority for each element. For example, different financial analysts may assign different ratings (i.e., priorities) to a particular asset, such as a share of stock.
- 386. 364 Chapter 9.Priority Queues 9.1.2 The Priority Queue ADT Formally, we model an element and its priority as a key-value pair. We define the priority queue ADT to support the following methods for a priority queue P: P.add(k, v): Insert an item with key k and value v into priority queue P. P.min(): Return a tuple, (k,v), representing the key and value of an item in priority queue P with minimum key (but do not re- move the item); an error occurs if the priority queue is empty. P.remove min(): Remove an item with minimum key from priority queue P, and return a tuple, (k,v), representing the key and value of the removed item; an error occurs if the priority queue is empty. P.is empty(): Return True if priority queue P does not contain any items. len(P): Return the number of items in priority queue P. A priority queue may have multiple entries with equivalent keys, in which case methods min and remove min may report an arbitrary choice of item having mini- mum key. Values may be any type of object. In our initial model for a priority queue, we assume that an element’s key re- mains fixed once it has been added to a priority queue. In Section 9.5, we consider an extension that allows a user to update an element’s key within the priority queue. Example 9.1: The following table shows a series of operations and their effects on an initially empty priority queue P. The “Priority Queue” column is somewhat deceiving since it shows the entries as tuples and sorted by key. Such an internal representation is not required of a priority queue. Operation Return Value Priority Queue P.add(5,A) {(5,A)} P.add(9,C) {(5,A), (9,C)} P.add(3,B) {(3,B), (5,A), (9,C)} P.add(7,D) {(3,B), (5,A), (7,D), (9,C)} P.min() (3,B) {(3,B), (5,A), (7,D), (9,C)} P.remove min() (3,B) {(5,A), (7,D), (9,C)} P.remove min() (5,A) {(7,D), (9,C)} len(P) 2 {(7,D), (9,C)} P.remove min() (7,D) {(9,C)} P.remove min() (9,C) { } P.is empty() True { } P.remove min() “error” { }
- 387. 9.2. Implementing aPriority Queue 365 9.2 Implementing a Priority Queue In this section, we show how to implement a priority queue by storing its entries in a positional list L. (See Section 7.4.) We provide two realizations, depending on whether or not we keep the entries in L sorted by key. 9.2.1 The Composition Design Pattern One challenge in implementing a priority queue is that we must keep track of both an element and its key, even as items are relocated within our data structure. This is reminiscent of a case study from Section 7.6 in which we maintain access counts with each element. In that setting, we introduced the composition design pattern, defining an Item class that assured that each element remained paired with its associated count in our primary data structure. For priority queues, we will use composition to store items internally as pairs consisting of a key k and a value v. To implement this concept for all priority queue implementations, we provide a PriorityQueueBase class (see Code Fragment 9.1) that includes a definition for a nested class named Item. We define the syntax a b, for item instances a and b, to be based upon the keys. 1 class PriorityQueueBase: 2 ”””Abstract base class for a priority queue.””” 3 4 class Item: 5 ”””Lightweight composite to store priority queue items.””” 6 slots = _key , _value 7 8 def init (self, k, v): 9 self. key = k 10 self. value = v 11 12 def lt (self, other): 13 return self. key other. key # compare items based on their keys 14 15 def is empty(self): # concrete method assuming abstract len 16 ”””Return True if the priority queue is empty.””” 17 return len(self) == 0 Code Fragment 9.1: A PriorityQueueBase class with a nested Item class that com- poses a key and a value into a single object. For convenience, we provide a concrete implementation of is empty that is based on a presumed len impelementation.
- 388. 366 Chapter 9.Priority Queues 9.2.2 Implementation with an Unsorted List In our first concrete implementation of a priority queue, we store entries within an unsorted list. Our UnsortedPriorityQueue class is given in Code Fragment 9.2, inheriting from the PriorityQueueBase class introduced in Code Fragment 9.1. For internal storage, key-value pairs are represented as composites, using instances of the inherited Item class. These items are stored within a PositionalList, identified as the data member of our class. We assume that the positional list is implemented with a doubly-linked list, as in Section 7.4, so that all operations of that ADT execute in O(1) time. We begin with an empty list when a new priority queue is constructed. At all times, the size of the list equals the number of key-value pairs currently stored in the priority queue. For this reason, our priority queue len method simply returns the length of the internal data list. By the design of our PriorityQueueBase class, we inherit a concrete implementation of the is empty method that relies on a call to our len method. Each time a key-value pair is added to the priority queue, via the add method, we create a new Item composite for the given key and value, and add that item to the end of the list. Such an implementation takes O(1) time. The remaining challenge is that when min or remove min is called, we must locate the item with minimum key. Because the items are not sorted, we must inspect all entries to find one with a minimum key. For convenience, we define a nonpublic find min utility that returns the position of an item with minimum key. Knowledge of the position allows the remove min method to invoke the delete method on the positional list. The min method simply uses the position to retrieve the item when preparing a key-value tuple to return. Due to the loop for finding the minimum key, both min and remove min methods run in O(n) time, where n is the number of entries in the priority queue. A summary of the running times for the UnsortedPriorityQueue class is given in Table 9.1. Operation Running Time len O(1) is empty O(1) add O(1) min O(n) remove min O(n) Table 9.1: Worst-case running times of the methods of a priority queue of size n, realized by means of an unsorted, doubly linked list. The space requirement is O(n).
- 389. 9.2. Implementing aPriority Queue 367 1 class UnsortedPriorityQueue(PriorityQueueBase): # base class defines Item 2 ”””A min-oriented priority queue implemented with an unsorted list.””” 3 4 def find min(self): # nonpublic utility 5 ”””Return Position of item with minimum key.””” 6 if self.is empty(): # is empty inherited from base class 7 raise Empty( Priority queue is empty ) 8 small = self. data.first() 9 walk = self. data.after(small) 10 while walk is not None: 11 if walk.element( ) small.element(): 12 small = walk 13 walk = self. data.after(walk) 14 return small 15 16 def init (self): 17 ”””Create a new empty Priority Queue.””” 18 self. data = PositionalList() 19 20 def len (self): 21 ”””Return the number of items in the priority queue.””” 22 return len(self. data) 23 24 def add(self, key, value): 25 ”””Add a key-value pair.””” 26 self. data.add last(self. Item(key, value)) 27 28 def min(self): 29 ”””Return but do not remove (k,v) tuple with minimum key.””” 30 p = self. find min() 31 item = p.element() 32 return (item. key, item. value) 33 34 def remove min(self): 35 ”””Remove and return (k,v) tuple with minimum key.””” 36 p = self. find min() 37 item = self. data.delete(p) 38 return (item. key, item. value) Code Fragment 9.2: An implementation of a priority queue using an unsorted list. The parent class PriorityQueueBase is given in Code Fragment 9.1, and the PositionalList class is from Section 7.4.
- 390. 368 Chapter 9.Priority Queues 9.2.3 Implementation with a Sorted List An alternative implementation of a priority queue uses a positional list, yet main- taining entries sorted by nondecreasing keys. This ensures that the first element of the list is an entry with the smallest key. Our SortedPriorityQueue class is given in Code Fragment 9.3. The implemen- tation of min and remove min are rather straightforward given knowledge that the first element of a list has a minimum key. We rely on the first method of the posi- tional list to find the position of the first item, and the delete method to remove the entry from the list. Assuming that the list is implemented with a doubly linked list, operations min and remove min take O(1) time. This benefit comes at a cost, however, for method add now requires that we scan the list to find the appropriate position to insert the new item. Our implementation starts at the end of the list, walking backward until the new key is smaller than an existing item; in the worst case, it progresses until reaching the front of the list. Therefore, the add method takes O(n) worst-case time, where n is the number of entries in the priority queue at the time the method is executed. In summary, when using a sorted list to implement a priority queue, insertion runs in linear time, whereas finding and removing the minimum can be done in constant time. Comparing the Two List-Based Implementations Table 9.2 compares the running times of the methods of a priority queue realized by means of a sorted and unsorted list, respectively. We see an interesting trade- off when we use a list to implement the priority queue ADT. An unsorted list supports fast insertions but slow queries and deletions, whereas a sorted list allows fast queries and deletions, but slow insertions. Operation Unsorted List Sorted List len O(1) O(1) is empty O(1) O(1) add O(1) O(n) min O(n) O(1) remove min O(n) O(1) Table 9.2: Worst-case running times of the methods of a priority queue of size n, realized by means of an unsorted or sorted list, respectively. We assume that the list is implemented by a doubly linked list. The space requirement is O(n).
- 391. 9.2. Implementing aPriority Queue 369 1 class SortedPriorityQueue(PriorityQueueBase): # base class defines Item 2 ”””A min-oriented priority queue implemented with a sorted list.””” 3 4 def init (self): 5 ”””Create a new empty Priority Queue.””” 6 self. data = PositionalList() 7 8 def len (self): 9 ”””Return the number of items in the priority queue.””” 10 return len(self. data) 11 12 def add(self, key, value): 13 ”””Add a key-value pair.””” 14 newest = self. Item(key, value) # make new item instance 15 walk = self. data.last( ) # walk backward looking for smaller key 16 while walk is not None and newest walk.element(): 17 walk = self. data.before(walk) 18 if walk is None: 19 self. data.add first(newest) # new key is smallest 20 else: 21 self. data.add after(walk, newest) # newest goes after walk 22 23 def min(self): 24 ”””Return but do not remove (k,v) tuple with minimum key.””” 25 if self.is empty(): 26 raise Empty( Priority queue is empty. ) 27 p = self. data.first() 28 item = p.element() 29 return (item. key, item. value) 30 31 def remove min(self): 32 ”””Remove and return (k,v) tuple with minimum key.””” 33 if self.is empty(): 34 raise Empty( Priority queue is empty. ) 35 item = self. data.delete(self. data.first()) 36 return (item. key, item. value) Code Fragment 9.3: An implementation of a priority queue using a sorted list. The parent class PriorityQueueBase is given in Code Fragment 9.1, and the PositionalList class is from Section 7.4.
- 392. 370 Chapter 9.Priority Queues 9.3 Heaps The two strategies for implementing a priority queue ADT in the previous section demonstrate an interesting trade-off. When using an unsorted list to store entries, we can perform insertions in O(1) time, but finding or removing an element with minimum key requires an O(n)-time loop through the entire collection. In contrast, if using a sorted list, we can trivially find or remove the minimum element in O(1) time, but adding a new element to the queue may require O(n) time to restore the sorted order. In this section, we provide a more efficient realization of a priority queue using a data structure called a binary heap. This data structure allows us to perform both insertions and removals in logarithmic time, which is a significant improvement over the list-based implementations discussed in Section 9.2. The fundamental way the heap achieves this improvement is to use the structure of a binary tree to find a compromise between elements being entirely unsorted and perfectly sorted. 9.3.1 The Heap Data Structure A heap (see Figure 9.1) is a binary tree T that stores a collection of items at its positions and that satisfies two additional properties: a relational property defined in terms of the way keys are stored in T and a structural property defined in terms of the shape of T itself. The relational property is the following: Heap-Order Property: In a heap T, for every position p other than the root, the key stored at p is greater than or equal to the key stored at p’s parent. As a consequence of the heap-order property, the keys encountered on a path from the root to a leaf of T are in nondecreasing order. Also, a minimum key is always stored at the root of T. This makes it easy to locate such an item when min or remove min is called, as it is informally said to be “at the top of the heap” (hence, the name “heap” for the data structure). By the way, the heap data structure defined here has nothing to do with the memory heap (Section 15.1.1) used in the run-time environment supporting a programming language like Python. For the sake of efficiency, as will become clear later, we want the heap T to have as small a height as possible. We enforce this requirement by insisting that the heap T satisfy an additional structural property—it must be what we term complete. Complete Binary Tree Property: A heap T with height h is a complete binary tree if levels 0,1,2,...,h− 1 of T have the maximum number of nodes possible (namely, level i has 2i nodes, for 0 ≤ i ≤ h− 1) and the remaining nodes at level h reside in the leftmost possible positions at that level.
- 393. 9.3. Heaps 371 (14,E) (5,A)(6,Z) (20,B) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (4,C) Figure 9.1: Example of a heap storing 13 entries with integer keys. The last position is the one storing entry (13,W). The tree in Figure 9.1 is complete because levels 0, 1, and 2 are full, and the six nodes in level 3 are in the six leftmost possible positions at that level. In formalizing what we mean by the leftmost possible positions, we refer to the discussion of level numbering from Section 8.3.2, in the context of an array-based representation of a binary tree. (In fact, in Section 9.3.3 we will discuss the use of an array to represent a heap.) A complete binary tree with n elements is one that has positions with level numbering 0 through n − 1. For example, in an array-based representation of the above tree, its 13 entries would be stored consecutively from A[0] to A[12]. The Height of a Heap Let h denote the height of T. Insisting that T be complete also has an important consequence, as shown in Proposition 9.2. Proposition 9.2: A heap T storing n entries has height h = logn. Justification: From the fact that T is complete, we know that the number of nodes in levels 0 through h−1 of T is precisely 1+2+4+···+2h−1 = 2h −1, and that the number of nodes in level h is at least 1 and at most 2h. Therefore n ≥ 2h −1+1 = 2h and n ≤ 2h −1+2h = 2h+1 −1. By taking the logarithm of both sides of inequality 2h ≤ n, we see that height h ≤ logn. By rearranging terms and taking the logarithm of both sides of inequality n ≤ 2h+1 − 1, we see that log(n + 1) − 1 ≤ h. Since h is an integer, these two inequalities imply that h = logn.
- 394. 372 Chapter 9.Priority Queues 9.3.2 Implementing a Priority Queue with a Heap Proposition 9.2 has an important consequence, for it implies that if we can perform update operations on a heap in time proportional to its height, then those opera- tions will run in logarithmic time. Let us therefore turn to the problem of how to efficiently perform various priority queue methods using a heap. We will use the composition pattern from Section 9.2.1 to store key-value pairs as items in the heap. The len and is empty methods can be implemented based on examination of the tree, and the min operation is equally trivial because the heap property assures that the element at the root of the tree has a minimum key. The interesting algorithms are those for implementing the add and remove min methods. Adding an Item to the Heap Let us consider how to perform add(k,v) on a priority queue implemented with a heap T. We store the pair (k,v) as an item at a new node of the tree. To maintain the complete binary tree property, that new node should be placed at a position p just beyond the rightmost node at the bottom level of the tree, or as the leftmost position of a new level, if the bottom level is already full (or if the heap is empty). Up-Heap Bubbling After an Insertion After this action, the tree T is complete, but it may violate the heap-order property. Hence, unless position p is the root of T (that is, the priority queue was empty before the insertion), we compare the key at position p to that of p’s parent, which we denote as q. If key kp ≥ kq, the heap-order property is satisfied and the algorithm terminates. If instead kp kq, then we need to restore the heap-order property, which can be locally achieved by swapping the entries stored at positions p and q. (See Figure 9.2c and d.) This swap causes the new item to move up one level. Again, the heap-order property may be violated, so we repeat the process, going up in T until no violation of the heap-order property occurs. (See Figure 9.2e and h.) The upward movement of the newly inserted entry by means of swaps is con- ventionally called up-heap bubbling. A swap either resolves the violation of the heap-order property or propagates it one level up in the heap. In the worst case, up- heap bubbling causes the new entry to move all the way up to the root of heap T. Thus, in the worst case, the number of swaps performed in the execution of method add is equal to the height of T. By Proposition 9.2, that bound is logn.
- 395. 9.3. Heaps 373 (14,E) (5,A)(6,Z) (20,B) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (4,C) (2,T) (5,A) (6,Z) (20,B) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (4,C) (a) (b) (20,B) (5,A) (6,Z) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (2,T) (4,C) (2,T) (5,A) (6,Z) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (20,B) (4,C) (c) (d) (2,T) (5,A) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (20,B) (6,Z) (4,C) (6,Z) (5,A) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (20,B) (2,T) (4,C) (e) (f) (4,C) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (20,B) (6,Z) (2,T) (5,A) (6,Z) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (13,W) (12,H) (14,E) (20,B) (2,T) (4,C) (5,A) (g) (h) Figure 9.2: Insertion of a new entry with key 2 into the heap of Figure 9.1: (a) initial heap; (b) after performing operation add; (c and d) swap to locally restore the partial order property; (e and f) another swap; (g and h) final swap.
- 396. 374 Chapter 9.Priority Queues Removing the Item with Minimum Key Let us now turn to method remove min of the priority queue ADT. We know that an entry with the smallest key is stored at the root r of T (even if there is more than one entry with smallest key). However, in general we cannot simply delete node r, because this would leave two disconnected subtrees. Instead, we ensure that the shape of the heap respects the complete binary tree property by deleting the leaf at the last position p of T, defined as the rightmost position at the bottommost level of the tree. To preserve the item from the last position p, we copy it to the root r (in place of the item with minimum key that is being removed by the operation). Figure 9.3a and b illustrates an example of these steps, with minimal item (4,C) being removed from the root and replaced by item (13,W) from the last position. The node at the last position is removed from the tree. Down-Heap Bubbling After a Removal We are not yet done, however, for even though T is now complete, it likely violates the heap-order property. If T has only one node (the root), then the heap-order property is trivially satisfied and the algorithm terminates. Otherwise, we distin- guish two cases, where p initially denotes the root of T: • If p has no right child, let c be the left child of p. • Otherwise (p has both children), let c be a child of p with minimal key. If key kp ≤ kc, the heap-order property is satisfied and the algorithm terminates. If instead kp kc, then we need to restore the heap-order property. This can be locally achieved by swapping the entries stored at p and c. (See Figure 9.3c and d.) It is worth noting that when p has two children, we intentionally consider the smaller key of the two children. Not only is the key of c smaller than that of p, it is at least as small as the key at c’s sibling. This ensures that the heap-order property is locally restored when that smaller key is promoted above the key that had been at p and that at c’s sibling. Having restored the heap-order property for node p relative to its children, there may be a violation of this property at c; hence, we may have to continue swapping down T until no violation of the heap-order property occurs. (See Figure 9.3e–h.) This downward swapping process is called down-heap bubbling. A swap either resolves the violation of the heap-order property or propagates it one level down in the heap. In the worst case, an entry moves all the way down to the bottom level. (See Figure 9.3.) Thus, the number of swaps performed in the execution of method remove min is, in the worst case, equal to the height of heap T, that is, it is logn by Proposition 9.2.
- 397. 9.3. Heaps 375 (13,W) (6,Z) (20,B) (7,Q) (9,F) (15,K) (11,S) (16,X)(25,J) (12,H) (14,E) (4,C) (5,A) (13,W) (14,E) (12,H) (25,J) (16,X) (11,S) (15,K) (9,F) (7,Q) (20,B) (6,Z) (5,A) (a) (b) (13,W) (20,B) (7,Q) (9,F) (15,K) (11,S) (16,X) (25,J) (12,H) (14,E) (5,A) (6,Z) (13,W) (14,E) (12,H) (25,J) (16,X) (11,S) (15,K) (9,F) (7,Q) (20,B) (6,Z) (5,A) (c) (d) (9,F) (20,B) (7,Q) (15,K) (11,S) (16,X) (25,J) (12,H) (14,E) (5,A) (13,W) (6,Z) (13,W) (14,E) (12,H) (25,J) (16,X) (11,S) (15,K) (7,Q) (20,B) (6,Z) (5,A) (9,F) (e) (f) (13,W) (20,B) (7,Q) (15,K) (5,A) (9,F) (11,S) (14,E) (25,J) (16,X) (12,H) (6,Z) (13,W) (20,B) (7,Q) (15,K) (5,A) (9,F) (12,H) (11,S) (14,E) (25,J) (16,X) (6,Z) (g) (h) Figure 9.3: Removal of the entry with the smallest key from a heap: (a and b) deletion of the last node, whose entry gets stored into the root; (c and d) swap to locally restore the heap-order property; (e and f) another swap; (g and h) final swap.
- 398. 376 Chapter 9.Priority Queues 9.3.3 Array-Based Representation of a Complete Binary Tree The array-based representation of a binary tree (Section 8.3.2) is especially suitable for a complete binary tree T. We recall that in this implementation, the elements of T are stored in an array-based list A such that the element at position p in T is stored in A with index equal to the level number f(p) of p, defined as follows: • If p is the root of T, then f(p) = 0. • If p is the left child of position q, then f(p) = 2f(q)+1. • If p is the right child of position q, then f(p) = 2f(q)+2. With this implementation, the elements of T have contiguous indices in the range [0,n−1] and the last position of T is always at index n−1, where n is the number of positions of T. For example, Figure 9.4 illustrates the array-based representation of the heap structure originally portrayed in Figure 9.1. 0 1 2 3 4 5 6 7 8 9 10 11 12 (4,C) (6,Z) (15,K) (9,F) (7,Q) (20,B) (16,X) (25,J) (14,E) (12,H) (11,S) (8,W) (5,A) Figure 9.4: An array-based representation of the heap from Figure 9.1. Implementing a priority queue using an array-based heap representation allows us to avoid some complexities of a node-based tree structure. In particular, the add and remove min operations of a priority queue both depend on locating the last index of a heap of size n. With the array-based representation, the last position is at index n− 1 of the array. Locating the last position of a complete binary tree implemented with a linked structure requires more effort. (See Exercise C-9.34.) If the size of a priority queue is not known in advance, use of an array-based representation does introduce the need to dynamically resize the array on occasion, as is done with a Python list. The space usage of such an array-based representation of a complete binary tree with n nodes is O(n), and the time bounds of methods for adding or removing elements become amortized. (See Section 5.3.1.) 9.3.4 Python Heap Implementation We provide a Python implementation of a heap-based priority queue in Code Frag- ments 9.4 and 9.5. We use an array-based representation, maintaining a Python list of item composites. Although we do not formally use the binary tree ADT, Code Fragment 9.4 includes nonpublic utility functions that compute the level numbering of a parent or child of another. This allows us to describe the rest of our algorithms using tree-like terminology of parent, left, and right. However, the relevant vari- ables are integer indexes (not “position” objects). We use recursion to implement the repetition in the upheap and downheap utilities.
- 399. 9.3. Heaps 377 1class HeapPriorityQueue(PriorityQueueBase): # base class defines Item 2 ”””A min-oriented priority queue implemented with a binary heap.””” 3 #------------------------------ nonpublic behaviors ------------------------------ 4 def parent(self, j): 5 return (j−1) // 2 6 7 def left(self, j): 8 return 2 j + 1 9 10 def right(self, j): 11 return 2 j + 2 12 13 def has left(self, j): 14 return self. left(j) len(self. data) # index beyond end of list? 15 16 def has right(self, j): 17 return self. right(j) len(self. data) # index beyond end of list? 18 19 def swap(self, i, j): 20 ”””Swap the elements at indices i and j of array.””” 21 self. data[i], self. data[j] = self. data[j], self. data[i] 22 23 def upheap(self, j): 24 parent = self. parent(j) 25 if j 0 and self. data[j] self. data[parent]: 26 self. swap(j, parent) 27 self. upheap(parent) # recur at position of parent 28 29 def downheap(self, j): 30 if self. has left(j): 31 left = self. left(j) 32 small child = left # although right may be smaller 33 if self. has right(j): 34 right = self. right(j) 35 if self. data[right] self. data[left]: 36 small child = right 37 if self. data[small child] self. data[j]: 38 self. swap(j, small child) 39 self. downheap(small child) # recur at position of small child Code Fragment 9.4: An implementation of a priority queue using an array-based heap (continued in Code Fragment 9.5). The extends the PriorityQueueBase class from Code Fragment 9.1.
- 400. 378 Chapter 9.Priority Queues 40 #------------------------------ public behaviors ------------------------------ 41 def init (self): 42 ”””Create a new empty Priority Queue.””” 43 self. data = [ ] 44 45 def len (self): 46 ”””Return the number of items in the priority queue.””” 47 return len(self. data) 48 49 def add(self, key, value): 50 ”””Add a key-value pair to the priority queue.””” 51 self. data.append(self. Item(key, value)) 52 self. upheap(len(self. data) − 1) # upheap newly added position 53 54 def min(self): 55 ”””Return but do not remove (k,v) tuple with minimum key. 56 57 Raise Empty exception if empty. 58 ””” 59 if self.is empty(): 60 raise Empty( Priority queue is empty. ) 61 item = self. data[0] 62 return (item. key, item. value) 63 64 def remove min(self): 65 ”””Remove and return (k,v) tuple with minimum key. 66 67 Raise Empty exception if empty. 68 ””” 69 if self.is empty(): 70 raise Empty( Priority queue is empty. ) 71 self. swap(0, len(self. data) − 1) # put minimum item at the end 72 item = self. data.pop( ) # and remove it from the list; 73 self. downheap(0) # then fix new root 74 return (item. key, item. value) Code Fragment 9.5: An implementation of a priority queue using an array-based heap (continued from Code Fragment 9.4).
- 401. 9.3. Heaps 379 9.3.5Analysis of a Heap-Based Priority Queue Table 9.3 shows the running time of the priority queue ADT methods for the heap implementation of a priority queue, assuming that two keys can be compared in O(1) time and that the heap T is implemented with an array-based or linked-based tree representation. In short, each of the priority queue ADT methods can be performed in O(1) or in O(logn) time, where n is the number of entries at the time the method is exe- cuted. The analysis of the running time of the methods is based on the following: • The heap T has n nodes, each storing a reference to a key-value pair. • The height of heap T is O(logn), since T is complete (Proposition 9.2). • The min operation runs in O(1) because the root of the tree contains such an element. • Locating the last position of a heap, as required for add and remove min, can be performed in O(1) time for an array-based representation, or O(logn) time for a linked-tree representation. (See Exercise C-9.34.) • In the worst case, up-heap and down-heap bubbling perform a number of swaps equal to the height of T. Operation Running Time len(P), P.is empty() O(1) P.min() O(1) P.add() O(logn)∗ P.remove min() O(logn)∗ ∗amortized, if array-based Table 9.3: Performance of a priority queue, P, realized by means of a heap. We let n denote the number of entries in the priority queue at the time an operation is executed. The space requirement is O(n). The running time of operations min and remove min are amortized for an array-based representation, due to occasional re- sizing of a dynamic array; those bounds are worst case with a linked tree structure. We conclude that the heap data structure is a very efficient realization of the priority queue ADT, independent of whether the heap is implemented with a linked structure or an array. The heap-based implementation achieves fast running times for both insertion and removal, unlike the implementations that were based on using an unsorted or sorted list.
- 402. 380 Chapter 9.Priority Queues 9.3.6 Bottom-Up Heap Construction If we start with an initially empty heap, n successive calls to the add operation will run in O(nlogn) time in the worst case. However, if all n key-value pairs to be stored in the heap are given in advance, such as during the first phase of the heap- sort algorithm, there is an alternative bottom-up construction method that runs in O(n) time. (Heap-sort, however, still requires Θ(nlogn) time because of the second phase in which we repeatedly remove the remaining element with smallest key.) In this section, we describe the bottom-up heap construction, and provide an implementation that can be used by the constructor of a heap-based priority queue. For simplicity of exposition, we describe this bottom-up heap construction as- suming the number of keys, n, is an integer such that n = 2h+1 − 1. That is, the heap is a complete binary tree with every level being full, so the heap has height h = log(n + 1) − 1. Viewed nonrecursively, bottom-up heap construction consists of the following h+1 = log(n+1) steps: 1. In the first step (see Figure 9.5b), we construct (n+ 1)/2 elementary heaps storing one entry each. 2. In the second step (see Figure 9.5c–d), we form (n+1)/4 heaps, each storing three entries, by joining pairs of elementary heaps and adding a new entry. The new entry is placed at the root and may have to be swapped with the entry stored at a child to preserve the heap-order property. 3. In the third step (see Figure 9.5e–f), we form (n+ 1)/8 heaps, each storing 7 entries, by joining pairs of 3-entry heaps (constructed in the previous step) and adding a new entry. The new entry is placed initially at the root, but may have to move down with a down-heap bubbling to preserve the heap-order property. . . . i. In the generic ith step, 2 ≤ i ≤ h, we form (n+1)/2i heaps, each storing 2i −1 entries, by joining pairs of heaps storing (2i−1 −1) entries (constructed in the previous step) and adding a new entry. The new entry is placed initially at the root, but may have to move down with a down-heap bubbling to preserve the heap-order property. . . . h+1. In the last step (see Figure 9.5g–h), we form the final heap, storing all the n entries, by joining two heaps storing (n− 1)/2 entries (constructed in the previous step) and adding a new entry. The new entry is placed initially at the root, but may have to move down with a down-heap bubbling to preserve the heap-order property. We illustrate bottom-up heap construction in Figure 9.5 for h = 3.
- 403. 9.3. Heaps 381 4 1512 6 7 23 20 16 (a) (b) 4 16 15 9 12 6 7 11 23 17 20 25 20 16 25 9 4 12 11 7 6 23 17 15 (c) (d) 25 12 11 23 20 17 15 16 8 4 9 5 6 7 25 12 11 23 20 17 15 16 8 5 9 4 6 7 (e) (f) 25 12 11 8 23 20 17 7 15 6 16 5 14 4 9 25 12 11 8 23 20 17 7 15 6 16 14 4 5 9 (g) (h) Figure 9.5: Bottom-up construction of a heap with 15 entries: (a and b) we begin by constructing 1-entry heaps on the bottom level; (c and d) we combine these heaps into 3-entry heaps, and then (e and f) 7-entry heaps, until (g and h) we create the final heap. The paths of the down-heap bubblings are highlighted in (d, f, and h). For simplicity, we only show the key within each node instead of the entire entry.
- 404. 382 Chapter 9.Priority Queues Python Implementation of a Bottom-Up Heap Construction Implementing a bottom-up heap construction is quite easy, given the existence of a “down-heap” utility function. The “merging” of two equally sized heaps that are subtrees of a common position p, as described in the opening of this section, can be accomplished simply by down-heaping p’s entry. For example, that is what happened to the key 14 in going from Figure 9.5(f) to (g). With our array-based representation of a heap, if we initially store all n items in arbitrary order within the array, we can implement the bottom-up heap construction process with a single loop that makes a call to downheap from each position of the tree, as long as those calls are ordered starting with the deepest level and ending with the root of the tree. In fact, that loop can start with the deepest nonleaf, since there is no effect when down-heap is called at a leaf position. In Code Fragment 9.6, we augment the original HeapPriorityQueue class from Section 9.3.4 to provide support for the bottom-up construction of an initial col- lection. We introduce a nonpublic utility method, heapify, that calls downheap on each nonleaf position, beginning with the deepest and concluding with a call at the root of the tree. We have redesigned the constructor of the class to accept an optional parameter that can be any sequence of (k,v) tuples. Rather than initializing self. data to an empty list, we use a list comprehension syntax (see Section 1.9.2) to create an initial list of item composites based on the given contents. We de- clare an empty sequence as the default parameter value so that the default syntax HeapPriorityQueue() continues to result in an empty priority queue. def init (self, contents=()): ”””Create a new priority queue. By default, queue will be empty. If contents is given, it should be as an iterable sequence of (k,v) tuples specifying the initial contents. ””” self. data = [ self. Item(k,v) for k,v in contents ] # empty by default if len(self. data) 1: self. heapify() def heapify(self): start = self. parent(len(self) − 1) # start at PARENT of last leaf for j in range(start, −1, −1): # going to and including the root self. downheap(j) Code Fragment 9.6: Revision to the HeapPriorityQueue class of Code Frag- ments 9.4 and 9.5 to support a linear-time construction given an initial sequence of entries.
- 405. 9.3. Heaps 383 AsymptoticAnalysis of Bottom-Up Heap Construction Bottom-up heap construction is asymptotically faster than incrementally inserting n keys into an initially empty heap. Intuitively, we are performing a single down- heap operation at each position in the tree, rather than a single up-heap operation from each. Since more nodes are closer to the bottom of a tree than the top, the sum of the downward paths is linear, as shown in the following proposition. Proposition 9.3: Bottom-up construction of a heap with n entries takes O(n) time, assuming two keys can be compared in O(1) time. Justification: The primary cost of the construction is due to the down-heap steps performed at each nonleaf position. Let πv denote the path of T from nonleaf node v to its “inorder successor” leaf, that is, the path that starts at v, goes to the right child of v, and then goes down leftward until it reaches a leaf. Although, πv is not necessarily the path followed by the down-heap bubbling step from v, the length πv (its number of edges) is proportional to the height of the subtree rooted at v, and thus a bound on the complexity of the down-heap operation at v. We can bound the total running time of the bottom-up heap construction algorithm based on the sum of the sizes of paths, ∑v πv. For intuition, Figure 9.6 illustrates the justification “visually,” marking each edge with the label of the nonleaf node v whose path πv contains that edge. We claim that the paths πv for all nonleaf v are edge-disjoint, and thus the sum of the path lengths is bounded by the number of total edges in the tree, hence O(n). To show this, we consider what we term “right-leaning” and “left-leaning” edges (i.e., those going from a parent to a right, respectively left, child). A particular right- leaning edge e can only be part of the path πv for node v that is the parent in the relationship represented by e. Left-leaning edges can be partitioned by considering the leaf that is reached if continuing down leftward until reaching a leaf. Each nonleaf node only uses left-leaning edges in the group leading to that nonleaf node’s inorder successor. Since each nonleaf node must have a different inorder successor, no two such paths can contain the same left-leaning edge. We conclude that the bottom-up construction of heap T takes O(n) time. 15 6 4 16 5 25 14 12 11 8 23 20 17 7 9 15 7 17 5 4 6 5 4 6 4 9 Figure 9.6: Visual justification of the linear running time of bottom-up heap con- struction. Each edge e is labeled with a node v for which πv contains e (if any).
- 406. 384 Chapter 9.Priority Queues 9.3.7 Python’s heapq Module Python’s standard distribution includes a heapq module that provides support for heap-based priority queues. That module does not provide any priority queue class; instead it provides functions that allow a standard Python list to be managed as a heap. Its model is essentially the same as our own, with n elements stored in list cells L[0] through L[n − 1], based on the level-numbering indices with the small- est element at the root in L[0]. We note that heapq does not separately manage associated values; elements serve as their own key. The heapq module supports the following functions, all of which presume that existing list L satisfies the heap-order property prior to the call: heappush(L, e): Push element e onto list L and restore the heap-order property. The function executes in O(logn) time. heappop(L): Pop and return the element with smallest value from list L, and reestablish the heap-order property. The operation executes in O(logn) time. heappushpop(L, e): Push element e on list L and then pop and return the smallest item. The time is O(logn), but it is slightly more efficient than separate calls to push and pop because the size of the list never changes. If the newly pushed el- ement becomes the smallest, it is immediately returned. Otherwise, the new element takes the place of the popped element at the root and a down-heap is performed. heapreplace(L, e): Similar to heappushpop, but equivalent to the pop be- ing performed before the push (in other words, the new element cannot be returned as the smallest). Again, the time is O(logn), but it is more efficient that two separate operations. The module supports additional functions that operate on sequences that do not previously satisfy the heap-order property. heapify(L): Transform unordered list to satisfy the heap-order prop- erty. This executes in O(n) time by using the bottom-up construction algorithm. nlargest(k, iterable): Produce a list of the k largest values from a given iterable. This can be implemented to run in O(n + klogn) time, where we use n to denote the length of the iterable (see Exercise C-9.42). nsmallest(k, iterable): Produce a list of the k smallest values from a given it- erable. This can be implemented to run in O(n+klogn) time, using similar technique as with nlargest.
- 407. 9.4. Sorting witha Priority Queue 385 9.4 Sorting with a Priority Queue In defining the priority queue ADT, we noted that any type of object can be used as a key, but that any pair of keys must be comparable to each other, and that the set of keys be naturally ordered. In Python, it is common to rely on the operator to define such an order, in which case the following properties must be satisfied: • Irreflexive property: k k. • Transitive property: if k1 k2 and k2 k3, then k1 k3. Formally, such a relationship defines what is known as a strict weak order, as it allows for keys to be considered equal to each other, but the broader equivalence classes are totally ordered, as they can be uniquely arranged from smallest to largest due to the transitive property. As our first application of priority queues, we demonstrate how they can be used to sort a collection C of comparable elements. That is, we can produce a sequence of elements of C in increasing order (or at least in nondecreasing order if there are duplicates). The algorithm is quite simple—we insert all elements into an initially empty priority queue, and then we repeatedly call remove min to retrieve the elements in nondecreasing order. An implementation of this algorithm is given in Code Fragment 9.7, assuming that C is a positional list. (See Chapter 7.4.) We use an original element of the collection as both a key and value when calling P.add(element, element). 1 def pq sort(C): 2 ”””Sort a collection of elements stored in a positional list.””” 3 n = len(C) 4 P = PriorityQueue() 5 for j in range(n): 6 element = C.delete(C.first()) 7 P.add(element, element) # use element as key and value 8 for j in range(n): 9 (k,v) = P.remove min() 10 C.add last(v) # store smallest remaining element in C Code Fragment 9.7: An implementation of the pq sort function, assuming an ap- propriate implementation of a PriorityQueue class. Note that each element of the input list C serves as its own key in the priority queue P. With a minor modification to this code, we can provide more general sup- port, sorting elements according to an ordering other than the default. For exam- ple, when working with strings, the operator defines a lexicographic ordering, which is an extension of the alphabetic ordering to Unicode. For example, we have that 12 4 because of the order of the first character of each string, just as
- 408. 386 Chapter 9.Priority Queues apple banana . Suppose that we have an application in which we have a list of strings that are all known to represent integral values (e.g., 12 ), and our goal is to sort the strings according to those integral values. In Python, the standard approach for customizing the order for a sorting algo- rithm is to provide, as an optional parameter to the sorting function, an object that is itself a one-parameter function that computes a key for a given element. (See Sections 1.5 and 1.10 for a discussion of this approach in the context of the built- in max function.) For example, with a list of (numeric) strings, we might wish to use the value of int(s) as a key for a string s of the list. In this case, the con- structor for the int class can serve as the one-parameter function for computing a key. In that way, the string 4 will be ordered before string 12 because its key int( 4 ) int( 12 ). We leave it as an exercise to support such an optional key parameter for the pq sort function. (See Exercise C-9.46.) 9.4.1 Selection-Sort and Insertion-Sort Our pq sort function works correctly given any valid implementation of the pri- ority queue class. However, the running time of the sorting algorithm depends on the running times of the operations add and remove min for the given priority queue class. We next discuss a choice of priority queue implementations that in effect cause the pq sort computation to behave as one of several classic sorting algorithms. Selection-Sort If we implement P with an unsorted list, then Phase 1 of pq sort takes O(n) time, for we can add each element in O(1) time. In Phase 2, the running time of each remove min operation is proportional to the size of P. Thus, the bottleneck com- putation is the repeated “selection” of the minimum element in Phase 2. For this reason, this algorithm is better known as selection-sort. (See Figure 9.7.) As noted above, the bottleneck is in Phase 2 where we repeatedly remove an entry with smallest key from the priority queue P. The size of P starts at n and incrementally decreases with each remove min until it becomes 0. Thus, the first operation takes time O(n), the second one takes time O(n− 1), and so on. There- fore, the total time needed for the second phase is O(n+(n−1)+···+2+1) = O(∑n i=1 i). By Proposition 3.3, we have ∑n i=1 i = n(n+1)/2. Thus, Phase 2 takes time O(n2), as does the entire selection-sort algorithm.
- 409. 9.4. Sorting witha Priority Queue 387 Collection C Priority Queue P Input (7,4,8,2,5,3) () Phase 1 (a) (4,8,2,5,3) (7) (b) (8,2,5,3) (7,4) . . . . . . . . . (f) () (7,4,8,2,5,3) Phase 2 (a) (2) (7,4,8,5,3) (b) (2,3) (7,4,8,5) (c) (2,3,4) (7,8,5) (d) (2,3,4,5) (7,8) (e) (2,3,4,5,7) (8) (f) (2,3,4,5,7,8) () Figure 9.7: Execution of selection-sort on collection C = (7,4,8,2,5,3). Insertion-Sort If we implement the priority queue P using a sorted list, then we improve the run- ning time of Phase 2 to O(n), for each remove min operation on P now takes O(1) time. Unfortunately, Phase 1 becomes the bottleneck for the running time, since, in the worst case, each add operation takes time proportional to the current size of P. This sorting algorithm is better known as insertion-sort (see Figure 9.8); in fact, our implementation for adding an element to a priority queue is almost identical to a step of insertion-sort as presented in Section 7.5. The worst-case running time of Phase 1 of insertion-sort is O(1+2+...+(n−1)+n) = O(∑n i=1 i). Again, by Proposition 3.3, this implies a worst-case O(n2) time for Phase 1, and thus, the entire insertion-sort algorithm. However, unlike selection-sort, insertion- sort has a best-case running time of O(n). Collection C Priority Queue P Input (7,4,8,2,5,3) () Phase 1 (a) (4,8,2,5,3) (7) (b) (8,2,5,3) (4,7) (c) (2,5,3) (4,7,8) (d) (5,3) (2,4,7,8) (e) (3) (2,4,5,7,8) (f) () (2,3,4,5,7,8) Phase 2 (a) (2) (3,4,5,7,8) (b) (2,3) (4,5,7,8) . . . . . . . . . (f) (2,3,4,5,7,8) () Figure 9.8: Execution of insertion-sort on collection C = (7,4,8,2,5,3).
- 410. 388 Chapter 9.Priority Queues 9.4.2 Heap-Sort As we have previously observed, realizing a priority queue with a heap has the advantage that all the methods in the priority queue ADT run in logarithmic time or better. Hence, this realization is suitable for applications where fast running times are sought for all the priority queue methods. Therefore, let us again consider the pq sort scheme, this time using a heap-based implementation of the priority queue. During Phase 1, the ith add operation takes O(logi) time, since the heap has i entries after the operation is performed. Therefore this phase takes O(nlogn) time. (It could be improved to O(n) with the bottom-up heap construction described in Section 9.3.6.) During the second phase of pq sort, the jth remove min operation runs in O(log(n − j + 1)), since the heap has n − j + 1 entries at the time the operation is performed. Summing over all j, this phase takes O(nlogn) time, so the entire priority-queue sorting algorithm runs in O(nlogn) time when we use a heap to im- plement the priority queue. This sorting algorithm is better known as heap-sort, and its performance is summarized in the following proposition. Proposition 9.4: The heap-sort algorithm sorts a collection C of n elements in O(nlogn) time, assuming two elements of C can be compared in O(1) time. Let us stress that the O(nlogn) running time of heap-sort is considerably better than the O(n2) running time of selection-sort and insertion-sort (Section 9.4.1). Implementing Heap-Sort In-Place If the collection C to be sorted is implemented by means of an array-based se- quence, most notably as a Python list, we can speed up heap-sort and reduce its space requirement by a constant factor using a portion of the list itself to store the heap, thus avoiding the use of an auxiliary heap data structure. This is accomplished by modifying the algorithm as follows: 1. We redefine the heap operations to be a maximum-oriented heap, with each position’s key being at least as large as its children. This can be done by recoding the algorithm, or by adjusting the notion of keys to be negatively oriented. At any time during the execution of the algorithm, we use the left portion of C, up to a certain index i−1, to store the entries of the heap, and the right portion of C, from index i to n − 1, to store the elements of the sequence. Thus, the first i elements of C (at indices 0,...,i − 1) provide the array-list representation of the heap. 2. In the first phase of the algorithm, we start with an empty heap and move the boundary between the heap and the sequence from left to right, one step at a time. In step i, for i = 1,...,n, we expand the heap by adding the element at index i−1.
- 411. 9.4. Sorting witha Priority Queue 389 3. In the second phase of the algorithm, we start with an empty sequence and move the boundary between the heap and the sequence from right to left, one step at a time. At step i, for i = 1,...,n, we remove a maximum element from the heap and store it at index n−i. In general, we say that a sorting algorithm is in-place if it uses only a small amount of memory in addition to the sequence storing the objects to be sorted. The variation of heap-sort above qualifies as in-place; instead of transferring elements out of the sequence and then back in, we simply rearrange them. We illustrate the second phase of in-place heap-sort in Figure 9.9. (e) 4 5 7 9 6 2 4 2 5 7 6 2 4 (c) 6 4 5 7 9 6 7 9 5 4 4 5 6 7 9 6 2 (f) 2 5 (b) 7 6 5 2 4 9 2 2 4 5 6 7 4 2 4 2 9 (a) 9 7 5 2 6 4 2 5 (d) Figure 9.9: Phase 2 of an in-place heap-sort. The heap portion of each sequence representation is highlighted. The binary tree that each sequence (implicitly) repre- sents is diagrammed with the most recent path of down-heap bubbling highlighted.
- 412. 390 Chapter 9.Priority Queues 9.5 Adaptable Priority Queues The methods of the priority queue ADT given in Section 9.1.2 are sufficient for most basic applications of priority queues, such as sorting. However, there are situations in which additional methods would be useful, as shown by the scenarios below involving the standby airline passenger application. • A standby passenger with a pessimistic attitude may become tired of waiting and decide to leave ahead of the boarding time, requesting to be removed from the waiting list. Thus, we would like to remove from the priority queue the entry associated with this passenger. Operation remove min does not suffice since the passenger leaving does not necessarily have first priority. Instead, we want a new operation, remove, that removes an arbitrary entry. • Another standby passenger finds her gold frequent-flyer card and shows it to the agent. Thus, her priority has to be modified accordingly. To achieve this change of priority, we would like to have a new operation update allowing us to replace the key of an existing entry with a new key. We will see another application of adaptable priority queues when implementing certain graph algorithms in Sections 14.6.2 and 14.7.1. In this section, we develop an adaptable priority queue ADT and demonstrate how to implement this abstraction as an extension to our heap-based priority queue. 9.5.1 Locators In order to implement methods update and remove efficiently, we need a mecha- nism for finding a user’s element within a priority queue that avoids performing a linear search through the entire collection. To support our goal, when a new ele- ment is added to the priority queue, we return a special object known as a locator to the caller. We then require the user to provide an appropriate locator as a parameter when invoking the update or remove method, as follows, for a priority queue P: P.update(loc, k, v): Replace key and value for the item identified by locator loc. P.remove(loc): Remove the item identified by locator loc from the priority queue and return its (key,value) pair. The locator abstraction is somewhat akin to the Position abstraction used in our positional list ADT from Section 7.4, and our tree ADT from Chapter 8. However, we differentiate between a locator and a position because a locator for a priority queue does not represent a tangible placement of an element within the structure. In our priority queue, an element may be relocated within our data structure during an operation that does not seem directly relevant to that element. A locator for an item will remain valid, as long as that item remains somewhere in the queue.
- 413. 9.5. Adaptable PriorityQueues 391 9.5.2 Implementing an Adaptable Priority Queue In this section, we provide a Python implementation of an adaptable priority queue as an extension of our HeapPriorityQueue class from Section 9.3.4. To implement a Locator class, we will extend the existing Item composite to add an additional field designating the current index of the element within the array-based representation of our heap, as shown in Figure 9.10. 0 1 2 3 4 5 6 7 token (15,K,3) (16,X,7) (9,F,4) (5,A,1) (20,B,6) (6,Z,2) (7,Q,5) (4,C,0) Figure 9.10: Representing a heap using a sequence of locators. The third element of each locator instance corresponds to the index of the item within the array. Iden- tifier token is presumed to be a locator reference in the user’s scope. The list is a sequence of references to locator instances, each of which stores a key, value, and the current index of the item within the list. The user will be given a reference to the Locator instance for each inserted element, as portrayed by the token identifier in Figure 9.10. When we perform priority queue operations on our heap, and items are relo- cated within our structure, we reposition the locator instances within the list and we update the third field of each locator to reflect its new index within the list. As an ex- ample, Figure 9.11 shows the state of the above heap after a call to remove min(). The heap operation caused the minimum entry, (4,C), to be removed, and the en- try, (16,X), to be temporarily moved from the last position to the root, followed by a down-heap bubble phase. During the down-heap, element (16,X) was swapped 1 2 3 4 5 6 7 0 token (9,F,1) (16,X,4) (7,Q,5) (15,K,3) (20,B,6) (6,Z,2) (5,A,0) Figure 9.11: The result of a call to remove min() on the heap originally portrayed in Figure 9.10. Identifier token continues to reference the same locator instance as in the original configuration, but the placement of that locator in the list has changed, as has the third field of the locator.
- 414. 392 Chapter 9.Priority Queues with its left child, (5,A), at index 1 of the list, then swapped with its right child, (9,F), at index 4 of the list. In the final configuration, the locator instances for all affected elements have been modified to reflect their new location. It is important to emphasize that the locator instances have not changed iden- tity. The user’s token reference, portrayed in Figures 9.10 and 9.11, continues to reference the same instance; we have simply changed the third field of that instance, and we have changed where that instance is referenced within the list sequence. With this new representation, providing the additional support for the adaptable priority queue ADT is rather straightforward. When a locator instance is sent as a parameter to update or remove, we may rely on the third field of that structure to designate where the element resides in the heap. With that knowledge, the update of a key may simply require an up-heap or down-heap bubbling step to reestablish the heap-order property. (The complete binary tree property remains intact.) To implement the removal of an arbitrary element, we move the element at the last position to the vacated location, and again perform an appropriate bubbling step to satisfy the heap-order property. Python Implementation Code Fragments 9.8 and 9.9 present a Python implementation of an adaptable pri- ority queue, as a subclass of the HeapPriorityQueue class from Section 9.3.4. Our modifications to the original class are relatively minor. We define a public Locator class that inherits from the nonpublic Item class and augments it with an addi- tional index field. We make it a public class because we will be using locators as return values and parameters; however, the public interface for the locator class does not include any other functionality for the user. To update locators during the flow of our heap operations, we rely on an inten- tional design decision that our original class uses a nonpublic swap method for all data movement. We override that utility to execute the additional step of updating the stored indices within the two swapped locator instances. We provide a new bubble utility that manages the reinstatement of the heap- order property when a key has changed at an arbitrary position within the heap, either due to a key update, or the blind replacement of a removed element with the item from the last position of the tree. The bubble utility determines whether to apply up-heap or down-heap bubbling, depending on whether the given location has a parent with a smaller key. (If an updated key coincidentally remains valid for its current location, we technically call downheap but no swaps result.) The public methods are provided in Code Fragment 9.9. The existing add method is overridden, both to make use of a Locator instance rather than an Item instance for storage of the new element, and to return the locator to the caller. The remainder of that method is similar to the original, with the management of locator indices enacted by the use of the new version of swap. There is no reason to over-
- 415. 9.5. Adaptable PriorityQueues 393 ride the remove min method because the only change in behavior for the adaptable priority queue is again provided by the overridden swap method. The update and remove methods provide the core new functionality for the adaptable priority queue. We perform robust checking of the validity of a locator that is sent by a caller (although in the interest of space, our displayed code does not do preliminary type-checking to ensure that the parameter is indeed a Locator instance). To ensure that a locator is associated with a current element of the given priority queue, we examine the index that is encapsulated within the locator object, and then verify that the entry of the list at that index is the very same locator. In conclusion, the adaptable priority queue provides the same asymptotic effi- ciency and space usage as the nonadaptive version, and provides logarithmic per- formance for the new locator-based update and remove methods. A summary of the performance is given in Table 9.4. 1 class AdaptableHeapPriorityQueue(HeapPriorityQueue): 2 ”””A locator-based priority queue implemented with a binary heap.””” 3 4 #------------------------------ nested Locator class ------------------------------ 5 class Locator(HeapPriorityQueue. Item): 6 ”””Token for locating an entry of the priority queue.””” 7 slots = _index # add index as additional field 8 9 def init (self, k, v, j): 10 super(). init (k,v) 11 self. index = j 12 13 #------------------------------ nonpublic behaviors ------------------------------ 14 # override swap to record new indices 15 def swap(self, i, j): 16 super(). swap(i,j) # perform the swap 17 self. data[i]. index = i # reset locator index (post-swap) 18 self. data[j]. index = j # reset locator index (post-swap) 19 20 def bubble(self, j): 21 if j 0 and self. data[j] self. data[self. parent(j)]: 22 self. upheap(j) 23 else: 24 self. downheap(j) Code Fragment 9.8: An implementation of an adaptable priority queue (continued in Code Fragment 9.9). This extends the HeapPriorityQueue class of Code Frag- ments 9.4 and 9.5
- 416. 394 Chapter 9.Priority Queues 25 def add(self, key, value): 26 ”””Add a key-value pair.””” 27 token = self.Locator(key, value, len(self. data)) # initiaize locator index 28 self. data.append(token) 29 self. upheap(len(self. data) − 1) 30 return token 31 32 def update(self, loc, newkey, newval): 33 ”””Update the key and value for the entry identified by Locator loc.””” 34 j = loc. index 35 if not (0 = j len(self) and self. data[j] is loc): 36 raise ValueError( Invalid locator ) 37 loc. key = newkey 38 loc. value = newval 39 self. bubble(j) 40 41 def remove(self, loc): 42 ”””Remove and return the (k,v) pair identified by Locator loc.””” 43 j = loc. index 44 if not (0 = j len(self) and self. data[j] is loc): 45 raise ValueError( Invalid locator ) 46 if j == len(self) − 1: # item at last position 47 self. data.pop( ) # just remove it 48 else: 49 self. swap(j, len(self)−1) # swap item to the last position 50 self. data.pop( ) # remove it from the list 51 self. bubble(j) # fix item displaced by the swap 52 return (loc. key, loc. value) Code Fragment 9.9: An implementation of an adaptable priority queue (continued from Code Fragment 9.8). Operation Running Time len(P), P.is empty(), P.min() O(1) P.add(k,v) O(logn)∗ P.update(loc, k, v) O(logn) P.remove(loc) O(logn)∗ P.remove min() O(logn)∗ ∗amortized with dynamic array Table 9.4: Running times of the methods of an adaptable priority queue, P, of size n, realized by means of our array-based heap representation. The space requirement is O(n).
- 417. 9.6. Exercises 395 9.6Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-9.1 How long would it take to remove the logn smallest elements from a heap that contains n entries, using the remove min operation? R-9.2 Suppose you label each position p of a binary tree T with a key equal to its preorder rank. Under what circumstances is T a heap? R-9.3 What does each remove min call return within the following sequence of priority queue ADT methods: add(5,A), add(4,B), add(7,F), add(1,D), remove min(), add(3,J), add(6,L), remove min(), remove min(), add(8,G), remove min(), add(2,H), remove min(), remove min()? R-9.4 An airport is developing a computer simulation of air-traffic control that handles events such as landings and takeoffs. Each event has a time stamp that denotes the time when the event will occur. The simulation program needs to efficiently perform the following two fundamental operations: • Insert an event with a given time stamp (that is, add a future event). • Extract the event with smallest time stamp (that is, determine the next event to process). Which data structure should be used for the above operations? Why? R-9.5 The min method for the UnsortedPriorityQueue class executes in O(n) time, as analyzed in Table 9.2. Give a simple modification to the class so that min runs in O(1) time. Explain any necessary modifications to other methods of the class. R-9.6 Can you adapt your solution to the previous problem to make remove min run in O(1) time for the UnsortedPriorityQueue class? Explain your an- swer. R-9.7 Illustrate the execution of the selection-sort algorithm on the following input sequence: (22,15,36,44,10,3,9,13,29,25). R-9.8 Illustrate the execution of the insertion-sort algorithm on the input se- quence of the previous problem. R-9.9 Give an example of a worst-case sequence with n elements for insertion- sort, and show that insertion-sort runs in Ω(n2) time on such a sequence. R-9.10 At which positions of a heap might the third smallest key be stored? R-9.11 At which positions of a heap might the largest key be stored?
- 418. 396 Chapter 9.Priority Queues R-9.12 Consider a situation in which a user has numeric keys and wishes to have a priority queue that is maximum-oriented. How could a standard (min- oriented) priority queue be used for such a purpose? R-9.13 Illustrate the execution of the in-place heap-sort algorithm on the follow- ing input sequence: (2,5,16,4,10,23,39,18,26,15). R-9.14 Let T be a complete binary tree such that position p stores an element with key f(p), where f(p) is the level number of p (see Section 8.3.2). Is tree T a heap? Why or why not? R-9.15 Explain why the description of down-heap bubbling does not consider the case in which position p has a right child but not a left child. R-9.16 Is there a heap H storing seven entries with distinct keys such that a pre- order traversal of H yields the entries of H in increasing or decreasing order by key? How about an inorder traversal? How about a postorder traversal? If so, give an example; if not, say why. R-9.17 Let H be a heap storing 15 entries using the array-based representation of a complete binary tree. What is the sequence of indices of the array that are visited in a preorder traversal of H? What about an inorder traversal of H? What about a postorder traversal of H? R-9.18 Show that the sum n ∑ i=1 logi, which appears in the analysis of heap-sort, is Ω(nlogn). R-9.19 Bill claims that a preorder traversal of a heap will list its keys in nonde- creasing order. Draw an example of a heap that proves him wrong. R-9.20 Hillary claims that a postorder traversal of a heap will list its keys in non- increasing order. Draw an example of a heap that proves her wrong. R-9.21 Show all the steps of the algorithm for removing the entry (16,X) from the heap of Figure 9.1, assuming the entry had been identified with a locator. R-9.22 Show all the steps of the algorithm for replacing key of entry (5,A) with 18 in the heap of Figure 9.1, assuming the entry had been identified with a locator. R-9.23 Draw an example of a heap whose keys are all the odd numbers from 1 to 59 (with no repeats), such that the insertion of an entry with key 32 would cause up-heap bubbling to proceed all the way up to a child of the root (replacing that child’s key with 32). R-9.24 Describe a sequence of n insertions in a heap that requires Ω(nlogn) time to process. R-9.25 Complete Figure 9.9 by showing all the steps of the in-place heap-sort algorithm. Show both the array and the associated heap at the end of each step.
- 419. 9.6. Exercises 397 Creativity C-9.26Show how to implement the stack ADT using only a priority queue and one additional integer instance variable. C-9.27 Show how to implement the FIFO queue ADT using only a priority queue and one additional integer instance variable. C-9.28 Professor Idle suggests the following solution to the previous problem. Whenever an item is inserted into the queue, it is assigned a key that is equal to the current size of the queue. Does such a strategy result in FIFO semantics? Prove that it is so or provide a counterexample. C-9.29 Reimplement the SortedPriorityQueue using a Python list. Make sure to maintain remove min’s O(1) performance. C-9.30 Give a nonrecursive implementation of the upheap method for the class HeapPriorityQueue. C-9.31 Give a nonrecursive implementation of the downheap method for the class HeapPriorityQueue. C-9.32 Assume that we are using a linked representation of a complete binary tree T, and an extra reference to the last node of that tree. Show how to update the reference to the last node after operations add or remove min in O(logn) time, where n is the current number of nodes of T. Be sure and handle all possible cases, as illustrated in Figure 9.12. C-9.33 When using a linked-tree representation for a heap, an alternative method for finding the last node during an insertion in a heap T is to store, in the last node and each leaf node of T, a reference to the leaf node immedi- ately to its right (wrapping to the first node in the next lower level for the rightmost leaf node). Show how to maintain such references in O(1) time per operation of the priority queue ADT assuming that T is implemented with a linked structure. (11,S) (2,B) (5,A) (4,C) (6,Z) (9,F) (15,K) (25,J) (12,H) (14,E) (16,X) (7,Q) (8,W) (10,L) (20,B) z w (5,A) (6,Z) (20,B) (9,F) (15,K) (25,J) (14,E) (16,X) (7,Q) (12,H) (4,C) w z (a) (b) Figure 9.12: Updating the last node in a complete binary tree after operation add or remove. Node w is the last node before operation add or after operation remove. Node z is the last node after operation add or before operation remove.
- 420. 398 Chapter 9.Priority Queues C-9.34 We can represent a path from the root to a given node of a binary tree by means of a binary string, where 0 means “go to the left child” and 1 means “go to the right child.” For example, the path from the root to the node storing (8,W) in the heap of Figure 9.12a is represented by “101.” Design an O(logn)-time algorithm for finding the last node of a complete binary tree with n nodes, based on the above representation. Show how this algorithm can be used in the implementation of a complete binary tree by means of a linked structure that does not keep a reference to the last node. C-9.35 Given a heap T and a key k, give an algorithm to compute all the entries in T having a key less than or equal to k. For example, given the heap of Figure 9.12a and query k = 7, the algorithm should report the entries with keys 2, 4, 5, 6, and 7 (but not necessarily in this order). Your algorithm should run in time proportional to the number of entries returned, and should not modify the heap C-9.36 Provide a justification of the time bounds in Table 9.4. C-9.37 Give an alternative analysis of bottom-up heap construction by showing the following summation is O(1), for any positive integer h: h ∑ i=1 i/2i . C-9.38 Suppose two binary trees, T1 and T2, hold entries satisfying the heap-order property (but not necessarily the complete binary tree property). Describe a method for combining T1 and T2 into a binary tree T, whose nodes hold the union of the entries in T1 and T2 and also satisfy the heap-order prop- erty. Your algorithm should run in time O(h1 + h2) where h1 and h2 are the respective heights of T1 and T2. C-9.39 Implement a heappushpop method for the HeapPriorityQueue class, with semantics akin to that described for the heapq module in Section 9.3.7. C-9.40 Implement a heapreplace method for the HeapPriorityQueue class, with semantics akin to that described for the heapq module in Section 9.3.7. C-9.41 Tamarindo Airlines wants to give a first-class upgrade coupon to their top logn frequent flyers, based on the number of miles accumulated, where n is the total number of the airlines’ frequent flyers. The algorithm they currently use, which runs in O(nlogn) time, sorts the flyers by the number of miles flown and then scans the sorted list to pick the top logn flyers. Describe an algorithm that identifies the top logn flyers in O(n) time. C-9.42 Explain how the k largest elements from an unordered collection of size n can be found in time O(n+klogn) using a maximum-oriented heap. C-9.43 Explain how the k largest elements from an unordered collection of size n can be found in time O(nlogk) using O(k) auxiliary space.
- 421. 9.6. Exercises 399 C-9.44Given a class, PriorityQueue, that implements the minimum-oriented pri- ority queue ADT, provide an implementation of a MaxPriorityQueue class that adapts to provide a maximum-oriented abstraction with methods add, max, and remove max. Your implementation should not make any as- sumption about the internal workings of the original PriorityQueue class, nor the type of keys that might be used. C-9.45 Write a key function for nonnegative integers that determines order based on the number of 1’s in each integer’s binary expansion. C-9.46 Give an alternative implementation of the pq sort function, from Code Fragment 9.7, that accepts a key function as an optional parameter. C-9.47 Describe an in-place version of the selection-sort algorithm for an array that uses only O(1) space for instance variables in addition to the array. C-9.48 Assuming the input to the sorting problem is given in an array A, describe how to implement the insertion-sort algorithm using only the array A and at most a constant number of additional variables. C-9.49 Give an alternate description of the in-place heap-sort algorithm using the standard minimum-oriented priority queue (instead of a maximum- oriented one). C-9.50 An online computer system for trading stocks needs to process orders of the form “buy 100 shares at $x each” or “sell 100 shares at $y each.” A buy order for $x can only be processed if there is an existing sell order with price $y such that y ≤ x. Likewise, a sell order for $y can only be processed if there is an existing buy order with price $x such that y ≤ x. If a buy or sell order is entered but cannot be processed, it must wait for a future order that allows it to be processed. Describe a scheme that allows buy and sell orders to be entered in O(logn) time, independent of whether or not they can be immediately processed. C-9.51 Extend a solution to the previous problem so that users are allowed to update the prices for their buy or sell orders that have yet to be processed. C-9.52 A group of children want to play a game, called Unmonopoly, where in each turn the player with the most money must give half of his/her money to the player with the least amount of money. What data structure(s) should be used to play this game efficiently? Why? Projects P-9.53 Implement the in-place heap-sort algorithm. Experimentally compare its running time with that of the standard heap-sort that is not in-place. P-9.54 Use the approach of either Exercise C-9.42 or C-9.43 to reimplement the top method of the FavoritesListMTF class from Section 7.6.2. Make sure that results are generated from largest to smallest.
- 422. 400 Chapter 9.Priority Queues P-9.55 Write a program that can process a sequence of stock buy and sell orders as described in Exercise C-9.50. P-9.56 Let S be a set of n points in the plane with distinct integer x- and y- coordinates. Let T be a complete binary tree storing the points from S at its external nodes, such that the points are ordered left to right by in- creasing x-coordinates. For each node v in T, let S(v) denote the subset of S consisting of points stored in the subtree rooted at v. For the root r of T, define top(r) to be the point in S = S(r) with maximum y-coordinate. For every other node v, define top(r) to be the point in S with highest y- coordinate in S(v) that is not also the highest y-coordinate in S(u), where u is the parent of v in T (if such a point exists). Such labeling turns T into a priority search tree. Describe a linear-time algorithm for turning T into a priority search tree. Implement this approach. P-9.57 One of the main applications of priority queues is in operating systems— for scheduling jobs on a CPU. In this project you are to build a program that schedules simulated CPU jobs. Your program should run in a loop, each iteration of which corresponds to a time slice for the CPU. Each job is assigned a priority, which is an integer between −20 (highest priority) and 19 (lowest priority), inclusive. From among all jobs waiting to be pro- cessed in a time slice, the CPU must work on a job with highest priority. In this simulation, each job will also come with a length value, which is an integer between 1 and 100, inclusive, indicating the number of time slices that are needed to process this job. For simplicity, you may assume jobs cannot be interrupted—once it is scheduled on the CPU, a job runs for a number of time slices equal to its length. Your simulator must output the name of the job running on the CPU in each time slice and must process a sequence of commands, one per time slice, each of which is of the form “add job name with length n and priority p” or “no new job this slice”. P-9.58 Develop a Python implementation of an adaptable priority queue that is based on an unsorted list and supports location-aware entries. Chapter Notes Knuth’s book on sorting and searching [65] describes the motivation and history for the selection-sort, insertion-sort, and heap-sort algorithms. The heap-sort algorithm is due to Williams [103], and the linear-time heap construction algorithm is due to Floyd [39]. Additional algorithms and analyses for heaps and heap-sort variations can be found in papers by Bentley [15], Carlsson [24], Gonnet and Munro [45], McDiarmid and Reed [74], and Schaffer and Sedgewick [88].
- 423. Chapter 10 Maps, HashTables, and Skip Lists Contents 10.1 Maps and Dictionaries . . . . . . . . . . . . . . . . . . . . . 402 10.1.1 The Map ADT . . . . . . . . . . . . . . . . . . . . . . . 403 10.1.2 Application: Counting Word Frequencies . . . . . . . . . . 405 10.1.3 Python’s MutableMapping Abstract Base Class . . . . . . 406 10.1.4 Our MapBase Class . . . . . . . . . . . . . . . . . . . . . 407 10.1.5 Simple Unsorted Map Implementation . . . . . . . . . . . 408 10.2 Hash Tables . . . . . . . . . . . . . . . . . . . . . . . . . . 410 10.2.1 Hash Functions . . . . . . . . . . . . . . . . . . . . . . . 411 10.2.2 Collision-Handling Schemes . . . . . . . . . . . . . . . . . 417 10.2.3 Load Factors, Rehashing, and Efficiency . . . . . . . . . . 420 10.2.4 Python Hash Table Implementation . . . . . . . . . . . . 422 10.3 Sorted Maps . . . . . . . . . . . . . . . . . . . . . . . . . . 427 10.3.1 Sorted Search Tables . . . . . . . . . . . . . . . . . . . . 428 10.3.2 Two Applications of Sorted Maps . . . . . . . . . . . . . 434 10.4 Skip Lists . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437 10.4.1 Search and Update Operations in a Skip List . . . . . . . 439 10.4.2 Probabilistic Analysis of Skip Lists . . . . . . . . . . . . 443 10.5 Sets, Multisets, and Multimaps . . . . . . . . . . . . . . . 446 10.5.1 The Set ADT . . . . . . . . . . . . . . . . . . . . . . . . 446 10.5.2 Python’s MutableSet Abstract Base Class . . . . . . . . . 448 10.5.3 Implementing Sets, Multisets, and Multimaps . . . . . . . 450 10.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452
- 424. 402 Chapter 10.Maps, Hash Tables, and Skip Lists 10.1 Maps and Dictionaries Python’s dict class is arguably the most significant data structure in the language. It represents an abstraction known as a dictionary in which unique keys are mapped to associated values. Because of the relationship they express between keys and values, dictionaries are commonly known as associative arrays or maps. In this book, we use the term dictionary when specifically discussing Python’s dict class, and the term map when discussing the more general notion of the abstract data type. As a simple example, Figure 10.1 illustrates a map from the names of countries to their associated units of currency. Rupee Turkey Spain China United States India Greece Lira Euro Yuan Dollar Figure 10.1: A map from countries (the keys) to their units of currency (the values). We note that the keys (the country names) are assumed to be unique, but the values (the currency units) are not necessarily unique. For example, we note that Spain and Greece both use the euro for currency. Maps use an array-like syntax for in- dexing, such as currency[ Greece ] to access a value associated with a given key or currency[ Greece ] = Drachma to remap it to a new value. Unlike a stan- dard array, indices for a map need not be consecutive nor even numeric. Common applications of maps include the following. • A university’s information system relies on some form of a student ID as a key that is mapped to that student’s associated record (such as the student’s name, address, and course grades) serving as the value. • The domain-name system (DNS) maps a host name, such as www.wiley.com, to an Internet-Protocol (IP) address, such as 208.215.179.146. • A social media site typically relies on a (nonnumeric) username as a key that can be efficiently mapped to a particular user’s associated information. • A computer graphics system may map a color name, such as turquoise , to the triple of numbers that describes the color’s RGB (red-green-blue) rep- resentation, such as (64,224,208). • Python uses a dictionary to represent each namespace, mapping an identifying string, such as pi , to an associated object, such as 3.14159. In this chapter and the next we demonstrate that a map may be implemented so that a search for a key, and its associated value, can be performed very efficiently, thereby supporting fast lookup in such applications.
- 425. 10.1. Maps andDictionaries 403 10.1.1 The Map ADT In this section, we introduce the map ADT, and define its behaviors to be consistent with those of Python’s built-in dict class. We begin by listing what we consider the most significant five behaviors of a map M as follows: M[k]: Return the value v associated with key k in map M, if one exists; otherwise raise a KeyError. In Python, this is implemented with the special method getitem . M[k] = v: Associate value v with key k in map M, replacing the ex- isting value if the map already contains an item with key equal to k. In Python, this is implemented with the special method setitem . del M[k]: Remove from map M the item with key equal to k; if M has no such item, then raise a KeyError. In Python, this is implemented with the special method delitem . len(M): Return the number of items in map M. In Python, this is implemented with the special method len . iter(M): The default iteration for a map generates a sequence of keys in the map. In Python, this is implemented with the special method iter , and it allows loops of the form, for k in M. We have highlighted the above five behaviors because they demonstrate the core functionality of a map—namely, the ability to query, add, modify, or delete a key- value pair, and the ability to report all such pairs. For additional convenience, map M should also support the following behaviors: k in M: Return True if the map contains an item with key k. In Python, this is implemented with the special contains method. M.get(k, d=None): Return M[k] if key k exists in the map; otherwise return default value d. This provides a form to query M[k] with- out risk of a KeyError. M.setdefault(k, d): If key k exists in the map, simply return M[k]; if key k does not exist, set M[k] = d and return that value. M.pop(k, d=None): Remove the item associated with key k from the map and return its associated value v. If key k is not in the map, return default value d (or raise KeyError if parameter d is None).
- 426. 404 Chapter 10.Maps, Hash Tables, and Skip Lists M.popitem(): Remove an arbitrary key-value pair from the map, and re- turn a (k,v) tuple representing the removed pair. If map is empty, raise a KeyError. M.clear(): Remove all key-value pairs from the map. M.keys(): Return a set-like view of all keys of M. M.values(): Return a set-like view of all values of M. M.items(): Return a set-like view of (k,v) tuples for all entries of M. M.update(M2): Assign M[k] = v for every (k,v) pair in map M2. M == M2: Return True if maps M and M2 have identical key-value associations. M != M2: Return True if maps M and M2 do not have identical key- value associations. Example 10.1: In the following, we show the effect of a series of operations on an initially empty map storing items with integer keys and single-character values. We use the literal syntax for Python’s dict class to describe the map contents. Operation Return Value Map len(M) 0 { } M[ K ] = 2 – { K : 2} M[ B ] = 4 – { K : 2, B : 4} M[ U ] = 2 – { K : 2, B : 4, U : 2} M[ V ] = 8 – { K : 2, B : 4, U : 2, V : 8} M[ K ] = 9 – { K : 9, B : 4, U : 2, V : 8} M[ B ] 4 { K : 9, B : 4, U : 2, V : 8} M[ X ] KeyError { K : 9, B : 4, U : 2, V : 8} M.get( F ) None { K : 9, B : 4, U : 2, V : 8} M.get( F , 5) 5 { K : 9, B : 4, U : 2, V : 8} M.get( K , 5) 9 { K : 9, B : 4, U : 2, V : 8} len(M) 4 { K : 9, B : 4, U : 2, V : 8} del M[ V ] – { K : 9, B : 4, U : 2} M.pop( K ) 9 { B : 4, U : 2} M.keys() B , U { B : 4, U : 2} M.values() 4, 2 { B : 4, U : 2} M.items() ( B , 4), ( U , 2) { B : 4, U : 2} M.setdefault( B , 1) 4 { B : 4, U : 2} M.setdefault( A , 1) 1 { A : 1, B : 4, U : 2} M.popitem() ( B , 4) { A : 1, U : 2}
- 427. 10.1. Maps andDictionaries 405 10.1.2 Application: Counting Word Frequencies As a case study for using a map, consider the problem of counting the number of occurrences of words in a document. This is a standard task when performing a statistical analysis of a document, for example, when categorizing an email or news article. A map is an ideal data structure to use here, for we can use words as keys and word counts as values. We show such an application in Code Fragment 10.1. We break apart the original document using a combination of file and string methods that results in a loop over a lowercased version of all whitespace separated pieces of the document. We omit all nonalphabetic characters so that parentheses, apostrophes, and other such punctuation are not considered part of a word. In terms of map operations, we begin with an empty Python dictionary named freq. During the first phase of the algorithm, we execute the command freq[word] = 1 + freq.get(word, 0) for each word occurrence. We use the get method on the right-hand side because the current word might not exist in the dictionary; the default value of 0 is appropriate in that case. During the second phase of the algorithm, after the full document has been pro- cessed, we examine the contents of the frequency map, looping over freq.items() to determine which word has the most occurrences. 1 freq = { } 2 for piece in open(filename).read().lower().split(): 3 # only consider alphabetic characters within this piece 4 word = .join(c for c in piece if c.isalpha()) 5 if word: # require at least one alphabetic character 6 freq[word] = 1 + freq.get(word, 0) 7 8 max word = 9 max count = 0 10 for (w,c) in freq.items(): # (key, value) tuples represent (word, count) 11 if c max count: 12 max word = w 13 max count = c 14 print( The most frequent word is , max word) 15 print( Its number of occurrences is , max count) Code Fragment 10.1: A program for counting word frequencies in a document, and reporting the most frequent word. We use Python’s dict class for the map. We convert the input to lowercase and ignore any nonalphabetic characters.
- 428. 406 Chapter 10.Maps, Hash Tables, and Skip Lists 10.1.3 Python’s MutableMapping Abstract Base Class Section 2.4.3 provides an introduction to the concept of an abstract base class and the role of such classes in Python’s collections module. Methods that are de- clared to be abstract in such a base class must be implemented by concrete sub- classes. However, an abstract base class may provide concrete implementation of other methods that depend upon use of the presumed abstract methods. (This is an example of the template method design pattern.) The collections module provides two abstract base classes that are relevant to our current discussion: the Mapping and MutableMapping classes. The Mapping class includes all nonmutating methods supported by Python’s dict class, while the MutableMapping class extends that to include the mutating methods. What we define as the map ADT in Section 10.1.1 is akin to the MutableMapping abstract base class in Python’s collections module. The significance of these abstract base classes is that they provide a framework to assist in creating a user-defined map class. In particular, the MutableMapping class provides concrete implementations for all behaviors other than the first five outlined in Section 10.1.1: getitem , setitem , delitem , len , and iter . As we implement the map abstraction with various data structures, as long as we provide the five core behaviors, we can inherit all other derived behav- iors by simply declaring MutableMapping as a parent class. To better understand the MutableMapping class, we provide a few examples of how concrete behaviors can be derived from the five core abstractions. For example, the contains method, supporting the syntax k in M, could be implemented by making a guarded attempt to retrieve self[k] to determine if the key exists. def contains (self, k): try: self[k] # access via getitem (ignore result) return True except KeyError: return False # attempt failed A similar approach might be used to provide the logic of the setdefault method. def setdefault(self, k, d): try: return self[k] # if getitem succeeds, return value except KeyError: # otherwise: self[k] = d # set default value with setitem return d # and return that newly assigned value We leave as exercises the implementations of the remaining concrete methods of the MutableMapping class.
- 429. 10.1. Maps andDictionaries 407 10.1.4 Our MapBase Class We will be providing many different implementations of the map ADT, in the re- mainder of this chapter and next, using a variety of data structures demonstrating a trade-off of advantages and disadvantages. Figure 10.2 provides a preview of those classes. The MutableMapping abstract base class, from Python’s collections module and discussed in the preceding pages, is a valuable tool when implementing a map. However, in the interest of greater code reuse, we define our own MapBase class, which is itself a subclass of the MutableMapping class. Our MapBase class pro- vides additional support for the composition design pattern. This is a technique we introduced when implementing a priority queue (see Section 9.2.1) in order to group a key-value pair as a single instance for internal use. More formally, our MapBase class is defined in Code Fragment 10.2, extend- ing the existing MutableMapping abstract base class so that we inherit the many useful concrete methods that class provides. We then define a nonpublic nested Item class, whose instances are able to store both a key and value. This nested class is reasonably similar in design to the Item class that was defined within our PriorityQueueBase class in Section 9.2.1, except that for a map we provide sup- port for both equality tests and comparisons, both of which rely on the item’s key. The notion of equality is necessary for all of our map implementations, as a way to determine whether a key given as a parameter is equivalent to one that is already stored in the map. The notion of comparisons between keys, using the operator, will become relevant when we later introduce a sorted map ADT (Section 10.3). ProbeHashMap MutableMapping (Section 10.1.4) MapBase (Section 10.3.1) SortedTableMap (Chapter 11) TreeMap (Section 10.2.4) HashMapBase (Section 10.2.4) ChainHashMap (Section 10.2.4) (collections module) (Section 10.1.5) UnsortedTableMap (additional subclasses) Figure 10.2: Our hierarchy of map types (with references to where they are defined).
- 430. 408 Chapter 10.Maps, Hash Tables, and Skip Lists 1 class MapBase(MutableMapping): 2 ”””Our own abstract base class that includes a nonpublic Item class.””” 3 4 #------------------------------- nested Item class ------------------------------- 5 class Item: 6 ”””Lightweight composite to store key-value pairs as map items.””” 7 slots = _key , _value 8 9 def init (self, k, v): 10 self. key = k 11 self. value = v 12 13 def eq (self, other): 14 return self. key == other. key # compare items based on their keys 15 16 def ne (self, other): 17 return not (self == other) # opposite of eq 18 19 def lt (self, other): 20 return self. key other. key # compare items based on their keys Code Fragment 10.2: Extending the MutableMapping abstract base class to provide a nonpublic Item class for use in our various map implementations. 10.1.5 Simple Unsorted Map Implementation We demonstrate the use of the MapBase class with a very simple concrete imple- mentation of the map ADT. Code Fragment 10.3 presents an UnsortedTableMap class that relies on storing key-value pairs in arbitrary order within a Python list. An empty table is initialized as self. table within the constructor for our map. When a new key is entered into the map, via line 22 of the setitem method, we create a new instance of the nested Item class, which is inherited from our MapBase class. This list-based map implementation is simple, but it is not particularly efficient. Each of the fundamental methods, getitem , setitem , and delitem , relies on a for loop to scan the underlying list of items in search of a matching key. In a best-case scenario, such a match may be found near the beginning of the list, in which case the loop terminates; in the worst case, the entire list will be examined. Therefore, each of these methods runs in O(n) time on a map with n items.
- 431. 10.1. Maps andDictionaries 409 1 class UnsortedTableMap(MapBase): 2 ”””Map implementation using an unordered list.””” 3 4 def init (self): 5 ”””Create an empty map.””” 6 self. table = [ ] # list of Item’s 7 8 def getitem (self, k): 9 ”””Return value associated with key k (raise KeyError if not found).””” 10 for item in self. table: 11 if k == item. key: 12 return item. value 13 raise KeyError( Key Error: + repr(k)) 14 15 def setitem (self, k, v): 16 ”””Assign value v to key k, overwriting existing value if present.””” 17 for item in self. table: 18 if k == item. key: # Found a match: 19 item. value = v # reassign value 20 return # and quit 21 # did not find match for key 22 self. table.append(self. Item(k,v)) 23 24 def delitem (self, k): 25 ”””Remove item associated with key k (raise KeyError if not found).””” 26 for j in range(len(self. table)): 27 if k == self. table[j]. key: # Found a match: 28 self. table.pop(j) # remove item 29 return # and quit 30 raise KeyError( Key Error: + repr(k)) 31 32 def len (self): 33 ”””Return number of items in the map.””” 34 return len(self. table) 35 36 def iter (self): 37 ”””Generate iteration of the map s keys.””” 38 for item in self. table: 39 yield item. key # yield the KEY Code Fragment 10.3: An implementation of a map using a Python list as an unsorted table. Parent class MapBase is given in Code Fragment 10.2.
- 432. 410 Chapter 10.Maps, Hash Tables, and Skip Lists 10.2 Hash Tables In this section, we introduce one of the most practical data structures for imple- menting a map, and the one that is used by Python’s own implementation of the dict class. This structure is known as a hash table. Intuitively, a map M supports the abstraction of using keys as indices with a syntax such as M[k]. As a mental warm-up, consider a restricted setting in which a map with n items uses keys that are known to be integers in a range from 0 to N −1 for some N ≥ n. In this case, we can represent the map using a lookup table of length N, as diagrammed in Figure 10.3. 0 1 2 3 4 5 6 7 8 9 10 D Z C Q Figure 10.3: A lookup table with length 11 for a map containing items (1,D), (3,Z), (6,C), and (7,Q). In this representation, we store the value associated with key k at index k of the table (presuming that we have a distinct way to represent an empty slot). Basic map operations of getitem , setitem , and delitem can be implemented in O(1) worst-case time. There are two challenges in extending this framework to the more general set- ting of a map. First, we may not wish to devote an array of length N if it is the case that N n. Second, we do not in general require that a map’s keys be integers. The novel concept for a hash table is the use of a hash function to map general keys to corresponding indices in a table. Ideally, keys will be well distributed in the range from 0 to N −1 by a hash function, but in practice there may be two or more distinct keys that get mapped to the same index. As a result, we will conceptualize our table as a bucket array, as shown in Figure 10.4, in which each bucket may manage a collection of items that are sent to a specific index by the hash function. (To save space, an empty bucket may be replaced by None.) 0 1 2 3 4 5 6 7 8 9 10 (1,D) (25,C) (3,F) (14,Z) (39,C) (6,A) (7,Q) Figure 10.4: A bucket array of capacity 11 with items (1,D), (25,C), (3,F), (14,Z), (6,A), (39,C), and (7,Q), using a simple hash function.
- 433. 10.2. Hash Tables411 10.2.1 Hash Functions The goal of a hash function, h, is to map each key k to an integer in the range [0,N − 1], where N is the capacity of the bucket array for a hash table. Equipped with such a hash function, h, the main idea of this approach is to use the hash function value, h(k), as an index into our bucket array, A, instead of the key k (which may not be appropriate for direct use as an index). That is, we store the item (k,v) in the bucket A[h(k)]. If there are two or more keys with the same hash value, then two different items will be mapped to the same bucket in A. In this case, we say that a collision has occurred. To be sure, there are ways of dealing with collisions, which we will discuss later, but the best strategy is to try to avoid them in the first place. We say that a hash function is “good” if it maps the keys in our map so as to sufficiently minimize collisions. For practical reasons, we also would like a hash function to be fast and easy to compute. It is common to view the evaluation of a hash function, h(k), as consisting of two portions—a hash code that maps a key k to an integer, and a compression function that maps the hash code to an integer within a range of indices, [0,N −1], for a bucket array. (See Figure 10.5.) -1 hash code 1 2 0 -2 ... ... compression function 1 2 0 N-1 ... Arbitrary Objects Figure 10.5: Two parts of a hash function: a hash code and a compression function. The advantage of separating the hash function into two such components is that the hash code portion of that computation is independent of a specific hash table size. This allows the development of a general hash code for each object that can be used for a hash table of any size; only the compression function depends upon the table size. This is particularly convenient, because the underlying bucket array for a hash table may be dynamically resized, depending on the number of items currently stored in the map. (See Section 10.2.3.)
- 434. 412 Chapter 10.Maps, Hash Tables, and Skip Lists Hash Codes The first action that a hash function performs is to take an arbitrary key k in our map and compute an integer that is called the hash code for k; this integer need not be in the range [0,N −1], and may even be negative. We desire that the set of hash codes assigned to our keys should avoid collisions as much as possible. For if the hash codes of our keys cause collisions, then there is no hope for our compression function to avoid them. In this subsection, we begin by discussing the theory of hash codes. Following that, we discuss practical implementations of hash codes in Python. Treating the Bit Representation as an Integer To begin, we note that, for any data type X that is represented using at most as many bits as our integer hash codes, we can simply take as a hash code for X an integer interpretation of its bits. For example, the hash code for key 314 could simply be 314. The hash code for a floating-point number such as 3.14 could be based upon an interpretation of the bits of the floating-point representation as an integer. For a type whose bit representation is longer than a desired hash code, the above scheme is not immediately applicable. For example, Python relies on 32-bit hash codes. If a floating-point number uses a 64-bit representation, its bits cannot be viewed directly as a hash code. One possibility is to use only the high-order 32 bits (or the low-order 32 bits). This hash code, of course, ignores half of the information present in the original key, and if many of the keys in our map only differ in these bits, then they will collide using this simple hash code. A better approach is to combine in some way the high-order and low-order por- tions of a 64-bit key to form a 32-bit hash code, which takes all the original bits into consideration. A simple implementation is to add the two components as 32- bit numbers (ignoring overflow), or to take the exclusive-or of the two components. These approaches of combining components can be extended to any object x whose binary representation can be viewed as an n-tuple (x0,x1,...,xn−1) of 32-bit inte- gers, for example, by forming a hash code for x as ∑n−1 i=0 xi, or as x0 ⊕x1 ⊕···⊕xn−1, where the ⊕ symbol represents the bitwise exclusive-or operation (which is ˆ in Python). Polynomial Hash Codes The summation and exclusive-or hash codes, described above, are not good choices for character strings or other variable-length objects that can be viewed as tuples of the form (x0,x1,...,xn−1), where the order of the xi’s is significant. For example, consider a 16-bit hash code for a character string s that sums the Unicode values of the characters in s. This hash code unfortunately produces lots of unwanted
- 435. 10.2. Hash Tables413 collisions for common groups of strings. In particular, temp01 and temp10 collide using this function, as do stop, tops, pots, and spot. A better hash code should somehow take into consideration the positions of the xi’s. An alternative hash code, which does exactly this, is to choose a nonzero constant, a = 1, and use as a hash code the value x0an−1 +x1an−2 +···+xn−2a+xn−1. Mathematically speaking, this is simply a polynomial in a that takes the compo- nents (x0,x1,...,xn−1) of an object x as its coefficients. This hash code is therefore called a polynomial hash code. By Horner’s rule (see Exercise C-3.50), this poly- nomial can be computed as xn−1 +a(xn−2 +a(xn−3 +···+a(x2 +a(x1 +ax0))···)). Intuitively, a polynomial hash code uses multiplication by different powers as a way to spread out the influence of each component across the resulting hash code. Of course, on a typical computer, evaluating a polynomial will be done using the finite bit representation for a hash code; hence, the value will periodically over- flow the bits used for an integer. Since we are more interested in a good spread of the object x with respect to other keys, we simply ignore such overflows. Still, we should be mindful that such overflows are occurring and choose the constant a so that it has some nonzero, low-order bits, which will serve to preserve some of the information content even as we are in an overflow situation. We have done some experimental studies that suggest that 33, 37, 39, and 41 are particularly good choices for a when working with character strings that are English words. In fact, in a list of over 50,000 English words formed as the union of the word lists provided in two variants of Unix, we found that taking a to be 33, 37, 39, or 41 produced less than 7 collisions in each case! Cyclic-Shift Hash Codes A variant of the polynomial hash code replaces multiplication by a with a cyclic shift of a partial sum by a certain number of bits. For example, a 5-bit cyclic shift of the 32-bit value 00111101100101101010100010101000 is achieved by taking the leftmost five bits and placing those on the rightmost side of the representation, resulting in 10110010110101010001010100000111. While this operation has little natural meaning in terms of arithmetic, it accomplishes the goal of varying the bits of the calculation. In Python, a cyclic shift of bits can be accomplished through careful use of the bitwise operators and , taking care to truncate results to 32-bit integers.
- 436. 414 Chapter 10.Maps, Hash Tables, and Skip Lists An implementation of a cyclic-shift hash code computation for a character string in Python appears as follows: def hash code(s): mask = (1 32) − 1 # limit to 32-bit integers h = 0 for character in s: h = (h 5 mask) | (h 27) # 5-bit cyclic shift of running sum h += ord(character) # add in value of next character return h As with the traditional polynomial hash code, fine-tuning is required when using a cyclic-shift hash code, as we must wisely choose the amount to shift by for each new character. Our choice of a 5-bit shift is justified by experiments run on a list of just over 230,000 English words, comparing the number of collisions for various shift amounts (see Table 10.1). Collisions Shift Total Max 0 234735 623 1 165076 43 2 38471 13 3 7174 5 4 1379 3 5 190 3 6 502 2 7 560 2 8 5546 4 9 393 3 10 5194 5 11 11559 5 12 822 2 13 900 4 14 2001 4 15 19251 8 16 211781 37 Table 10.1: Comparison of collision behavior for the cyclic-shift hash code as ap- plied to a list of 230,000 English words. The “Total” column records the total num- ber of words that collide with at least one other, and the “Max” column records the maximum number of words colliding at any one hash code. Note that with a cyclic shift of 0, this hash code reverts to the one that simply sums all the characters.
- 437. 10.2. Hash Tables415 Hash Codes in Python The standard mechanism for computing hash codes in Python is a built-in function with signature hash(x) that returns an integer value that serves as the hash code for object x. However, only immutable data types are deemed hashable in Python. This restriction is meant to ensure that a particular object’s hash code remains constant during that object’s lifespan. This is an important property for an object’s use as a key in a hash table. A problem could occur if a key were inserted into the hash table, yet a later search were performed for that key based on a different hash code than that which it had when inserted; the wrong bucket would be searched. Among Python’s built-in data types, the immutable int, float, str, tuple, and frozenset classes produce robust hash codes, via the hash function, using tech- niques similar to those discussed earlier in this section. Hash codes for character strings are well crafted based on a technique similar to polynomial hash codes, except using exclusive-or computations rather than additions. If we repeat the ex- periment described in Table 10.1 using Python’s built-in hash codes, we find that only 8 strings out of the set of more than 230,000 collide with another. Hash codes for tuples are computed with a similar technique based upon a combination of the hash codes of the individual elements of the tuple. When hashing a frozenset, the order of the elements should be irrelevant, and so a natural option is to compute the exclusive-or of the individual hash codes without any shifting. If hash(x) is called for an instance x of a mutable type, such as a list, a TypeError is raised. Instances of user-defined classes are treated as unhashable by default, with a TypeError raised by the hash function. However, a function that computes hash codes can be implemented in the form of a special method named hash within a class. The returned hash code should reflect the immutable attributes of an in- stance. It is common to return a hash code that is itself based on the computed hash of the combination of such attributes. For example, a Color class that maintains three numeric red, green, and blue components might implement the method as: def hash (self): return hash( (self. red, self. green, self. blue) ) # hash combined tuple An important rule to obey is that if a class defines equivalence through eq , then any implementation of hash must be consistent, in that if x == y, then hash(x) == hash(y). This is important because if two instances are considered to be equivalent and one is used as a key in a hash table, a search for the second instance should result in the discovery of the first. It is therefore important that the hash code for the second match the hash code for the first, so that the proper bucket is examined. This rule extends to any well-defined comparisons between objects of different classes. For example, since Python treats the expression 5 == 5.0 as true, it ensures that hash(5) and hash(5.0) are the same.
- 438. 416 Chapter 10.Maps, Hash Tables, and Skip Lists Compression Functions The hash code for a key k will typically not be suitable for immediate use with a bucket array, because the integer hash code may be negative or may exceed the ca- pacity of the bucket array. Thus, once we have determined an integer hash code for a key object k, there is still the issue of mapping that integer into the range [0,N −1]. This computation, known as a compression function, is the second action per- formed as part of an overall hash function. A good compression function is one that minimizes the number of collisions for a given set of distinct hash codes. The Division Method A simple compression function is the division method, which maps an integer i to i mod N, where N, the size of the bucket array, is a fixed positive integer. Additionally, if we take N to be a prime number, then this compression function helps “spread out” the distribution of hashed values. Indeed, if N is not prime, then there is greater risk that patterns in the distribution of hash codes will be repeated in the distribution of hash values, thereby causing collisions. For example, if we insert keys with hash codes {200,205,210,215,220,... ,600} into a bucket array of size 100, then each hash code will collide with three others. But if we use a bucket array of size 101, then there will be no collisions. If a hash function is chosen well, it should ensure that the probability of two different keys getting hashed to the same bucket is 1/N. Choosing N to be a prime number is not always enough, however, for if there is a repeated pattern of hash codes of the form pN + q for several different p’s, then there will still be collisions. The MAD Method A more sophisticated compression function, which helps eliminate repeated pat- terns in a set of integer keys, is the Multiply-Add-and-Divide (or “MAD”) method. This method maps an integer i to [(ai+b) mod p] mod N, where N is the size of the bucket array, p is a prime number larger than N, and a and b are integers chosen at random from the interval [0, p− 1], with a 0. This compression function is chosen in order to eliminate repeated patterns in the set of hash codes and get us closer to having a “good” hash function, that is, one such that the probability any two different keys collide is 1/N. This good behavior would be the same as we would have if these keys were “thrown” into A uniformly at random.
- 439. 10.2. Hash Tables417 10.2.2 Collision-Handling Schemes The main idea of a hash table is to take a bucket array, A, and a hash function, h, and use them to implement a map by storing each item (k,v) in the “bucket” A[h(k)]. This simple idea is challenged, however, when we have two distinct keys, k1 and k2, such that h(k1) = h(k2). The existence of such collisions prevents us from simply inserting a new item (k,v) directly into the bucket A[h(k)]. It also complicates our procedure for performing insertion, search, and deletion operations. Separate Chaining A simple and efficient way for dealing with collisions is to have each bucket A[ j] store its own secondary container, holding items (k,v) such that h(k) = j. A natural choice for the secondary container is a small map instance implemented using a list, as described in Section 10.1.5. This collision resolution rule is known as separate chaining, and is illustrated in Figure 10.6. A 1 2 3 4 5 6 7 8 9 10 0 11 12 12 38 25 90 54 28 41 36 18 10 Figure 10.6: A hash table of size 13, storing 10 items with integer keys, with colli- sions resolved by separate chaining. The compression function is h(k) = k mod 13. For simplicity, we do not show the values associated with the keys. In the worst case, operations on an individual bucket take time proportional to the size of the bucket. Assuming we use a good hash function to index the n items of our map in a bucket array of capacity N, the expected size of a bucket is n/N. Therefore, if given a good hash function, the core map operations run in O( n/N). The ratio λ = n/N, called the load factor of the hash table, should be bounded by a small constant, preferably below 1. As long as λ is O(1), the core operations on the hash table run in O(1) expected time.
- 440. 418 Chapter 10.Maps, Hash Tables, and Skip Lists Open Addressing The separate chaining rule has many nice properties, such as affording simple im- plementations of map operations, but it nevertheless has one slight disadvantage: It requires the use of an auxiliary data structure—a list—to hold items with collid- ing keys. If space is at a premium (for example, if we are writing a program for a small handheld device), then we can use the alternative approach of always storing each item directly in a table slot. This approach saves space because no auxiliary structures are employed, but it requires a bit more complexity to deal with colli- sions. There are several variants of this approach, collectively referred to as open addressing schemes, which we discuss next. Open addressing requires that the load factor is always at most 1 and that items are stored directly in the cells of the bucket array itself. Linear Probing and Its Variants A simple method for collision handling with open addressing is linear probing. With this approach, if we try to insert an item (k,v) into a bucket A[ j] that is already occupied, where j = h(k), then we next try A[(j +1) mod N]. If A[(j +1) mod N] is also occupied, then we try A[(j + 2) mod N], and so on, until we find an empty bucket that can accept the new item. Once this bucket is located, we simply in- sert the item there. Of course, this collision resolution strategy requires that we change the implementation when searching for an existing key—the first step of all getitem , setitem , or delitem operations. In particular, to attempt to locate an item with key equal to k, we must examine consecutive slots, starting from A[h(k)], until we either find an item with that key or we find an empty bucket. (See Figure 10.7.) The name “linear probing” comes from the fact that accessing a cell of the bucket array can be viewed as a “probe.” 26 1 2 3 4 5 6 7 8 9 10 0 New element with key = 15 to be inserted Must probe 4 times before finding empty slot 5 37 16 21 13 Figure 10.7: Insertion into a hash table with integer keys using linear probing. The hash function is h(k) = k mod 11. Values associated with keys are not shown.
- 441. 10.2. Hash Tables419 To implement a deletion, we cannot simply remove a found item from its slot in the array. For example, after the insertion of key 15 portrayed in Figure 10.7, if the item with key 37 were trivially deleted, a subsequent search for 15 would fail because that search would start by probing at index 4, then index 5, and then index 6, at which an empty cell is found. A typical way to get around this diffi- culty is to replace a deleted item with a special “available” marker object. With this special marker possibly occupying spaces in our hash table, we modify our search algorithm so that the search for a key k will skip over cells containing the available marker and continue probing until reaching the desired item or an empty bucket (or returning back to where we started from). Additionally, our algorithm for setitem should remember an available cell encountered during the search for k, since this is a valid place to put a new item (k,v), if no existing item is found. Although use of an open addressing scheme can save space, linear probing suffers from an additional disadvantage. It tends to cluster the items of a map into contiguous runs, which may even overlap (particularly if more than half of the cells in the hash table are occupied). Such contiguous runs of occupied hash cells cause searches to slow down considerably. Another open addressing strategy, known as quadratic probing, iteratively tries the buckets A[(h(k)+ f(i)) mod N], for i = 0,1,2,..., where f(i) = i2, until finding an empty bucket. As with linear probing, the quadratic probing strategy compli- cates the removal operation, but it does avoid the kinds of clustering patterns that occur with linear probing. Nevertheless, it creates its own kind of clustering, called secondary clustering, where the set of filled array cells still has a non-uniform pattern, even if we assume that the original hash codes are distributed uniformly. When N is prime and the bucket array is less than half full, the quadratic probing strategy is guaranteed to find an empty slot. However, this guarantee is not valid once the table becomes at least half full, or if N is not chosen as a prime number; we explore the cause of this type of clustering in an exercise (C-10.36). An open addressing strategy that does not cause clustering of the kind produced by linear probing or the kind produced by quadratic probing is the double hashing strategy. In this approach, we choose a secondary hash function, h, and if h maps some key k to a bucket A[h(k)] that is already occupied, then we iteratively try the buckets A[(h(k) + f(i)) mod N] next, for i = 1,2,3,..., where f(i) = i · h(k). In this scheme, the secondary hash function is not allowed to evaluate to zero; a common choice is h(k) = q−(k mod q), for some prime number q N. Also, N should be a prime. Another approach to avoid clustering with open addressing is to iteratively try buckets A[(h(k) + f(i)) mod N] where f(i) is based on a pseudo-random number generator, providing a repeatable, but somewhat arbitrary, sequence of subsequent probes that depends upon bits of the original hash code. This is the approach cur- rently used by Python’s dictionary class.
- 442. 420 Chapter 10.Maps, Hash Tables, and Skip Lists 10.2.3 Load Factors, Rehashing, and Efficiency In the hash table schemes described thus far, it is important that the load factor, λ = n/N, be kept below 1. With separate chaining, as λ gets very close to 1, the probability of a collision greatly increases, which adds overhead to our operations, since we must revert to linear-time list-based methods in buckets that have col- lisions. Experiments and average-case analyses suggest that we should maintain λ 0.9 for hash tables with separate chaining. With open addressing, on the other hand, as the load factor λ grows beyond 0.5 and starts approaching 1, clusters of entries in the bucket array start to grow as well. These clusters cause the probing strategies to “bounce around” the bucket array for a considerable amount of time before they find an empty slot. In Exercise C-10.36, we explore the degradation of quadratic probing when λ ≥ 0.5. Experiments sug- gest that we should maintain λ 0.5 for an open addressing scheme with linear probing, and perhaps only a bit higher for other open addressing schemes (for ex- ample, Python’s implementation of open addressing enforces that λ 2/3). If an insertion causes the load factor of a hash table to go above the specified threshold, then it is common to resize the table (to regain the specified load factor) and to reinsert all objects into this new table. Although we need not define a new hash code for each object, we do need to reapply a new compression function that takes into consideration the size of the new table. Each rehashing will generally scatter the items throughout the new bucket array. When rehashing to a new table, it is a good requirement for the new array’s size to be at least double the previous size. Indeed, if we always double the size of the table with each rehashing operation, then we can amortize the cost of rehashing all the entries in the table against the time used to insert them in the first place (as with dynamic arrays; see Section 5.3). Efficiency of Hash Tables Although the details of the average-case analysis of hashing are beyond the scope of this book, its probabilistic basis is quite intuitive. If our hash function is good, then we expect the entries to be uniformly distributed in the N cells of the bucket array. Thus, to store n entries, the expected number of keys in a bucket would be n/N, which is O(1) if n is O(N). The costs associated with a periodic rehashing, to resize a table after occasional insertions or deletions can be accounted for separately, leading to an additional O(1) amortized cost for setitem and getitem . In the worst case, a poor hash function could map every item to the same bucket. This would result in linear-time performance for the core map operations with sepa- rate chaining, or with any open addressing model in which the secondary sequence of probes depends only on the hash code. A summary of these costs is given in Table 10.2.
- 443. 10.2. Hash Tables421 Operation List Hash Table expected worst case getitem O(n) O(1) O(n) setitem O(n) O(1) O(n) delitem O(n) O(1) O(n) len O(1) O(1) O(1) iter O(n) O(n) O(n) Table 10.2: Comparison of the running times of the methods of a map realized by means of an unsorted list (as in Section 10.1.5) or a hash table. We let n denote the number of items in the map, and we assume that the bucket array supporting the hash table is maintained such that its capacity is proportional to the number of items in the map. In practice, hash tables are among the most efficient means for implementing a map, and it is essentially taken for granted by programmers that their core oper- ations run in constant time. Python’s dict class is implemented with hashing, and the Python interpreter relies on dictionaries to retrieve an object that is referenced by an identifier in a given namespace. (See Sections 1.10 and 2.5.) The basic com- mand c = a + b involves two calls to getitem in the dictionary for the local namespace to retrieve the values identified as a and b, and a call to setitem to store the result associated with name c in that namespace. In our own algorithm analysis, we simply presume that such dictionary operations run in constant time, independent of the number of entries in the namespace. (Admittedly, the number of entries in a typical namespace can almost surely be bounded by a constant.) In a 2003 academic paper [31], researchers discuss the possibility of exploiting a hash table’s worst-case performance to cause a denial-of-service (DoS) attack of Internet technologies. For many published algorithms that compute hash codes, they note that an attacker could precompute a very large number of moderate-length strings that all hash to the identical 32-bit hash code. (Recall that by any of the hashing schemes we describe, other than double hashing, if two keys are mapped to the same hash code, they will be inseparable in the collision resolution.) In late 2011, another team of researchers demonstrated an implementation of just such an attack [61]. Web servers allow a series of key-value parameters to be embedded in a URL using a syntax such as ?key1=val1key2=val2key3=val3. Typically, those key-value pairs are immediately stored in a map by the server, and a limit is placed on the length and number of such parameters presuming that storage time in the map will be linear in the number of entries. If all keys were to collide, that storage requires quadratic time (causing the server to perform an inordinate amount of work). In spring of 2012, Python developers distributed a security patch that introduces randomization into the computation of hash codes for strings, making it less tractable to reverse engineer a set of colliding strings.
- 444. 422 Chapter 10.Maps, Hash Tables, and Skip Lists 10.2.4 Python Hash Table Implementation In this section, we develop two implementations of a hash table, one using sepa- rate chaining and the other using open addressing with linear probing. While these approaches to collision resolution are quite different, there are a great many com- monalities to the hashing algorithms. For that reason, we extend the MapBase class (from Code Fragment 10.2), to define a new HashMapBase class (see Code Fragment 10.4), providing much of the common functionality to our two hash table implementations. The main design elements of the HashMapBase class are: • The bucket array is represented as a Python list, named self. table, with all entries initialized to None. • We maintain an instance variable self. n that represents the number of dis- tinct items that are currently stored in the hash table. • If the load factor of the table increases beyond 0.5, we double the size of the table and rehash all items into the new table. • We define a hash function utility method that relies on Python’s built-in hash function to produce hash codes for keys, and a randomized Multiply- Add-and-Divide (MAD) formula for the compression function. What is not implemented in the base class is any notion of how a “bucket” should be represented. With separate chaining, each bucket will be an independent structure. With open addressing, however, there is no tangible container for each bucket; the “buckets” are effectively interleaved due to the probing sequences. In our design, the HashMapBase class presumes the following to be abstract methods, which must be implemented by each concrete subclass: • bucket getitem(j, k) This method should search bucket j for an item having key k, returning the associated value, if found, or else raising a KeyError. • bucket setitem(j, k, v) This method should modify bucket j so that key k becomes associated with value v. If the key already exists, the new value overwrites the existing value. Otherwise, a new item is inserted and this method is responsible for incre- menting self. n. • bucket delitem(j, k) This method should remove the item from bucket j having key k, or raise a KeyError if no such item exists. (self. n is decremented after this method.) • iter This is the standard map method to iterate through all keys of the map. Our base class does not delegate this on a per-bucket basis because “buckets” in open addressing are not inherently disjoint.
- 445. 10.2. Hash Tables423 1 class HashMapBase(MapBase): 2 ”””Abstract base class for map using hash-table with MAD compression.””” 3 4 def init (self, cap=11, p=109345121): 5 ”””Create an empty hash-table map.””” 6 self. table = cap [ None ] 7 self. n = 0 # number of entries in the map 8 self. prime = p # prime for MAD compression 9 self. scale = 1 + randrange(p−1) # scale from 1 to p-1 for MAD 10 self. shift = randrange(p) # shift from 0 to p-1 for MAD 11 12 def hash function(self, k): 13 return (hash(k) self. scale + self. shift) % self. prime % len(self. table) 14 15 def len (self): 16 return self. n 17 18 def getitem (self, k): 19 j = self. hash function(k) 20 return self. bucket getitem(j, k) # may raise KeyError 21 22 def setitem (self, k, v): 23 j = self. hash function(k) 24 self. bucket setitem(j, k, v) # subroutine maintains self. n 25 if self. n len(self. table) // 2: # keep load factor = 0.5 26 self. resize(2 len(self. table) − 1) # number 2ˆx - 1 is often prime 27 28 def delitem (self, k): 29 j = self. hash function(k) 30 self. bucket delitem(j, k) # may raise KeyError 31 self. n −= 1 32 33 def resize(self, c): # resize bucket array to capacity c 34 old = list(self.items()) # use iteration to record existing items 35 self. table = c [None] # then reset table to desired capacity 36 self. n = 0 # n recomputed during subsequent adds 37 for (k,v) in old: 38 self[k] = v # reinsert old key-value pair Code Fragment 10.4: A base class for our hash table implementations, extending our MapBase class from Code Fragment 10.2.
- 446. 424 Chapter 10.Maps, Hash Tables, and Skip Lists Separate Chaining Code Fragment 10.5 provides a concrete implementation of a hash table with sepa- rate chaining, in the form of the ChainHashMap class. To represent a single bucket, it relies on an instance of the UnsortedTableMap class from Code Fragment 10.3. The first three methods in the class use index j to access the potential bucket in the bucket array, and a check for the special case in which that table entry is None. The only time we need a new bucket structure is when bucket setitem is called on an otherwise empty slot. The remaining functionality relies on map behaviors that are already supported by the individual UnsortedTableMap instances. We need a bit of forethought to determine whether the application of setitem on the chain causes a net increase in the size of the map (that is, whether the given key is new). 1 class ChainHashMap(HashMapBase): 2 ”””Hash map implemented with separate chaining for collision resolution.””” 3 4 def bucket getitem(self, j, k): 5 bucket = self. table[j] 6 if bucket is None: 7 raise KeyError( Key Error: + repr(k)) # no match found 8 return bucket[k] # may raise KeyError 9 10 def bucket setitem(self, j, k, v): 11 if self. table[j] is None: 12 self. table[j] = UnsortedTableMap( ) # bucket is new to the table 13 oldsize = len(self. table[j]) 14 self. table[j][k] = v 15 if len(self. table[j]) oldsize: # key was new to the table 16 self. n += 1 # increase overall map size 17 18 def bucket delitem(self, j, k): 19 bucket = self. table[j] 20 if bucket is None: 21 raise KeyError( Key Error: + repr(k)) # no match found 22 del bucket[k] # may raise KeyError 23 24 def iter (self): 25 for bucket in self. table: 26 if bucket is not None: # a nonempty slot 27 for key in bucket: 28 yield key Code Fragment 10.5: Concrete hash map class with separate chaining.
- 447. 10.2. Hash Tables425 Linear Probing Our implementation of a ProbeHashMap class, using open addressing with linear probing, is given in Code Fragments 10.6 and 10.7. In order to support deletions, we use a technique described in Section 10.2.2 in which we place a special marker in a table location at which an item has been deleted, so that we can distinguish between it and a location that has always been empty. In our implementation, we declare a class-level attribute, AVAIL, as a sentinel. (We use an instance of the built-in object class because we do not care about any behaviors of the sentinel, just our ability to differentiate it from other objects.) The most challenging aspect of open addressing is to properly trace the series of probes when collisions occur during an insertion or search for an item. To this end, we define a nonpublic utility, find slot, that searches for an item with key k in “bucket” j (that is, where j is the index returned by the hash function for key k). 1 class ProbeHashMap(HashMapBase): 2 ”””Hash map implemented with linear probing for collision resolution.””” 3 AVAIL = object( ) # sentinal marks locations of previous deletions 4 5 def is available(self, j): 6 ”””Return True if index j is available in table.””” 7 return self. table[j] is None or self. table[j] is ProbeHashMap. AVAIL 8 9 def find slot(self, j, k): 10 ”””Search for key k in bucket at index j. 11 12 Return (success, index) tuple, described as follows: 13 If match was found, success is True and index denotes its location. 14 If no match found, success is False and index denotes first available slot. 15 ””” 16 firstAvail = None 17 while True: 18 if self. is available(j): 19 if firstAvail is None: 20 firstAvail = j # mark this as first avail 21 if self. table[j] is None: 22 return (False, firstAvail) # search has failed 23 elif k == self. table[j]. key: 24 return (True, j) # found a match 25 j = (j + 1) % len(self. table) # keep looking (cyclically) Code Fragment 10.6: Concrete ProbeHashMap class that uses linear probing for collision resolution (continued in Code Fragment 10.7).
- 448. 426 Chapter 10.Maps, Hash Tables, and Skip Lists 26 def bucket getitem(self, j, k): 27 found, s = self. find slot(j, k) 28 if not found: 29 raise KeyError( Key Error: + repr(k)) # no match found 30 return self. table[s]. value 31 32 def bucket setitem(self, j, k, v): 33 found, s = self. find slot(j, k) 34 if not found: 35 self. table[s] = self. Item(k,v) # insert new item 36 self. n += 1 # size has increased 37 else: 38 self. table[s]. value = v # overwrite existing 39 40 def bucket delitem(self, j, k): 41 found, s = self. find slot(j, k) 42 if not found: 43 raise KeyError( Key Error: + repr(k)) # no match found 44 self. table[s] = ProbeHashMap. AVAIL # mark as vacated 45 46 def iter (self): 47 for j in range(len(self. table)): # scan entire table 48 if not self. is available(j): 49 yield self. table[j]. key Code Fragment 10.7: Concrete ProbeHashMap class that uses linear probing for collision resolution (continued from Code Fragment 10.6). The three primary map operations each rely on the find slot utility. When at- tempting to retrieve the value associated with a given key, we must continue probing until we find the key, or until we reach a table slot with the None value. We cannot stop the search upon reaching an AVAIL sentinel, because it represents a location that may have been filled when the desired item was once inserted. When a key-value pair is being assigned in the map, we must attempt to find an existing item with the given key, so that we might overwrite its value, before adding a new item to the map. Therefore, we must search beyond any occurrences of the AVAIL sentinel when inserting. However, if no match is found, we prefer to repurpose the first slot marked with AVAIL, if any, when placing the new element in the table. The find slot method enacts this logic, continuing the search until a truly empty slot, but returning the index of the first available slot for an insertion. When deleting an existing item within bucket delitem, we intentionally set the table entry to the AVAIL sentinel in accordance with our strategy.
- 449. 10.3. Sorted Maps427 10.3 Sorted Maps The traditional map ADT allows a user to look up the value associated with a given key, but the search for that key is a form known as an exact search. For example, computer systems often maintain information about events that have occurred (such as financial transactions), organizing such events based upon what are known as time stamps. If we can assume that time stamps are unique for a particular system, then we might organize a map with a time stamp serving as the key, and a record about the event that occurred at that time as the value. A particular time stamp could serve as a reference ID for an event, in which case we can quickly retrieve information about that event from the map. However, the map ADT does not provide any way to get a list of all events ordered by the time at which they occur, or to search for which event occurred closest to a particular time. In fact, the fast performance of hash-based implementations of the map ADT relies on the intentionally scattering of keys that may seem very “near” to each other in the original domain, so that they are more uniformly distributed in a hash table. In this section, we introduce an extension known as the sorted map ADT that includes all behaviors of the standard map, plus the following: M.find min(): Return the (key,value) pair with minimum key (or None, if map is empty). M.find max(): Return the (key,value) pair with maximum key (or None, if map is empty). M.find lt(k): Return the (key,value) pair with the greatest key that is strictly less than k (or None, if no such item exists). M.find le(k): Return the (key,value) pair with the greatest key that is less than or equal to k (or None, if no such item exists). M.find gt(k): Return the (key,value) pair with the least key that is strictly greater than k (or None, if no such item exists). M.find ge(k): Return the (key,value) pair with the least key that is greater than or equal to k (or None, if no such item). M.find range(start, stop): Iterate all (key,value) pairs with start = key stop. If start is None, iteration begins with minimum key; if stop is None, iteration concludes with maximum key. iter(M): Iterate all keys of the map according to their natural order, from smallest to largest. reversed(M): Iterate all keys of the map in reverse order; in Python, this is implemented with the reversed method.
- 450. 428 Chapter 10.Maps, Hash Tables, and Skip Lists 10.3.1 Sorted Search Tables Several data structures can efficiently support the sorted map ADT, and we will examine some advanced techniques in Section 10.4 and Chapter 11. In this section, we begin by exploring a simple implementation of a sorted map. We store the map’s items in an array-based sequence A so that they are in increasing order of their keys, assuming the keys have a naturally defined order. (See Figure 10.8.) We refer to this implementation of a map as a sorted search table. 9 2 4 5 7 8 12 14 17 19 22 25 27 28 33 5 37 0 1 2 3 4 6 7 8 9 10 11 12 13 14 15 Figure 10.8: Realization of a map by means of a sorted search table. We show only the keys for this map, so as to highlight their ordering. As was the case with the unsorted table map of Section 10.1.5, the sorted search table has a space requirement that is O(n), assuming we grow and shrink the array to keep its size proportional to the number of items in the map. The primary advan- tage of this representation, and our reason for insisting that A be array-based, is that it allows us to use the binary search algorithm for a variety of efficient operations. Binary Search and Inexact Searches We originally presented the binary search algorithm in Section 4.1.3, as a means for detecting whether a given target is stored within a sorted sequence. In our original presentation (Code Fragment 4.3 on page 156), a binary search function returned True of False to designate whether the desired target was found. While such an approach could be used to implement the contains method of the map ADT, we can adapt the binary search algorithm to provide far more useful information when performing forms of inexact search in support of the sorted map ADT. The important realization is that while performing a binary search, we can de- termine the index at or near where a target might be found. During a successful search, the standard implementation determines the precise index at which the tar- get is found. During an unsuccessful search, although the target is not found, the algorithm will effectively determine a pair of indices designating elements of the collection that are just less than or just greater than the missing target. As a motivating example, our original simulation from Figure 4.5 on page 156 shows a successful binary search for a target of 22, using the same data we portray in Figure 10.8. Had we instead been searching for 21, the first four steps of the algorithm would be the same. The subsequent difference is that we would make an additional call with inverted parameters high=9 and low=10, effectively conclud- ing that the missing target lies in the gap between values 19 and 22 in that example.
- 451. 10.3. Sorted Maps429 Implementation In Code Fragments 10.8 through 10.10, we present a complete implementation of a class, SortedTableMap, that supports the sorted map ADT. The most notable fea- ture of our design is the inclusion of a find index utility function. This method using the binary search algorithm, but by convention returns the index of the left- most item in the search interval having key greater than or equal to k. Therefore, if the key is present, it will return the index of the item having that key. (Recall that keys are unique in a map.) When the key is missing, the function returns the index of the item in the search interval that is just beyond where the key would have been located. As a technicality, the method returns index high + 1 to indicate that no items of the interval had a key greater than k. We rely on this utility method when implementing the traditional map opera- tions and the new sorted map operations. The body of each of the getitem , setitem , and delitem methods begins with a call to find index to deter- mine a candidate index at which a matching key might be found. For getitem , we simply check whether that is a valid index containing the target to determine the result. For setitem , recall that the goal is to replace the value of an existing item, if one with key k is found, but otherwise to insert a new item into the map. The index returned by find index will be the index of the match, if one exists, or oth- erwise the exact index at which the new item should be inserted. For delitem , we again rely on the convenience of find index to determine the location of the item to be popped, if any. Our find index utility is equally valuable when implementing the various in- exact search methods given in Code Fragment 10.10. For each of the methods find lt, find le, find gt, and find ge, we begin with a call to find index utility, which locates the first index at which there is an element with key ≥ k, if any. This is precisely what we want for find ge, if valid, and just beyond the index we want for find lt. For find gt and find le we need some extra case analysis to distinguish whether the indicated index has a key equal to k. For example, if the indicated item has a matching key, our find gt implementation increments the index before continuing with the process. (We omit the implementation of find le, for brevity.) In all cases, we must properly handle boundary cases, reporting None when unable to find a key with the desired property. Our strategy for implementing find range is to use the find index utility to locate the first item with key ≥ start (assuming start is not None). With that knowl- edge, we use a while loop to sequentially report items until reaching one that has a key greater than or equal to the stopping value (or until reaching the end of the table). It is worth noting that the while loop may trivially iterate zero items if the first key that is greater than or equal to start also happens to be greater than or equal to stop. This represents an empty range in the map.
- 452. 430 Chapter 10.Maps, Hash Tables, and Skip Lists 1 class SortedTableMap(MapBase): 2 ”””Map implementation using a sorted table.””” 3 4 #----------------------------- nonpublic behaviors ----------------------------- 5 def find index(self, k, low, high): 6 ”””Return index of the leftmost item with key greater than or equal to k. 7 8 Return high + 1 if no such item qualifies. 9 10 That is, j will be returned such that: 11 all items of slice table[low:j] have key k 12 all items of slice table[j:high+1] have key = k 13 ””” 14 if high low: 15 return high + 1 # no element qualifies 16 else: 17 mid = (low + high) // 2 18 if k == self. table[mid]. key: 19 return mid # found exact match 20 elif k self. table[mid]. key: 21 return self. find index(k, low, mid − 1) # Note: may return mid 22 else: 23 return self. find index(k, mid + 1, high) # answer is right of mid 24 25 #----------------------------- public behaviors ----------------------------- 26 def init (self): 27 ”””Create an empty map.””” 28 self. table = [ ] 29 30 def len (self): 31 ”””Return number of items in the map.””” 32 return len(self. table) 33 34 def getitem (self, k): 35 ”””Return value associated with key k (raise KeyError if not found).””” 36 j = self. find index(k, 0, len(self. table) − 1) 37 if j == len(self. table) or self. table[j]. key != k: 38 raise KeyError( Key Error: + repr(k)) 39 return self. table[j]. value Code Fragment 10.8: An implementation of a SortedTableMap class (continued in Code Fragments 10.9 and 10.10).
- 453. 10.3. Sorted Maps431 40 def setitem (self, k, v): 41 ”””Assign value v to key k, overwriting existing value if present.””” 42 j = self. find index(k, 0, len(self. table) − 1) 43 if j len(self. table) and self. table[j]. key == k: 44 self. table[j]. value = v # reassign value 45 else: 46 self. table.insert(j, self. Item(k,v)) # adds new item 47 48 def delitem (self, k): 49 ”””Remove item associated with key k (raise KeyError if not found).””” 50 j = self. find index(k, 0, len(self. table) − 1) 51 if j == len(self. table) or self. table[j]. key != k: 52 raise KeyError( Key Error: + repr(k)) 53 self. table.pop(j) # delete item 54 55 def iter (self): 56 ”””Generate keys of the map ordered from minimum to maximum.””” 57 for item in self. table: 58 yield item. key 59 60 def reversed (self): 61 ”””Generate keys of the map ordered from maximum to minimum.””” 62 for item in reversed(self. table): 63 yield item. key 64 65 def find min(self): 66 ”””Return (key,value) pair with minimum key (or None if empty).””” 67 if len(self. table) 0: 68 return (self. table[0]. key, self. table[0]. value) 69 else: 70 return None 71 72 def find max(self): 73 ”””Return (key,value) pair with maximum key (or None if empty).””” 74 if len(self. table) 0: 75 return (self. table[−1]. key, self. table[−1]. value) 76 else: 77 return None Code Fragment 10.9: An implementation of a SortedTableMap class (together with Code Fragments 10.8 and 10.10).
- 454. 432 Chapter 10.Maps, Hash Tables, and Skip Lists 78 def find ge(self, k): 79 ”””Return (key,value) pair with least key greater than or equal to k.””” 80 j = self. find index(k, 0, len(self. table) − 1) # j s key = k 81 if j len(self. table): 82 return (self. table[j]. key, self. table[j]. value) 83 else: 84 return None 85 86 def find lt(self, k): 87 ”””Return (key,value) pair with greatest key strictly less than k.””” 88 j = self. find index(k, 0, len(self. table) − 1) # j s key = k 89 if j 0: 90 return (self. table[j−1]. key, self. table[j−1]. value) # Note use of j-1 91 else: 92 return None 93 94 def find gt(self, k): 95 ”””Return (key,value) pair with least key strictly greater than k.””” 96 j = self. find index(k, 0, len(self. table) − 1) # j s key = k 97 if j len(self. table) and self. table[j]. key == k: 98 j += 1 # advanced past match 99 if j len(self. table): 100 return (self. table[j]. key, self. table[j]. value) 101 else: 102 return None 103 104 def find range(self, start, stop): 105 ”””Iterate all (key,value) pairs such that start = key stop. 106 107 If start is None, iteration begins with minimum key of map. 108 If stop is None, iteration continues through the maximum key of map. 109 ””” 110 if start is None: 111 j = 0 112 else: 113 j = self. find index(start, 0, len(self. table)−1) # find first result 114 while j len(self. table) and (stop is None or self. table[j]. key stop): 115 yield (self. table[j]. key, self. table[j]. value) 116 j += 1 Code Fragment 10.10: An implementation of a SortedTableMap class (continued from Code Fragments 10.9 and 10.10). We omit the find le method due to space.
- 455. 10.3. Sorted Maps433 Analysis We conclude by analyzing the performance of our SortedTableMap implementa- tion. A summary of the running times for all methods of the sorted map ADT (including the traditional map operations) is given in Table 10.3. It should be clear that the len , find min, and find max methods run in O(1) time, and that iter- ating the keys of the table in either direction can be peformed in O(n) time. The analysis for the various forms of search all depend on the fact that a binary search on a table with n entries runs in O(logn) time. This claim was originally shown as Proposition 4.2 in Section 4.2, and that analysis clearly applies to our find index method as well. We therefore claim an O(logn) worst-case running time for methods getitem , find lt, find gt, find le, and find ge. Each of these makes a single call to find index, followed by a constant number of additional steps to determine the appropriate answer based on the index. The analysis of find range is a bit more interesting. It begins with a binary search to find the first item within the range (if any). After that, it executes a loop that takes O(1) time per iteration to report subsequent values until reaching the end of the range. If there are s items reported in the range, the total running time is O(s+logn). In contrast to the efficient search operations, update operations for a sorted table may take considerable time. Although binary search can help identify the index at which an update occurs, both insertions and deletions require, in the worst case, that linearly many existing elements be shifted in order to maintain the sorted order of the table. Specifically, the potential call to table.insert from within setitem and table.pop from within delitem lead to O(n) worst-case time. (See the discussion of corresponding operations of the list class in Section 5.4.1.) In conclusion, sorted tables are primarily used in situations where we expect many searches but relatively few updates. Operation Running Time len(M) O(1) k in M O(logn) M[k] = v O(n) worst case; O(logn) if existing k del M[k] O(n) worst case M.find min(), M.find max() O(1) M.find lt(k), M.find gt(k) O(logn) M.find le(k), M.find ge(k) M.find range(start, stop) O(s+logn) where s items are reported iter(M), reversed(M) O(n) Table 10.3: Performance of a sorted map, as implemented with SortedTableMap. We use n to denote the number of items in the map at the time the operation is performed. The space requirement is O(n).
- 456. 434 Chapter 10.Maps, Hash Tables, and Skip Lists 10.3.2 Two Applications of Sorted Maps In this section, we explore applications in which there is particular advantage to using a sorted map rather than a traditional (unsorted) map. To apply a sorted map, keys must come from a domain that is totally ordered. Furthermore, to take advantage of the inexact or range searches afforded by a sorted map, there should be some reason why nearby keys have relevance to a search. Flight Databases There are several Web sites on the Internet that allow users to perform queries on flight databases to find flights between various cities, typically with the intent to buy a ticket. To make a query, a user specifies origin and destination cities, a depar- ture date, and a departure time. To support such queries, we can model the flight database as a map, where keys are Flight objects that contain fields corresponding to these four parameters. That is, a key is a tuple k = (origin,destination,date,time). Additional information about a flight, such as the flight number, the number of seats still available in first (F) and coach (Y) class, the flight duration, and the fare, can be stored in the value object. Finding a requested flight is not simply a matter of finding an exact match for a requested query. Although a user typically wants to exactly match the ori- gin and destination cities, he or she may have flexibility for the departure date, and certainly will have some flexibility for the departure time on a specific day. We can handle such a query by ordering our keys lexicographically. Then, an ef- ficient implementation for a sorted map would be a good way to satisfy users’ queries. For instance, given a user query key k, we could call find ge(k) to return the first flight between the desired cities, having a departure date and time match- ing the desired query or later. Better yet, with well-constructed keys, we could use find range(k1, k2) to find all flights within a given range of times. For exam- ple, if k1 = (ORD, PVD, 05May, 09:30), and k2 = (ORD, PVD, 05May, 20:00), a respective call to find range(k1, k2) might result in the following sequence of key-value pairs: (ORD, PVD, 05May, 09:53) : (AA 1840, F5, Y15, 02:05, 251), (ORD, PVD, 05May, 13:29) : (AA 600, F2, Y0, 02:16, 713), (ORD, PVD, 05May, 17:39) : (AA 416, F3, Y9, 02:09, 365), (ORD, PVD, 05May, 19:50) : (AA 1828, F9, Y25, 02:13, 186)
- 457. 10.3. Sorted Maps435 Maxima Sets Life is full of trade-offs. We often have to trade off a desired performance measure against a corresponding cost. Suppose, for the sake of an example, we are interested in maintaining a database rating automobiles by their maximum speeds and their cost. We would like to allow someone with a certain amount of money to query our database to find the fastest car they can possibly afford. We can model such a trade-off problem as this by using a key-value pair to model the two parameters that we are trading off, which in this case would be the pair (cost,speed) for each car. Notice that some cars are strictly better than other cars using this measure. For example, a car with cost-speed pair (20000,100) is strictly better than a car with cost-speed pair (30000,90). At the same time, there are some cars that are not strictly dominated by another car. For example, a car with cost-speed pair (20000,100) may be better or worse than a car with cost-speed pair (30000,120), depending on how much money we have to spend. (See Figure 10.9.) Figure 10.9: Illustrating the cost-performance trade-off with pairs represented by points in the plane. Notice that point p is strictly better than points c, d, and e, but may be better or worse than points a, b, f, g, and h, depending on the price we are willing to pay. Thus, if we were to add p to our set, we could remove the points c, d, and e, but not the others. Formally, we say a cost-performance pair (a,b) dominates pair (c,d) = (a,b) if a ≤ c and b ≥ d, that is, if the first pair has no greater cost and at least as good performance. A pair (a,b) is called a maximum pair if it is not dominated by any other pair. We are interested in maintaining the set of maxima of a collection of cost-performance pairs. That is, we would like to add new pairs to this collection (for example, when a new car is introduced), and to query this collection for a given dollar amount, d, to find the fastest car that costs no more than d dollars. Performance Cost d f h a p g b e c
- 458. 436 Chapter 10.Maps, Hash Tables, and Skip Lists Maintaining a Maxima Set with a Sorted Map We can store the set of maxima pairs in a sorted map, M, so that the cost is the key field and performance (speed) is the value field. We can then implement opera- tions add(c, p), which adds a new cost-performance pair (c, p), and best(c), which returns the best pair with cost at most c, as shown in Code Fragment 10.11. 1 class CostPerformanceDatabase: 2 ”””Maintain a database of maximal (cost,performance) pairs.””” 3 4 def init (self): 5 ”””Create an empty database.””” 6 self. M = SortedTableMap( ) # or a more efficient sorted map 7 8 def best(self, c): 9 ”””Return (cost,performance) pair with largest cost not exceeding c. 10 11 Return None if there is no such pair. 12 ””” 13 return self. M.find le(c) 14 15 def add(self, c, p): 16 ”””Add new entry with cost c and performance p.””” 17 # determine if (c,p) is dominated by an existing pair 18 other = self. M.find le(c) # other is at least as cheap as c 19 if other is not None and other[1] = p: # if its performance is as good, 20 return # (c,p) is dominated, so ignore 21 self. M[c] = p # else, add (c,p) to database 22 # and now remove any pairs that are dominated by (c,p) 23 other = self. M.find gt(c) # other more expensive than c 24 while other is not None and other[1] = p: 25 del self. M[other[0]] 26 other = self. M.find gt(c) Code Fragment 10.11: An implementation of a class maintaining a set of maxima cost-performance pairs using a sorted map. Unfortunately, if we implement M using the SortedTableMap, the add behavior has O(n) worst-case running time. If, on the other hand, we implement M using a skip list, which we next describe, we can perform best(c) queries in O(logn) expected time and add(c, p) updates in O((1+ r)logn) expected time, where r is the number of points removed.
- 459. 10.4. Skip Lists437 10.4 Skip Lists An interesting data structure for realizing the sorted map ADT is the skip list. In Section 10.3.1, we saw that a sorted array will allow O(logn)-time searches via the binary search algorithm. Unfortunately, update operations on a sorted array have O(n) worst-case running time because of the need to shift elements. In Chapter 7 we demonstrated that linked lists support very efficient update operations, as long as the position within the list is identified. Unfortunately, we cannot perform fast searches on a standard linked list; for example, the binary search algorithm requires an efficient means for direct accessing an element of a sequence by index. Skip lists provide a clever compromise to efficiently support search and update operations. A skip list S for a map M consists of a series of lists {S0,S1,...,Sh}. Each list Si stores a subset of the items of M sorted by increasing keys, plus items with two sentinel keys denoted −∞ and +∞, where −∞ is smaller than every possible key that can be inserted in M and +∞ is larger than every possible key that can be inserted in M. In addition, the lists in S satisfy the following: • List S0 contains every item of the map M (plus sentinels −∞ and +∞). • For i = 1,...,h−1, list Si contains (in addition to −∞ and +∞) a randomly generated subset of the items in list Si−1. • List Sh contains only −∞ and +∞. An example of a skip list is shown in Figure 10.10. It is customary to visualize a skip list S with list S0 at the bottom and lists S1,...,Sh above it. Also, we refer to h as the height of skip list S. Intuitively, the lists are set up so that Si+1 contains more or less alternate items of Si. As we shall see in the details of the insertion method, the items in Si+1 are chosen at random from the items in Si by picking each item from Si to also be in Si+1 with probability 1/2. That is, in essence, we “flip a coin” for each item in Si 31 25 25 -∞ -∞ -∞ -∞ -∞ -∞ 17 17 17 17 12 S5 S4 S3 S2 S1 S0 55 55 55 55 12 17 20 25 31 38 39 44 50 +∞ +∞ +∞ +∞ +∞ +∞ 44 38 31 25 Figure 10.10: Example of a skip list storing 10 items. For simplicity, we show only the items’ keys, not their associated values.
- 460. 438 Chapter 10.Maps, Hash Tables, and Skip Lists and place that item in Si+1 if the coin comes up “heads.” Thus, we expect S1 to have about n/2 items, S2 to have about n/4 items, and, in general, Si to have about n/2i items. In other words, we expect the height h of S to be about logn. The halving of the number of items from one list to the next is not enforced as an explicit property of skip lists, however. Instead, randomization is used. Functions that generate numbers that can be viewed as random numbers are built into most modern computers, because they are used extensively in computer games, cryptography, and computer simulations, Some functions, called pseudo- random number generators, generate random-like numbers, starting with an initial seed. (See discusion of random module in Section 1.11.1.) Other methods use hardware devices to extract “true” random numbers from nature. In any case, we will assume that our computer has access to numbers that are sufficiently random for our analysis. The main advantage of using randomization in data structure and algorithm design is that the structures and functions that result are usually simple and efficient. The skip list has the same logarithmic time bounds for searching as is achieved by the binary search algorithm, yet it extends that performance to update methods when inserting or deleting items. Nevertheless, the bounds are expected for the skip list, while binary search has a worst-case bound with a sorted table. A skip list makes random choices in arranging its structure in such a way that search and update times are O(logn) on average, where n is the number of items in the map. Interestingly, the notion of average time complexity used here does not depend on the probability distribution of the keys in the input. Instead, it depends on the use of a random-number generator in the implementation of the insertions to help decide where to place the new item. The running time is averaged over all possible outcomes of the random numbers used when inserting entries. Using the position abstraction used for lists and trees, we view a skip list as a two-dimensional collection of positions arranged horizontally into levels and ver- tically into towers. Each level is a list Si and each tower contains positions storing the same item across consecutive lists. The positions in a skip list can be traversed using the following operations: next(p): Return the position following p on the same level. prev(p): Return the position preceding p on the same level. below(p): Return the position below p in the same tower. above(p): Return the position above p in the same tower. We conventionally assume that the above operations return None if the position requested does not exist. Without going into the details, we note that we can eas- ily implement a skip list by means of a linked structure such that the individual traversal methods each take O(1) time, given a skip-list position p. Such a linked structure is essentially a collection of h doubly linked lists aligned at towers, which are also doubly linked lists.
- 461. 10.4. Skip Lists439 10.4.1 Search and Update Operations in a Skip List The skip-list structure affords simple map search and update algorithms. In fact, all of the skip-list search and update algorithms are based on an elegant SkipSearch method that takes a key k and finds the position p of the item in list S0 that has the largest key less than or equal to k (which is possibly −∞). Searching in a Skip List Suppose we are given a search key k. We begin the SkipSearch method by setting a position variable p to the topmost, left position in the skip list S, called the start position of S. That is, the start position is the position of Sh storing the special entry with key −∞. We then perform the following steps (see Figure 10.11), where key(p) denotes the key of the item at position p: 1. If S.below(p) is None, then the search terminates—we are at the bottom and have located the item in S with the largest key less than or equal to the search key k. Otherwise, we drop down to the next lower level in the present tower by setting p = S.below(p). 2. Starting at position p, we move p forward until it is at the rightmost position on the present level such that key(p) ≤ k. We call this the scan forward step. Note that such a position always exists, since each level contains the keys +∞ and −∞. It may be that p remains where it started after we perform such a forward scan for this level. 3. Return to step 1. 55 S1 S2 S3 S4 S5 +∞ +∞ +∞ +∞ +∞ +∞ -∞ -∞ -∞ 12 12 -∞ 17 17 25 25 20 17 31 38 39 -∞ -∞ 17 17 25 25 31 31 38 44 44 50 55 55 55 S0 Figure 10.11: Example of a search in a skip list. The positions examined when searching for key 50 are highlighted. We give a pseudo-code description of the skip-list search algorithm, SkipSearch, in Code Fragment 10.12. Given this method, the map operation M[k] is performed by computing p = SkipSearch(k) and testing whether or not key(p) = k. If these two keys are equal, we return the associated value; otherwise, we raise a KeyError.
- 462. 440 Chapter 10.Maps, Hash Tables, and Skip Lists Algorithm SkipSearch(k): Input: A search key k Output: Position p in the bottom list S0 with the largest key such that key(p) ≤ k p = start {begin at start position} while below(p) = None do p = below(p) {drop down} while k ≥ key(next(p)) do p = next(p) {scan forward} return p. Code Fragment 10.12: Algorthm to search a skip list S for key k. As it turns out, the expected running time of algorithm SkipSearch on a skip list with n entries is O(logn). We postpone the justification of this fact, however, until after we discuss the implementation of the update methods for skip lists. Navigation starting at the position identified by SkipSearch(k) can be easily used to provide the additional forms of searches in the sorted map ADT (e.g., find gt, find range). Insertion in a Skip List The execution of the map operation M[k] = v begins with a call to SkipSearch(k). This gives us the position p of the bottom-level item with the largest key less than or equal to k (note that p may hold the special item with key −∞). If key(p) = k, the associated value is overwritten with v. Otherwise, we need to create a new tower for item (k,v). We insert (k,v) immediately after position p within S0. After inserting the new item at the bottom level, we use randomization to decide the height of the tower for the new item. We “flip” a coin, and if the flip comes up tails, then we stop here. Else (the flip comes up heads), we backtrack to the previous (next higher) level and insert (k,v) in this level at the appropriate position. We again flip a coin; if it comes up heads, we go to the next higher level and repeat. Thus, we continue to insert the new item (k,v) in lists until we finally get a flip that comes up tails. We link together all the references to the new item (k,v) created in this process to create its tower. A coin flip can be simulated with Python’s built-in pseudo-random number generator from the random module by calling randrange(2), which returns 0 or 1, each with probability 1/2. We give the insertion algorithm for a skip list S in Code Fragment 10.13 and we illustrate it in Figure 10.12. The algorithm uses an insertAfterAbove(p,q,(k,v)) method that inserts a position storing the item (k,v) after position p (on the same level as p) and above position q, returning the new position r (and setting internal references so that next, prev, above, and below methods will work correctly for p, q, and r). The expected running time of the insertion algorithm on a skip list with n entries is O(logn), which we show in Section 10.4.2.
- 463. 10.4. Skip Lists441 Algorithm SkipInsert(k,v): Input: Key k and value v Output: Topmost position of the item inserted in the skip list p = SkipSearch(k) q = None {q will represent top node in new item’s tower} i = −1 repeat i = i+1 if i ≥ h then h = h+1 {add a new level to the skip list} t = next(s) s = insertAfterAbove(None,s,(−∞,None)) {grow leftmost tower} insertAfterAbove(s,t,(+∞,None)) {grow rightmost tower} while above(p) is None do p = prev(p) {scan backward} p = above(p) {jump up to higher level} q = insertAfterAbove(p,q,(k,v)) {increase height of new item’s tower} until coinFlip() == tails n = n+1 return q Code Fragment 10.13: Insertion in a skip list. Method coinFlip() returns “heads” or “tails”, each with probability 1/2. Instance variables n, h, and s hold the number of entries, the height, and the start node of the skip list. 55 S1 S2 S3 S4 S5 +∞ +∞ +∞ +∞ +∞ +∞ -∞ -∞ -∞ 12 12 -∞ 17 17 25 25 20 17 31 -∞ -∞ 17 17 25 25 31 31 38 44 44 42 42 42 55 55 55 38 39 42 50 S0 Figure 10.12: Insertion of an entry with key 42 into the skip list of Figure 10.10. We assume that the random “coin flips” for the new entry came up heads three times in a row, followed by tails. The positions visited are highlighted. The positions inserted to hold the new entry are drawn with thick lines, and the positions preceding them are flagged.
- 464. 442 Chapter 10.Maps, Hash Tables, and Skip Lists Removal in a Skip List Like the search and insertion algorithms, the removal algorithm for a skip list is quite simple. In fact, it is even easier than the insertion algorithm. That is, to per- form the map operation del M[k] we begin by executing method SkipSearch(k). If the position p stores an entry with key different from k, we raise a KeyError. Otherwise, we remove p and all the positions above p, which are easily accessed by using above operations to climb up the tower of this entry in S starting at posi- tion p. While removing levels of the tower, we reestablish links between the hor- izontal neighbors of each removed position. The removal algorithm is illustrated in Figure 10.13 and a detailed description of it is left as an exercise (R-10.24). As we show in the next subsection, deletion operation in a skip list with n entries has O(logn) expected running time. Before we give this analysis, however, there are some minor improvements to the skip-list data structure we would like to discuss. First, we do not actually need to store references to values at the levels of the skip list above the bottom level, because all that is needed at these levels are references to keys. In fact, we can more efficiently represent a tower as a single object, storing the key-value pair, and maintaining j previous references and j next references if the tower reaches level Sj. Second, for the horizontal axes, it is possible to keep the list singly linked, storing only the next references. We can perform insertions and removals in strictly a top-down, scan-forward fashion. We explore the details of this optimization in Exercise C-10.44. Neither of these optimizations improve the asymptotic perfor- mance of skip lists by more than a constant factor, but these improvements can, nevertheless, be meaningful in practice. In fact, experimental evidence suggests that optimized skip lists are faster in practice than AVL trees and other balanced search trees, which are discussed in Chapter 11. 31 S5 S4 S3 S2 S1 -∞ -∞ -∞ 12 12 -∞ 17 17 25 25 31 31 42 55 50 55 +∞ +∞ +∞ +∞ +∞ -∞ -∞ 17 38 38 39 42 42 42 44 44 55 55 +∞ 17 17 20 25 25 S0 Figure 10.13: Removal of the entry with key 25 from the skip list of Figure 10.12. The positions visited after the search for the position of S0 holding the entry are highlighted. The positions removed are drawn with dashed lines.
- 465. 10.4. Skip Lists443 Maintaining the Topmost Level A skip list S must maintain a reference to the start position (the topmost, left po- sition in S) as an instance variable, and must have a policy for any insertion that wishes to continue inserting a new entry past the top level of S. There are two possible courses of action we can take, both of which have their merits. One possibility is to restrict the top level, h, to be kept at some fixed value that is a function of n, the number of entries currently in the map (from the analysis we will see that h = max{10,2 logn} is a reasonable choice, and picking h = 3 logn is even safer). Implementing this choice means that we must modify the insertion algorithm to stop inserting a new position once we reach the topmost level (unless logn log(n+1), in which case we can now go at least one more level, since the bound on the height is increasing). The other possibility is to let an insertion continue inserting a new position as long as heads keeps getting returned from the random number generator. This is the approach taken by algorithm SkipInsert of Code Fragment 10.13. As we show in the analysis of skip lists, the probability that an insertion will go to a level that is more than O(logn) is very low, so this design choice should also work. Either choice will still result in the expected O(logn) time to perform search, insertion, and removal, however, which we show in the next section. 10.4.2 Probabilistic Analysis of Skip Lists As we have shown above, skip lists provide a simple implementation of a sorted map. In terms of worst-case performance, however, skip lists are not a superior data structure. In fact, if we do not officially prevent an insertion from continuing signif- icantly past the current highest level, then the insertion algorithm can go into what is almost an infinite loop (it is not actually an infinite loop, however, since the prob- ability of having a fair coin repeatedly come up heads forever is 0). Moreover, we cannot infinitely add positions to a list without eventually running out of memory. In any case, if we terminate position insertion at the highest level h, then the worst- case running time for performing the getitem , setitem , and delitem map operations in a skip list S with n entries and height h is O(n+h). This worst- case performance occurs when the tower of every entry reaches level h−1, where h is the height of S. However, this event has very low probability. Judging from this worst case, we might conclude that the skip-list structure is strictly inferior to the other map implementations discussed earlier in this chapter. But this would not be a fair analysis, for this worst-case behavior is a gross overestimate.
- 466. 444 Chapter 10.Maps, Hash Tables, and Skip Lists Bounding the Height of a Skip List Because the insertion step involves randomization, a more accurate analysis of skip lists involves a bit of probability. At first, this might seem like a major undertaking, for a complete and thorough probabilistic analysis could require deep mathemat- ics (and, indeed, there are several such deep analyses that have appeared in data structures research literature). Fortunately, such an analysis is not necessary to un- derstand the expected asymptotic behavior of skip lists. The informal and intuitive probabilistic analysis we give below uses only basic concepts of probability theory. Let us begin by determining the expected value of the height h of a skip list S with n entries (assuming that we do not terminate insertions early). The probability that a given entry has a tower of height i ≥ 1 is equal to the probability of getting i consecutive heads when flipping a coin, that is, this probability is 1/2i. Hence, the probability Pi that level i has at least one position is at most Pi ≤ n 2i , for the probability that any one of n different events occurs is at most the sum of the probabilities that each occurs. The probability that the height h of S is larger than i is equal to the probability that level i has at least one position, that is, it is no more than Pi. This means that h is larger than, say, 3logn with probability at most P3logn ≤ n 23logn = n n3 = 1 n2 . For example, if n = 1000, this probability is a one-in-a-million long shot. More generally, given a constant c 1, h is larger than clogn with probability at most 1/nc−1. That is, the probability that h is smaller than clogn is at least 1−1/nc−1. Thus, with high probability, the height h of S is O(logn). Analyzing Search Time in a Skip List Next, consider the running time of a search in skip list S, and recall that such a search involves two nested while loops. The inner loop performs a scan forward on a level of S as long as the next key is no greater than the search key k, and the outer loop drops down to the next level and repeats the scan forward iteration. Since the height h of S is O(logn) with high probability, the number of drop-down steps is O(logn) with high probability.
- 467. 10.4. Skip Lists445 So we have yet to bound the number of scan-forward steps we make. Let ni be the number of keys examined while scanning forward at level i. Observe that, after the key at the starting position, each additional key examined in a scan-forward at level i cannot also belong to level i + 1. If any of these keys were on the previous level, we would have encountered them in the previous scan-forward step. Thus, the probability that any key is counted in ni is 1/2. Therefore, the expected value of ni is exactly equal to the expected number of times we must flip a fair coin before it comes up heads. This expected value is 2. Hence, the expected amount of time spent scanning forward at any level i is O(1). Since S has O(logn) levels with high probability, a search in S takes expected time O(logn). By a similar analysis, we can show that the expected running time of an insertion or a removal is O(logn). Space Usage in a Skip List Finally, let us turn to the space requirement of a skip list S with n entries. As we observed above, the expected number of positions at level i is n/2i, which means that the expected total number of positions in S is h ∑ i=0 n 2i = n h ∑ i=0 1 2i . Using Proposition 3.5 on geometric summations, we have h ∑ i=0 1 2i = 1 2 h+1 −1 1 2 −1 = 2· 1− 1 2h+1 2 for all h ≥ 0. Hence, the expected space requirement of S is O(n). Table 10.4 summarizes the performance of a sorted map realized by a skip list. Operation Running Time len(M) O(1) k in M O(logn) expected M[k] = v O(logn) expected del M[k] O(logn) expected M.find min(), M.find max() O(1) M.find lt(k), M.find gt(k) O(logn) expected M.find le(k), M.find ge(k) M.find range(start, stop) O(s+logn) expected, with s items reported iter(M), reversed(M) O(n) Table 10.4: Performance of a sorted map implemented with a skip list. We use n to denote the number of entries in the dictionary at the time the operation is performed. The expected space requirement is O(n).
- 468. 446 Chapter 10.Maps, Hash Tables, and Skip Lists 10.5 Sets, Multisets, and Multimaps We conclude this chapter by examining several additional abstractions that are closely related to the map ADT, and that can be implemented using data structures similar to those for a map. • A set is an unordered collection of elements, without duplicates, that typi- cally supports efficient membership tests. In essence, elements of a set are like keys of a map, but without any auxiliary values. • A multiset (also known as a bag) is a set-like container that allows duplicates. • A multimap is similar to a traditional map, in that it associates values with keys; however, in a multimap the same key can be mapped to multiple val- ues. For example, the index of this book maps a given term to one or more locations at which the term occurs elsewhere in the book. 10.5.1 The Set ADT Python provides support for representing the mathematical notion of a set through the built-in classes frozenset and set, as originally discussed in Chapter 1, with frozenset being an immutable form. Both of those classes are implemented using hash tables in Python. Python’s collections module defines abstract base classes that essentially mirror these built-in classes. Although the choice of names is counterintuitive, the abstract base class collections.Set matches the concrete frozenset class, while the abstract base class collections.MutableSet is akin to the concrete set class. In our own discussion, we equate the “set ADT” with the behavior of the built- in set class (and thus, the collections.MutableSet base class). We begin by listing what we consider to be the five most fundamental behaviors for a set S: S.add(e): Add element e to the set. This has no effect if the set already contains e. S.discard(e): Remove element e from the set, if present. This has no effect if the set does not contain e. e in S: Return True if the set contains element e. In Python, this is implemented with the special contains method. len(S): Return the number of elements in set S. In Python, this is implemented with the special method len . iter(S): Generate an iteration of all elements of the set. In Python, this is implemented with the special method iter .
- 469. 10.5. Sets, Multisets,and Multimaps 447 In the next section, we will see that the above five methods suffice for deriving all other behaviors of a set. Those remaining behaviors can be naturally grouped as follows. We begin by describing the following additional operations for removing one or more elements from a set: S.remove(e): Remove element e from the set. If the set does not contain e, raise a KeyError. S.pop(): Remove and return an arbitrary element from the set. If the set is empty, raise a KeyError. S.clear(): Remove all elements from the set. The next group of behaviors perform Boolean comparisons between two sets. S == T: Return True if sets S and T have identical contents. S != T: Return True if sets S and T are not equivalent. S = T: Return True if set S is a subset of set T. S T: Return True if set S is a proper subset of set T. S = T: Return True if set S is a superset of set T. S T: Return True if set S is a proper superset of set T. S.isdisjoint(T): Return True if sets S and T have no common elements. Finally, there exists a variety of behaviors that either update an existing set, or compute a new set instance, based on classical set theory operations. S | T: Return a new set representing the union of sets S and T. S |= T: Update set S to be the union of S and set T. S T: Return a new set representing the intersection of sets S and T. S = T: Update set S to be the intersection of S and set T. S ˆ T: Return a new set representing the symmetric difference of sets S and T, that is, a set of elements that are in precisely one of S or T. S ˆ= T: Update set S to become the symmetric difference of itself and set T. S − T: Return a new set containing elements in S but not T. S −= T: Update set S to remove all common elements with set T.
- 470. 448 Chapter 10.Maps, Hash Tables, and Skip Lists 10.5.2 Python’s MutableSet Abstract Base Class To aid in the creation of user-defined set classes, Python’s collections module pro- vides a MutableSet abstract base class (just as it provides the MutableMapping ab- stract base class discussed in Section 10.1.3). The MutableSet base class provides concrete implementations for all methods described in Section 10.5.1, except for five core behaviors (add, discard, contains , len , and iter ) that must be implemented by any concrete subclass. This design is an example of what is known as the template method pattern, as the concrete methods of the MutableSet class rely on the presumed abstract methods that will subsequently be provided by a subclass. For the purpose of illustration, we examine algorithms for implementing several of the derived methods of the MutableSet base class. For example, to determine if one set is a proper subset of another, we must verify two conditions: a proper subset must have size strictly smaller than that of its superset, and each element of a subset must be contained in the superset. An implementation of the corresponding lt method based on this logic is given in Code Fragment 10.14. def lt (self, other): # supports syntax S T ”””Return true if this set is a proper subset of other.””” if len(self) = len(other): return False # proper subset must have strictly smaller size for e in self: if e not in other: return False # not a subset since element missing from other return True # success; all conditions are met Code Fragment 10.14: A possible implementation of the MutableSet. lt method, which tests if one set is a proper subset of another. As another example, we consider the computation of the union of two sets. The set ADT includes two forms for computing a union. The syntax S | T should produce a new set that has contents equal to the union of existing sets S and T. This operation is implemented through the special method or in Python. Another syntax, S |= T is used to update existing set S to become the union of itself and set T. Therefore, all elements of T that are not already contained in S should be added to S. We note that this “in-place” operation may be implemented more efficiently than if we were to rely on the first form, using the syntax S = S | T, in which identifier S is reassigned to a new set instance that represents the union. For convenience, Python’s built-in set class supports named version of these behaviors, with S.union(T) equivalent to S | T, and S.update(T) equivalent to S |= T (yet, those named versions are not formally provided by the MutableSet abstract base class).
- 471. 10.5. Sets, Multisets,and Multimaps 449 def or (self, other): # supports syntax S | T ”””Return a new set that is the union of two existing sets.””” result = type(self)( ) # create new instance of concrete class for e in self: result.add(e) for e in other: result.add(e) return result Code Fragment 10.15: An implementation of the MutableSet. or method, which computes the union of two existing sets. An implementation of the behavior that computes a new set as a union of two others is given in the form of the or special method, in Code Fragment 10.15. An important subtlety in this implementation is the instantiation of the resulting set. Since the MutableSet class is designed as an abstract base class, instances must belong to a concrete subclass. When computing the union of two such con- crete instances, the result should presumably be an instance of the same class as the operands. The function type(self) returns a reference to the actual class of the in- stance identified as self, and the subsequent parentheses in expression type(self)() call the default constructor for that class. In terms of efficiency, we analyze such set operations while letting n denote the size of S and m denote the size of set T for an operation such as S | T. If the concrete sets are implemented with hashing, the expected running time of the implementation in Code Fragment 10.15 is O(m + n), because it loops over both sets, performing constant-time operations in the form of a containment check and a possible insertion into the result. Our implementation of the in-place version of a union is given in Code Frag- ment 10.16, in the form of the ior special method that supports syntax S |= T. Notice that in this case, we do not create a new set instance, instead we modify and return the existing set, after updating its contents to reflect the union operation. The in-place version of the union has expected running time O(m) where m is the size of the second set, because we only have to loop through that second set. def ior (self, other): # supports syntax S |= T ”””Modify this set to be the union of itself an another set.””” for e in other: self.add(e) return self # technical requirement of in-place operator Code Fragment 10.16: An implementation of the MutableSet. ior method, which performs an in-place union of one set with another.
- 472. 450 Chapter 10.Maps, Hash Tables, and Skip Lists 10.5.3 Implementing Sets, Multisets, and Multimaps Sets Although sets and maps have very different public interfaces, they are really quite similar. A set is simply a map in which keys do not have associated values. Any data structure used to implement a map can be modified to implement the set ADT with similar performance guarantees. We could trivially adapt any map class by storing set elements as keys, and using None as an irrelevant value, but such an implementation is unnecessarily wasteful. An efficient set implementation should abandon the Item composite that we use in our MapBase class and instead store set elements directly in a data structure. Multisets The same element may occur several times in a multiset. All of the data structures we have seen can be reimplemented to allow for duplicates to appear as separate elements. However, another way to implement a multiset is by using a map in which the map key is a (distinct) element of the multiset, and the associated value is a count of the number of occurrences of that element within the multiset. In fact, that is essentially what we did in Section 10.1.2 when computing the frequency of words within a document. Python’s standard collections module includes a definition for a class named Counter that is in essence a multiset. Formally, the Counter class is a subclass of dict, with the expectation that values are integers, and with additional functionality like a most common(n) method that returns a list of the n most common elements. The standard iter reports each element only once (since those are formally the keys of the dictionary). There is another method named elements() that iterates through the multiset with each element being repeated according to its count. Multimaps Although there is no multimap in Python’s standard libraries, a common imple- mentation approach is to use a standard map in which the value associated with a key is itself a container class storing any number of associated values. We give an example of such a MultiMap class in Code Fragment 10.17. Our implementation uses the standard dict class as the map, and a list of values as a composite value in the dictionary. We have designed the class so that a different map implementation can easily be substituted by overriding the class-level MapType attribute at line 3.
- 473. 10.5. Sets, Multisets,and Multimaps 451 1 class MultiMap: 2 ”””A multimap class built upon use of an underlying map for storage.””” 3 MapType = dict # Map type; can be redefined by subclass 4 5 def init (self): 6 ”””Create a new empty multimap instance.””” 7 self. map = self. MapType( ) # create map instance for storage 8 self. n = 0 9 10 def iter (self): 11 ”””Iterate through all (k,v) pairs in multimap.””” 12 for k,secondary in self. map.items(): 13 for v in secondary: 14 yield (k,v) 15 16 def add(self, k, v): 17 ”””Add pair (k,v) to multimap.””” 18 container = self. map.setdefault(k, [ ]) # create empty list, if needed 19 container.append(v) 20 self. n += 1 21 22 def pop(self, k): 23 ”””Remove and return arbitrary (k,v) with key k (or raise KeyError).””” 24 secondary = self. map[k] # may raise KeyError 25 v = secondary.pop() 26 if len(secondary) == 0: 27 del self. map[k] # no pairs left 28 self. n −= 1 29 return (k, v) 30 31 def find(self, k): 32 ”””Return arbitrary (k,v) pair with given key (or raise KeyError).””” 33 secondary = self. map[k] # may raise KeyError 34 return (k, secondary[0]) 35 36 def find all(self, k): 37 ”””Generate iteration of all (k,v) pairs with given key.””” 38 secondary = self. map.get(k, [ ]) # empty list, by default 39 for v in secondary: 40 yield (k,v) Code Fragment 10.17: An implementation of a MultiMap using a dict for storage. The len method, which returns self. n, is omitted from this listing.
- 474. 452 Chapter 10.Maps, Hash Tables, and Skip Lists 10.6 Exercises For help with exercises, please visit the site, www.wiley.com/college/goodrich. Reinforcement R-10.1 Give a concrete implementation of the pop method in the context of the MutableMapping class, relying only on the five primary abstract methods of that class. R-10.2 Give a concrete implementation of the items() method in the context of the MutableMapping class, relying only on the five primary abstract meth- ods of that class. What would its running time be if directly applied to the UnsortedTableMap subclass? R-10.3 Give a concrete implementation of the items() method directly within the UnsortedTableMap class, ensuring that the entire iteration runs in O(n) time. R-10.4 What is the worst-case running time for inserting n key-value pairs into an initially empty map M that is implemented with the UnsortedTableMap class? R-10.5 Reimplement the UnsortedTableMap class from Section 10.1.5, using the PositionalList class from Section 7.4 rather than a Python list. R-10.6 Which of the hash table collision-handling schemes could tolerate a load factor above 1 and which could not? R-10.7 Our Position classes for lists and trees support the eq method so that two distinct position instances are considered equivalent if they refer to the same underlying node in a structure. For positions to be allowed as keys in a hash table, there must be a definition for the hash method that is consistent with this notion of equivalence. Provide such a hash method. R-10.8 What would be a good hash code for a vehicle identification number that is a string of numbers and letters of the form “9X9XX99X9XX999999,” where a “9” represents a digit and an “X” represents a letter? R-10.9 Draw the 11-entry hash table that results from using the hash function, h(i) = (3i+5) mod 11, to hash the keys 12, 44, 13, 88, 23, 94, 11, 39, 20, 16, and 5, assuming collisions are handled by chaining. R-10.10 What is the result of the previous exercise, assuming collisions are han- dled by linear probing? R-10.11 Show the result of Exercise R-10.9, assuming collisions are handled by quadratic probing, up to the point where the method fails.
- 475. 10.6. Exercises 453 R-10.12What is the result of Exercise R-10.9 when collisions are handled by dou- ble hashing using the secondary hash function h(k) = 7−(k mod 7)? R-10.13 What is the worst-case time for putting n entries in an initially empty hash table, with collisions resolved by chaining? What is the best case? R-10.14 Show the result of rehashing the hash table shown in Figure 10.6 into a table of size 19 using the new hash function h(k) = 3k mod 17. R-10.15 Our HashMapBase class maintains a load factor λ ≤ 0.5. Reimplement that class to allow the user to specify the maximum load, and adjust the concrete subclasses accordingly. R-10.16 Give a pseudo-code description of an insertion into a hash table that uses quadratic probing to resolve collisions, assuming we also use the trick of replacing deleted entries with a special “deactivated entry” object. R-10.17 Modify our ProbeHashMap to use quadratic probing. R-10.18 Explain why a hash table is not suited to implement a sorted map. R-10.19 Describe how a sorted list implemented as a doubly linked list could be used to implement the sorted map ADT. R-10.20 What is the worst-case asymptotic running time for performing n deletions from a SortedTableMap instance that initially contains 2n entries? R-10.21 Consider the following variant of the find index method from Code Frag- ment 10.8, in the context of the SortedTableMap class: def find index(self, k, low, high): if high low: return high + 1 else: mid = (low + high) // 2 if self. table[mid]. key k: return self. find index(k, mid + 1, high) else: return self. find index(k, low, mid − 1) Does this always produce the same result as the original version? Justify your answer. R-10.22 What is the expected running time of the methods for maintaining a max- ima set if we insert n pairs such that each pair has lower cost and perfor- mance than one before it? What is contained in the sorted map at the end of this series of operations? What if each pair had a lower cost and higher performance than the one before it? R-10.23 Draw an example skip list S that results from performing the following series of operations on the skip list shown in Figure 10.13: del S[38], S[48] = x , S[24] = y , del S[55]. Record your coin flips, as well.
- 476. 454 Chapter 10.Maps, Hash Tables, and Skip Lists R-10.24 Give a pseudo-code description of the delitem map operation when using a skip list. R-10.25 Give a concrete implementation of the pop method, in the context of a MutableSet abstract base class, that relies only on the five core set behav- iors described in Section 10.5.2. R-10.26 Give a concrete implementation of the isdisjoint method in the context of the MutableSet abstract base class, relying only on the five primary abstract methods of that class. Your algorithm should run in O(min(n,m)) where n and m denote the respective cardinalities of the two sets. R-10.27 What abstraction would you use to manage a database of friends’ birth- days in order to support efficient queries such as “find all friends whose birthday is today” and “find the friend who will be the next to celebrate a birthday”? Creativity C-10.28 On page 406 of Section 10.1.3, we give an implementation of the method setdefault as it might appear in the MutableMapping abstract base class. While that method accomplishes the goal in a general fashion, its effi- ciency is less than ideal. In particular, when the key is new, there will be a failed search due to the initial use of getitem , and then a subse- quent insertion via setitem . For a concrete implementation, such as the UnsortedTableMap, this is twice the work because a complete scan of the table will take place during the failed getitem , and then an- other complete scan of the table takes place due to the implementation of setitem . A better solution is for the UnsortedTableMap class to over- ride setdefault to provide a direct solution that performs a single search. Give such an implementation of UnsortedTableMap.setdefault. C-10.29 Repeat Exercise C-10.28 for the ProbeHashMap class. C-10.30 Repeat Exercise C-10.28 for the ChainHashMap class. C-10.31 For an ideal compression function, the capacity of the bucket array for a hash table should be a prime number. Therefore, we consider the problem of locating a prime number in a range [M,2M]. Implement a method for finding such a prime by using the sieve algorithm. In this algorithm, we allocate a 2M cell Boolean array A, such that cell i is associated with the integer i. We then initialize the array cells to all be “true” and we “mark off” all the cells that are multiples of 2, 3, 5, 7, and so on. This process can stop after it reaches a number larger than √ 2M. (Hint: Consider a bootstrapping method for finding the primes up to √ 2M.)
- 477. 10.6. Exercises 455 C-10.32Perform experiments on our ChainHashMap and ProbeHashMap classes to measure its efficiency using random key sets and varying limits on the load factor (see Exercise R-10.15). C-10.33 Our implementation of separate chaining in ChainHashMap conserves memory by representing empty buckets in the table as None, rather than as empty instances of a secondary structure. Because many of these buck- ets will hold a single item, a better optimization is to have those slots of the table directly reference the Item instance, and to reserve use of sec- ondary containers for buckets that have two or more items. Modify our implementation to provide this additional optimization. C-10.34 Computing a hash code can be expensive, especially for lengthy keys. In our hash table implementations, we compute the hash code when first in- serting an item, and recompute each item’s hash code each time we resize our table. Python’s dict class makes an interesting trade-off. The hash code is computed once, when an item is inserted, and the hash code is stored as an extra field of the item composite, so that it need not be recom- puted. Reimplement our HashTableBase class to use such an approach. C-10.35 Describe how to perform a removal from a hash table that uses linear probing to resolve collisions where we do not use a special marker to represent deleted elements. That is, we must rearrange the contents so that it appears that the removed entry was never inserted in the first place. C-10.36 The quadratic probing strategy has a clustering problem related to the way it looks for open slots. Namely, when a collision occurs at bucket h(k), it checks buckets A[(h(k)+i2) mod N], for i = 1,2,...,N −1. a. Show that i2 mod N will assume at most (N + 1)/2 distinct values, for N prime, as i ranges from 1 to N − 1. As a part of this justifica- tion, note that i2 mod N = (N −i)2 mod N for all i. b. A better strategy is to choose a prime N such that N mod 4 = 3 and then to check the buckets A[(h(k) ± i2) mod N] as i ranges from 1 to (N − 1)/2, alternating between plus and minus. Show that this alternate version is guaranteed to check every bucket in A. C-10.37 Refactor our ProbeHashMap design so that the sequence of secondary probes for collision resolution can be more easily customized. Demon- strate your new framework by providing separate concrete subclasses for linear probing and quadratic probing. C-10.38 Design a variation of binary search for performing the multimap opera- tion find all(k) implemented with a sorted search table that includes du- plicates, and show that it runs in time O(s+logn), where n is the number of elements in the dictionary and s is the number of items with given key k.
- 478. 456 Chapter 10.Maps, Hash Tables, and Skip Lists C-10.39 Although keys in a map are distinct, the binary search algorithm can be applied in a more general setting in which an array stores possibly duplica- tive elements in nondecreasing order. Consider the goal of identifying the index of the leftmost element with key greater than or equal to given k. Does the find index method as given in Code Fragment 10.8 guarantee such a result? Does the find index method as given in Exercise R-10.21 guarantee such a result? Justify your answers. C-10.40 Suppose we are given two sorted search tables S and T, each with n entries (with S and T being implemented with arrays). Describe an O(log2 n)- time algorithm for finding the kth smallest key in the union of the keys from S and T (assuming no duplicates). C-10.41 Give an O(logn)-time solution for the previous problem. C-10.42 Suppose that each row of an n×n array A consists of 1’s and 0’s such that, in any row of A, all the 1’s come before any 0’s in that row. Assuming A is already in memory, describe a method running in O(nlogn) time (not O(n2) time!) for counting the number of 1’s in A. C-10.43 Given a collection C of n cost-performance pairs (c, p), describe an algo- rithm for finding the maxima pairs of C in O(nlogn) time. C-10.44 Show that the methods above(p) and prev(p) are not actually needed to efficiently implement a map using a skip list. That is, we can imple- ment insertions and deletions in a skip list using a strictly top-down, scan- forward approach, without ever using the above or prev methods. (Hint: In the insertion algorithm, first repeatedly flip the coin to determine the level where you should start inserting the new entry.) C-10.45 Describe how to modify a skip-list representation so that index-based operations, such as retrieving the item at index j, can be performed in O(logn) expected time. C-10.46 For sets S and T, the syntax S ˆ T returns a new set that is the symmet- ric difference, that is, a set of elements that are in precisely one of S or T. This syntax is supported by the special xor method. Provide an implementation of that method in the context of the MutableSet abstract base class, relying only on the five primary abstract methods of that class. C-10.47 In the context of the MutableSet abstract base class, describe a concrete implementation of the and method, which supports the syntax S T for computing the intersection of two existing sets. C-10.48 An inverted file is a critical data structure for implementing a search en- gine or the index of a book. Given a document D, which can be viewed as an unordered, numbered list of words, an inverted file is an ordered list of words, L, such that, for each word w in L, we store the indices of the places in D where w appears. Design an efficient algorithm for construct- ing L from D.
- 479. 10.6. Exercises 457 C-10.49Python’s collections module provides an OrderedDict class that is unre- lated to our sorted map abstraction. An OrderedDict is a subclass of the standard hash-based dict class that retains the expected O(1) performance for the primary map operations, but that also guarantees that the iter method reports items of the map according to first-in, first-out (FIFO) order. That is, the key that has been in the dictionary the longest is re- ported first. (The order is unaffected when the value for an existing key is overwritten.) Describe an algorithmic approach for achieving such per- formance. Projects P-10.50 Perform a comparative analysis that studies the collision rates for various hash codes for character strings, such as various polynomial hash codes for different values of the parameter a. Use a hash table to determine collisions, but only count collisions where different strings map to the same hash code (not if they map to the same location in this hash table). Test these hash codes on text files found on the Internet. P-10.51 Perform a comparative analysis as in the previous exercise, but for 10-digit telephone numbers instead of character strings. P-10.52 Implement an OrderedDict class, as described in Exercise C-10.49, en- suring that the primary map operations run in O(1) expected time. P-10.53 Design a Python class that implements the skip-list data structure. Use this class to create a complete implementation of the sorted map ADT. P-10.54 Extend the previous project by providing a graphical animation of the skip-list operations. Visualize how entries move up the skip list during insertions and are linked out of the skip list during removals. Also, in a search operation, visualize the scan-forward and drop-down actions. P-10.55 Write a spell-checker class that stores a lexicon of words, W, in a Python set, and implements a method, check(s), which performs a spell check on the string s with respect to the set of words, W. If s is in W, then the call to check(s) returns a list containing only s, as it is assumed to be spelled correctly in this case. If s is not in W, then the call to check(s) returns a list of every word inW that might be a correct spelling of s. Your program should be able to handle all the common ways that s might be a misspelling of a word in W, including swapping adjacent characters in a word, inserting a single character in between two adjacent characters in a word, deleting a single character from a word, and replacing a character in a word with another character. For an extra challenge, consider phonetic substitutions as well.
- 480. 458 Chapter 10.Maps, Hash Tables, and Skip Lists Chapter Notes Hashing is a well-studied technique. The reader interested in further study is encouraged to explore the book by Knuth [65], as well as the book by Vitter and Chen [100]. Skip lists were introduced by Pugh [86]. Our analysis of skip lists is a simplification of a pre- sentation given by Motwani and Raghavan [80]. For a more in-depth analysis of skip lists, please see the various research papers on skip lists that have appeared in the data structures literature [59, 81, 84]. Exercise C-10.36 was contributed by James Lee.
- 481. Chapter 11 Search Trees Contents 11.1Binary Search Trees . . . . . . . . . . . . . . . . . . . . . . 460 11.1.1 Navigating a Binary Search Tree . . . . . . . . . . . . . . 461 11.1.2 Searches . . . . . . . . . . . . . . . . . . . . . . . . . . . 463 11.1.3 Insertions and Deletions . . . . . . . . . . . . . . . . . . . 465 11.1.4 Python Implementation . . . . . . . . . . . . . . . . . . . 468 11.1.5 Performance of a Binary Search Tree . . . . . . . . . . . . 473 11.2 Balanced Search Trees . . . . . . . . . . . . . . . . . . . . 475 11.2.1 Python Framework for Balancing Search Trees . . . . . . . 478 11.3 AVL Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . 481 11.3.1 Update Operations . . . . . . . . . . . . . . . . . . . . . 483 11.3.2 Python Implementation . . . . . . . . . . . . . . . . . . . 488 11.4 Splay Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 11.4.1 Splaying . . . . . . . . . . . . . . . . . . . . . . . . . . . 490 11.4.2 When to Splay . . . . . . . . . . . . . . . . . . . . . . . . 494 11.4.3 Python Implementation . . . . . . . . . . . . . . . . . . . 496 11.4.4 Amortized Analysis of Splaying . . . . . . . . . . . . . 497 11.5 (2,4) Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . 502 11.5.1 Multiway Search Trees . . . . . . . . . . . . . . . . . . . 502 11.5.2 (2,4)-Tree Operations . . . . . . . . . . . . . . . . . . . . 505 11.6 Red-Black Trees . . . . . . . . . . . . . . . . . . . . . . . . 512 11.6.1 Red-Black Tree Operations . . . . . . . . . . . . . . . . . 514 11.6.2 Python Implementation . . . . . . . . . . . . . . . . . . . 525 11.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 528
- 482. 460 Chapter 11.Search Trees 11.1 Binary Search Trees In Chapter 8 we introduced the tree data structure and demonstrated a variety of applications. One important use is as a search tree (as described on page 332). In this chapter, we use a search tree structure to efficiently implement a sorted map. The three most fundamental methods of a map M (see Section 10.1.1) are: M[k]: Return the value v associated with key k in map M, if one exists; otherwise raise a KeyError; implemented with getitem method. M[k] = v: Associate value v with key k in map M, replacing the existing value if the map already contains an item with key equal to k; implemented with setitem method. del M[k]: Remove from map M the item with key equal to k; if M has no such item, then raise a KeyError; implemented with delitem method. The sorted map ADT includes additional functionality (see Section 10.3), guar- anteeing that an iteration reports keys in sorted order, and supporting additional searches such as find gt(k) and find range(start, stop). Binary trees are an excellent data structure for storing items of a map, assuming we have an order relation defined on the keys. In this context, a binary search tree is a binary tree T with each position p storing a key-value pair (k,v) such that: • Keys stored in the left subtree of p are less than k. • Keys stored in the right subtree of p are greater than k. An example of such a binary search tree is given in Figure 11.1. As a matter of convenience, we will not diagram the values associated with keys in this chapter, since those values do not affect the placement of items within a search tree. 80 82 54 28 93 65 32 97 44 88 17 8 76 29 Figure 11.1: A binary search tree with integer keys. We omit the display of associ- ated values in this chapter, since they are not relevant to the order of items within a search tree.
- 483. 11.1. Binary SearchTrees 461 11.1.1 Navigating a Binary Search Tree We begin by demonstrating that a binary search tree hierarchically represents the sorted order of its keys. In particular, the structural property regarding the place- ment of keys within a binary search tree assures the following important conse- quence regarding an inorder traversal (Section 8.4.3) of the tree. Proposition 11.1: An inorder traversal of a binary search tree visits positions in increasing order of their keys. Justification: We prove this by induction on the size of a subtree. If a subtree has at most one item, its keys are trivially visited in order. More generally, an inorder traversal of a (sub)tree consists of a recursive traversal of the (possibly empty) left subtree, followed by a visit of the root, and then a recursive traversal of the (possibly empty) right subtree. By induction, a recursive inorder traversal of the left subtree will produce an iteration of the keys in that subtree in increasing order. Furthermore, by the binary search tree property, all keys in the left subtree have keys strictly smaller than that of the root. Therefore, visiting the root just after that subtree extends the increasing order of keys. Finally, by the search tree property, all keys in the right subtree are strictly greater than the root, and by induction, an inorder traversal of that subtree will visit those keys in increasing order. Since an inorder traversal can be executed in linear time, a consequence of this proposition is that we can produce a sorted iteration of the keys of a map in linear time, when represented as a binary search tree. Although an inorder traversal is typically expressed using a top-down recur- sion, we can provide nonrecursive descriptions of operations that allow more fine- grained navigation among the positions of a binary search relative to the order of their keys. Our generic binary tree ADT from Chapter 8 is defined as a positional structure, allowing direct navigation using methods such as parent(p), left(p), and right(p). With a binary search tree, we can provide additional navigation based on the natural order of the keys stored in the tree. In particular, we can support the following methods, akin to those provided by a PositionalList (Section 7.4.1). first(): Return the position containing the least key, or None if the tree is empty. last(): Return the position containing the greatest key, or None if empty tree. before(p): Return the position containing the greatest key that is less than that of position p (i.e., the position that would be visited immediately before p in an inorder traversal), or None if p is the first position. after(p): Return the position containing the least key that is greater than that of position p (i.e., the position that would be visited immediately after p in an inorder traversal), or None if p is the last position.
- 484. 462 Chapter 11.Search Trees The “first” position of a binary search tree can be located by starting a walk at the root and continuing to the left child, as long as a left child exists. By symmetry, the last position is reached by repeated steps rightward starting at the root. The successor of a position, after(p), is determined by the following algorithm. Algorithm after(p): if right(p) is not None then {successor is leftmost position in p’s right subtree} walk = right(p) while left(walk) is not None do walk = left(walk) return walk else {successor is nearest ancestor having p in its left subtree} walk = p ancestor = parent(walk) while ancestor is not None and walk == right(ancestor) do walk = ancestor ancestor = parent(walk) return ancestor Code Fragment 11.1: Computing the successor of a position in a binary search tree. The rationale for this process is based purely on the workings of an inorder traversal, given the correspondence of Proposition 11.1. If p has a right subtree, that right subtree is recursively traversed immediately after p is visited, and so the first position to be visited after p is the leftmost position within the right subtree. If p does not have a right subtree, then the flow of control of an inorder traversal returns to p’s parent. If p were in the right subtree of that parent, then the parent’s subtree traversal is complete and the flow of control progresses to its parent and so on. Once an ancestor is reached in which the recursion is returning from its left subtree, then that ancestor becomes the next position visited by the inorder traversal, and thus is the successor of p. Notice that the only case in which no such ancestor is found is when p was the rightmost (last) position of the full tree, in which case there is no successor. A symmetric algorithm can be defined to determine the predecessor of a po- sition, before(p). At this point, we note that the running time of single call to after(p) or before(p) is bounded by the height h of the full tree, because it is found after either a single downward walk or a single upward walk. While the worst-case running time is O(h), we note that either of these methods run in O(1) amortized time, in that series of n calls to after(p) starting at the first position will execute in a total of O(n) time. We leave a formal justification of this fact to Exercise C-11.34, but intuitively the upward and downward paths mimic steps of the inorder traversal (a related argument was made in the justification of Proposition 9.3).
- 485. 11.1. Binary SearchTrees 463 11.1.2 Searches The most important consequence of the structural property of a binary search tree is its namesake search algorithm. We can attempt to locate a particular key in a binary search tree by viewing it as a decision tree (recall Figure 8.7). In this case, the question asked at each position p is whether the desired key k is less than, equal to, or greater than the key stored at position p, which we denote as p.key(). If the answer is “less than,” then the search continues in the left subtree. If the answer is “equal,” then the search terminates successfully. If the answer is “greater than,” then the search continues in the right subtree. Finally, if we reach an empty subtree, then the search terminates unsuccessfully. (See Figure 11.2.) 80 82 54 28 93 65 32 97 44 88 17 8 76 29 82 54 28 93 65 32 97 44 88 17 8 80 76 29 (a) (b) Figure 11.2: (a) A successful search for key 65 in a binary search tree; (b) an unsuccessful search for key 68 that terminates because there is no subtree to the left of the key 76. We describe this approach in Code Fragment 11.2. If key k occurs in a subtree rooted at p, a call to TreeSearch(T, p, k) results in the position at which the key is found; in this case, the getitem map operation would return the associated value at that position. In the event of an unsuccessful search, the TreeSearch al- gorithm returns the final position explored on the search path (which we will later make use of when determining where to insert a new item in a search tree). Algorithm TreeSearch(T, p, k): if k == p.key() then return p {successful search} else if k p.key() and T.left(p) is not None then return TreeSearch(T, T.left(p), k) {recur on left subtree} else if k p.key() and T.right(p) is not None then return TreeSearch(T, T.right(p), k) {recur on right subtree} return p {unsuccessful search} Code Fragment 11.2: Recursive search in a binary search tree.
- 486. 464 Chapter 11.Search Trees Analysis of Binary Tree Searching The analysis of the worst-case running time of searching in a binary search tree T is simple. Algorithm TreeSearch is recursive and executes a constant number of primitive operations for each recursive call. Each recursive call of TreeSearch is made on a child of the previous position. That is, TreeSearch is called on the positions of a path of T that starts at the root and goes down one level at a time. Thus, the number of such positions is bounded by h+1, where h is the height of T. In other words, since we spend O(1) time per position encountered in the search, the overall search runs in O(h) time, where h is the height of the binary search tree T. (See Figure 11.3.) Tree T: Time per level Total time: Height h O(h) O(1) O(1) O(1) Figure 11.3: Illustrating the running time of searching in a binary search tree. The figure uses standard caricature of a binary search tree as a big triangle and a path from the root as a zig-zag line. In the context of the sorted map ADT, the search will be used as a subroutine for implementing the getitem method, as well as for the setitem and delitem methods, since each of these begins by trying to locate an existing item with a given key. To implement sorted map operations such as find lt and find gt, we will combine this search with traversal methods before and after. All of these operations will run in worst-case O(h) time for a tree with height h. We can use a variation of this technique to implement the find range method in time O(s+h), where s is the number of items reported (see Exercise C-11.34). Admittedly, the height h of T can be as large as the number of entries, n, but we expect that it is usually much smaller. Indeed, later in this chapter we show various strategies to maintain an upper bound of O(logn) on the height of a search tree T.
- 487. 11.1. Binary SearchTrees 465 11.1.3 Insertions and Deletions Algorithms for inserting or deleting entries of a binary search tree are fairly straight- forward, although not trivial. Insertion The map command M[k] = v, as supported by the setitem method, begins with a search for key k (assuming the map is nonempty). If found, that item’s existing value is reassigned. Otherwise, a node for the new item can be inserted into the underlying tree T in place of the empty subtree that was reached at the end of the failed search. The binary search tree property is sustained by that placement (note that it is placed exactly where a search would expect it). Pseudo-code for such a TreeInsert algorithm is given in in Code Fragment 11.3. Algorithm TreeInsert(T, k, v): Input: A search key k to be associated with value v p = TreeSearch(T,T.root(),k) if k == p.key() then Set p’s value to v else if k p.key() then add node with item (k,v) as left child of p else add node with item (k,v) as right child of p Code Fragment 11.3: Algorithm for inserting a key-value pair into a map that is represented as a binary search tree. An example of insertion into a binary search tree is shown in Figure 11.4. 82 54 28 93 65 32 97 44 88 17 8 80 76 29 68 54 28 93 65 32 97 44 88 17 8 80 76 82 29 (a) (b) Figure 11.4: Insertion of an item with key 68 into the search tree of Figure 11.2. Finding the position to insert is shown in (a), and the resulting tree is shown in (b).
- 488. 466 Chapter 11.Search Trees Deletion Deleting an item from a binary search tree T is a bit more complex than inserting a new item because the location of the deletion might be anywhere in the tree. (In contrast, insertions are always enacted at the bottom of a path.) To delete an item with key k, we begin by calling TreeSearch(T, T.root(), k) to find the position p of T storing an item with key equal to k. If the search is successful, we distinguish between two cases (of increasing difficulty): • If p has at most one child, the deletion of the node at position p is easily implemented. When introducing update methods for the LinkedBinaryTree class in Section 8.3.1, we declared a nonpublic utility, delete(p), that deletes a node at position p and replaces it with its child (if any), presuming that p has at most one child. That is precisely the desired behavior. It removes the item with key k from the map while maintaining all other ancestor-descendant relationships in the tree, thereby assuring the upkeep of the binary search tree property. (See Figure 11.5.) • If position p has two children, we cannot simply remove the node from T since this would create a “hole” and two orphaned children. Instead, we proceed as follows (see Figure 11.6): ◦ We locate position r containing the item having the greatest key that is strictly less than that of position p, that is, r = before(p) by the notation of Section 11.1.1. Because p has two children, its predecessor is the rightmost position of the left subtree of p. ◦ We use r’s item as a replacement for the one being deleted at position p. Because r has the immediately preceding key in the map, any items in p’s right subtree will have keys greater than r and any other items in p’s left subtree will have keys less than r. Therefore, the binary search tree property is satisfied after the replacement. ◦ Having used r’s as a replacement for p, we instead delete the node at position r from the tree. Fortunately, since r was located as the right- most position in a subtree, r does not have a right child. Therefore, its deletion can be performed using the first (and simpler) approach. As with searching and insertion, this algorithm for a deletion involves the traversal of a single path downward from the root, possibly moving an item between two positions of this path, and removing a node from that path and promoting its child. Therefore, it executes in time O(h) where h is the height of the tree.
- 489. 11.1. Binary SearchTrees 467 r 32 8 28 29 54 93 68 65 76 82 p 44 17 88 80 97 r 28 8 54 93 68 65 76 82 44 17 88 80 29 97 (a) (b) Figure 11.5: Deletion from the binary search tree of Figure 11.4b, where the item to delete (with key 32) is stored at a position p with one child r: (a) before the deletion; (b) after the deletion. r 28 8 54 93 68 76 82 44 17 88 80 29 65 p 97 r 68 97 28 8 54 93 76 44 17 82 29 65 p 80 (a) (b) Figure 11.6: Deletion from the binary search tree of Figure 11.5b, where the item to delete (with key 88) is stored at a position p with two children, and replaced by its predecessor r: (a) before the deletion; (b) after the deletion.
- 490. 468 Chapter 11.Search Trees 11.1.4 Python Implementation In Code Fragments 11.4 through 11.8 we define a TreeMap class that implements the sorted map ADT using a binary search tree. In fact, our implementation is more general. We support all of the standard map operations (Section 10.1.1), all ad- ditional sorted map operations (Section 10.3), and positional operations including first(), last(), find position(k), before(p), after(p), and delete(p). Our TreeMap class takes advantage of multiple inheritance for code reuse, inheriting from the LinkedBinaryTree class of Section 8.3.1 for our representation as a positional binary tree, and from the MapBase class from Code Fragment 10.2 of Section 10.1.4 to provide us with the key-value composite item and the concrete behaviors from the collections.MutableMapping abstract base class. We subclass the nested Position class to support more specific p.key() and p.value() accessors for our map, rather than the p.element() syntax inherited from the tree ADT. We define several nonpublic utilities, most notably a subtree search(p, k) method that corresponds to the TreeSearch algorithm of Code Fragment 11.2. That returns a position, ideally one that contains the key k, or otherwise the last position that is visited on the search path. We rely on the fact that the final position dur- ing an unsuccessful search is either the nearest key less than k or the nearest key greater than k. This search utility becomes the basis for the public find position(k) method, and also for internal use when searching, inserting, or deleting items from a map, as well as for the robust searches of the sorted map ADT. When making structural modifications to the tree, we rely on nonpublic update methods, such as add right, that are inherited from the LinkedBinaryTree class (see Section 8.3.1). It is important that these inherited methods remain nonpublic, as the search tree property could be violated through misuse of such operations. Finally, we note that our code is peppered with calls to presumed methods named rebalance insert, rebal