I
A
Creative
i
Approach
i
'
-\"\\'v
mVi
INTRODUCTION
TO ALGORITHMS
A Creative Approach
INTRODUCTION TO ALGORITHMS A
Creative
Approach
UDIMANBER University
of Arizona
\342\226\274\342\226\274
ADDISON-WESLEY
PUBLISHING
Menlo Reading, Massachusetts \342\200\242 Don Mills, Ontario \342\200\242 Wokingham,
Bonn
\342\200\242
Sydney
\342\200\242
Singapore
COMPANY Park, California England
\342\200\242
Tokyo
\342\200\242 New
York
\342\200\242 Amsterdam
\342\200\242 Madrid
\342\200\242 San Juan
Cataloging-in-PublicationData
of Congress
Library
Udi.
Manber,
to algorithms.
Introduction
Includes bibliographiesand index. 1. Data structures (Computer science)
2. Algorithms.
I. Title.
'
005.7 3
1989
QA76.9.D35M36
88-2186
0-201-12037-2
ISBN
by Addison-Wesley
Reproduced
from camera-ready
copysuppliedby
the
author.
programs and applications presented in this book have been included for their value. but are not guaranteed for any care, They have been tested with The publisher does not offer or warranties purpose. any representation, nor does it accept liabilities with to or the any respect programs applications. The
instructional
Reprinted
with
Copyright
All
rights
system,
corrections
\302\251 1989
No part
transmitted,
photocopying, recording, Printed in the United States EFGHIJ-DO-943210
Publishing Company
by Addison-Wesley
reserved.
or
1989
October,
in
of
this
any
publication
form
or otherwise, of America.
or
without
Inc.
be reproduced, stored in a retrieval means, electronic, mechanical, by any written of the publisher. permission prior may
Published
simultaneously
in Canada.
To my
parents Eva and Meshulam
PREFACE
book
grew
Like many
other
This
out of my teachers,
able to explain algorithms clearly. hard for some students to solve for them to but it is also hard
not being
with
frustrations
that not
I discovered
is it
only
seemed to me) simple problems by themselves, these two parts \342\200\224 the that are given to them. I believe that understand the solutions \342\200\224 and should not be separated. are related and the explanation It is essential to creation in order to understand it fully. to It is not sufficient follow the steps leading to a solution look at the finished product. (what
book emphasizes the creative how to design a new
This
show
of
sequence
Instead,
'
the
'problem
but
rather
Our goals in
provides
depth. use
intellectual
looks more
problems
when
an algorithm is
not
they arise
as important
how
only in
the
as teaching
as finished
to solve future.
program,
principles.
algorithm's than
presentan
into a
to translate the
drawbacks,
The goal is to
so on.
particular
Teaching
the details
a
improvements,
turns),
wrong
in
so on.
X, the
\"problem these
overcoming
through a creativeprocess,rather are to show
like
and
P\\\"
of the
The
products. problems,
the
thinking
solution.
involved in creating algorithms, an process in this book. This is used designing algorithms for and it also known covers algorithms, many techniques designing in more an elegant intuitive for explaining framework the design of algorithms we do It does not, however, of designing and cover all possible ways algorithms, it exclusively. between the The heart of the methodology lies in an analogy of proving mathematical theorems by induction and that of designing process further
To
methodology
program
is to
purpose
described
are not P, program
(including possible
algorithm
algorithms
in designing
\"old-new\"
A\\
(although not always) its drawbacks, the difficulties
explained
teaching
how to solve new
involved
algorithm
better
a
thus
are
but also
A, algorithm
design. Its main
Algorithms
and relation to other methods and algorithms,\" it easier for a programmer not in a way that makes in a way makes that it easier to understand
algorithms
not
at
of algorithm
algorithm.
usually
algorithm,
first attempts algorithm
X,
sequence
straightforward
analysis,
side
reader
the
help
methodology
combinatorial
the
thinking
for
two processes serve different and purposes similar than they may appear to be. This has been observed by many people. The novelty of this book is the to analogy degree We show that the analogy which this analogy is exploited. encompasses many known in the process of algorithm creation. algorithm-design techniques,and helps considerably 1 and is introduced in Chapter The methodology is discussed briefly more in formally achieve
algorithms.
Although
different types of results, they
Chapter 5.
these are
more
vi
Preface
a
car, and
the following analogy.
Consider
to get
directions
want
to
your
Suppose that
you
at an unfamiliar
arrive
would be
You
hotel.
impatient
quite
rent
city,
were
if you
and so on. You of the city, its general layout, the traffic patterns, for two turn right, go of the form directions blocks, \"go straight would would outlook so on. and three However, miles,\" for change if you your straight around for a while You could for a time. that in to live probably get city long planned who those of the second form (if you find someone directions), directions gives you with the city. This book is not a source of but eventually you will need to know more about easy directions. It does contain explanations of how to solve many particular problems, methods. As a result, the is book is on general principles and the but emphasis I the well believe that extra effort is and involvement demands It thinking. challenging. told
the history
about
rather
have
worthwhile. nonnumeric The design of efficient diverse fields, including mathematics, This book can serve as an introduction
general.
is becoming important molecular biology, and
in
algorithms statistics,
and to
to algorithms
Many professionals, and even scientists
not
nonnumeric
deeply
many
engineering.
computations
involved
with
in
computers,
more than nonintellectual work. It grungy programming is nothing But such a belief may lead to straightforward, solutions, trivial, inefficient of to convince solutions exist. One this book is more efficient where goal elegant, is an elegant discipline, as well as an important one. readersthat algorithm design The bookis self-contained. The is mostly intuitive, and technicalities presentation are either kept to a minimum In particular, or are separated from the main discussion. from the algorithm-design ideas as much as implementation details are separated There are of that were possible. many algorithms designed especially to examples illustrate the principles in the book. The in this book is not material emphasized as to be series of mastered and memorized. It is as a presented something presented and so on. ideas, examples, counterexamples, modifications, improvements, Pseudocodesfor most the descriptions. Numerous exercises and algorithms are given following a discussionof further reading, with a relevant bibliography, follow In each chapter. most chapters, the exercises are divided into two drill exercises and creative classes, exercises. Drill exercises are meant to test the reader's of the specific understanding
believe
that
is.
sometimes
examplesand the reader's particular (those
ability
algorithms, whose
presented
algorithms
numbers
to
use
to solve
new
problems.
are underlined)
also include a summary of the main The book is organized as
material. Chapter 2 is
Creative exercisesare meant to test developed in that chapter, in addition to the Sketches of solutions to selected exercises
in that chapter.
the techniques
an
ideas follows.
introduction
are
at
given
the
end of
the
book.
The
chapters
introduced. Chapters
to
1
through
mathematical
4 present
induction.
introductory
Mathematical
to algorithm with is, as we will see, very important design. Experience induction proofs is thereforevery helpful. few computer-science students Unfortunately, 2 may be quite for some difficult get proofs. enough exposure to induction Chapter the more difficult examples at first reading, students. We suggestskipping and returning to them later. Chapter 3 is an introduction to the analysis of algorithms. It describes the and gives the basic tools one needs to be able to perform process of analyzing algorithms, induction
Preface of the algorithms
analysis
simple
Readers
the book. Chapter 4 is with basic data structures
presented in
vii
a brief introduction
and who have a from 5 is a good idea background directly Chapter (it always 5 the basic ideas behind the though). Chapter presents approach the analogy to induction It gives several of designing through algorithms proofs. and describes their creation. If you read only one chapter algorithms, examples of simple
structures.
data
to
basic mathematical to read the introduction
in
There are book
the
can start
read Chapter
book,
this
5.
ways to organize a book on to the subject of the algorithms,
basic
two
according
geometric algorithms. Even
Another
6
Chapters
organization.
way
of
the emphasis
though
familiar
are
who
is to
divide the
this
book
is on
through
9
present
for
book
graph algorithms,
example,
to design
according
design techniques, I have algorithms
techniques.
chosenthe
areas:
four
in
divide
way is to
One
algorithms.
former
for
algorithms
sequence comparisons, data compression),graph trees, shortest paths, matching), geometric algorithms (e.g., spanning (e.g., and numerical and matrix intersection problems), algebraic algorithms (e.g., Fourier I believe that this organization is clearer and fast transform).
sequences
sets
and
algorithms
convex hull, multiplication,
sorting,
(e.g.,
easier to follow. 10
Chapter
earlier
chapters,
to reductions. Although the subject is unique and important is devoted
chapter also serves as an opening of This of NP-completeness. aspect subject
own. This
of algorithm
part
theory.
and
completeness
the
algorithms.
of parallel
computation.
The material which leaves include
parts
book is
this
of Chapters
for the
more
parts
course,
of these
and can
appear
of its
a chapter
theory has become should know algorithms
the
with
deals
in
essential
an
NP-
about
this property. Chapter 12is an introduction different models interesting algorithms under
than
instructor.
3, 5, 6, 7, and
advanced
for a first
several
contains
choices
many
them. The more are optional
in
It
enough to warrant to Chapter 11, which
complexity
designs
for proving
techniques
to parallel
who
Anyone
act
of reductions
examples
8 in
can A
be covered first
some depth,
chapters, along
not
although
with
be used as a basisfor
a one-semester
in
in algorithm
course
Chapters a more
course,
design
should
all of
necessarily 9, 10,
advanced
11, and course.
12,
Acknowledgments and
First
here
foremost
on which
methodology \342\200\224
more
I thank this
throughout
important
my wife Rachel for helping She was instrumental
adventure.
the book is based. Shecontributed else \342\200\224 sound advice.
than anything
me
in more in
the
suggestions,
I could
ways
than
corrections,
not have
list
I can
development
done it
of
the
and without
her.
thanks are due to Jan van Leeuwen for an excellent and thorough of review of this book. His detailed comments, numerous and suggestions, many the book enormously. I also thank corrections have improved Eric Bach, Darrah Chavey, Kirk read and Sun Wu, who and made Pruhs, parts of the manuscript many helpful T. Almes H. Chan comments, and the reviewers (Rice Guy University), Agnes Dan Gusfield of David Harel California, (Northeastern University), Davis), (University of Israel), Daniel Hirschberg (University California, (Weizmann Institute, Irvine), Special a large portion
Preface
viii
of Iowa), of Washington), Victor Klee (University H. Kingston (University of of New Michael J. Martel California, Davis), (University Quinn (University Charles M. Diane Madison and ( James Spresser University). Hampshire), who failed to supply I thank the people at Addison-Wesley pie with any examples authors are so fond of that were stories of horror very helpful and telling. They In I and understanding. thank particular, my production supervisor patient incredibly
Jefferey
Bette Aaronson, my editor guided me the
Tektronix,
Award,
a Linotronic
300
Jim
do
DeWolf,
and AT&T,
for Digital
and way
my copy editor even when they
financial
Lyn
Hewlett
Corporation,
Equipment
who
not only
thank
I also
better.
a Presidential
through
support,
Dupr6,
knew
Young
Packard,
and
funds.
designed at
my
things
Foundation
for matching
The book was on
let me
Science
National
Investigator
also
but
the
typeset by me. It of Computer Department and
was
formatted
Science,
University
in troff, and of
printed
Arizona.
I
Ralph Griswold for his advice, and John Luiten, Allen Peckham, and Andrey \342\200\22 the typesetting. The figures were preparedwith gremlin for technical help with Yeatts for Fig. 12.22, which at the University of California,Berkeley\342\200\224 was except developed was drawn Townsend. The index and with the of a by Gregg compiled designed help I thank and Brian for me the Kernighan [1988]. Kernighan system by Bentley supplying within minutes after I (indirectly) requestedit. The cover code was done by Marshall Henrichs, basedon an idea by the author. I must was prepared by the typesetter. stress, however, that the final manuscript He was the one who decided to overlook many comments and suggestions of the people listed here. And he is the one who should bear the consequences. thank
Tucson,
Udi Manber
Arizona (Internet
address:
[email protected].)
CONTENTS
Chapter 1
Introduction
Chapter2
Mathematical
l
2.1
Introduction
2.2
Three
2.3 2.4 2.5
9
Counting Regions in the Plane A Simple Coloring Problem A
More
A
Simple
2.7
Euler's
2.8
A
2.9
Gray Codes
2.10
Finding
2.11
Arithmetic
Summation
Complicated
Problem
16 17
Inequality
Formula in Graph
Problem
Paths Geometric
versus
18 20 23
Theory
Edge-Disjoint
2.12 Loop Invariants:
2.14
11 13 14 15
Examples
Simple
2.6
2.13
9
Induction
Converting
in a Graph Mean Theorem a Decimal
Number to
24
26
Binary
28
Errors
Common
29
Summary
Bibliographic Notes and Further
30
Reading
31
Exercises
Chapter
3
Analysis
3.1
Introduction
3.2
The
3.3 3.4 3.5
3.6 3.7
37
of Algorithms
0 Notation
Time and SpaceComplexity Summations
Recurrence
Relations
3.5.1
Intelligent
3.5.2
Divide and
3.5.3
Recurrence
Guesses
Conquer Relations Relations
with
Full History
Useful Facts Summary
Bibliographic
Exercises
Notes
and Further Reading
37 39 42 43 46 47 50
51 53 55 55 56
IX
x
Contents
Chapter
4
61
Data Structures
4.1 4.2
4.3
61
Introduction
4.2.1
Elements
4.2.2
Arrays
4.2.3
Records
4.2.4
Linked
Lists
Trees
of Trees
4.3.1
Representation
4.3.2
Heaps
4.3.3
Binary Search Trees
4.3.4
AVL
71
Trees
4.4
Hashing
4.5
The Union-Find
4.6 4.7
62 62 63 63 64 66 67 68
Structures
Data
Elementary
75
78
Problem
80
83
Graphs
84
Summary
85
and Further Reading
Notes
Bibliographic
86
Exercises
Chapter 5 Designof Algorithms 5.1
91
Induction
by
91 92 95 96 98
Introduction
5.2
Evaluating
5.3
Maximal InducedSubgraph
5.4
Finding
5.5
The Celebrity
5.6
A Divide-and-Conquer
5.7 5.8
Computing BalanceFactors
Polynomials One-to-One
the
Finding
Mappings
Problem
Strengthening
5.10 5.11
Dynamic Programming: Common Errors
5.12
Summary
the
Induction
Skyline
102 104
Problem
Trees
in Binary
Consecutive
Maximum
5.9
The
Algorithm:
106
Subsequence
107
Hypothesis
The
Problem
Knapsack
108
111
112
Bibliographic
113
and Further Reading
Notes
114
Exercises
Chapter 6
6.1 6.2 6.3
6.4
and
Sequences
Involving
Algorithms
Sets
119
119
Introduction
120 125 127 127
Search and Variations Binary Search Interpolation Sorting
Sort and
6.4.1
Bucket
6.4.2
Insertion
6.4.3
Mergesort
Sort
and
Radix
Sort
Selection
Sort
130
130
xi
Contents
6.4.4
131
Quicksort
137
6.4.5 Heapsort
6.5
6.4.6
A
Order
Statistics
141
for Sorting
Bound
Lower
143
6.5.1
Maximum
6.5.2
Finding
Minimum
Elements
143
fcth-Smallest
Element
144
and
the
6.6
Data Compression
145
6.7
String Matching
148
6.8
SequenceComparisons
155
6.9
Probabilistic
158
Algorithms
6.9.1
Random
6.9.2
A Coloring Problem
6.9.3
A
6.11
6.12
161 Probabilistic
for Transforming
Technique
into
Algorithms
6.10
160
Numbers
Deterministic
161
Algorithms
164
a Majority
Finding
Three ProblemsExhibiting 6.11.1 LongestIncreasing 6.11.2
Finding
6.11.3
Computing the
Interesting
167
Techniques
167
Subsequence
Two
the
Proof
Largest
Elements
of a
Mode
in
169
a Set
171
Multiset
173
Summary
Notes and
Bibliographic
Further
173
Reading
175
Exercises
Chapter 7
Graph Algorithms
185
7.1
Introduction
185
7.2
Eulerian
187
7.3
Graph Traversal
Graphs
189
7.3.1 Depth-First 7.3.2
7.4
190
Search
198
Breadth-First Search
199
Sorting
Topological
7.6
Single-Source Shortest Paths Minimum-Cost Spanning Trees
7.7
All
7.8
Transitive Closure
7.5
7.9
Shortest
7.9.2 7.9.3
7.10
208 212
Paths
of
Decompositions
7.9.1
201
214 217
Graphs
217
Biconnected Components Connected
Strongly
Examples of the
Use
Components of Graph
226
Decomposition
7.10.1
Perfect
7.10.2
Bipartite
Matching Matching
7.11
Network Flows
7.12
Hamiltonian
7.12.1
230 234
Matching
Very
Dense
Graphs
.234 235
238 243
Tours
Reversed
in
Induction
244
xii
Contents
7.12.2
7.13
Hamiltonian
Finding
Cycles
in Very
Dense Graphs
Summary
and Further Reading
Notes
Bibliographic
Exercises
Chapter 8 GeometricAlgorithms 8.1
Introduction
8.2
Determining
8.3
Constructing
8.4
Convex
8.5
265
a Point
Whether
265 266 270
Is Insidea Polygon
Polygons
Simple
273
Hulls
8.4.1
A Straightforward
8.4.2
Gift
8.4.3
Graham's
Approach
273
274
Wrapping
Scan
275
Closest Pair
278
8.6 Intersectionsof
8.7
and Vertical
Horizontal
281
Line Segments
285 286 287
Summary
and Further Reading
Notes
Bibliographic
Exercises
Chapter 9
Numeric
and
Algebraic
293
Algorithms
9.1
Introduction
293
9.2
Exponentiation
294
9.3 9.4
Euclid's Algorithm Polynomial Multiplication
297
Matrix
Multiplication
301
9.5
9.6 9.7
298
Algorithm
301
Algorithm
301
9.5.1
Winograd's
9.5.2
Strassen's
9.5.3
Boolean Matrices
304
Transform
309
The
Fourier
Fast
316
Summary
Bibliographic Notesand
Further
316
Reading
317
Exercises
Chapter
244 246 247 248
321
10 Reductions
10.1
Introduction
10.2
Examples
10.2.1
10.3
321
323
of Reductions A
Simple
String-Matching
Problem
10.2.2 Systems of Distinct Representatives A Reduction Involving 10.2.3 Sequence Comparisons 10.2.4 Finding a Triangle in Undirected Graphs Reductions Involving Linear Programming 10.3.1 Introduction and Definitions of Reductions to Linear Programming 10.3.2 Examples
323
323
324
325
327
327
329
Contents
10.4
10.4.1 A
for Finding
Bound
Lower
10.4.2
Reductions
Simple
331
Bounds
for Lower
Reductions
331
Simple Polygons
333
Matrices
Involving
10.5 Common Errors
334
10.6
336
Summary
Notes and
Bibliographic
Further
336
Reading
337
Exercises
Chapter 11
341
NP-Completeness
11.1
Introduction
11.2
Polynomial-Time
11.3
Nondeterminism
341
342
Reductions Cook's
and
11.4 Examplesof
Theorem
344
Proofs
347
NP-Completeness
11.4.1 Vertex Cover
348
11.4.2 Dominating
348
11.4.3
Set
350
3SAT
351
11.4.4 Clique
11.4.5
352
3-Coloring
11.5
11.4.6
General
11.4.7
More
355
Observations
356
Problems
NP-Complete
Problems Techniques For Dealingwith NP-Complete 11.5.1 Backtracking and Branch-and-Bound
11.5.2 Approximation
with
Algorithms
357
358
Guaranteed
363
Performance
11.6
368
Summary
Notes and
Bibliographic
Further
368
Reading
370
Exercises
Chapter 12
xiii
Parallel Algorithms 12.1
Introduction
12.2
Models
12.3
375
375 of Parallel for
Algorithms
376
Computation
378 379 380 382
Machines
Shared-Memory
12.3.1 Parallel Addition 12.3.2
Algorithms
Maximum-Finding
12.3.3 The Parallel-PrefixProblem
12.4
Ranks
12.3.4
Finding
12.3.5
The Euler's
in
for Interconnection
Algorithms
Lists
Linked
385
Tour Technique
12.4.1
Sorting
on
12.4.2
Sorting
Networks
12.4.3
Finding the
12.4.4
Matrix
12.4.5
Routing
387
Networks
389
390
an Array
Multiplication in a
393
fcth-SmallestElement Hypercube
on the
Mesh
on
a Tree
396
398
401
xiv
Contents
12.5
SystolicComputation
404
Matrix-Vector
12.5.1
12.5.2 The Convolution 12.5.3
12.6
Sequence
404
Multiplication
405
Problem
407
Comparisons
409 409
Summary
Bibliographic Notesand
Further
Reading
411
Exercises
tions
to Selected
Exercises
417
References
445
Index
465
1
CHAPTER
INTRODUCTION
Great
of
has been
importance
necessary
and
For a
claim
condition
sufficient
sciences.
exact
of
no
Necessary,
be useful to serve as a
mental effort, for it things, it must first of all possess to more than see something
but
doubt,
to
construction
to this process to see in it the the progress of the
attached
rightly
some
and
\"construction,\"
and
not mere
not
to higher
stepping-stone
a kind the
sufficient! waste of
enabling us juxtaposition of its of unity
elements. Henri
The Webster's
solving a
of steps
number
New
Ninth
mathematical
by-stepprocedure
starting
fires,
as \"a procedure for dictionary defines an algorithm of the common divisor) in a finite (as finding greatest involves or broadly, a stepa repetition of an operation;
frequently for
building
a problem or accomplishingsome end.\" We of algorithms is thus an old field of study.
solving
the broad definition. The always been interested in
1902
Collegiate problem
that
Poincare,
design
pyramids,
methods
better
finding
or
the
sorting
to achieve
mail.
their
The study
goals,
stick
will People
whether
to
have
those be
of computer algorithms
is
new. Some computer algorithms use methods developed before the invention of computers, but most problemsrequire new For one thing, it is not enough approaches. to tell a computer to \"look over the hill and sound the alarm if an army is advancing.\" A computer must know the exact meaning of \"look,\" how to identify an army, and how of course
the alarm (for receives its instructions process to translate regular
to sound
some reason, via
instructions
necessary process, called programming, millions
of people.
sounding
well-defined,
an alarm
limited
primitive
to a language that is now performed
is
always
operations. a computer
on one
easy).
A computer It is a difficult
understands.
level or
another
This by
Introduction
2
understood
instructions
to devise totally
to a
\"talk\"
what
possible.
with
dozens,
perform
of
used to
We are
counterintuitive.
of
millions
this order
learning
magnitude
is a tendency when result, and feel. that works very well approach straightforward that work well for small problemsmay algorithms and inefficiency of the complexity easy to lose sight
in
designing
terrible
be
of
need
weird
is knowing
it
performed was
than
by
ever
of items, and, at or even trillions, instructions per primitive) is something new. It is in terms of things we can see an algorithm to use the
small
for
more
the
hundreds
with
(their
thinking
there
a
As
cases, we
to program;
much
maybe
deal
can
Computers
on
algorithms
Designing
respects
many
instructions.
and can
information,
of
bits
second.
of
thousands
past dealt
the
of
Algorithms
with
most,
not just
hard
well-
translating
just
In most
that were previously
operations
only
it
than
is
speed, computers can do
enormous
their
with
that makes
computer
to say. Computers execute not
humans;
a problem. It
for solving
methods
new
in which we
language
of
however, requires more computer, a computer can understand. to a language
a
Programming
an
billions,
problems. Unfortunately, for large problems. It is algorithm when applied to
large-scalecomputations. too
not
to expend
a
efficient and
of
the problem
the
algorithm.
perfect
grocery it,
we perform
algorithms
It is
too often.
performed
unpacking
is organized.
kitchen
The
problem. not
ways of doing
efficient
more
the
way
to develop
effort
of
lot
consider
example,
the
aspect to this and are complicated
is another
There
life are
depending
in
our
daily
worthwhile
The payoff is too small. For There are obviously less bags. on the contents of the bags and
Few people spend time for
usually
not
even
thinking
about
this
other
hand, people who do developing algorithms commercial must and large-scale develop good methods. Another packing unpacking is the lawn. We can the the number of example mowing improve mowing by minimizing the total time for or the of the to the cans. turns, mowing, trips Again, length garbage unless one really hates the lawn, one would not spend out how an hour figuring mowing to save a minute of on the other can deal with hand, mowing. Computers, very have to perform those tasks many times. It is tasks, and they complicated may worthwhile to spend a lot of time designing better methods, even if the resulting to understand. The potential of a payoff is algorithms are more complicatedand harder much greater. (Of course, we should not overoptimize, of hours spending programming time to save overall a few seconds of computer time.)
problem,
These algorithms
learning always
less
much
and
\342\200\224 the
issues
two
need
this subject. the best. It is
for
complexities of First, we must realize
the possible
important
of course, to learn new methods for algorithm design. need
to continue methods.
it.
the
On
approaches to large-scale
counterintuitive
these algorithms that
\342\200\224
straightforward
the search for better This
book
surveys
point
difficulties in methods are not
to the
intuitive
methods. To do that, and
illustrates
we
numerous
But it is not enough of to learn even a large number to memorize many of chess in order to be a good games the principles behind the methods. One must know how to player. One must understand when to apply them. apply them and, more important, of an algorithm A and implementation is analogous to a design and design methods,
just as
it
is not
enough
3
Introduction
house.
of a
construction
the basic
with
start
We
concepts, based on
the
for
requirements
architect's job to present a plan that satisfies the requirements. It is (so that the house will job to make sure that the plan is feasible and correct after a short It is then the builder's not collapse based while). job to construct the house on these plans. Of course,all along the way, the costs associatedwith each step must be into account. Each job is different, but analyzed and taken they are all related and A design of an algorithm also starts with the basic ideas and methods. intertwined. a is made. We must the correctness of the and make sure that its Then, plan prove plan is to implement the algorithm for a particular computer. cost is effective. The last step we can divide the process into four steps: design, proof of oversimplification, Risking and implementation. each of these steps is different, but correctness, analysis, Again, in are all related. None of them can be made a without a to the vacuum, they regard in these linear in all order. Difficulties arise One rarely goes through others. steps
the house.
It
is the
the engineer's
in which They usually require modifications to the design, phases of the construction. another of and of turn require costs, feasibility proof, adjustment change implementation. This book concentrates on the first step, the design of algorithms. our Following been entitled The Architecture of Algorithms. However, analogy, the book could have has a different so using this term would be confusing. architecture meaning, computer A all the other discussion of The book does not, however, correctness, aspects. ignore \342\200\224 in detail the description of most algorithms for follows analysis, and implementation for others. The emphasis is on methods of design. some algorithms, briefly to be a good architect and to be able to to learn many algorithms It is not enough behind the design. We One must understand the principles design new algorithms. of explaining employ a different way algorithms in this book. First, we try to lead the we strongly believe that the best way to learn how reader to find his or her own solution; to create something is to try to create it. and more important, we follow a Second,
for
methodology
The methodology, process. algorithms that helps this creative the intuitive framework for explaining an [1988], provides elegant the to approach in more It also provides a unified depth. way and their methods that are encompassed by this methodology,
designing
in Manber
introduced
of algorithms
design
design. The different numerous
variations,
are
among those many
possible
for a
however,
great
analogy between
the
possible
of the
majority
The methodology
and applying them
methods
not cover all
does
methodology
mathematical
process
of
ways
algorithms
is based on
intellectual
of choosing same technique. The process becomes more methodical.This
of the
instances
in
this
designing book.
induction.
of proving
algorithms. The
heart of
It is it
lies
useful,
in an
mathematical theorems and that in the principle of mathematical
combinatorial algorithms. The main idea statement need not be proven from scratch: It is sufficient to show the correctness of the statement follows from the correctness of the same statement smaller instances and the correctness of the statement for a small base case. Translating this to algorithm design suggestsan approach that concentrates on extending principle
designing induction
1
The
two
(Houghton
is that a
wonderful Mifflin,
books 1985),
by Tracy Kidder, inspired this analogy.
The Soul of a New
Machine
(Little
Brown, 1981),
and
of
that
for
House
4
Introduction
Given a problem, if we can of large problems. of small problems to solutions we for smaller inputs, then of the same a solution it by using solve to how problem show a solution on on rather than is to concentrate idea basic The building extending are done. of doing As we will show in the following chapters, there are many ways it from scratch. design techniques. this, leading to many algorithm as a tool for explaining and induction We use mathematical designing mainly to formalize or axiomizethe approach. make little We attempt algorithms. high-level solutions
Gries done by several people, including Dijkstra [1976], Manna [1980], others. This and Paull book Dershowitz [1988], [1983], among complements [1981], but of course whenever something Our goal is mainly books. other these pedagogical, we better. is understood it better can be explained Among the proof techniques usually
has been
This
the induction discuss are strengthening sequence hypothesis, choosing the induction is The significance of our approach and reverse induction. wisely, double induction, one of algorithm design under different two-fold. First, we collect seemingly techniques known mathematical second, we utilize umbrella; proof techniques for algorithm design. it opens the door to the of powerful since use The latter is especially important, in for another have been that developed discipline. many years techniques One notable weaknessof this approach is that it is not a universal approach. Not the principle be designed with induction in mind. can or should all algorithms However, in it is worthwhile to is so prevalent the of algorithms that of induction design concentrate on it. The other principles are not ignored in this book. A common criticism of almost any new methodology is that, although an interesting way to it may present that were is in This is a valid it of no them. created, explain things help already creating since only the future will is and how tell how effective a certain methodology criticism, used it becomes. I is another tool for believe that induction not widely strongly only just is in but it order to understand them. explaining algorithms, necessary Personally, even I
though
methodology, algorithms
a good
had
I found more
experience it
quickly
and,
helpful,
(Manber
and
in
algorithms
developing
at least McVoy
in
two
[1988],
cases,
Manber
without
following
this
me to develop new and Myers [1989]).
it led
Notation for Describing Algorithms In
addition
development, including
to we
the describing also include
programs
is to
algorithms
pseudocodes for
enhance
optimize the programs, and we do we made a consciousdecisionnot it introduces
the
through many
creative algorithms.
process The
of
purpose
their
of
made a great effort to descriptions. not recommend simply copying them. In some cases, to include the most optimized version of the program,
the
We
have not
additional which distracts from the main ideas of the complexity, in detail how we translate the algorithmic ideas algorithm. We sometimesdo not explain into a program. Such translations The sometimes are obvious and sometimes are not. in this book, as we mentioned, is on the principles of algorithm emphasis design. For the most part, we use a Pascal-like In language (sometimes even pure Pascal). into a table,\" or \"check as \"insert many cases, we include high-level descriptions(such the set is empty\") inside a Pascal code to make whether it more readable. One notable exception we make to the rules of Pascal is the use of begin and end to encompass
because
Exercises 5 at the beginning and end of the programs, and This convention saves without separate space causing We usually do not include declarations of variables and data types precise ambiguities. in cases where such declarations are clear (e.g.,we may say that G is a graph, or that T is
let the
these statements
include
We
blocks.
only
blocks.
the
indentation
a tree).
Exercises whose Exercises
are underlined have
numbers
by a star
marked
are judged
by the
solutions at
author to be substantially
the back more
of the
difficult
book. Exercisesthat are than other exercises.
The exercisesin
this chapter do not require any previous knowledge of algorithms. They address for specific relatively simple problems inputs. The reader is asked to find the answers by hand. is to illustrate in dealing with exercises the difficulty a very large The main purpose of these In other words, one of the goals of these exercises is to cause frustration number of possibilities. here will be discussed in the following with straightforward methods. The problems given
chapters.
1.1
Write
1.2
Write about
the numbers
down
them in
1
100 each
to
on a
separate
card.
Shuffle
the cards
and
rearrange
order again.
down the following 100 numbers each on a separate card 1.1. between this exercise and Exercise
and
sort
the cards.
Think
the differences
11 72 971 8234 2223849283 3295 32918 21192 11923423388231 8312 29347 3102 32883 20938293016823 9234 9236 29372 2218 9222 21202 2831 83721 9238 8221 30234 93920 81102 101118152 29133 9229 10039 9235 48395283237927 73492 8402 48201 38024 2800 321552273 82930 2221 3841 311 30223809929920 74212 7011 1823 903 29919335 28349 29123 28910 29281 3772 20012 7045830572 38013 72032 28001 83835 3017 9262673825 29263 2017 262 8362 77302 85933826 9374 2001 444 83261 48402 4845 79794 27271 3999222836 2937 37201 37322
49472 113292253 1.3
the following list of numbers. of those numbers as Your job is to erase as few that the remaining numbers appear in increasing order. For example,erasing everything except the first two numbers leaves an increasing sequence; erasing everything for first, third, sixth, and eighth does the same (but fewer numbers are numbers, except
Consider
possible
such
erased).
9 44 32
12 7 42 34 92 35 37 41 8 20 27 83 6461 28 39 93 29 17 13 1455 72 23 73 99 1 2 88 77 3 65 83 84 62 5 11 74 68 76 78 67 75 69 70 22 25 26
21 66
7124 1 -4
Solve Exercise
1-5
Suppose
that
29, 41, and and
8 cents
1.3, such
in a 67
strange
the remaining
that
Find
(1808 cents).You
a combination
have
decreasing
order.
are five types of coins with denominations of these coins to pay the sum enough coins of each type in your pocket.
there
country
(all cents).
numbers are in
of 15, 23, of 18 dollars
Introduction
The
list of
is a
input
waiting
it
particular, k will
answer
cannot
problem is to (any
of integers
pairs
an answer from
for
y.
any questions
a sequence
find
do). If such a
since everyone
is waiting
You can use a
pencil
of pairs
(You may draw figure. general method of solving
of
a piece
such
1
proceed,
any kind of computation, involving you cannot draw any kind of a a particular input, to help you design
make
and
paper,
creating
however,
tables);
unrelated to
figures,
some k >
can
else.
for someone and
x |), for
No one
is a deadlock.
there
for
waiting
x is
and, in it. The
else,
anything
be
\342\200\242 \342\200\242 \342\200\242, (**_, xk\\ (xk
x^),
(x | jc2), (x2
is that
of a pair (jt, y)
do
that may
others
from
sequenceexists, then
(e.g., comparisons,
numbers
given below. The meaning it cannot x is waiting,
When
this
a problem.)
24 1, 25 10,35
45, 36 37, 3842, 22, 29 2, 3048, 31 15,32 4,48 17,6 45,6 1,5 35,5 20,5 10,49 32,7 31,7 4, 5 33, 6 29,6 12, 6 11, 6 3, 6 17,45 27, 47 34, 48 20, 7 40, 7 34, 8 11, 9 19, II 30, 11 4, 11 22, 11 25,20 24, 21 23, 21 46, 22 47, 23 49, 3 39, 3 34, 4 14,4 37, 5 42, 5 8, 15 2, 15 50, 154, 1537, 16 13, 17 38, 18 28, 198, 26 15, 26 42, 27 18,28 35, 13 36, 13 50, 1334, 13 22, 29 34, 29 38,29 30, 29 16, 44 33, 44 36, 44 7, 44 3, 44 32,44 21, 33 9, 33 21, 33 35, 33 19, 33 41, 16, 2
I
39 41,
21, 2 25, 2 22,23 12 37, 1223, 123,
50,
23 47,
12 20,
9, 42 3,
14 25,41
7, 36
43 5,43
28,511,48
26 10,2644,2616,26 39,2617 two-dimensional 15 by 15 table given in Fig. 1.1. The /th row and the /th correspond to the same place. Each entry in the table indicates the direct between the places in the corresponding distance row and column. The \"-\" symbol indicates that there is no direct link between the two places. The direct distance may not be the shortest distance. There may be a shorter path between two places going through a third 1 and 6 is through 5 and the shortest route between place (or several places). For example, 12. Find the shortest route 1 and between 15, between 4 and 3, and between 15 and 8. input is the
The
column
Consider
(for any /)
the table
in
Fig.
1.1. Find the
shortest route
between
5 and
all
other
places.
Consider the graph shown in Fig. 1.2. Find a closed route along the edges of the graph which includes every vertex exactly of a once. (This graph corresponds to the edges Sir William R. dodecahedron; this puzzle was first described by the Irish mathematician Hamilton, and we discuss it further in Section 7.12.)
The
is a regular maze problem, with that the maze is given in the exception in a rectangle with 11 representation (rather than a picture). The maze is contained rows and columns, numbered the rows and from 0 to 10. The maze is traversed along columns \342\200\224 The starting point is 0,0 and the target is 10,10. The up, down, right, or left. are obstacles you cannot traverse through: following points following
numeric
(3,2) (6,6)(7,0) (2,8)(5,9) (8,4) (2,4) (0,8) (1,3) (6,3) (9,3)(1,9)(3,0) (3,7) (4,2) (7,8) (2,2) (4,5) (5,6)(10,5)(6,2) (4,0) (7,5) (7,9) (8,1) (5,7) (6,10)
(4,4)(8,7) a. b.
Find
a path from
Find
a shortest
obstacles.
(9,2)
(10,9) (2,6)
the starting point to the
target
path from the starting point
to
that does not include the
target
that does
any
not
of the include
obstacles. any
of the
Exercises
1
3 3
2
1
0
?
4
5
6
7
8
9
10
n
12
13
14
15
-
1
9
6
2
1
7
4
2
8
3
-
9 -
1
7
2
8
8
-
3
2
7
0
?
-
-
-
-
2
1
6
3
8
-
8
9
3
6
8
5
7
4
-
8
0 -
0
-
5
-
1
1
9
-
8
-
5
9
-
8
-
0
3
4 ?
7
5
8
-
1
-
4
2
6
2
8
7
3 2
-
-
-
8
1
1
-
9
4
-
9
10
-
-
-
3
8
7
12
3
-
1
2
8
13
-
3
1
6
14
7 2
9
6
-
15
2
9
2
1
11
3 2
-
1
6
0
5
3
2
-
8
7
2
-
8 2
-
0
6
2
-
8
8
2
-
4
3
8
1
1
-
2
7
-
3
-
-
-
1
9 8
0 2
-
-
-
-
2
0
4
9
7
1
0
3
-
-
-
2
3
8
-
-
0
2
9
2
1
1
1
-
5
1
9
0
2
-
9
-
-
2
-
3
-
9
0
2
-
7
-
9
-
3
-
1
1
9
0
-
-
-
1
-
4
3
6
5
1
-
0
Figure 1.1 The table
1.11
Find
the
of two
1.12
greatest
for Exercises
1.7 and 1.8.
common divisor of 225277and 178794. (The greatest common of that divides both number them.) largest
integers is the
Compute the
value
of 2M. Try
Figure
to find
1.2
a way
to minimize the number of multiplications.
Hamilton's puzzle.
divisor
7
8
Introduction
1.13
the number of electoral votes for each state in the 1988 candidate (the Presidential receiving the majority of the votes in a state collects all There are altogether 538 electoralvotes. Determine for that votes state). electoral the for the election to end up in a tie. (This problem is it is (mathematically) whether possible it is a special case of the known as the partition problem, and problem knapsack The
list represents
following
election
discussed
in
Section
5.10.)
Alabama
9
Alaska
3
Arizona
Arkansas
California
47
Colorado
Connecticut
6 8
Delaware
3
Florida
21
Georgia
12
Hawaii
4
Idaho
Illinois
24
Indiana
4 8
Kansas
7
Maine
4
Michigan
20
Minnesota
Missouri
11
Montana
Nevada
South Carolina
4 5 3 7 8
Texas
29
Utah
Virginia
12 Washington 6 Wisconsin
New Mexico Dakota
North
Oregon
West
Virginia
12
7 8
Iowa
9 10 10 4 4
New York
36
North
Ohio
23
Oklahoma
13 8
4
Kentucky
Maryland
New
Hampshire
Pennsylvania
South
Dakota
Louisiana
10
Massachusetts
13
Mississippi
7
Nebraska New Jersey
5
16
Carolina
25
Rhode Island
3
Tennessee
11
5
Vermont
3
10
Washington,
11
Wyoming
D.C.
3
3
2
CHAPTER
INDUCTION
MATHEMATICAL
one believes an hypothesis except its originator, but everyone believes an experiment except the experimenter.
No
Anon
is always the
Obviousness
enemy
of correctness.
Bertrand Russell (1872-1970)
Introduction
2.1
in the
see
will
We
In this
design.
The
examples.
proofs may
this
find
n
T be
value
whose
a theorem
can be
of proving
Instead
natural
any
directly
that
T
Readers
difficult.
algorithm through not
We claim
are
algorithms
constructing
role in
induction who have similar,
seen the
that
and
thus
helpful. powerful
technique.
proof
It usually works
as
to prove. Suppose that T includes a parameter is number natural number a (a positive integer). holds for all values of ny we prove the following two want
we
that
plays a major to mathematical
to be relatively hard.
chapter
of constructing proofs and with induction proofs is very experience Mathematical induction is a very processes
follows. Let
induction
chapter, we presenta brief introduction examples range from easy to quite
induction
many
that
chapters
following
conditions:
1. T 2.
holds for
For every
reason these that T holds for n
The
and
so on.
n
two
= 2.
n
= 1 > 1,
if T
conditions
If T
holds for n
- 1,
are sufficient
holds for
The induction principle
n
= 2,
then T holdsfor is clear.
Conditions1 and
then condition
itself is so basic that
n
2 implies that it is
usually
2 imply T holds
directly for n = 3,
not proved; rather,
it
9
Induction
Mathematical
10
definition of the natural numbers. 2 is easier in many condition cases Condition simple to prove. Proving - 1. n we can use the T for since that holds theorem the assumption directly, than proving some In we induction the induction the called sense, hypothesis. get This assumption is to one with to reduce the theorem value of ny smaller free. It is enough for hypothesis We concentrate on this reduction. start right scratch. Let's than proving it from rather
is stated as an
1
is usually
an example.
with
away
in the
axiom
Theorem \342\226\241
2.1
For all
numbers
natural
x and
-
n, xn
1
by x
is divisible
- 1.
is trivially induction on n. The theorem true for n = 1. We assume that the theorem is true for n 1; namely, we assume that xn~x - 1 is divisible by x. We now have to prove that xn -\\ is divisible by x- 1. numbers x- 1 for all natural The idea is to try to write the expression xn -\\ using xn~{ -1, which, by the induction - 1: x divisible is by hypothesis,
Proof: The
is by
proof
+ (x-\\).
xn-\\=x(xn-l-\\) But
just
jc
is divisible
left term
the
by
x
-
1
hypothesis, and
induction
the
by
the
term
right
\342\200\224 1.
The
If a
is
\342\226\2
induction
statement
P,
the truth of
is
principle with a n - 1
P for
as follows:
defined
thus
parameter implies
n,
for n =
is true
its truth
for
n,
then
\\,
and
P is
every
if for
true for
n
> 1,
all natural
numbers.
of using
Instead
n
-1
and
P,
with a
we \302\253,
use
sometimes
n and
/i
which
+ l,
is completely
equivalent:
If a statement the
of P
truth
for
n
parameter
implies
n,
its truth
for n =
is true
for
n
+ \\, then
\\,
and
P is
every n>\\,
if for true
for
all natural
numbers.
The proof of
Theorem 2.1 illustrates a simple application of induction. have been developed. For example, the many variations of induction called strong induction, is very common.
If a statement
P,
the truth of true
for
P for all
all natural
The differenceis that < n useful.
in
proving
Another
with a
parameter
natural
n,
numbers
is true
< n
for n =
\\,
implies
its
and truth
if for for
Over following
the years, variation,
every n > 1, n, then P is
numbers.
the assumption that the statement is true for all numbers for n. In many cases, this stronger assumption can be very simple variation is the following: we
can use
the statement
Three
2.2
If a
statement
a parameter
with
P,
the truth every n>2, all natural numbers.
for for
This variation step imply for all even
is true
for n =
implies
for all
numbers.
statement P, with a parameter n, is true for that n is an integer power of 2, the truth n, then P is true for all natural numbers that
If a such
for
This variation
1
n=2, and if, n, then P is true
and
truth
its
for
for
in two parallel tracks. The base casefor odd numbers; the base case for n =2 and the Another common variation is the following:
''works\" P
for
ofP
n,
n-2
the induction
induction
step
imply P
n>
\\,
1, and
of P
for n/2 implies its truth integer powers of 2.
if,
every
for
n as 2k, and carrying the parameter from k = 0). different can also be used in many Induction ways to prove propertiesof structures n that measures the other than numbers. In most cases, the induction is on some number which of the measure to the induction of the instance the size right problem. Finding should be applied is not straightforward. (For example, we could have appliedinduction much than to n\\ this would have made the proof to x in the previous example, rather is not natural, and it has to be invented more complicated.) Sometimes, this measure just The common thread to all these proofs is the extension for the purpose of the induction. for larger structures. of claims for smaller structures to claims
Three
2.2 The
+ 2
1
+
the
to
S(l)=l = l-(1+
the
of
the
first
n natural
first
is by
proof
We
l)/2.
numbers is n{n + \\)l2.
induction on n. If n-
now assume
that
the
sum
1, then of the
the first
is true
claim
(n +
because
numbers
n natural
l)/2, and prove that this assumption implies that the sum of the numbers is S(/i + 1)= (/i + 1)(/i+2)/2. We know from the definition + l) = S(\302\253) + \302\253 +1. S(\302\253 But, +1)/2, by the assumption, 5(\302\253)= \302\253(\302\253 + l)/2, which is exactly what S(/i + 1)= /i(/i+ 1)/2 + /i + 1 = (\302\253+2)(\302\253 n
numbers
natural
n
theorem.
following
2.2
The sum of the
Proof: The
for the sum
expression
We prove
\342\226\241 Theorem
is
the
find
writing
Examples
\342\200\242 \342\200\242 \342\200\242
+n.
by
k (starting
parameter
Simple is
problem
S(n)=
from
follows
induction for
out the
the first one
n = 1 and
n =
are
11
Simple Examples
n +
first
1
S(n) natural
and
S(n) that therefore
we
wanted
of
prove.
to
\342\226\
We
compute
continue the
sum
with a slightly more sum. Suppose complicated = $ + 13+ 18+ 23+ \342\200\242\342\200\242\342\200\242 sum +(3 + 5\302\253). The
T(n)
that
we
in the
want
to
previous
of the elements to n2l2 + nl2. Each in the current is example the corresponding element in the previous Hence, example. it is reasonable to guess that T(n) is also a quadratic expression. Let's try the implicit That the parameters c,, c2, and c3, and guess G(n) = cln2 + is, we introduce it is convenient we can determine the to do so. For example, determine their values when
example,S(\302\253), is slightly
more
than
equal f\\\\e
times
c2n+c3.
Induction
Mathematical
12
the first few terms. If n = parameters by checking two equations: n = 1 and n - 2, we get the following
(2) 4-c, + If we
We
expression.
verified + l)
We have
now
=
+ l)2
77^
series
5\302\253w fl/f/je
18 + 23+
end this
Ifn is a
+3
5.5\302\253+5.5
1).
We have to
example.
simple
1
+jc>
0, then
1
(2.1)
+ (n
prove that
+ 1)jc +
assumption
+\302\253jc for
(1 +
(by induction)
nx2
> 1 + (n
multiply
that
the
l+*>0.
+
n
= 1,
all x
jc)/,+i >
>
+JC)\302\253
were able to
clearly nonnegative.
induction on n. If
that (1 +jc)\"> 1
(1 +xy?+l = (i +JC)(i
that we
and
is by
proof
+ 1.
=
another
with
1+fljt.
assume
the
right
already to prove that
\342\226\2
We
because of
the
is
have
theorem.
number
natural
Proof: The
Notice
try
that
2.4
(\\+x)n>
case of n
we
and
+ 5(/i+l) +
+ 5\302\253 + 2.5 2.5\302\2532
We
implies
\342\200\242\342\200\242\342\200\242 + 5\302\253) +(3
section
\342\226\241 Theorem
+*.
For
+ 5.5\302\253.
is2.5\302\2532
1
=
2c, =5, which = 2.5n2+5.5n
induction.
by
C(/i)
induction)
+5.5(/i + 1)= C(/i+
following
2.3
8+ 13+
to
l) + 3 = (by
= 2.5(/i
Theorem \342\226\241
= T(n)
G(n) = T(n),
that
+ 5.5\302\253+5\302\253 + 8
the
guess
assume
We
(2), we get that G(n)
from
that G(n)
prove
= 2.5a?2
proved
We
therefore
to
5(/? +
7\302\273 +
it
subtract
We
try
a base case. = 7,(/i + l): + l)
7,(/?
0.
8
2 and
by
and c2 = 5.5.
Cj=2.5,
C(/?
= 13 +
2-c2 (1)
multiply
be
lc2 = 8
l-c, +
(1)
so c3 must
sum is 0,
0, the
1
then both
such
+ (n
+
1
that 1 )a\\
sides of (2.1)are and
+jc>0,
for all
x such
that
equal
consider 1+
the
x> 0:
(1 +jc)(1 +\302\253jc) 1 )jc.
inequality
(implied
The last step was
by
the
induction)
possible
because
by (1 +jc) nx2 is
\342\226\2
2.3
2.3
Counting
Regions in the
Counting
Regions
in the
Plane
13
Plane
if no two lines are parallel and is said to be in general set of lines in the plane position is to compute the number no three lines intersect at a common point. The next problem for the right of regions in the plane formed by n lines in general position. Good hints lines cases. When n = 1, there are 2. Two intersecting from small can be obtained guess a 7 at form It do not intersect form 4 regions; three lines that regions. seems, at point < least for / 3, that the /th line adds / regions. If this is true for all /, then the number of section. from S(n), which was computed in the previous computed regions can be easily line is one more of the number of regionswhen Therefore, we concentrate on the growth added. The claim we are trying to prove is the following: A
Guess:
Adding
increasesthe
one more line to number
As we have already seen, the induction hypothesis, and try
n-\\ lines in
is true
guess
general
position
We
can now
in the
plane
by n.
of regions
for
n
1) is in the previous section that number of regions is n(n + l)/2+ 1.\342\226\2 adds
line
total
n more
of
number
seen
We \342\200\242\342\200\242\342\200\242+/!. have
\342\200\242 \342\200\242
nth
total
the
position is
in general
n lines
by
that the
hence,
regions;
formed
plane
proved
already
two
introduces
line
the
regions
in this proof. First, the hypothesis dealt with rather than the function. As after, directly growth the induction of the growth of the function. a result, proof concentrated on the growth is no need to define the hypothesis There such that it proves the theorem directly. We can achieve the proof in two or more steps. As long as we are learning more about the There is no need to hurry, or to attempt too much too situation, we are making progress. Patience quickly. usually pays. Second, the same induction hypothesis was used twice in two different + l)th once for the A?th line and once for the (\302\253 line configurations: as A?th an line. This use is not double the lesson it is and teaches uncommon, 4'acting\" that we should utilize our assumptions to their fullest. There are
Comments
of the
the
with
2.4
A Simple
We are
again
n distinct in
neighboring
we
lines
assigning
were
Problem a plane,
in
colors
have different
regions
points
interesting
Coloring
interested
Consider
two
function
to
colors
this time the (two
regions regions
not
necessarily
in general
position.
by these lines such are considered neighbors if
formed
that
and
edge say possible to color\" the follow this rule, and of colors a valid coloring. assignment In general, it is possible to color any planar colors of this fact with four (the proof map has occupied mathematicians for about a hundred years, and was found only recently). The regionsformed as is shown lines, however, have specialcharacteristics, by (infinite)
if they have if we can regions
only
in
the
next
in common).
an
We will we call the
that
\"it is
theorem.
\342\226\241 Theorem
2.6
It is possible to color with only two colors.
the regions
formed
by
any
number
of lines
in
the
plane
2.5 Proof: We
Induction hypothesis:It is lines in the It is
clear
plane
15
Problem
to color
possible
the regions
formed by
<
n
two colors.
only
are necessary
colors
two
that
with
Summation
Complicated
induction hypothesis.
the natural
use
More
A
n=
for
sufficient
and
1.
the
Assume
induction
to modify and consider the coloring the nth line is added. when side according to which the same as before, and of the nth line they lie. Leave all regions on one sidecolored To prove that this is a valid coloring, the colors of all regions on the other side. reverse R If both are on the same side of the A?th and we considertwo neighboring \\ R2. regions line was added (by the induction then were colored before the line, they differently the reverse but have If the edge colors, They may they are still different. hypothesis). between them is part of the nth line, then they belonged to the same region before the line colored Since the color of one region was reversed, they are now was added. differently.
the only question is how Again, Dividethe regions into two groups
lines.
n
hypothesis,
\342\226\2
The
Comments
flexibility,
as much
as
given a was
general
or for more coloring,
the
to handle
used
degrees
of
of new
formation
next
example
idea is
The
by the
regions
complicated. Considerthe
is more
following =
The of
is to
the
64
21+23+25+27+29
=
125
the
rows
seem
This idea
+
11
for the sum to follow
a
of the
regular
ith
row,
pattern;
and prove its correctness. They look like a sequence
cubes.
Induction The
27
=
expression
for
1
=
an
search
the hypothesis idea was that,
triangle.
13+15+17+19
find
sums of
the
stretch
=8
5
+
7 + 9
problem
usually
to
added line.
1
3
The
is
example
Summation Problem
2.5 A More Complicated The
this
in
illustrated
freedom.
in order to get the most out of it. In this case, the key we can reverse all colors and still have a valid coloring.
possible
valid
method
problem problem
to be
vaguely
hypothesis:
The sum of row
i in
the triangle
is i3.
the hypothesis are definedin terms of a picture. It is not easy to define let alone to solve it. In practice, for problems it is not uncommon precisely, A major part of any defined. is to extract the solution right problem. and
that are consistent with the picture, and solve assumptions other The zth row (It is possible to make assumptions.) i numbers. The numbers contains are the odd numbers in order. let's concentrate Again, on the difference two consecutive rows. To prove between that the sum of row i is to show indeed z3, we need only that the difference between row z'+l and row i is is true for i 0 and a > 1, and for all f(n), we have (f (n))c =o(aJ{n)).In other than does a polynomial function grows faster function. functions
The
that
growing
monotonically
words,
an exponential
\342\226\2
OO Symbol
The O notation
has
of course,
in
convenient
notation
that
received reality
to forget
gives
only
a lot
constants
of criticism over the years. The main objection to it is, do matter. The wide use of the O notation makes it
about constants altogether. It As such, approximation.
a first
is essential it
serves
to remember
a useful
purpose,
that and
the its
O use
Analysis of
42
has
was also
of complexity
development
efficiency. to distinguish important
however,
is,
O
are
notation
by
It
measures.
all
light on
sheds
which
theory,
book a
in this
large and
is efficient
new
in
It is
symbol.
where
case
the
not meant to be
constants
the
they are
this
make
To
practice.
where
case
the
between
prohibitively
algorithm
corresponding
introduce
are practical
that
algorithms
many
of algorithm
by the
ignored the
the
in
instrumental
aspects It
of
the development
prompted
many
Algorithms
small and we
distinction,
a precisemathematically
(or at least only to replace some prose that accompanies whose as measured times, should running by the O algorithms accompany) value We to denote theoretical of are notation, only. suggest by CO(f(n)) (pronounced that is O (/(\302\253)), but with constants that are too large for \"Oh Oh of/(\302\253)\") a function should be easy to remember since it resembles a big uses. (This notation most practical
meant
\342\200\224 it is
notation
defined
some
oo.)
The use of not a certain of
intention
the
reader
to tighten our
of
symbol is to stressthat
Time
3.3 How
do
need to
we count
and Space
example, a
The
not
the
whole
of
main
the
Another
We have no
is to indicate to
purpose goal
or
Whether
writer.
defined.
well
not
the
this
in introducing
story.
Complexity
analyze an algorithm's the number of steps the
many different types of
judgment
use\" is
concise form.
is
O notation
to the
definition.
in a
writer
the
the
left
a \"practical
to
leads
attempting
opinion
should be
CD notation
the
constant
steps, and
time
running
algorithm each
may
without
performs.
require
a
running the algorithm? The problem is that there
We
of time.
For
amount
different
are
to compute than an addition does. One way to an algorithm is to count the number of different steps separately. But listing all analyze the types of steps separately will the be, in most cases, too cumbersome. Furthermore, of the different on the or the implementation steps depends specific computer in used the We to that are avoid programming language implementation. trying division
may
take
longer
dependency.
Instead of counting all steps, to us to we focus on the one type of step that seems be the major step. For example, if we are analyzing a sorting then we choose algorithm, of sorting; comparisons as the major step. Intuitively, comparing elements is the essence all the rest can be regarded that sure as overhead. Of course, we have to make indeed constitute comparisons constant factors anyway, it
the suffices
major part of to check that
the
algorithm.
the
number
Since
we
will
ignore
operations is is a bound for and if 0(f(n)) for the total number of steps. of
all other
to the number of comparisons. If this is true, of comparisons, then 0(f(n)) is also a bound We that the time complexity of the algorithm, or the running time, is 0(f(n)). say This the of also solves different that different problem approach steps require computation time, as long as the difference is no more than a constant. The space complexity of an algorithm indicates the amount of temporary storage In most cases, we do not count for running the algorithm. the storage required required as part of the for the input or for the output space complexity. This is so, because the proportional
the
number
3.4 Summations
spacecomplexity
43
same problem, in which do without the input or to we want count the and that be saved. We also do not count only storage output, may for the program itself, since it is independent of the size of the input. the required storage refers to worst denoted as Like time complexity, space complexity case, and it is usually in the size of the input. an 0(\302\253)-space an Thus, expression asymptotic algorithm of memory a constant amount per input primitive. An 0(l)-space requires algorithm amount of space independent of the size of the input. requires a constant the number of major steps may not be easy. In the next sections we Counting several for times. mathematical In discuss briefly techniques running computing the of a is contrast, estimating complexity particular space algorithm usually to compare
is used
different
straightforward, and,
in
most
cases,
we
will
for the
algorithms
requirements are fixed.
the input/output
case
not
cannot
we
Also,
it.
discuss
Summations
3.4
is composed of several parts, then its complexity is the sum of the it In this is as as The algorithm of its not sounds. simple complexities parts. many cases, a different We many times, each time with complexity. may consist of a loop executed in order to analyze such cases. Probably the need techniques for summing expressions that the /th step (i2).
(3.17)
+
to
most of
cancel
both sides of (3.17)by
the
T(i)
+ 1):
(n>2).
-^yX7(/)
we multiply
want
for T(n
expression
corresponding
l)-l
We
terms technique.
canceling
the
+ (\302\253
convenience,
so simple, of quicksort which
is not
is
relation
\302\2437(1),
one
i=\\
and
shifting
l) =
case.
is very
/i-1 + -
Let's look
terms.
the
double
canceled 7(1) is not necessarily by c. One when a parameter (c in this case) that appears in the expression final solution. the To solve this problem we note that correctly, and that the proof above is correct for all n>2. Hence, definition),
n
use the
This is base
proof ignoring for T(2), since
in important. It appears Section 6.4.4. The recurrence
7(/i) =
7(2)
We can, for
incorrect!
is
it
= 6*27(1). the
work
not
recurrence
This
We
but
+1), since we
for T(n
+ c)2''-1.
= (7(l)
+ l) 7(\342\200\236
''clear,\"
is true
it
suspicious
not appear in + c* (by
does
be
may
5, and see
c =
the proof
that
then
time.)
value every
going
T{n),
for
true
(3.18) and \302\253,
both
sides of
(3.18) by
n +
1:
n-\\
(/i +
+2
= n(n-\\)
nT(n)
l)r(/i + l) = (/i
(n>2).
\302\2437(/)
+ l)/i+2
now
(n +
which
(3.19)
\\)T{n +1)
-
(3.20), and
from =
nT(n)
(n +
(3.20)
!
- n
\\)n
obtain
(n
- 1)+
2T(n)
= 2n
+ 2T{n)
(n
>2),
implies that + r(\302\253
This
subtract
(n>2).
2;r(i) /=
We can
(3.19)
l) =
recurrence
+ ^7(\302\253) n+\\ is easier to
-^n+\\
solve.
(n>2). First,
we
substitute
2 for \302\253 + l
, and get a close
approximation: + r(\302\253
If we
l)2).
(3.21)
3.6
\302\253 + l 7\302\273og/,a
(3 (3 of logarithms
Sum
ILlog2'J
+ =(\302\273
!)Llog2\"J
-2Llog!\"J
+ 1
+2
=
(3
0(wlog\302\253).
1=1
a summation
Bounding If
is a
f(x)
integral
monotonically increasing continuous
\342\200\236
function,
then
.v=n+l
(3
J /(*)4[2/+l].
(1) The
by induction: that is stored
The
in
A [i]
of
advantage
this
saves On the other hand, if no pointers are required, which storage. representation much from if some leaves are farther the root than is unbalanced, away namely, the tree nodes must be represented. An unbalanced tree is others are, then nonexisting many in node indicate its the An below each shown in Fig. 4.6; the numbers position array. or 8 nodes. The implicit representation thus may array of size 30 is neededto represent
is that
may
not save
operations
storage, depending
the
in
can be reasonably
on
of the
middle
if they
supported
Also, since
tree.
the
tree are costly.
other
the
On
are limited to nodes that
are used, dynamic hand, dynamic operations to the end of the correspond arrays
array.
4.3.2
Heaps
A heap is a binary
tree
whose
keys satisfy
node is greater
The key of every
the
than
property:
heap
following
to
or equal
the
of
of any
key
its
children.
By the transitivity or equal to
law,
than
a priority
implementing two
the
the
of
all
queue,
which
keys
node's
that
is an
data
abstract
type
in
by the following
defined
operations:
insert a key x
Insert(x):
Remove{)\\ remove the can be
Heaps will
of every node is greater descendants. Heaps are useful
that the key
implies
property
heap
use
into
largest
the
structure.
data
key from
the data
structure.
implemented with either the explicit or the representation, since we can ensure
implicit
the implicit
We assume
elements
the
the
that
heap
is array will ever
Figure 4.6
i4[l..\302\243],
where
contain (if
Implicit
an
representation
k is an upper
upper
bound
of an
tree
representation.
the heaps
that
bound
is not
be
will
on known,
unbalanced tree.
the then
We
balanced.
number
of
a linked
4.3 Trees 69 Let n denote required). A [\\..n ] is of interest array
is
representation
only how to implement insert namely,
the
We key
largest
from
The
root.
the
first
[n ],
:= A
We
the leaf
take
two
have
still
A
decrement
and
heaps
separate
we
property,
now
jc
propagate
This is done by
maximum.
the
proceed
the heap; to describe
heaps. the node
property,
heap
in
always
operation
the
with
removes
the key
the
property
heap
on top, which or may not satisfy the may the at this is if the satisfy heap property point was contained the value jc.) To restore the heap a value
plus
(The only way property. path from the root to where
whole
By
use of
now
after the key of the root has have to two separate heaps. an array A [2..n ], which corresponds it in place of the root. That is, we let A [1] [n ], delete it, and put the value of n by one. Denote the value of the new A [1] by jc. for
heap
the
with
A[\\]. So, a Remove
is to restore
problem
We now
been deleted. We
efficiently
operation.
is the root,
a heap
in
Remove
the
with
start
remove
and
We
moment.
at any
of elements
number
current
the
jc jc
to
the tree,
down
jc
comparing
a subtree for
it reaches
until
with
the
values
of
its
which
children
two
it is
04 [2]
a
and
A [1] with the largest and, if jc is not the maximal among the three, by exchanging of them. Assume that A [2] is the maximal. Then,A [2] is clearly the maximal key in the rooted at A [3] whole heap, so it can be put in the root position. Furthermore, the subtree
A [3])
remains unchanged, the subtree
about
continue key
jc
again
thus
rooted
at
inductively
is now
if
jc
is
in
the
it also A [2]
same
satisfies the
now it Assume that
(because way.
We have to
property.
heap has we
jc
in
its root).
But
only
worry
now
we
for / steps, and that satisfy the heap property.
continue
Only the tree rootedatA[j] may not jc to its two new children, A[2j] and terminates not the maximal. The algorithm
atA[j].
compare
exchange
and
A[2j+\\]
either
(if they when jc
exist), becomes
can the
We and
the
number of comparisons a leaf. The maximal of a subtree,or when it reaches tree. The algorithm the of the for which is twice a deletionis 2|\"log2\302\253~|, required height for removing a heap is given in Fig. 4.7. a maximum element from the new key n by one, and insert An Insert operation is similar. We first increment as the new leaf A [n ]. We then compare and exchange if the the new leaf with its parent, new of its subtree the new key is the maximal leaf is larger than its parent. At this point, it was found to be larger). We was the maximal and (since the parent assume, and A the tree satisfies that the rooted condition, [n ]) heap ?xA[j] (initially inductively, that if we remove We continue the this tree the rest of the heap satisfies heap property. this its the new key up the tree, until the new key is not larger than process, promoting The it reaches the root). At this point, the whole tree is a valid parent (or until heap. maximal number of comparisonsrequired is riog2\302\253l, which is the height for an insertion of the tree. The algorithm for inserting in Fig. 4.8. into a heap is given an element in time of and Insert Remove Overall, we can perform sequence operations any O (\\ogn) On is the other it other not to hand, per operation. possible operations perform a heap. For example, if we want with to search for a given key, the hierarchy efficiently A Heap is a good exampleof an implementation of an given by the heap is not useful. a limited abstract data type. A heap supports number of specific operations very we need these particular we can impose the Whenever operations, efficiently. heap structure on the data whatever its type is.
maximal
70
Data Structures
(A, n);
Remove_MaxJrom_Heap
Algorithm
Input: Output:
A
of size
array
(an
n
maximal
Top_of_the_Heap (the
heap), and
n
new
(the
begin ifn = 0 then
size of
the
is
\"the heap
print
a heap). element
representing
if n
heap;
of the
= 0, then
heap), A
the
heap
empty\"
else
:=A[\\];
Top_of_the_Heap
A[\\]:=A[n]; n := n -1 ;
parent :=
1;
2 ;
:=
child
-1 do
while child A[parent]
ifA[child]
swap(A[parent], parent := child;
child := 2*child child :=
else
then
< A[child+J]
n
A
;
[child])
;
the loop
stop
{to
}
end Figure 4.7 Algorithm _Heap (A,n,x);
Insert Jo
Algorithm
Input: A Output:
of size
(an
RemoveJAaxJromJieap.
array A (the new
n
heap),
representing and
n (the
a heap), and x (a new size of the heap).
number).
begin
n :=
n
A [n ] :=
while
assume that
the
array
does not
overflow
}
x;
child := n parent
; { we
+ 1
;
:= n div 2 parent
> 1
; do
if A[parent] < A[child]
A[child]);
[parent],
swap(A child
then
:=
parent
{ see alsoExercise4.6}
;
parent := parent div else parent := 0 {to stop
2 ; the
loop }
end
Figure 4.8
Algorithm
Insert
jojieap.
(the
new
is empty).
4.3
Binary Search Trees
4.3.3
trees
search
Binary
implement
there (for simplicity, the
delete(x):deletethe data
Abstract
insertions
holds
(or to
nil).
are
leaves node
be removed
may
simplicity,
Binary
section, since dynamic We do not want to trees. parts binary of elements. We assume that each for the number at least three fields: key, left, and right, such that are pointers to other nodes node, and left and right in this
search
of
may be
pointers
that all keys
assume
we
the
and
dictionaries.
called
as other more complicated
as well
a given upper bound tree is a record containing the key associated with the search trees are more complicatedthan heaps, Binary whereas in added or removed and keys exchanged,
in the
already there).
is there.
if it
structure
efficiently,
limit ourselves to
key
it is
(unless
these three operationsare
are important
deletions
and
structure
explicit representation of trees
use the
will
We
operations.
structure, or determine that x is not that each key appears at most once).
the data
dictionaries
implement
operations:
following
the data
x from
key
that handle
types
search trees
x into
key
data
the
in
will assume
we
insert
insert(x):
the
efficiently
find the key x
search(x)\\
node
71
Trees
binary
in
manipulated
in heaps trees, other ways.
because
only
search
many
any
For
are distinct.
Search tree is a structure to facilitate searching. The structure search that we have a key x Assume search procedureis understood. and we want to know whether it is currently a key of a node in the tree, and if it is, we x against the want to find that node. This operation is called a search. We first compare = If x root we of the tree, whose value If x then are done. r. m (p should smaller than | U |). than m to be effective,but it should also be sufficiently function As we have already mentioned, no hash can be good for all inputs. Using since most data in practice as described is fairly have no structure related to safe, primes it is always numbers. On the other hand, that, in a possible (although unlikely) prime on integers made certain application, one will want to store results of some experiments r. All these numbers of course will are of the form r + kp for a constant have all of which data the same hash values if p is usedas described.We can take the idea of scrambling function! For with hashing one step further, and use a random procedure to selecta hash from list in the be selected at random of the can a prime p primes appropriate example, list of primes, however, is not easy. Another is the a large possibility range. Finding a let At select two numbers a such and and that and /?, a,b *0, random, n the
the
operations iterated
total
number
(either
logarithm
find
of or
function,
\342\226\2
4.6
(b)
(a)
Figure 4.17 function
The
=
\\og*n
log*\302\253is
defined
1 +log*([log2\302\253]).
= 1+log*
log* 60000 all
virtually
find
operation
0(n
\\og*n)
is almost
finds
We
as follows,
log*
(b)
1 =
4.
For
any
we have
number
such \302\253,
log*4 = 3, and a? XLlog2/J
il\\og2i\\
,=1
height
is
A[i]
[\\og2(n/i)\\,
is
of
number
the
at most |_log2'J ^l^g2n\\ The /th step requires time is 0(n\\ogn). Moreover, 0(n\\ogn) is not an shows. as the following argument
running
running
of
construction
2|_log2(\302\253//)J.
down)
(top
Complexity
=
>\302\253/2Llog2(\302\253/2)J
Q(/ilog/i).
i=n/2
most twice
the
comparedwith complexity is at evaluate
//(/),
look at complete
Let's in
the
tree of height i, and
+
of this
binary complete construction heap
trees
each
in
//(0) = 0.
tree of
i
height
is
2/+l
-1,
it
is at
step
the
Since
of
the complexity
of
sum
the
a recurrence i - 1 and a
(by induction) that number of nodes
verify
that
follows
trees of height
of two can
We
recurrence is //(i) = 2/+l-(i+2).
We
/.
height
i consists
by //(/) can derive
denote
and
first,
binary tree of
complete
noting that a
root. Hence,//(/) =2//(/- l) solution
involved
comparisons
node the corresponding node (since each may have to be the the tree). its two children, Therefore, exchanged, and so on down We want to of all nodes in the tree. most twice the sum of the heights
heights of all nodes for
of
number
of
height
sum.
this
The
Up)
(bottom
Complexity
relation
case is
worst
the
in
the
of A[i]
value
old
the
A bottom-up
its children.
both
than
6.15. Since
in Fig.
comparisons
it is larger
where
in a heap
a deletion
to
the in
a
bottom-up
2k -1 trees is O(n) for complete binary nodes). (namely, heaps with The complexity for a heap with n nodes such that 2k /2>
Given a characters
the
for
minimizes
characters.
different
Problem
The
Information
typical
texts,
instead
isrfi. 1=1
Our
goal
L(\302\243,F).
is to
find
an
encoding
E that satisfies
the
prefix
constraint
and minimizes
6.6
The prefix a decoding
to an
is equal
each node has either
which
in this
the path from
root
constraint
file is
encoded
character.
scanned
a graphic
is
not
n -1
define
the
characters induction
Consider
characters.
of
1
and
is based
to solve
problem
(and of
reduction
and
in
the
that minimizes It is
problem.
a problem As
order
which
to
tree
binary
in
of 0s
and
Is on
(see Fig. 6.17). leaves. When the corresponding
The
L(\302\243,F).
tree
useful, however, to
its constraints).
case is trivial).
(the base hypothesis
in order
on a
correspond
a
0, or no emanating
can safely determine the
a tree
such
construct
the
must
is reached, we
a leaf
and
necessary
illustration
The algorithm with
that all characters
says
Our problem is to
representation have
the
so let's look at unambiguous, of bits one by one until we get a
labeled
two
147
Compression
emanating edges by tree correspond to the characters. The sequence to a leaf corresponds to the character's encoding
edges. The leaves
The prefix
decoding
scan the sequence encoding of one of the
We need to
procedure.
sequencethat
to make the
is needed
constraint
Data
with
usual,
n characters
the main
to eliminate
difficulty
to a
problem
is how
to
characters. The reduction
from the ones we have seen so far. Instead of simply one eliminating from consideration, we introduce a new \"artificially made\" character in place is a little more complicated, but it serves the of two existing characters. This technique is reduced. the size of the input Let C, and Cj be two characters with same purpose \342\200\224 two such characters, then ties are broken minimal (if there are more than frequency We claim that there exists a tree that minimizes these L{E, F) in which arbitrarily). if distance from the root. characters correspond to leaves with the maximal Otherwise, lower in the tree, it can be exchangedwith C, there is a characterwith higher frequency or Cj decreasing L(\302\243,F). (If its frequency is equal, it can still be exchanged without or no children (or L each node in the tree has either two children Since (\302\243, F).) changing else we can shorten the tree), we can assume that C, and Cj are together. We now is the sum/) C, and Cj with a new character, called C,7, whosefrequency replace +/y. The problem now has n - 1 characters (n-2 old and one new), and as such can be of the original solved the solution problem by by the induction hypothesis. We obtain in to an internal node the reduced with two leaves C, problem corresponding substituting and in of as an of the leaf to We leave the proof optimality place corresponding Cj C,y. here
is different
character
exercise.
010
Figure 6.17
The tree
representation
Oil of encoding.
Algorithms
148
Sets
and
Sequences
Involving
for Huffman's The operationsrequired encoding are (1) insertions Impl6ni6ntati0n with of minimal from the deletions the two characters data structure, a (2) into frequency a the tree. is data structure for the first two A and (3) building data structure, heap good in the worst case. of which The is each requires steps 0(\\ogn) algorithm operations, known as This is Huffman's 6.18. encoding after compression technique given in Fig. D. Huffman [1952], who proposed this algorithm.
(S, f);
Huffman_Encoding
Algorithm
Input; S (a string
Output;
T
of characters),
begin insert all characters while H is
not
make X
one
only
root
the
H according to their
;
frequencies
do
empty
ifH contains
frequencies).
S).
a heap
into
array of
and/(an
tree for
Huffman
(the
X then
character
ofT
else
pick two replace
a
lowest
with
Y
frequencies
H ;
from
new
Z whose
character
of the frequencies
ZtoH
make X
them
Y with
and
X
sum
the
insert
X and
characters
and delete
ofX and Y
frequency is
;
; Y children
and
{Z has no
ofZ inT
yet}
parent
end
Figure 6.18
Example
that the data contains six
Suppose in
1, respectively
The Huffman
6.19.
The internal
nodes are numbered
O (log
6.7 Let
the tree takes constant each. Overall,the running steps
n)
String A -a
assume
\\
time
characters
although
sequence of
and
an
the
it
is
characters
a i a2
time
time
to the
per node.
of the
frequencies
characters
created.
were
they
5, 2, is given
\342\226\2
Insertions and deletions is O
algorithm
(n log
n).
Matching \342\200\242 \342\200\242 \342\200\242
a2
that
characters, substring
according
Building
Complexity take
characters 4, \302\243, C, D, E, and F, with tree correspondingto these
10,and
Fig.
Encoding.
Huffman
6.1
\342\226\241
3, 4,
Algorithm
' '
' a{ (h\\
\342\200\242 \342\200\224
B=bxb2 come from a not
b2
'
m anc* to this rule is when the mismatch is against b {; in this case,
1
2
3
4
5
6
7
8
9
10
11
x
y
x
v
y
x
y
x
y
x
x
-1001201234 Figure
6.22
3
The values
of next.
6.7 String Matching to proceed in A. This case can be determined The program for string is given in matching
we want
is -1.
which
Algorithm
Input: {
A
We
Output:
at A
value of next(\\),
the
by
Fig.
153
special 6.23.
String_Match (A, n, B, m) ; of size n), and B (a string of size m). assume that next has been computed;seeFig.6.25) Start (the first index such that B is a substring of A starting (a string
[Start]).
begin Start :=
0; and i0do
;
next(i):=j
end Figure 6.25 Algorithm
A character
Complexity
there is a pointed
mismatch,
then
the same
by the next table. the same character of to
of A
may
Compute
be compared
character of
If there is another .4 until there is
A
Next.
against
many
is compared mismatch,
then
characters
of B.
If
against the character of B we continue comparing
of either a match or we reach the beginning How we claim that the running time of this algorithm is still O(n). times can we backtrack for one characterfrom Let's assume that the many A> say aft first mismatch involved bk. Since each backtrack leads us to a smaller index in By we can to reach bk we must have gone forward k times backtrack only k times. However, without moves, to the forward any backtracking! If we assign the costs of backtracking then we at most double the cost of the forward moves. But there are exactly n forward \302\27 so the number of comparisonsis O (n). moves, against
B. Nevertheless,
6.8
155
Comparisons
Sequence
This algorithm was developed by Knuth, Morris, and Pratt [1977J, and it is known for this problem was developed by KMP algorithm. Another fast algorithm We The is and Moore sketch it difference between the algorithms [1977]. briefly. Boyer B from the end rather than scans from the the that Boyer-Moore algorithm beginning. be of bm against then the next That is, the first comparison will am. If there is a match, we use the am_x, and so on. If there is a mismatch, comparison will be of bm_\\ against information, much as we did in the previous algorithm, to shift the whole pattern to the - \"Z,\" and Z does not appear if am For example, at all in B, then the whole right. as the
to the right by m steps, and the next comparison will be of a2m If Z at bh then we can shift does by m-i say steps. The appear in \302\243, against bm. becomes more when there how much to shift are several decision complicated partial the matches matches. On the one hand, we want to utilize already found. On the other the whole even if the same hand, it is more efficient to shift pattern as far as possible, comparisonsmay have to be performed twice. We omit the details. The interesting it is likely to make fewer than n comparisons of this algorithm is that characteristic (in one allows us This is bad mismatch to without more because shift, any regular text)! be shifted
can
pattern
comparisons, 6.8 The
by
m.
Comparisons
Sequence
that
attention
concentrate here on required
section
assume would
comparisons has received a lot of is the applications to problems
of sequence
subject
reason for to
that
one string
change
is dynamic Let A=a{a2
one
only
programming (discussed \"' '\" an and B-b\\b2 come
change
A
character
steps
throughout
this
of characters.
We
5.10). two
be
bm
number of edit
strings
characters, for example). We it becomes equal to B. We allow a cost of 1 to each: (1) insert \342\200\22
set (English
a finite
from
Section
We
biology.
technique used
main
The
another.
into
The main
lately.
molecular
minimum
the
finding
in
the characters
like to
\342\200\224
problem
attention in
such
by character
that
types of changes (or edit steps),and we assign insert a character from the string, and (3) delete a character into the string, (2) delete \342\200\224 with For a different character. one character replace \342\200\224 example, to change the replace three
string
abbe
insert an a
into the
between
two
bs (babe),
However, we can changes. and then delete the last c, for a total three
of single-character
also insert of
first
the
replace
b at
a new
a, forming the
last c
the
beginning
the with
bbc\\
string
a b
for a
of
babbc),
(forming
Our goal is to minimize
two changes.
then
total
the
number
changes.
The string-edit maintenance. We modification of the
delete and then
we can
babb,
string the
problem
has
also applications
to
file
comparisons
and
revisions
is a It is convenient to extract the differences between the two nles. There may be several versions of the same program, and, if the versions are similar and need to be archived, it is more efficient to store only the differences instead of they In such cases, we may allow only insertions and deletions. the whole storing programs. to each of the edit steps. costs In other cases, we may assign different may
first
have
one.
a text
file
(or
a
program)
and
another
file
that
156
Involving Sequences
Algorithms
and Sets
possible changes,and it seems difficult to find the best one. We denote by A (/) (B (/)) the prefix substrings a { a2 ' '' a{ As usual, ''' hi)- Our problem is to change A (n) to B (m) with a minimum number of edit (b\\b2 steps. Supposethat we know the best way to change A (n- 1) to B(m) by induction. we assume only that we know one of best solutions; may be several different (There to change A (n) to B (m). But one more deletion, that of an, we have a way With them.) be that it is better to replace an with bm, this may not be the best way of doing it. It could or better yet, an may even be equal to bm. all the different possibilities of constructing the minimum We need to consider of smaller best A and A to B with the aid of the from changes sequences involving change B the minimum cost of A to Let's assume for now C B. Denote by (/) (/, j) changing (J). in finding A to B and not in the change the cost of changing that we are interested only in finding a relation between C(n, m) and C(/, y)s for some itself. We are interested is and js. It is not hard to see that there are four possibilities, of smaller combination are
There
quite
a few
try induction.
we
corresponding to the
delete: if an is deleted scenario above holds.
B(m)
insert:if
nothing:
change
from
be the
would
deletion.
more
one
minimum
the
the
induction)
from
change
bm, then we
match
A
to
then \302\243,
one from
A
(n
other
In
the
- 1) to words,
have C(n, m)
minimum
change
to B
A
insertion of
involves
1)+ 1. That from A(n) to B(m-\\) = C(n,
m-
a character
is, we find
(by
insert
a
and
equal to bm.
character
replace: if change from match:
in the minimum
to doing
= C(fl-l,m)+l.
C(/i, m)
to
and
The best change
then
and
edit steps
different
three
if
an
an
is
replacing
- 1) to B (m
A (n
to
is equal
bm,
bm, -
then
we
then 1) and C(n,
then
first
to add
m) = C(n
to
need 1
find the minimum
if a\342\200\236 * bm.
- 1,m- 1).
Denote f
o
', to
at
of
the
7.27,
Fig. v
if if
how
DFS,
perform
hence
and
High
vertices
higher
the tree;
v in
than
the
We claim
values.
High
= w$;
edges from
hence,
in exactly
graph
is an
v
that a
the
vertex v
that High(Wj) is
v such
of
u>,
then there are no
w, exists,
vertex
the
(v) = r.
computed all the High only if there is a child
such
whether one
can easily compute all = v, and High(w2) High(w2) we
have
is that it traverses
DFS
= rJ
High(wl)
to q,
goes we
(v) if we know all the back among
High
compute
all High{wt) and among to determine efficiently
highest
shortly
So, if we
than another.) is higher For values. example, in highest
We can easily
w {, u>2, \342\200\242\342\200\242\342\200\242, wk (see Fig. 7.27).
the
articulation order
right
not
in the
vertices
point. for
our
purposes.)
Computing
the
induction
following
High
goes hand
values
we visit the kth vertex Induction hypothesis: When to find the High values of vertices that have already below
this
of the induction follows a perform (recursively) DFSfor all whether
compute
High(y)
a vertex
the
DFS,
by
we know
been visited
how
and are
vertex.
The order and
according to
the DFS,
with
hand
in
hypothesis.
is an articulation
Figure
children
to
according
of DFS.
the order the
of v,
find
(by
At
definition.
we
When
induction)
the same
point.
7.27
Computing
the
High
values.
a vertex v, we their High values,
reach
time, we can
decide
7.9
223
of Graphs
Decompositions
DFS tree presents a special case. Obviously, no High value can see is to that the root an if articulation and easy point only if it has more than one child in the DFS tree. Of course, this is easy to determine. The key to the efficiency of the algorithm for computing the High values is that all is information available when DFS is we the necessary performed. The only problem in the DFS tree. We use than another have is how to decide whether one vertex is higher All the vertices involved in the computation of DFS numbers to make this determination. in the values are ancestors tree. the High Therefore, they already have a DFSnumber. lower the higher an ancestor is, the its number is! This is not true for Furthermore, in the tree; fortunately, we care only about vertices that are not related back however, is a to the to DFS values use the numbers. We So, practical way High manipulate edges. it as that refers not to the vertex but to that define High(v) before, itself, except highest to describe the algorithm in terms of DFS numbers, vertex's DFS number. It is confusing vertices because we define Therefore, correspond to lower DFS numbers. higher of is | V | and the number decreasing DFS numbers: the root has a DFS number time we visit a new vertex. We also can use decreased numbers: we every negative and we decrement the number a of-1, assign the root a DFS number every time we visit new vertex. The advantage of the latter scheme is that the value of | V | need not be known in advance. the actual The only remaining task is to find biconnected components. We could It is
root.
the
by brute
them
find
of the
root
The
exceed
Noticethat,
at the
force,
edges of B were the most vertices on a stack and add be an stack now program defined
to be
V
the edges
constant
|). The be remembered | + |E
that b is
traversed.
the
During
an
7.25.
back at Fig.
look
Let's
way.
articulation
we put
traversal,
the
point,
new
the
as
are
they
encountered.
per
space as they
amount
vertex.
of
the
Hence,
also
are
requirements
in addition to
work,
running
O (|
V | + |E
work
the
this
of
time
|) since
the
involved
in
algorithm
is
components
traversed.
are
7.3
\342\226\241
Example
An example
of
Fig. 7.29. The first Each DFS Numbers.
in
(decreasing) when
is discovered
a new
to be
for
Biconnected^Components
algorithm
repeated here, is given gives their as updated
the extra
Clearly,
the DFS, is
it
recent
elegant
determined
algorithm
fully.)
Complexity
must
the
to When a vertex is found all the top edges going back in the we can remove from the stack articulation point, We can until that vertex is reached. This is exactly the biconnected component! in the and continue same way. The complete remove those edges from the graph is given in Fig. 7.28. for biconnected (The algorithm can be components in for completeness, we terms of and of DFS, but, preWORK postWORK merely
presentit
O (|
is also an
there
but
point where
an
call to
the
articulation
recursive
point.
line
gives
the graph in the vertices
Fig. and
successive line presentsthe
procedure
is made.
A
vertex
7.25,
High
is
which
second
the
line
numbers
is circled
when
\342\226\2
Graph
Algorithms
(G, Algorithm BiconnectedjComponents connected G = (V, E) (an undirected Input: the
root
Output:
n)
v,
graph), v (a vertex serving as DFS tree), and n (the number of vertices in G). and the High values are the biconnected components are marked of the
computed. begin
for
vertex vofGdo
every
v.DFS Number {the DFS numbers
serve to
also
indicate whether
vertices have been visited
corresponding DFSN
0 ;
:= will
not the
or
}
:= n ;
DFSnumbers; { we use decreasing
the explanation
see
in
the
text.}
BC(v)
end
procedure BC(v) ; begin
:= DFSN
v.DFSNumber
;
DFSN := DFSN -1 ; v into Stack ; { Stackis initially insert v.High := v.DFSNumber ; { initial for all edges (v, w) do insert (v, w)
into
Stack
}
;
twice (for { each edge will be inserted is not the then ifw parent ofv
if w.DFS Number BC( w) ;
if w.High {v
= 0
disconnects
wfrom
and mark
reached,
{ v.High
else
v
v
ofw's
:= max
{ (v, w) v.High :=
of the
the
is a
until
Stack
back edge
, w.High
Algorithm
is
;
)
or a forward , w.DFSNumber
}
edge
)
end
Figure 7.28
v
they form
subgraph
component and possiblyothers
( v.High
max ( v.High
}
graph
from
biconnectedcomponent back into Stack ;
is part
directions)}
then the rest
edges and vertices
remove all
insert
both
then
< v.DFSNumber
as a
}
empty value
Biconnected
Components.
}
7.9
Decompositions
of Graphs
J 9e
a
b
c
d
16
15
14
13
a
16
b
16
c
16
15
d
16
15
e
16
15
d
16
15
f
16
15
16
15
c
16
15
g
16
15
c
16
15
00
16
15
h
16
15
i
16
15
J
16
15
k
16
15
J
16
15
1
16
15
J
16
15
16
15
14 14 14 14 14 14 15 15 15 15 15 15 15 15 15 15 15 15
P
16
15
n
16
15
m
16
15
16
15
15 15 15 15 15 15 15 15 15 15 15
7.29
An
example
00 b a m
16 16 16 16
15 16 15 15
n
16
15
0
16
15
00
16
15
\302\251
Figure
e 12
f
g
h
i
11
10
9
8
J 7
k
1
m
n
0
P
6
5
4
3
2
1
15
d
CO
wl
wk
V
13
13
15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15
15 15 15 15 15 15 15 14 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15 15
14 14 14 14
15
14
15
13
15
14 14 14 14 14 14 14 14
15
14 14 14 14 14 14 14 14 14 14 14
of computing
16
15
16
8
15
16
8
7
15
16
8
7
8
-
-
-
-
-
15
16
8
8
8
-
-
-
-
-
15
16
8
8
8
8
-
-
-
15
16
8
8
8
8
-
-
-
15
16
8
8
8
8
-
-
-
8
-
-
-
8
-
-
-
8
-
-
-
-
-
-
15 15 15 15
16 16 16 16
8 8 8 8
8 8 8 8
8 8 8 8
8
4
15
16
8
8
8
8
4
16
-
15
16
8
8
8
8
4
16
2
-
15
16
8
8
8
8
4
16
2
-
15
16
8
8
8
8
4
16
2
16
15
16
8
8
8
8
4
16
2
16
15
16
8
8
8
8
16
16
2
16
15
16
8
8
8
8
16
16
2
16
High values and
biconnected
components.
Graph Algorithms
226
7.9.2 Strongly Connected In
we discuss
section,
this
if, for every pair
Definition:A vertices such no subset that that,
Notice
as a set connected An
in a
components
in the
edge
components.
We
are similar to
the
(we
can
vertices
A circuit is
contained
and a
only
path
necessarily
a circuit.
in
Proof: If combining
contained
into
may
the
strongly
one component. two separate
connect two
which
claims,
connected component if and only (Recall that a circuit is a not necessarily simple; that is, it may include a of them.
is a simple
itself
circuit.)
A
connected.
strongly
of the
vertices of a circuit,
vertices of the
the same strongly By the definition
to the
circuit
connected of
component).
component
not be
it would
component,
connectivity,
strong
connected
strongly
since
Now,
maximal
given
we claim
that
there is a path results in a circuit
any
two
they
are
v to w
from
(but
not
\342\226\2
7.12
Each vertex belongs to are
it
by the following section. previous
Putting together these two paths since the paths not be vertex disjoint). may
a cycle,
\342\226\241 Lemma
there
connected component is defined
w to v.
from in
by
a subset
w from
and
graph).
be partitioned vertex belongs to exactly
in
of the
there is
strongly
both
is
that
add all the other v
(i.e.,
can
graph
partition
case
same
once. A cycle
than
more
include
subset
7.11
closed directedpath Proof:
the
component
Two vertices belongto the is a circuit containing if there
cannot
of
maximal
component, or
to one
belong
the existence
prove
Each
way.
unique
biconnected
Lemma \342\226\241
vertex
of every
w to v.
from
path
connected connected
a strongly
components,
may
graph
is a
strongly
induces a strongly
vertices
The
vertices.
and a
any other vertex.
component
is
connected
is strongly
graph
v to w
from
from
subgraph
it and
contains biconnected
unlike
of
induced
its
a path
vertex
connected
strongly that
A directed
graphs.
u\\ there is
v and
possible to reach any
it is
words,
In other
directed
only
of vertices
Components
a vertex
circuits those
in only
v belongs
containing circuits
one strongly
v
results
connected component.
one strongly
exactly
to more and in
connected
than
vertices another
one
from circuit,
component.
strongly
connected
component,
then
the other components. However, be which, by Lemma 7.11, must \342\226\2 This is a contradiction.
We can define the strongly connected to the similarly component (SCC)graph biconnected component tree. (This is also called a condensation graph.) The graph nodes of the SCC graph call them nodes to distinguish the original them from (we vertices) correspondto the strongly connected components; there is a directed edge from node a to node b if there is a directededge(in the original graph) from any vertex in the to a to any to b. vertex in the component that corresponds component that corresponds since The SCC graph is acyclic involve cycles cannot more than one component. Figure a directed graph G and its SCC graph. 7.30 presents
7.9 Decompositions
Figure7.30A As was
directed
biconnected
with
case
the
and its
graph
connected
strongly
graph.
component
we can
components,
of Graphs 227
design an
by
algorithm
induction.
Induction
hypothesis: We of graphs with
components
how
know <
m edges,
to find
and
the
connected
strongly
construct
to
how
their
SCC
graphs.
The basecaseis remove x from now
have
case is
and find,
graph
to determine
what
effect
adding
and pick
m edges
with
x would the
have on the
same
an
arbitrary
edge
x. We
connected components.
the strongly
induction,
by
two vertices from
x connects
when
a graph
Consider
trivial. the
Again,
partition.
In this
component.
We
the easy
case,
x
adding
is when x connects the SCC graph. The difficult case partition in Fig. 7.31, in which an verticesfrom two different components. This case is illustrated x is in the SCC graph of Fig. 7.30. edge Clearly, x merges connecting two components has no
effect on
these two
if and
components
this case, all one
or on
the
the
components
and we
component,
no changes are made we can improve this DFS
plays
a major
Let's algorithm,
try
and
to
are
only
if
it completes
corresponding done. If x does
the component.
to
nodes
cycle the
in
in
cycle
SCC graph. are combined
In
the
into
not complete a cycle in the SCC graph, then As was the case with biconnected components,
by considering
algorithm
a (directed)
to the
the
in a
edges
particular order.
Again,
role. the same steps as we them when necessary. When
follow
modify
want to determine whether than vertices that are higher
it
of a circuit the DFS tree. The
is part
it in
did we with notion
the biconnected component a vertex through we DFS, other vertices \342\200\224 in particular,
in
visit
of High
values can
be used in
Algorithms
Graph
228
Figure 7.31
Adding
a similar way. of the graph
from
can
the
We
identify
an
edge
connecting
two
different
are looking for vertices such them or from their descendants.
\"breakpoints\"
in a similar
way
is no
there
that
We to
connected
strongly
need a
to
way
components.
the articulation
other parts
reach
mechanism
by
which
points. Consider
we the
connected of the tree parts strongly connected components occupy in That all the a connected vertices must is, (Exercise7.88). component strongly belong to one connected subtree of the DFS tree. For a given component, consider its highest in the tree; we call this vertex vertex the root of the component. The root is the first vertex of the component to be visited by the DFS. (For example, the roots in Fig. 7.30 are a, d, #, and /.) If we can identify the roots similarly to the way we identified articulation points, to then we can find the partition. We will see that the roots are similar DFS
The
tree.
articulation
The
points. algorithm
is based
on induction
that
follows
the
order of
DFS. Let r be the
component visited in its entirety by the DFS. It is the lowest leftmost component in the usual picture of DFS (r -d in Fig. 7.30). The component must consist of all of r's descendantsin the tree (none of the descendants can belong to a smaller If, traversal would have been completed first). component, since that component's the DFS, we can identify r as the first root, then we can identify the component, during of the first
root
removeit
the graph, and continue This is not as simple as we stated by induction. the main idea. Let's first see if we can identify r. for a vertex r there cannot be any back edges to be a root of a component, First, a from a descendantof r to a vertex higher than r. Such a back edge completes leading with same the higher to the vertex which cycle vertex, belongs implies that the higher to the whether such back edges exist similarly component as r. We can determine \342\200\224 biconnected the High values. However, we need to be more using component case careful here since DFS in directed does not eliminate cross edges. Consider Fig. graphs is 7.32. Vertex g does not have any back edges, but it has a cross edge to e, which a vertex in a cycle with contained h. Consequently, g's parent higher (/) is not a root of a is no there even back though component, edge from any of its descendants. Thus, we it,
but
from
this is
7.9 Decompositionsof
7.32 The effect of
Figure must consider
cross
the
words, they must looking for found both
e. If
/ and
before we
take
that
means
a
to go
way
that
be a
vertex
higher from e. not a root. /is
back edge \342\200\224 we need of the edge from g to e,
only to is not
it
which
it would
then
the component
as a
the effect
considering
of /,
that
fact
the
must
It
been an ancestor
before /implies
visited
account
into
be /.
it cannot
In particular, that there is
/.
was
that
not
had
it
reached
discovered yet a vertex
root.
we claim
then
yet,
as well.
cross edges? Crossedgesmust go from right to left; in other that have already been visited. Rememberthat we are vertices If there is a cross edgefrom root has not been g to e and the
to
point
first
the
edges
edges.
effect of
is the
What
cross
229
Graphs
is an
have been
discovered
e has
containing
not been
So, a crossedgefrom But
this
is just
important
this
g to
as easy to
consider DFS numbers! whether
of
ancestor
When edge
is a
back edge. Only the DFS number of e (and its value relative to that of/) is important. We can define the High for an values as in the biconnected component case by looking edge leading to a vertex with the lowest DFS number. The High value of a vertex is the those of its children is and among its back edges or crossedges.A vertex highest among the first root if it is the first vertex whose High Notice that is not higher than value itself. the High values do not really point to the highest The High value of g will be vertices. the DFS number of e, even it is possible to reach b from e (and thus from g). We though care vertex whether we reacha can than it is not important to know higher g (or /); only do the identity we vertex. of the highest want to chase once a back (Nor pointers edge is encountered.)
Once
consists
component edges,
and
the
we find
of all the from
the
all the
first
root,
we can
descendants of graph.
edges
that
the
This is point
the
find
done
by
to them
connected component \342\200\224 it We can then remove this the component's vertices and
first strongly
in the
root
deleting
DFS tree. all
from other vertices.
We
can
ignore
edges
Graph
230
since there is no way since we now have
vertices,
other
from
Algorithms
be done by
induction
assumptions are still values is dynamic. Since we remove verify
all the
that
to
of
get outside
a smaller
the
graph!
Notice
valid.)
component.
The
reader
should
(The
that the
of
definition
rest can carefully
the
High
discovered pointing to the newly later. of the values in the computation (This is High they will play no part component, values for the biconnected component the \"static\" definition of the High from different there is no case, which did not depend on any of the previous components.) In practice, vertices or edges. We can simply mark the vertices of remove either need to actually to marked later on ignore edges pointing as they are discovered, and each component is in 7.33 connected The vertices. (we use algorithm given Fig. component strongly
decreasing DFS numbers
The algorithm
Complexity is
complexity
to avoid
again
the
The
same.
the
edges
confusion).
to the biconnected componentalgorithm space complexitiesare O (| V | + | E |).
is similar
time and
An example of algorithm StronglyjConnectedComponents The in Fig. 7.34. which is repeated here, is given second
line their
In
it
this
to be
of the
call to
whether
determine
seen
that
a cycle
Given
for
procedure connected
Use of Graph
a
must be
line presents is made. A
the
High
is
vertex
\342\226\2
component.
Decomposition
where problem
the use involves
undirected
connected
the graph in Fig. 7.32, the vertices and the
gives
successive
recursive
a root of a strongly
it contains
line
first
Each the
section,
The Problem
We have
a new
we present two examples simplifies the solutions. The first one involves directed graphs.
short
second
when
is discovered
Examples
significantly the
(decreasing) DFS numbers.
as updated
circled when
7.9.3
its
7.4
\342\226\241 Example
numbers
and
a cycle of
even
contained
in
of
graph
decomposition
undirected
graph G=(V,
graphs
and
\302\243),
length.
Hence, we component. then consider each components, that the graph is biconnected! assume at least it contains than one edge, then a biconnected
can first partition the graph into its biconnected component separately. In other words, we can now If the graph is biconnected and it contains more one cycle (in fact, every two edges are containedin a cycle). Let's find an arbitrary cycle C l = v,, v2,..., v*, v,. If A: is even, we are done. If there are no more edges \342\200\224 is, that the graph consists of exactly one odd cycle \342\200\224 then the answer is obviously negative. Otherwise,there is an edge not in the cycle such that one of its vertices is in the cycle. Let that edge be (v,, u>). Since the graph is biconnected, the edges (v,, w) and (v,, v,+i) are contained in another at w until we meet C{ again cycle C2. We traverse C2 starting
7.9
Input: of
(G, v, n)
Strongly_Connected_Components
Algorithm
G =(V, E) (a directed the DFS tree), and n the
marking
Output: the
of vertices
number connected
strongly
vertex serving as the
v (a
graph),
(the
of Graphs
Decompositions
in
components,
root
G).
and computing
values.
High
the case with directed { As is always have to be called several times until
this procedure may
DFS,
all vertices
have been visited.
}
begin
for
every vertex
vofGdo
v.DFS Number
:=
Current
{ we while
;
:= 0
Component
DFS_N := n
0 ;
:= 0
v.Component
;
;
use decreasing DFS numbers; there exists a vertex v such
see
that
the explanation v.DFS Number
in Section
7.9.1.}
= 0 do
SCC(v)
end SCC(v);
procedure
begin
:= DFS_N
v.DFSNumber
:= DFS_N
DFSN insert
v into :=
v.High
-
;
1 ;
STACK ;
v.DFS
Number
for all edges (v, w) do if w.DFS Number =
; {the
initial
value
}
0 then
SCC(w);
:= max
v.High
( v.High
, w.High
)
else > v.DFSNumber if w.DFS Number { (v, w) is a cross edgeora back
, w.DFS Number ) ; = v is root a v.DFS Number then a v.High { of component} Current := Current Component + 1 ; Component { mark the vertices of the new component} repeat remove xfrom the top of STACK ; v.High
if
edge
and w.Component = 0 then that we need to consider }
:=
max
( v.High
x.Component := Current until
x =
Component;
v
end
Figure
7.33
Algorithm
Strongly
Connected
^Components.
3
, (z, w), which Notice that alternating does not belong to A/. paths are The number of edges in P must be exactlywhat we used already to improve a matching. odd since P starts in V and ends in U. Furthermore, there is exactly one more of P edge in E-M than there is in A/. Therefore, if we replace all the edges of P that belong to M by the edges that do not belong to A/, we get another matching with one more edge. For the matching in Fig. 7.37(a) was example, the first alternating path we used to improve which was used to replace the edge A2 with the edges 1A and 2B\\ the (M, A2, 2\302\243), second alternating was (C5, 3D, D4, 4E, \302\2435),which was used to replace the edges path 3D and 4E with the edges C5, D4, and E5.
we have
an
For
improvement.
3D and
12
A
B
4E
with
example, the
3
4
5
6
12
C
D
E
F
A
B
4
5
6
C
D
E
F
(b)
(a)
Figure7.37Extending
3
a bipartite
matching.
7.10
It
then
now
be clear
should
M is
not maximum.
It
out that
A matching is will be
claim
The matching
a matching.
proved, in maximum
with the
of a
context
the
theorem
path
is not
only if it has no
if and
maximum
alternating
that
is an alternating path the opposite is also true.
for
a given
matching
A/,
Theorem
\342\226\241 Alternating-Path
This
if there
that,
turns
237
Matching
has
more generaltheorem, suggests
immediately an
path and
alternating
\342\226\2
paths.
alternating
an
any
in the algorithm,
alternating
next section. because
any can extend path
edges to the matching an alternating path, the until no more The and modify matching accordingly alternating paths can be found. alternating resulting matching is maximum. Since each path extends a matching by one of vertices), (where n is the number edge and there are at most nil edges in any matching is at most nil. The only of iterations is how to find the number remaining problem We We solve this as follows. transform the undirected problem alternating paths. graph the edges in M to point from U to V and directing G' by directing G to a directedgraph for the edges not in M to point from V to U. Figure 7.38(a) shows the matching obtained in Fig. 7.37(a), and Fig. 7.38(b) shows the directed the graph graph G'. An alternating from an unmatched vertex in V to an exactly to a directed path path corresponds can unmatched vertex in U. Such a directed path be found by any graph-search of a search is O (| V \\ + \\ E |); hence, the for example, DFS. The complexity procedure, as possible,
We
start
until
we
complexity of the An
greedy
We
get a maximal matching.
is O
algorithm
(|
V
|
as many
adding
algorithm,
(| V \\ + \\ E
for
search
then
|)).
Improvement
in Since a search can traverse the whole graph traverses one path, we might as well try to find We have to make sure, however, that these paths the independence of such alternating guarantee
the
same worst-case
several do
A
B
that
paths
U
3
4
5
6
12
C
D
E
F
A
B
3
4
5
6
C
D
E
F
V
V
(a)
(b)
Figure 7.38
Finding
alternating
it
alternating paths with one search. one another. One way to is to restrict them to be vertex
not modify
U
12
time
running
paths.
Graph Algorithms
238
disjoint. If
the
vertex
are
paths
disjoint,
improved applied concurrently. The new First, we perform BFS in G' following.
different
modify
they
algorithm
the set
so
vertices,
for finding
alternating
of all
unmatched
U are
found.
they
can
be
is
the
paths
in V, level we extract from Then, level, by in a set of vertex G' the BFS maximal induced (which are the disjoint paths by graph in G). This is done by finding any path, removing its vertices, alternating paths finding is not a maximum set, but and so on. (The result another path, removing its vertices, set in order to maximize the number a maximal of set.) We choosea maximal merely with one search to the added adds (each vertex-disjoint alternating matching path edges we modify the matching one edge to the matching). Finally, using this set of alternating no more alternating the new This process is repeated until (i.e., paths can be found paths. in V from the the unmatched vertices unmatched vertices G' disconnects directed graph a level
until
vertices
unmatched
which
in
from
in
vertices
inU).
Complexity 0(v\\V\\) [1973].
The overall
7.11
Network
The
Furthermore, E)
G=(V,
indegree weight
0,
that
1.
of
/ (the
an
edge.
0 to
satisfies
the
proof,
which
is thus
O ((|
V
+ \\ E
\\
|).
| )V|V
of
the
two
capacity of e. We
call such
The
flow
through
is a
the amount
function / on
an edge cannot
the
with
source)
(the
an associated
exceed
the
total
flow
that
can
we assign a the network
of
edges
positive
of flow
For convenience
a network.
graph
nonexisting edges. A flow following two conditions:
0, contrary there cannot be an edge (w, v) such that w$ A and v e A, and/ (w, v) > 0 (sinceit would extend an augmenting path). Hence, the value be a backward edge and it could of the of the cut defined by A, and /is maximum. flow /is equal to the capacity \342\226\
Algorithms
Graph
242
We have provedthe
Theorem
Min-Cut
Max-Flow \342\226\241
The value of a maximum of a cut. The
The \342\226\241
//
the
theorem
implies
maximum
matching,
augmenting path there is a maximum vertex
a
the
to the
the
is a
there
then
theorem.
augmenting-path
to an integral flow a flow of 0,
lead
paths
will
since
we
with
start
if
In fact, all the each
and
total flow.
\342\226\
Clearly, versa.
problem.
bipartite-matching
in
path
alternating
any
The augmenting-path augmenting path in the previous the alternating-path theorem given section. If M is a which then there is no alternating for it, path implies that there is no in G', which is the flow maximum. On the other hand, that implies to a matching since each flow, and it clearly integral corresponds in
of
flow
by only one 1. The
same
since, if
it
vice
and
G\\
edge (with
only
be maximum
must
matching
obvious
an
connected
V is
in
can support
integer
return to
now
We
G corresponds to
from
directly
is
This
are integers,
integer.
uses only augmenting an
capacity
theorem.
following
network
the
in
is an
follows
integers.
path adds
augmenting
edges
value
theorem
that
algorithm
capacities are
minimum
Theorem
whose
flow
Proof: The any
also implies the
of all
capacities
network is equal to the
\342\226\
Integral-Flow
maximum
in a
flow
theorem
augmenting-path
theorem.
fundamental
following
could
of V s; hence, each vertex the vertices of U. This be a larger then there would
1) to
capacity
for
holds
argument
be extended,
flow.
The augmenting-path 0, searchfor no more augmenting flow. Searching for flow of
residual
theorem
paths,
augmenting
We
paths. augmenting
with respect to
graph,
and augment
are always making can be done paths
a
network
in an augmenting
the
possible
belongs to F or a backward directed
regular
requires
\\E
|
the
augmenting it is either
if
edges
possible
flow through
a
forward
edge,
flow
Their
path.
those edges. case
More its
/,
the
precisely,
is c(v,
in which
graph
edge
w)-/(v, path
the
with
to
correspond an
case its capacity w). An augmenting is/(v, path from s to t in the residual graph. Constructing since each edge has to be checkedexactly once. steps edge,
increasing
residual
capacities
capacity
are
We define the as the network
the
in
a
with
start
until there
same edges, but
and the
The edges
capacities.
in which
and a
sink,
are
way.
following
E)
G=(V,
correspond to
different
the
in
We
accordingly,
progress since we
the same sourceand
= (V,
flow
the
F) with the same vertices, different directions and possibly
R
an algorithm.
suggests
immediately
residual
(v, w)
w),
is thus
a
graph
lead to a very in an arbitrary way may selecting augmenting paths be a The worst-case running time of such an algorithm may not even function of the size of the graph. Consider the network flow in Fig. 7.42. The maximum is obviously a 2Af. However, one might start with the path s-a-b-t, which can support one might take the augmenting 1. flow of only which again Then, path s-b-a-t, Unfortunately,
slow
algorithm.
Figure 7.42 augments
even
very large, of M can
case,
the
size of
number
of
(IV 13-
IV
next
has only
be
edges. I
)/4
polynomial
An
this algorithm is
above may be unlikely, to minimize the number
and Karp
upper
They
Some
this
We
started
the chapter
by taking
path
proved
chapter with
Problem
that includes
worst
the
in
precautions to avoid in
path
to speed
order
an
complexity not
describe
these
worst
whose
algorithms network
most
at
then
simple
of
up
things)
with the minimum
algorithm
others are relatively will
other
maintained,
different
Many
input.
are complicated;
is
policy
have been are really
(none
been algorithms here flow has
Bibliography section).
Tours with a
a discussion
discussion of a tour of a
tour
containing
all edges
containing
all the
also a famous problem, named after the Irish mathematician who designed a popular based on this problem in 1857. game
The
to take
augmenting
This leads to
required. the
the
this
if
that,
on the of 0(|V|3) of these algorithms. We
in the
have
we
of augmentations
bound
Hamiltonian
end
exponential,
[1972], for example,suggested(among
are augmentations in the size of
achieved by several (referencesare given
2M times, where M may be and five edges. (Since the value
repeated
vertices
four
bits,
O(logM)
augmenting
suggestedsince then.
7.12
processcan
input.)
Edmonds
the
selecting
the
we want
algorithm.
simple).
graph
of network flow.
example
the scenario
Furthermore,
case is
the
though
Although it.
1. This
by only
be representedby
the
in
flow
the
bad
A
243
Tours
Hamiltonian
7.12
Given a graph find G=(V, \302\243), once. every vertex of V exactly
vertices of
Sir
a simple
of a graph.
a graph.
William
cycle
R.
in
We
This
Hamilton,
G
is
Graph Algorithms
244
Such a cycle is called a graphs.
Hamiltonian
only the undirected
consider
the
Unlike
characterizing
interesting
Eulerian-tour
reversed
induction.
7.12.1
Reversed
We
have
(e.g., S =
set S the
theorem that
proving
an
algorithms
solve that
the
do
[2k),
base on
= 1,
affect
in
In
of finding
this
It
the
Hamiltonian
to
belongs
we present graphs that are
section,
special
only
is
example
use
as
hand,
base
values of
n
is much
case
the
it
Section
in
induction
2,...)
from n to
the other
problem
not
we
version;
will
an interesting
of
cycles
(or
of NP-
the class
a simple example The
dense.
very
called
technique
Induction k
going
infinite
cycles
this
P(n) holds for all of the validity
mathematics,
11.
in Chapter
seen reversed
already
are called
such cycles
undirected
an
is very
graphs)
of
the problem difficult.
problem,
discussed
part
directed and
version.
find Hamiltonian
we
which
most
containing
Graphs
cycle.
has a
problem
Hamiltonian
problems
complete in
Hamiltonian
The
n
case that
P(n) implies - 1 is not easier
more is almost
difficult
Then, we go of
validity
than going from than
a
can
n-\\ to w,
from
many
that
\"backward,\"
and
in
proving
When designing namely, to
nton-\\,
introduce in
infinite
Usually
P(n-\\).
one.
simple
always easy to go
for smaller inputs. For example,we the outcome. As a result, it is sufficient
use an
is, we prove
That
induction.
to S.
belong
the
The idea is to
2.11.
for the
\"dummy\" inputs cases to design the
not for inputs of all sizes, but only for sizes taken from an infinite set. The algorithm most common use of this principle is designing algorithms for inputs of size n which only is a power of 2. It makes the design much cleaner and eliminates many \"dirty\" details. will have to be resolved eventually. But it is more convenient to Obviously, these details solve the main problem first. We use the assumption that n is a power of 2 in several the book Sections and 8.2, algorithms throughout 9.4). (e.g., The same method is also useful when on the number of possible there is a bound elements. The base case of the theorem can be the instance with the maximal number of elements (rather For than the minimal number). The proof can then \"go backward.\" to apply about and we want example, supposethat we want to prove a theorem graphs induction on the number of edges. has the We can start with the complete graph, which maximal number of edges for a fixed number of vertices. We can then prove that the theorem continues to hold even if we remove an edge (as opposedto the usual adding of an This gives us extra flexibility in applying The next algorithm induction. edge). illustrates
this principle.
7.12.2
Finding
Hamiltonian Cycles in Very Dense Graphs
of the let d(y) denote the degree and G=(V, E) be a connected undirected graph, vertex v. The following involves problem finding Hamiltonian cycles in very dense is We will show that the conditions of the problem that the graph graphs. guarantee the to illustrate the principle of reversedinduction. We introduce Hamiltonian. problem Let
Given The Problem n > 3 vertices, such that satisfies d(v) + d(w)>n,
The
algorithm
is based on
the
complete
graph.
Hamiltonian
and
cycle
connect them
in a
Induction satisfying
undirected graph G = (V, of nonadjacent vertices pair
a connected
each find
a Hamiltonian
reversedinduction
Every complete it is easy to find
cycle
with
one
with
E) v
245
and
w
G.
of edges. The basecase is three vertices contains a all vertices in an arbitrary order and
the number
on
graph
in
Tours
Hamiltonian
7.12
(put
at
least
cycle).
hypothesis: the
given
how to find a with > m edges.
know
We
conditions
Hamiltonian
cycle
in graphs
how to find a Hamiltonian cycle in a graph with m -1 edges that a graph. of the problem. Take any pair of Let G =(V, E) be such w v in is and and consider the which the same as G vertices G', G, graph nonadjacent w we the know how v are connected. induction to find a that and By hypothesis, except a in in Let be such G' G'. Hamiltonian (see Fig. 7.43). If cycle x{, jc2,...,Jt\342\200\236,xx cycle in G and we in is contained the cycle, then the same cycle the (v, w) is not included edge we can assume that v=x{ and w=xn. loss of generality, are done. Otherwise,without The stage is now set to find a new for G, d(v) + d(w)>n. By the conditions given We
to show
have
satisfies
the conditions
Hamiltonian
cycle.
Consider all the the
two
conditions
vertices
of the
jc, and
edges
in G
problem).
jc/+1
, which
of
out
xh
the
edges
not */_!,...,
Figure 7.43 Modifying
v and
n-2
G contains
are neighbors
and w is connected to xh Using does Hamiltonian cycle that
v(=x!), xi+l,*/+2,...,w(=xn\\
coming
But
in
the
cycle,
(v, jc/+1)
use v (see
w. There are at least n of them (by other vertices. Therefore,there are such that
v
and (w, *,), we
the
edge Fig. 7.43).
Hamiltonian
(v,
w).
cycles.
can now
find
to jc,+1 a new
is
the
cycle
is connected
It
Graph Algorithms
246
Implementation
The
complete graph much smaller graph and
edge at a
one
replaces
of this
implementation
straightforward
as follows. Take
time.
graph G,
proof
do better
can
We
with
starts
by starting
the a
with
path (e.g., by DFS), to complete this path to a Hamiltonian cycle. a Hamiltonian have a larger graph now We cycle. Usually, only few edges n 1 will be added. We be added. However, even in the worst case, at most will edges with is G', until a Hamiltonian can apply the proof above iteratively, starting path of steps to replace an edge is O(n). There are 0(n) number obtained for G. The total add the
and
edges
from
(not
edgesto
replace;
7.13
Summary
hence,
the
input
a large
find
G) necessary has G', which
the algorithm
in time
runs
0(n2).
among pairs of objects.Sincemost algorithms an is input, the first issue involved in graph algorithms require We studied two types of graph traversal. traversals: search depth-first frequently graph search DFS was more (BFS). We saw several exampleswhere (DFS), and breadth-first we suggest trying DFS first (although there are many suitable than BFS. Therefore, BFS is superior). DFS is especiallysuited where for recursive examples algorithms on BFS also more this is not a rule \342\200\224 it usually space (although again, graphs. requires on We the have also seen an of which was search, depends graph). priority example from a single source. Priority used to compute shortest paths search is more expensive to model relationships
used
are
Graphs
examination of
than
for
have
for
useful
major difficulties directed acyclic graphs are
It is
important
different
many
for
much
usually
cycles (Exercise
7.54).
even
with
involving
to design a small number
easier
that
Algorithms
graphs.
weighted
Therefore,
algorithms.
graph
that graphs
realize
to
problems
optimization
usually cause
Cycles trees or
execute.
It is
search.
regular
whole
the
components,
and
consideration
may
properties not have
by
algorithms.
We
Network-flow
discuss
problems also
discuss a class
help
of
connectivity),
reasonably
biconnected
Decomposition basically allowsus to even the graphs under though
them.
this and
us to
components.
(such as
technique matrices there
represented
Reductions
connected
useful
Another
a
in
strongly
certain
assume
all or
checking
require
to
faster
of edges can
a graph can be very slow for most graphs. cycles useful. it is also usually Graph decomposition is very Fortunately, We have seen decompositionsinto connected inexpensive. components,
large
of the
fraction
algorithms
and
for graph
is a
called running
reductions
and
whether
Since graphs
is reduction.
relationship
problems
matching
solved by algorithms whose worst case. This class includes
natural
relationship
determine
problems,
algorithms
between
in general
are excellent
a problem
graph in
and
can be matrix
10.
Chapter
source for reductions.
is difficult.
In
11, we
Chapter
which probably NP-complete problems, in the size of the times are polynomial
be
cannot input
in
the
numerous The differences betweeneasy graph problems. sometimes seem minuscule. For example,we have seen an efficient algorithm whether a directed graph to determine a simple cycle of odd contains the same vertex length; problem with the extra constraint that the cycle contains a given to understand and develop an intuitive feeling (or edge) is NP-complete. It is essential problems
and
hard
problems
Bibliographic Notesand Further
for
differences.
these
Thus,
is a
theory only
this
field,
and
[1979],
for
important
very
now
graph books
the
Berge [1962], Ore [1963], Harary [1976], Chartrand [1977], Capobiancoand and Chartrand and Lesniak [1986]. There [1984], including
algorithms,
graph
and related
Even
results
basic
were
theory is a developedand on graph theory have been [1969], [1973], Deo Berge
Murty
Tutte
books devoted to
Most of
in mathematics.
field
new
Nevertheless, by century. with thousands of results. Many
them
among
[1974], Bondy Bollobas
relatively
in
well-understood published,
11 is
Chapter
and Further Reading
Notes
Bibliographic Graph
in
graph algorithms.
understanding
discovered
material
the
247
Reading
Molluzzo
[1979], Golumbic
[1978], also
are
several
[1980] (which
classes of
Gondran and Minoux [1984] graphs), Gibbons Nishizeki and Chiba [1985], (which [1988] emphasizes optimization problems), and a survey paper by van Leeuwen (which is devoted to planar [1986]. graphs), The notion of Eulerian graphs is due to Euler [1736], and it is regarded as the first An algorithm in graph theory. for finding Eulerian paths result can be obtained quite Even from the for or Ebert search (see, [1979] [1988]). easily proof example, Depth-first was first described by Lucas work by Tr6maux) and Tarry [1895], [1882] (describing where it was used to design algorithms to traverse a maze. The importance of depth-first in the of who also the search was made evident work algorithms presented Tarjan [1972], for biconnected and strongly connected components. tree has been studied extensively. The The minimum-cost spanning problem is due to Prim in Section 7.6 (although not its implementation) algorithm presented is the of Exercise is due to Kruskal 7.59) [1957]. Another (which algorithm subject were for the minimum-cost tree Other [1956]. finding algorithms spanning developed by and Gabow, Galil, Yao [1975], Cheriton and Tarjan [1976], Fredman and Tarjan [1987], emphasizes
perfect
graphs
Tarjan
[1986]
Spencer,and
in Section 7.5 was for single-source shortest paths presented a heap is due to Johnson [1977] The implementation using [1959]. the case in practice, this When the graph is sparse, as is usually (see also Tarjan [1983]). is a fast algorithm. If the number time of edges is proportional to | V |2, then the running of this algorithm with a is O (| V | 2log V for dense A better | \\. graphs, implementation The time of is the of Exercise 7.43. best-known 0(\\V |2), running asymptotic subject time for this data structures) is running (using problem quite complicated due to Fredman and Tarjan [1987]. The all-pair 0(\\E | + |V|log |V|), a result in Section 7.7 is due to Floyd [1962]. It works algorithm shortest-paths presented
The
algorithm
developed by
correctly
Dijkstra
for weighted
negative weight on the average 0(| and time survey
V an
|2 log2
\342\200\224
Spira
|V
earlier
is 0(\\
graphs
(Exercise
cycles
|),
and
by Deo
negative weights, provided that It is possible to find all the shortest
possibly
7.73).
[1960] to obtain |). For more information and Pang [1984] (which includes,
|2 log
|V
there
presented an algorithm whose average running Takaoka [1987] used a hybrid of Spira's
and
by Dantzig
algorithm V
[1973] Moffat
with
an on
algorithm shortest-path
among
are no
paths faster
whose average running see algorithms
other things,
is
time
algorithm the
222 references).
248 Graph Algorithms 7.8 is due to Warshall [1962]. algorithm presented in Section to network flows its theorem and were application augmenting-path of data the excellent and Fulkerson An Ford structures [1956]. discovered description by A new for flows is given in Tarjan network [1983]. and combinatorial algorithms and flow was for network Tarjan [1988]. recently developed by Goldberg algorithm on the network-flow More information problem and many of its extensions can be found Christofides in Ford and Fulkerson [1962], Hu [1975], Lawler [1976], Minieka [1969], The
closure
transitive The
and Plummer and
theory
in
Hamiltonian
finding
[1986] covers for various
algorithms algorithms
matching
[1982], and
and Steiglitz
Papadimitriou
[1978],
Lovasz
bipartite
in dense
cycles
mathematical
problems.
matching
and
and
Gondran
the
both
Galil
Minoux
[1984]. A book foundations of matching
general graphs
a
[1986] presents
in graphs. The algorithm is based on a theorem (and
of
survey
7.12.2 for
Section its
by
by Ore
proof)
[I960].
Two important
subjects
isomorphism.
The
algorithms were not discussed of characterizing planar graphs in graph theory. is one of the oldest in the plane problems Early Auslander and Parter and were [1961] developed by problem to determine whether Cederbaum [1966]. A linear-time algorithm graph
developed by
and Tarjan
Hopcroft
a graph
decompose
polynomial-time
isomorphism is
[1974].
It
a linear-time
uses
3-connected
the
NP-hard) is still see, for example,
unknown
(more
Hoffman
[1982],
on that or Luks
in
11).
Chapter
here: and
algorithms Lempel, a graph
and them for this
planarity
embedding
and
Even,
is planar was
to (DFS-based) algorithm and Tarjan [1973]). This and data structures. A
components (Hopcroft of many other development algorithms for graph isomorphism has not been algorithm one of the very few major problems whose status into
motivated
algorithm
in graph
problem
For a
found (either
Graph
yet.
or
polynomial
discussion on
this
topic
[1982].
Bruijn sequences (Exercise7.28)can be found in Even [1979]. and Vitter [1986]. Exercise7.55 is motivated Sedgewick by an exercise from Bollobas [1986], and Exercise from 7.58 is motivated by an exercise Lovasz [1979]. Ford of contains an algorithm that satisfies the requirements [1956] Exercise 7.75. The algorithm for transitive in Exercise closure hinted 7.81 is from Warren Exercise 7.97 is from Lovasz and Plummer Gabow and Tarjan [1975]. [1986]. The for in the bottleneck Exercise 7.100. [1988] presentan efficient algorithm problem theorem presented in Exercises 7.101 and 7.102 is known as theorem. Gomory's Exercise 7.105 is from of Lovasz Exercise 7.121 is related to a problem [1979]. in Manber and McVoy is solved designing space-efficient [1988]. routing tables, which A
on de
discussion
Exercise7.46 is
Drill
7.1
from
Exercises
Considerthe
problem
Solve this problem using
7.2
LetG = (V,
E) be a
of finding balance factors in binary trees discussed in Section DFS. You need only to define pre WORK and postWORK.
connected undirected graph,
and
let T
be a DFStree
of G
rooted
at
5.8.
v.
a. Let H be an b. Let R be Prove that
7.3
7.4
7.7
7.8
Characterize
all
and
identical
traversed
is
H is
let S be tree of S.
the
of G
subgraph
induced
vertices
in R.
the
by
graphs
at v that if
immaterial
is identical
7.14) in the following way. Assume that you if the graph is cyclic, a graph is acyclic. Obviously, that will output the topological-sort sort is impossible. Design an algorithm topological if the graph is acyclic,and will output a cycle if the graph is not. The running time labeling of the algorithm should be O (| E \\ + \\ V |). Topological
algorithm
Modify
know
or
whether
Sorting (Fig.
not the
Consider algorithm Source Shortest Paths (Fig. 7.17) Prove Single consisting of all the edges that belong to shortest paths from v, found of the algorithm, is a tree rooted at v.
the
that
Prove or show works correctly there are
a counterexample: for
weighted
no negative-weight
7.11
Prove
MCST (Fig.
algorithm
graphs some of
weighted
7.20) to
graphs someof whose
7.12 a. Give such rooted b.
an example that
graph.
minimum-cost find
a counterexample: algorithm have negative edges
or show
the
Single Source Shortest
whose
of a weighted
minimum-cost
Prove that, tree.
if
all the
a maximum-cost spanning
MCST (Fig. 7.20)works costs.
connected
undirected
graph
spanning tree of G is
the
same
(Fig.
7.17)
weights, provided
costs are distinct,
then
tree. for weighted
correctly
G =(V, E) and a vertex v, as the shortest-path tree
at v.
connected undirected an example of a weighted graph G the minimum-cost spanning tree of G is very different tree rooted at v. Can the two trees be completely disjoint?
=(V, E)
that
Describe the changes in deleting the vertex c from
7.14 a. Run
a constant
spanning
Give
such
Paths
have negative
edges
tree rooted at by
cycles.
Let G = (V, E) be an undirected there exists exactly one unique Modify
Algorithm
subgraph execution
the
during
and let T be the shortest-paths Let G = (V, E) be an undirected weighted graph, a vertex v (Exercise 7.6). Suppose now that all the weights in G are increased tree from v? number c. Is T still the shortest-paths
7.10
7.13
of T and
v such that there exists a DFS that contain a vertex to a BFS spanning tree rooted at v. (Two spanning same contain the set of the order in which they are they edges; here. However, both trees must have the same root v.)
undirected
rooted
tree
trees are
that
7.9
7\\
the intersection
input
no longer
7.6
of
be a DFS
R could
that
Show
of H.
tree
is a connected undirected graph a spanning tree T of G, and a vertex v. G = (V, \302\243), T a to is a determine whether valid DFS tree of G rooted at v. In other Design algorithm T can be the output of DFS under some order whether of the edges words, determine should be O (| E | + | V |). starting with v. The running time of the algorithm
The
spanning
7.5
a subtree
G.
subgraph of
induced
arbitrary
not necessarily a spanning
249
Exercises
Drill
the
biconnected
the
biconnected components in Fig. 7.25.
and
from the
and biconnected
tree
a vertex
shortest
resulting
from
the graph
components
algorithm on
the
graph
in Fig.
7.44. The
v,
path
algorithm
250
Algorithms
Graph
should b. Add
7.15
that
7.16 a.
the edge
there
are no the
Run
(4,8) to the
in the figure. Show
given
the
High
values
as
graph
and discuss the
changes this
to the
makes
algorithm.
of a biconnected tree in Section 7.9.1 is valid. You have to show are connected. cycles, and that the set of all biconnected components
strongly should
algorithm
b. Add the
connected components the DFS numbers
follow
values as computed
7.17
are
that
in each step.
the algorithm
the definition
that
Prove
by
DFS numbers
the
follow
computed
by the
edge (4,1)to
algorithm
the graph
in
each
algorithm
the graph
on
are given
that
in
the
in Fig. 7.45. The Show the High
figure.
step.
and discuss the
changes
this makes
to the
algorithm.
connected Let G = (V, E) be a strongly graph and let T be a DFS tree in G. Prove that, if all from G, the resulting graph is still the forward edges in G, with respect to 7, are removed strongly
connected.
Figure 7.44 An
undirected
graph with
DFS numbers
for Exercise
7.14.
1
Figure
7.45
A directed graph
with
DFS
numbers
for Exercise
7.16.
Creative Exercises
7.18 a.
the
Prove
of the algorithm for
correctness
an odd
finding
length cycle
in
a directed
251 graph
(Section 7.9.4).
b. Show 7.19
7.20
of a graph
an example
that is
not
connected
strongly
for which the algorithm
does
work.
not
Show an implementation of the algorithm discussed in Section 7.10.2 to find a perfect in with In with a each at least n. Your algorithm should vertices, matching degree graph in time O (| V \\ + \\ E \\) in the worst case. run somewhat the proof of existence of perfect in dense matchings are given a graph with 2n vertices such that not all of them have the sum of their degrees is at least In. Is vertices, but, for any two nonadjacent high degree, it still true that a perfect matching exists? Is the obtained in Exercise 7.19 always algorithm still valid? exercise
This
generalizes
graphs. Suppose that
you
Exercises
Creative
we assume that the graphs are given in an adjacency-lists Unless otherwise, specified O (| V | + | E \\) space; hence, we say that an requires representation. Such a representation time if its running time is 0(|V | + |\302\243 Unless specified runs in linear |). algorithm are In times worst case. some time is all the cases, a particular otherwise, running running for an that in other we ask and the exercise time; cases, requires achieving only given \"efficient algorithm.\" In the latter case, the reader should try to find the best possible is of course, the best running time is unknown when a problem In practice, algorithm.
encountered.
7.21
an undirected
Given
H of G
such
that
induced
subgraph
includes
all edges
induced subgraph graph G = (V, E) and an integer k, find the maximum that it does not exist. (An each vertex in H has degree > k, or determine F // = (\302\243/, and of a graph G = (V, E) is a graph F) such that (/cV, E both of
in
time. (This problem
7.22
is discussed
whose vertices are in Section 5.3.)
in
U.)
The algorithm
should run
in linear
undirected (V, E) be a connected graph. We want to pick a vertex of degree 1 of G, another vertex it and its incident edge from G, and continue this process (i.e., taking of degree 1 in the remaining graph, removing all edges are removed. If it, and so on) until this procedure is always for certain graphs, then designing possible algorithms by induction for these graphs easier. that these be Characterize connected undirected may graphs satisfy In other words, find necessary conditions. and sufficient conditions for a graph G on which the procedure described above is possible.
Let
G =
remove
7.23
Describe
7.2.
an efficient implementation of the Eulerian graph algorithm discussed The algorithm should run in linear time and space.
7.24 Let G = (V\\ linear-time
equal to the 7.25
A directed
E)
be an
algorithm
in
Section
undirected graph such that each vertex has an even degree. Design a to direct the edges of G such that, for each vertex, the outdegree is
indegree. Eulerian
circuit
is a directed circuit
that
contains
each edge exactly
once. Prove
252
Graph Algorithms
an efficient
circuit if and only if the indegree of each contains a directed Eulerian to its outdegree, and the underlying undirected graph is connected. Design algorithm to find such an Eulerian circuit if it exists.
Let G = (V,
E) be
a directed graph
that
is equal
vertex
7.26
a. Prove b.
7.27
graph
of odd degrees.
with k vertices
even.
k is
that
Design
an
exactly
one of these
to
algorithm
k/2
find
paths such
open
that
each
in G is
edge
included
in
paths.
a vertex in a connected undirected whose removal does graph The not algorithm should run in linear time. (Do not use the graph. As a consequence, prove that every connected biconnected components algorithm.) graph contains such a vertex.
Design an algorithm to
de
A binary
s of size
unique index i
n
somewhere
is represented
s =
that
such
sequence of 2\"
is a (cyclic)
sequence
Bruijn
string
binary
find
the
disconnect
7.28
an undirected connected
at
ai+x
\342\200\242 \342\200\242 \342\200\242
in
(where
fl/^-i
the
bits
\342\200\224 \342\200\242
a2\" such
axa2
that is,
sequence;
the indices are
taken
there
modulo
that each exists
a
2\.") For
11010001 is a binary de Bruijn sequence for n =3. Let Gn = (V, E) as follows. The vertex V defined set to the of all binary set corresponds graph \342\200\242 \342\200\242 - 1 A vertex corresponding to the string axa2 \342\200\242 (| V \\ =2\"~l). strings of size n an.x has ''' to a vertex corresponding to the string bxb2 an edge leading if and only if bn_x \342\200\242 ''' an-\\ -bxb2 \342\200\242 \342\200\242 Prove that Gn is a directed Eulerian and discuss bn_2. #2^3 graph, the implications for de Bruijn sequences.
example, the
be a
7.29
sequence
directed
/', =0, and
= 1 for 1
ik
integers n vertices of
positive with
dn.
d2,...,
dx,
Let (/',, o!),(/'2,o2),...,(/\342\200\236,on) a.
n
that
such
dny
degreesexactly 7.30
Given algorithm for the following problem: \342\200\242\342\200\242\342\200\242 construct a tree +dn = 2n-2, dx+d2+
efficient
an
Design
dx>d2,...,
be a sequence
of pairs
of integers
such
that
< k , and pk+\\ can be added. Are we done? Not quite. can be eliminated, the modified path is not necessarily convex. Indeed,Fig.
of the
side
\"left\"
277
(p \\,
p2,...,
in
8.12.
Fig
pn);
of points in the plane). 0,2,4,6(C04)
=
Po,2aA2,6(co4)
/>o,4(C08)
+ C04-/>2,6(C08)= +
^0,4(C012)
= -4
\342\200\242
CO6
same
co6):
= 4 + 8=12.
= />o,4(l)+l-/>2,6(l)
/>o,2,4,6(l)
the
In
/>2,6(C012)
+ co2(-4).
/>0,4(l)-l-/>2,6(l) = 4-8 = -4. =
/>0,4(C04)
-
CO2
\342\200\242
= ~*
/>2,6(C04)
~ ^HO-
So, overall
same
In the
CO4, CO6) = (
CO2,
^0,2,4,6(1,
P 1,3,5,7(1, co2,co4,co6)= ( To
compute
= 12-
co, co2,...,
1-16=-4; P0t
in
the
+co2)) +
= (-4(1
1,2,3,4,5,6,7(co)
we need to
co7),
12+ 1 \342\200\242 16 = 28, and,
= (-^4(1+co2))- co-(-4(l+ manner, ^0,1,2,3,4,5,6,7(co5) to the reader. The
Fourier
Inverse
The algorithm
for
the
evaluate
the
multiply
the resulting
two
).
16,-4(l+co2),-4, -4(1-co2)).
/>o,i,2,3,4,5,6,7(l\302\273
P 0,1,2,3,4,5,6,7(1)=
example,
-4, -4(1 -CO2)
find that
we
way,
12, -4(1+C02),
given
same
compute 8
manner,
P0,i,2,3,4,5,6,7(co4)
co-(-4(l +co2)),and, co2)),
and
so on.
For
values.
in
the
We leave the
same rest
\342\226\2
Transform
fast Fourier
values, and
solves
transform
polynomials find
p{x)
and
the values
q{x)
of
only half of at the points the
product
our
problem.
1, co,...,co\"\"1
polynomial
p(x)-q
We
can
quickly,
(x)
at
we still need to
But
those
points.
from
the evaluation points.
similar to the evaluation Consider again
vector
the
Fortunately,
the
and we
polynomial,
points 1, co,co2
at the n
algorithm
to
want
, we
co\"
can
we are
When
compute
out
to
solve
it.
values of
the
matrix
be very
coefficients
the
given
compute
the
polynomial
product
turns
problem
interpolation
notation.
matrix
of the
coefficients
and an almost identical
problem, the
...,0\342\200\236_i)of
(a0,ai, polynomial
the
interpolate
315
Transform
Fast Fourier
The
9.6
by
the
product:
1
1
co
CO\"
2 CO
oy
On
the
(v0,
v j,...,
other
system
following
2-2
co2'\"\"\"
oy((n-l
and we
are given, v\342\200\236_!)
1
1
1
1
co
CO\"
of
want
/>(0)2)
)\342\200\242(/.-!)
/>((0\"_l)
an-\\
the
(fO), f^co), ...,/> (cow_1))= the coefficients, we need to solvethe
polynomial
to compute
a0, a \\,...,0\342\200\236_i:
of equations for
1
\"
v0\"
\"tfo\"
o)\"-1
2-2
CO\"
P(l)
a)\"\"1
the values
when
hand,
1
co(\"_1)*2
co\"\"1
Vl
tfl
v2
a)2c-|>
CO
=
1 CO\"\"1
Solving case),
(vo.
where V(co) is
Tne \342\200\224\302\273 vi\302\273 vn-\\)-
V(co) has inverse
an
has a
simple
very
of
the
matrix
for a
form
out
that
(we omit
011-1 .
.
time
quite
in
the
left
can be written V(co) the
v\342\200\236-i -
consuming Let's write
equations.
always
(9.10)
(0(az3) for this
matrix
the
general
equation
as
side, a =(a0,a\\, ...,an_i), that a = [V(co)]_1 \342\200\242 v, provided its has an inverse; furthermore, and
as
proof):
9.1
\342\226\241 Theorem
[V(co)]\"1 =
usually
solution
It turns
inverse.
oy((n-l)(n-l)
CO*\"\"02
systems of equations is but this is a special system
V(co)-a=v, v =
the
following
- V(-). co
\342\226\2
n
to solve
Therefore,
by vector product.
the system of equations we need to compute (9.10), This task is greatly simplified by the following theorem.
Theorem \342\226\241
If (d is a primitive unity.
only
matrix
one
9.2 nth
root
of
unity,
then
1/co is
also a
primitive
nth
root
of
\342\226\2
and Numeric
Algebraic
316
Therefore,we
the product
V(l/(fl)v by substituting 1/(0 for co. This transform
can
compute
transform,
Fourier
Algorithms
the
using
for the inverse
algorithm
is called
fast
the
Fourier
transform.
Notice
operations.
0(n\\ogn)
product of two that we need to
the
Overall,
Complexity
be
computed with complex
and multiply
add
to
able
be
can
polynomials
numbers.
9.7 The
Summary
numerical
We have
algorithms.
Strassen's
best.
the
lecessarily
in this
presented
algorithms
nonintuitive
algorithm
examples of
the
for
chapter are a seen again that
a seemingly
and,
of induction,
use
is
algorithm
small
one
known
straightforward of the most striking
simple problem. in
of
sample
the
particular,
of the
use of
of
examples
seen
have
We
algebraic and algorithms are not a
more
several
divide-and-conquer
algorithms.
which is not based algorithm suggests an interesting technique, The main idea is to compute all possible combinations of certain terms, in them are This is useful cases where needed. even if not all of technique computing all combinations costs much less than each one separately. (or many) together computing Another for problems which is common particularly technique, involving matrices, is the use of reductions is described, between problems. This method with examples, in four-Russians
The
on
induction.
10.
Chapter
Notes
Bibliographic The include
Aho,
[1980],
for arithmetic
source
best
and
and
Hopcroft,
and and
probably algorithm
is Knuth Munro
and
[1981]. Other books Winograd
[1975],
[1981].
Lipson
schemes
encryption
algorithm
algorithms
algebraic
powers by writings circa 200 B.C. (seeKnuth scheme is due to Rivest, Shamir,
encryption
Reading
Ullman [1974], Borodin
The algorithm for computing in Hindu
key
Further
repeated
squaring
is very
old;
it
was introduced in Euclid's Elements,
Book 7 (circa 300 B.C.),but known even beforethen (see Knuth [1981], page 318). The divide-and-conquer for multiplying and two polynomials was developed by Karatsuba appeared
first
appeared
The RSA public-key [1981] page and Adleman [1978]. The idea of publicHellman [1976]. Euclid's Diffie and by 441).
it
was
Ofman
two large numbers). in Winograd [1970]). [1968] (see also Winograd Winograd's algorithm appeared Strassen's in in the Strassen The c constant [1969]. algorithm asymptotic appeared 1969 since has been reduced several times multiplication running time 0(nc) for matrix of asymptotic in terms at this time \342\200\224 (first by Pan [1978]). The best-known algorithm \342\200\224 is by Coppersmith times and Winograd [1987], and its running time is running as the CO notation indicates, this is not (D(n2316). Unfortunately, practical. algorithm of matrix multiplication and related For more on the complexity topics see Pan [1984]. A [1962] (in
the
context
of multiplying
discussion on Roth [1976].
the
The four-Russians
of
the
Myers
in
and
Cohen
to Arlazarov, Dinic, Kronrod, and Faradzev tables has algorithm by using addition it is without in mentioned, details, people; Rytter is used for context-free language recognition. The
been observed by many [1985], where a similar technique same idea was also used to improye [1983],
be found
317
four-Russians
probably
Paterson
algorithm can
Strassen's
Exercises
is due
algorithm
The improvement
[1970].
of
implementation
Drill
Exercise
of
solution
(Masek
algorithms 9.26 appears in
comparisons
sequence
[1988]). The
and
and
Atkinson
showed a reduction between Boolean matrix [1988]. Fischer and Meyer [1971] and the transitive-closure multiplication problem. The algorithm for the fast Fourier transform was introduced by Cooley and Tuckey can be traced to Runge and Konig [1965], [1924]. For although the origins of the method more information on the fast Fourier transform, see Brigham and Rao [1974] and Elliott Santoro
[1982].
Exercises
Drill
9.1
the
Discuss
computing
9.2
9.3
Algorithm Power_by_Repeated_Squaring (Fig. 9.2)for lead to the minimal number of multiplications. Show an of multiplications. with fewer number
Let x be a positive Design an algorithm
(a, then
9.4
between algorithm Power_by_Repeated_Squaring relationship nk and the binary oik. representation
not
of computing
example
necessarily nk
(k>
10)
= a/b. number that is represented by the pair (a, b) such that x of x\\ that is, the representation to compute the smallest representation b) with the smallest possible values of a and b. For example, if x = 24/84 = 6/21 =2/7, of x. (2, 7) is the smallest representation
Prove that
nk does
computing
(Fig. 9.2) for
that
the
computes as
operations
rational
divide-and-conquer algorithm for of the smaller polynomials products
straightforward all
does
four
the straightforward
algorithm
follows
that
polynomial
makes
(9.1).
multiplication
exactly
Assume
the same that
n is a
power of 2.
9.5
the
Find
P(x)=x
algorithm
+ 2x2 + 3x* +
\302\243)(.*:)=16+15*
How
9.6
P(x)-Q(x\\ presented
product
multiplication
+14*2
many operations
hand, Section
using 9.4.
the divide-and-conquer
polynomial
\342\200\242\342\200\242\342\226\240 +
15jcl\\
+
\342\226\240\342\200\242\342\200\242 + 2x,4 1jc'\\
+
are required
A divide-and-conquer technique and discuss an algorithm, such algorithm.
by in
overall? can the
be used
differences
to
multiply
between
two binary numbers. Describe it and the polynomial multiplication
318
and Numeric
Algebraic
9.7
Use the algorithm
9.8
A
way to
all the
Design
an
Derive the in
9.11
two numbers 4679x7114.
base b (not the recursion Carry in
any
two complex numbers
with
only three
scheme
described
(a+bi)(c+di)
of -1.)
root
2x2
for Strassen's
matrix
multiplication
9.5.2.
Section
Suppose that
you
be the
would
square
expressions
explicit
by 10111010.
10011011
multiply
can be used to multiply the decimal multiplication
to multiply
(/ is the
9.6 to
Exercise
numbers.
1 -digit
algorithm
multiplications.
9.10
discussed in
divide-and-conquer technique b = 2). Use it to perform only down
9.9
Algorithms
complexity of a
What
algorithm?
find an algorithm
value
matrices
4x4
multiply
matrix
general
maximal
is the
to
multiplication
will lead
of k that
to
with k multiplications. What algorithm based on the this an asymptotic improvement
over Strassen'salgorithm?
Exercises
Creative
9.12
Consider
the
two
for computing
algorithms
Let n
be an
nk
9.2 (simple iteration,
in Section
given
Assume
d digits.
and
repeated squaring). integer multiplications integer are performed by the regular algorithm, which requires J, \342\200\242 two integers d2 steps to multiply with dx and d2 digits. What is the number of steps required to compute nk by the two algorithms? (You can assume that k is a power of 2, and that a product of two integers with with d]-\\-d2 digits.) dx and d2 digits is another integer
9.13 9.14 9.15
9.16
to
an algorithm
Design
is a multiple
The Fibonaccinumbers
a.
Prove
of m and
n
integer
LCM of k given of all of them.)
> 2
n
is the
integers.
is a
that
multiple of
integers.
given
(The LCM of
k
integers
is the
recurrence relation:
by the following
can be
smallest integer
LCM of two
F{n) = F{n-\\) + F{n-2)
every
that
of k integers.
to find the
are defined
F(2)=l, that
(LCM) algorithm
to find the
Design an algorithm smallest integer that
F(\\)=\\%
the GCD
find
The least common multiple both n and m. Design an
with
(n>2).
written
as
a sum
of
at
most
\\og2n Fibonacci
numbers.
b. 9.17
an algorithm
Design
Let P(x)
to
find
such
a representation
for a given
number
n.
be two polynomials. We say that a polynomial D(x) divides P(x) if Q(x) another polynomial S(x) such that P(x) = D(x)-S(x). Similarly, we say that and there exists a Q(x) = R(x)modP(x) if R(x) has a smaller degree than P(x), such that Q(x) = D(x) P(x)+ R(x). P(x) The GCD of two polynomials D(x) polynomial and Q (x) is a polynomial R (x) such that R (x) is the highest-degree polynomial that divides bothP(.v)and(2(.v). there
and
exists
a.
Show
b.
Extend
that
the GCD
Euclid's
of two
algorithm
polynomials
to
find
is uniquely
the GCD
defined.
of two given
polynomials.
319
Creative Exercises
9.18
the
Modify
multiplications involving of 3. What is the
smaller complexity
power
9.19
the
Modify
polynomial multiplication into four equal parts, and
polynomial
parts. You can assume of the algorithm? complexity smaller
9.20
Hamilton's real
and
i, j,
componentwise, and
= -kj
ki
= -ik /,
symbols
j\\
a. Prove that b.
= j
P[i]
Considerthe
by
required
a, b, c, and d are subtract quaternions
where and
is an
n
x n
with
themselves).
the ordinary
procedure
and
the number
reduces
matrix such
each
of two permutation
to The outcome should
multiply
also
many
quaternion
to 12.
each column be represented
row and
is equal to 1. A permutation matrix can the ith row contains a 1 in the jth column.
if
How for
only five multiplications.
with that
of multiplications
that
the product
representation.
matrices is another
permutation
has
exactly
by an array
P
matrix.
two permutation matrices given be given in an array representation.
by
the
array
We can use Winograd's to modify Strassen's algorithm. with approximately k312 the product of two kxk matrices We can then use this as the basis for the divide-and-conquer multiplications. product strategy instead of the one using 2x2 matrices. If k is large enough we get a better time than Strassen's algorithm. this suggestion? What is wrong with asymptotic suggestion
following
to
algorithm
9.24
that
a linear-time algorithm
Design
+ cj + dky
a+bi
form
the
with real numbers
numbers are
matrix
one nonzero entry
such that
of
the square of a 2 x 2 matrix
to compute
permutation
of
problem is a
size of the
algorithm described in Section 9.4 by dividing each the number of multiplications minimizing involving the size of the problem is a power of 4. What is the
are
k commute
and
Give an algorithm
A
each
= i = j
multiplication?
9.22
the
9.4 by dividing the number
= -l
of real
Show how
minimizing
special symbols. We add them by using the following rules: k
multiplications
9.21
9.23
and
multiply
/' = y2 = k2 ij = -ji = k
jk
that
Section
(instead
are vectors
quaternions
numbers,
(the
algorithm described in of two), and can You assume that parts. of the algorithm?
polynomial multiplication into three equal parts
polynomial
compute
to compute all possible vectors Boolean inner products of two Boolean algorithm create a table of size 22*. The product of should (see Section 9.5.3). The algorithm the two vectors a and b should be stored at entry i, where i is an integer respresented by 2k bits such that the k most significant of a and the k least significant bits are bits of i are those those of b. The running time of the algorithm should be O (22*).
Design an of size k
9.25
the program
Complete
tables
explicitly,
significant 9.26
Design submatrices such
should
and
for
matrix multiplication to handle the case where
Boolean
how
(Fig. 9.6).
Show
is not an
n/k
how
to build the
integer
a
without
loss of efficiency. an algorithm for Boolean matrix multiplication of size kxk, and uses the idea of precomputing
submatrices.
The running time
require extra
operations on
numbers
of
the
space of 0(n\\ogn). with up to
bits log2\302\253
in one
can
assume
step.
divides the matrices
all possible
should
algorithm You
that
into
products between
be 0(w3/(logw)15), that you can perform
and
it
basic
320
Algebraic and Numeric
9.27
Let A and B chosen
be two
(independently) the product of A and
9.28
nxn
Algorithms
random
to be By such
Boolean matrices; each
either
that the
0 or 1 with
expected
probability
number
entry
in each
matrix is
randomly
Vi. Design an algorithm
of operations
to
find
will be 0(n2).
B be two 2n x In Boolean matrices that represent open Gray codes (seeSection to the strings in the Gray in the following way. The rows of each matrix correspond code, such that two consecutive rows differ by exactly one bit (the first row and the last row to find the product of the two differ by more than one bit). Design an O (n2) algorithm may
Let
A and
2.9)
matrices. be n matrices of real numbers. The dimensions of A/, are fl,Xfl/+1, so 9.29 Let Af M A/2> \342\200\242\342\200\242\342\200\242\302\273A/\342\200\236 is defined for each 1 ,
programming
(10.1)
c(x)
e
subset
linear
The
of size ny
represented as a specialcaseof
can be
P is
where
usually
they
though
the
add
We can
constraint.
equality
arbitrary
for
substitute
(10.5)
the following
two
constraints:
inequality
(10.6)
erx:^l^J;-
-r-: -r-\342\200\242\342\226\240-' -:-r-r:- - -: -V-CC->\\-V^
11
CHAPTER
NP-COMPLETENESS
Give
me where
to stand,
Archimedes
and
(287-212
I will
move the earth.
B.C.)
Introduction
11.1
This chapter
from other chapters. In the previous chapters, we mainly them to specific and applied solving algorithmic problems techniques can be that if efficient be nice all had would algorithms elegant problems problems. It discoveredby a small set of techniques. But life is rarely that simple. There are still that we have learned so that do not seem to succumb to the techniques many problems that we just have not tried hard enough, but we strongly far. It is possible suspect that we there are problems that have no good general efficient solutions. In this chapter, some of these problems. describe techniques for identifying that we have seen so far were bounded times of most of the algorithms The running in We call such algorithms efficient algorithms, the size of the by some polynomial input. is quite
different
for
studied
the corresponding is efficient if its algorithm size of the input n. Recall and
call
required to represent algorithms
is
denoted
that
problems tractable problems.In that the
input.
is 0{P{n)\\
time
running
The
by P (for
size of
the
input
class of all
polynomial
words,
where P(n) is is defined as
problems that
time).
other
This
can
we say
a polynomial the
in
an
the
of bits
number
be solved
that
by efficient
may seem to be
a
strange
are not efficient Surely, algorithms by any standard in run time 107a? that are are not even that efficient, matter, (for algorithms though they is definition valid for this two the reasons. it allows Nevertheless, First, linear). are we about to which and most of the explore; second, important, theory, development the vast majority It turns out that of the tractable it simply works in practice. problems In other words, the than others). have practical solutions (of course, some are better definition.
that run
in
time
O (nl0)
NP-Completeness
342
times of
we encounter
in practice are mostly smallThe is also true: opposite usually quadratic). degree polynomials not are times whose are than usually any polynomial practical running larger Algorithms for large inputs. is known. no are many problems for which There algorithm polynomial-time that are yet to be of these problems may be solved by efficient Some algorithms We strongly that many problems cannot be solved however, discovered. suspect, like to be able to identify so that we do not have to We would such problems, efficiently. In this we discuss how to deal for a nonexistent time searching algorithm. chapter, spend with problems that are not known to be in P. In particular, we discuss one specialclass in one class these called NP-complete of problems, problems problems. We can group
running
above
(seldom
becausethey any
all equivalent
are
one NP-complete problems.
complete
for
The
chapter
problems
and
show
several techniques
These solutions
a strong
sense
\342\200\224 there
an efficient
exists
proof
problems,
they would
Even
known.
be
surely
very
for
algorithm
exist efficient algorithms there is no efficient algorithm
of that belief is
no
NP-complete
have been
problems
but
for all NPfor
any
were efficient since they complicated, if there
of (maybe even thousands) which is is so this why NP-complete, subject important. consists of two we define the class of NP-complete First, parts. how to prove that a problem Then, we present belongs to the class.
for
researchers
eluded
have
in
problem if and only if there There is a general belief that
problem,
NP-complete algorithms
that
algorithms
polynomial
years.
hundreds
to be
found
and
may
many
So far,
for
examples
be optimal,
not
and
solving they
NP-complete problems approximately. may not always work, but they are better
than nothing.
11.2 We
will
Reductions
Polynomial-Time restrict
ourselves
those problemswhose
in
this
answer
section
is either
to decision problems; yes or no. This restriction
that
is,
we consider
makes
only
the discussion
Most problems can be easily to decision converted problems. in a for the size of the maximum looking matching given graph, > we can ask whether there exists a to solve the If how of size k. we know matching \342\200\224 decision problem,we can usually solve the original for by binary example, problem search. A decision Let U be can be viewed as a language-recognition problem problem. the set of all possible inputs \302\243 to the decision problem. Let L U be the set of all inputs for which to the the answer to the problem is yes. We call L the language corresponding The decision problem, and we use the terms problem and language interchangeably. the introduce We now to L. or not a given input problem is to recognizewhether belongs in notion of polynomial-time reduction use we is the main tool which between languages, and
the
theory
simpler.
For example,instead
this
of
chapter.
Definition: Let L, We U2. say that
polynomial-time
and
Lx
algorithm
L2
be two
languages from reducible
is polynomially that converts
each
the
to L2
input
spaces
input if
there
U\\e U{ to
U
\\
exists another
and
a
11.2
well defined is also
size of the
in the
polynomial
L{ if
such that u^e
U2
u2e
input
the
in
polynomial
input
spaces U x size of u x.
input the
in
and
u
x.
and
L2. The
if u2e
only We
that the
assume
so, \302\243/2,
343
Reductions
Polynomial-Time
is
algorithm
of
notion
size
is
the size of u2
in particular,
in the definition converts one problem to another. If we have then the we can two to an for algorithm compose algorithms produce algorithm Denote the conversion and denote the for L\\. algorithm by AC, algorithm L2 by AL2. an arbitrary input Given u x to an input u2 e U2\\ we uxe U\\ we can use AC to convert then use AL2 to determine whether to L2, which will tell us whether ux u2 belongs to L x. In particular, we have the following theorem. belongs
The
mentioned
algorithm
for L2,
an
11.1
\342\226\241 Theorem
reducible
If L\\ is polynomially algorithm for L2, then
comes from
an
cases likely,
the for
inputs
the definition
of L2 involved in the Thus, if L \\ is polynomially
problems are equivalent precise proof of this
If
L\\
and
can
We
L
essence
u3
in
reducible
relation
or
of
equivalent,
simply
nontrivial
all
particular,
polynomial-time
The
(we
algorithms
leave
reducibility\"
\"polynomial
if
tractable the
is
of the
an efficient
(This
to L2 and L2 to L 3.
the two
still
conversion
is one
a polynomial
of
the
method we present in algorithm
problems.
reducible
is polynomially
cannot
to
reducible
reasons this
be found.
whether we try to find cannot solve efficiently, to be hard. The class of NP-complete known equivalent
equivalent, In
to
algorithms
form
An input u { in L { will be converted first to Since we use polynomial reductions and L3. is
functions
conversionalgorithm. when
theorem.
compose
to L3.
x
to an input
two polynomial
The
next
the
in
have
exercise).
L \\ is polynomially
from
then
all
polynomially the other.
as an
is polynomially
Proof: algorithm
to
11.2
\342\226\241 Theorem
L 3, then
are
L2
because fact
is shown
as
and
reducible
polynomially
transitive,
It is
versa.
problem.
Two languages Lx
is
each
of L { can be possible, and in many reduction are only a small fraction of all reducible to L2, then we regard L2 to
inputs L2.
that L{ is polynomially to L \\. This asymmetry
that any input
requires
vice
not
but
x.
\342\226\
reducible
polynomially
of reducibility
of L2,
input
L
for
fact
the
symmetric;
is
a polynomial-time
discussion.
preceding
not
that L2
imply
equivalent
that
harder
not
that
fact
the
to
converted
be the
does
to L2
to L2 and there is polynomial time algorithm
the
is
of reducibility
notion
The reducible
possible
proof follows from
The
Proof:
is a
there
function,
we chose
the
result
an
is a
a conversion input
u2 in
a composition
L2 of
polynomial-time
to use polynomials.)
\342\226\2
is to look for equivalent problems When we are given a problem that we it is equivalent to other problems that are hundreds of such problems encompasses
chapter
NP-Completeness
344
state
we
Before
started with NP-completeness the theorem, we must explain
of
theory
The
Theorem
and Cooks
Nondeterminism
11.3
We
notions.
several
of Cook
theorem
remarkable
a
will skip several technical details. An excellent and Johnson [1979]. The theory of NP-completeness for this area most of which is beyond large theory, called computational complexity, that help us to use the theory. to some parts this book. We limit the discussion in precise detail describing mathematical We have not gone into great
discussion
and
intuitive
is
is Garey
an algorithm
is. This is not
the
a
of
part
of
scope
terms what
practical algorithms, as long as we (e.g., additions, comparisons, computers is algorithm very important, however, for
for describing
important
steps that are supported
reasonable
use
[1971].
try to keep the reference book
will
all
by
definition of an accesses). memory have used decisiontrees to prove lower bounds in Chapter bounds. lower (We 6, proving The most fundamental model of computation restricted is a but this is a very model.) used model is that of a random access machine. commonly Turing machine. Another for our purposes, these and other reasonable models are equivalent because Fortunately, an algorithm from one model to another without the we can transform changing running A precise
time the
more
by
We first
which
mystery.
One
and
leads
realistic goal. the
deterministic
name
has,
algorithm
non-
rather
to all
addition
in
of the
development the
of a not
theory the
for using
techniques
the regular operations of a will call nd-choice. As the
a very powerful primitive, which we the nd-choice primitive is used to handle choices,
algorithm,
suggests,
one.
This notion is
important to the class than it is to
existence of this
nondeterministic
A
a nondeterministic is more
was proved with will not use any
that NP-completeness is something as an abstract notion, and algorithm
to think
Nondeterminism
that require
details
any
nondeterminism.
of
notion
people
many
of the
explanation
theory.
the
think of
should
go into
not
will
to discuss
need
intuitive,
as a
it is
but
model
specific
factor. Cook's theorem, for example, valid for other models as well. We
a polynomial
than
Turing machines, here, since we
use of
but
it does
so
in
an
This primitive is associated with a fixed number of choices, such that, for each the algorithm follows a different We can assume, computation path. without loss of generality, that the number of choices is always two. Let L be a language unusual
that
way. choice,
deterministic
regular
end,
to recognize.
want
we
it decides
We say following
during
outcome
an
in the
nondeterministic
a
of the
The key
difference
a language. recognizes a language
recognize
they
way
algorithm performs primitive, and, at the and between deterministic
nd-choice
algorithm
L
if
the
is satisfied:
input
the
uses
with
nondeterministic
the
that
condition
Given
lies
algorithms
jc,
input
accept x.
not to
or
whether
an
interleaved
steps
nondeterministic
Given
x,
it is
execution
of the
algorithm
In other words, the algorithm to L to arrive at an accept at L to to arrive belonging
of
possible to convert each the
be to
will
must
into
algorithm
provide
outcome, an accept
and
accept
jc,
at least it
must
outcome.
nd-choiceencountered
a real if and
choice such
only
if
x e
one possibleway not provide
Notice
the
any
asymmetry
that
the
L. for way
inputs for in the
belonging
not definition.
inputs
11.3 Nondeterminismand Cook'sTheorem
345
L may have many to a reject outcome. We require only that the paths at least of choices for one xe L. On the other algorithm every \"good\" sequence a reject outcome, no matter which x 4 L, we must reach choices we hand, for every input substitute for the nd-choices. is sometimes called The nd-choice for primitive guessing reasons. x e L is the obvious The running time for an input of a minimum length An
input
xe
has
execution sequence nondeterministic to L
belonging
to
leads
that
accept outcome. The running xe running time for inputs
an
a
of
time
to worst-case
refers
algorithm
L (inputs
not
are ignored).
Let's see an
of a nondeterministic of example algorithm. Consider the problem whether a has a The is a G=(V, E) deciding given perfect matching. following graph We maintain nondeterministic algorithm for this problem. a set M of edges, which is We examine all the edges of G, one edge e at a time, and use an initially empty. nd-choice to whether or not we include e in M. When we are done corresponding all the edges, we check to see whether M is a perfect matching. The checking examining in linear time, since we have to determine only whether M contains can be done exactly whether and vertex is incident to one from M. The output V each | 112 edges exactly edge is yes if M is a perfect matching, no otherwise. of the algorithm and This is a correct nondeterministic then
there
for perfect
algorithm
is a
sequence of choicesthat
only if the existence of a will see more examplesof Nondeterministic Not all
nondeterministic
the
of size exactly
efficiently
is to
problem
in the
algorithms
are very
algorithms
(1) if
a perfect
exists,
matching
outputs put it in M; and (2) the algorithm was proved (because of the checking).
will
matching
perfect
problems can be solved
supposethat
matching because
but
powerful,
section.
next
their
power
the
whether
is
not
unlimited.
algorithm. For example,
by a nondeterministic
determine
yes We
maximum
matching
in
a
given
use the nondeterministic matching algorithm to find a of size k if it exists, but we cannot easily determine (even nondeterministically) matching of a larger size. that there is no matching whose The class of problems for which there exists a nondeterministic algorithm reasonable is is NP. It seems to time a in the size of the called running polynomial input than more deterministic believe that nondeterministic are much powerful algorithms an NP problem are is to exhibit But are they? One algorithms. they way to prove that to prove that that is not in P. Nobody has been able to do that yet. In contrast, if we want that the two classes are equal (i.e., P = NP), then we have to show that every problem deterministic algorithm. Nobody has belongs to NP can be solved by a polynomial-time the proved that either (and few believe it to be true). The problem of determining between P and NP is known as the P = NP problem. relation We now define two classes, which not only contain numerous important problems that are not known to one another) to be in P, but also contain the hardest (all equivalent in NP. problems graph is
Definition: NP
A
k.
problem
is polynomially
Definition:
A
belongs to NP,
We can
X is called an reducible to X.
problem and
called NP-hard.
X is
(2) X is
NP-hard
an
problem
NP-complete
if every
problem
problem in if
(1) X
NP-Completeness
346
of NP-hardness
definition
The
then
to P,
belong
that
proof
Cook [1971]proved exhibited NP-complete
implies that
one such problem, which proving that problem,
easier.Given
a new
NP-complete
problem,
problem
is
it is
Y,
if
that,
problem is ever
NP-hard
any
imply that P = NP. there exist NP-complete
would
we
other
problems
sufficient
to prove that to
reducible
polynomially
he problems; in particular, Once we have found an shortly. are also NP-complete becomes
describe
will
proved to
Y.
Cook's
or
problem,
other
any
follows from
This
next
the
lemma.
11.3
Lemma \342\226\241
A problem
Proof: is
to
reducible
is a
reducibility
the
definition
Y.
But
2 in
condition
By
polynomially
belongsto
problem if(J)X NP-complete reducible to X,for someproblem
is an
X
is polynomially
transitive
of
since
and (2') NP, is NP-complete.
Y that
every problem
NP-completeness,
Y is
relation, every problem in
reducible to
is polynomially
well.
X
as
\342\226\
much easier to
It is
prove that
news. As more choicesfor proving more good
[1972] found hundreds of same
24
problems
problems(maybe
even
have been
are polynomially reducible than for the whole theory. anchor
more problemsthat are 2'. Shortly after Cook's result that he proved to be NP-complete.
thousands,
it is
the
depending
discovered to be
NP-complete.
on
became
In
known,
Karp time, variations of
Since
count
the
to prove there is we have
NP-complete
we
how
And
that
next section, we
problems NP-completeness proof. We The without most difficult part of NP-complete problems proof. not 2 to condition (but (or 2'). always) verify mention the problem that Cook proved to be NP-complete, and The problem is known as satisfiability (SAT). Let 5 be a Boolean of normal form (CNF). That is, S is the product (and) conjunctive
present examples also list several other such proofsis usually We now describe the idea of the proof. in
problems
condition
important
problem) five
expression
two
Thus, Cook has found we find more and
2 directly.
condition
the
NP
in
X and
to
reducible
polynomially
NP
Y
of NP-complete
with
their
sums (or). For example,S = (x+y and +z) (x +y +z), where addition -(x+y+z)is variable to each the and and and or Boolean multiplication correspond operations, either 0 (false) or 1 (true). A into transformed Boolean can be CNF.) (Any expression Boolean expression is said to be satisfiable of Os and Is to if there exists an assignment its variables such that the value of the expression is 1. The SAT problem is to determine whether a given a satisfying is satisfiable (without expression necessarily finding *=1, S is satisfiable, since the assignment assignment). For example, the expression Is to the variables of a it. y=\\, and z=0 satisfy We call an assignment of Os and several
Boolean
expression
a truth
The SAT problem
assignment. NP because we can
guess a truth assignment and check that the expression in polynomial that SAT is NPtime. the proof The idea behind hard is that a Turing machine (even a nondeterministic on a all of its and one) operations * the that mean we 'described\" given input can be describedby a Boolean expression. By if and only if the Turing at an machine will terminate expression will be satisfiable it satisfies
is in
11.4 Examples
state
accepting becomes
for
the
quite large
number of
described by
steps the an
instance
is not easy to do, and such an expression size is no more than a polynomial its in the yet makes. NP can be Therefore, any algorithm
complicated,
machine
Turing
of a
347
This
input.
given
and
of NP-Completeness Proofs
SAT problem.
theorem:
\342\226\241 Cook's
The SAT problem is NP-complete. 11.4
In
of
Examples
this
we prove
section,
following
set, 3SAT, 3-coloring, and detail below. The techniques we at the end of the are summarized
dominating more they
we must
problem,
always!) easy,
first prove
then
reduce
Proofs
NP-Completeness the
that
\342\226\
that
a known
the
five problems are NP-complete:vertex cover, Each of these problems is described in
clique.
use for proving To
section.
problem
The reduction
11.1.
To
rather
than
NP-completeness to NP, which is usually
prove
belongs
NP-complete problem to our problem in
order used for the five problems we present make them easier to understand, This order is indicated in the tree order.
time.
are typical,
NP-completeness
in
this
Fig.
11.1
in
order
Figure
11.1
The order of NP-completeness
proofs
in the
text.
(but not in
Fig.
of difficulty
by the numbers
edges.
and new
polynomial
is illustrated
section
the proofs
of a
of
the
NP-Completeness
348
11.4.1 Vertex Cover E) be
G =(V,
Let
an
undirected
incident
G is
every edge in
The Problem k, determine
at
Given
one of
an
undirected
a vertex
cover of G is a set of
vertices
such
that
these vertices.
least
G has
whether
A vertex
graph.
to
graph G
=(V, E)
cover containing
< k
and
an
integer
vertices.
11.4
Theorem \342\226\241
The vertex-cover problemis NP-complete. Proof:
size G and an whether G contains a of size k. We have to transform k, graph integer clique an arbitrary instance of the clique into an instance of the vertex-cover problem problem such that the answer to
is positive if and only if the answer to the is positive. Let G=(V,E) and k represent an Let G =(V, E) be the complement of arbitrary jnstance of the clique problem. G; graph namely, G has the same set of vertices and two vertices are connected in G if and only if are not connected in G. We claim that the clique problem is reduced to the vertexthey cover problem represented by the graph G and n-k (where n is the number of vertices in that C = (U, F) is a clique in G. The set of vertices G). covers all the V-U Suppose in U (they are all in G). edges of G, becausein G there are no edges connectingvertices Thus, V - U is a vertex cover in G. Therefore, if G has a clique of size ky then G has a vertex let D be a vertex cover in G. Then, D covers all cover^f size n-k. Conversely, the in G, so in G there could be no edges connectingvertices in V -D. edges Thus, V-D generatesa clique in G. Therefore, if there is a vertex cover of size k in G, then there is a time, be performed in polynomial can obviously clique of size n -k in G. This reduction since it requires only the construction \342\226\2 of G' from G (and the computation of n - k).
11.4.2
Dominating
Let G =(V, E) be that
every
the
clique
vertex-cover
corresponding
vertex
an
problem
problem
Set undirected
of G is either
graph. in
D or
A dominating is adjacent to
at
set D is a least
one
set of vertices in G such vertex from D.
11.4 Examplesof
Given
The Problem k, determine whether
\342\226\241 Theorem
an
G has
undirected
graph
a dominating
Proofs
NP-Completeness
G = (V,
E)
set containing
and
< k
an
349
integer
vertices.
11.5
The dominating-set problemis NP-complete. Proof:
~->yk-s
each
clause
each
time.
in polynomial
expression
to
introduce
new
without
formulation
for each
clause. C is
variables
affecting
its
transformed
that
\"'
(x[+x2+y[)-(x3+yl+y2)'(x4+y2+y3)
(**-i+**+)'*-3)-
is satisfiable if and only if C is satisfiable. If C is satisfiable, then one of 1. In that case, we can set the values of the y(s in C such that all in C are satisfied as well. For if jc3 = 1, then we set y, = 1 (which takes example, = 1), and the rest of the care of the first clause), since (the second clause is okay ^2=0 jc3 if x,r = 1, then we set y\\,y2, the rest to be 0, to be 1, and .v/s to 0. In general, --^1-2 which satisfies C\". Conversely, if C is satisfiable,then we claim that at least one of the We
x,s
that C
claim
the *,s clauses
be
must
be
must
set to
_L
all
if
Indeed,
xfs
are 0,
then
the
expression
becomes
CVi)*CVi+>;2),CV2+>;3) Cy*-3). expression is clearly unsatisfiable. this variables three that has more than we can replace any clause reduction, Using with several clauses, each with clauses It remains to transform three variables. exactly with one or two variables. If C has only two variables, namely, C=(x{ +Jt2), then \"\342\226\240\342\200\242
This
C' = (xx
where z is a new C' =
where both
+jc2 + z\,
+x2+z)'(xx
(x{ +y y and
variable.
Finally,
+ z)-(x{ +y + z are
new
z)-(jr1
variables.
if C
=jc ,, then
+>>+z)-(jt1
+y + z),
11.4
a general instance of SAT if the other one and only
is satisfiable if
instance
done in
polynomial
11.4.4
reduced
have
we
Thus,
one
that
of NP-Completeness
Examples
of 3SAT such can clearly be
an instance
into
is.
The
351
Proofs
reduction
time.
\342\226\2
Clique
The clique problem was
definedin
Section
when we
11.4.1,
discussed
the
E)
integer
vertex-cover
problem.
Given
Problem
The
an
undirected
G contains
whether
ky determine
graph
a cliqueof
size
G =(V, > k.
and
an
11.7
\342\226\241 Theorem
The clique problemis NP'-complete. Proof:
The
and
check
vertices
problem. Consider
Let the
clique problem belongs to NP since we can guess a subset that it is a clique in polynomial time. We reduce SAT to the
E be clause
an
E,- = (x+y+z
We associate also appear in other they purposes).
arbitrary
a
\"column\"
Boolean
+ w)
(we of
four
expression use four vertices
in
CNF,
variables with
the
E =EXE2 only
variables
of >k clique
''
Em.
illustration
for
in
even \302\243,
if
is, the graph G will have a vertex for each The each variable. of question is how to connect these vertices such that G appearance a clique of size >k if and only if E is satisfiable. Notice that we are free to contains which choose the value of k because we want to reduce SAT to the clique problem, of a solution of the clique problem. A solution means that we want to solve SAT using that the clique problem should work for every value of k. This is an important flexibility of k to be equal to the number is used often in NP-completeness We will choose proofs. clauses m. The edges of G are as follows. Vertices from the same column (i.e., vertices Vertices from different of the same clause) are not connected. associatedwith variables variable the same unless they correspond to columns are almost connected always two vertices That is, the only time we do not connect form. appearing in complementary from different clauses is when one corresponds to a variable x and the other to x. An + z)-(y +z), is E = (x+y+~z)-(x+y which to the expression example, corresponds in in 11.3. G can be constructed time. presented Fig. clearly polynomial claim that G has a cliqueof size > m if and only if E is satisfiable. In fact, We now the construction m independent guarantees that the maximal clique size does not exceed that E is satisfiable. of E. Assume such that each Then, there existsa truth assignment clause contains at least one variable whose value is 1. We will choose the vertex for the clique. (If more than in a clause is set one variable corresponding to this variable The result is since a the indeed time two we one choose to 1, arbitrarily.) clique, only are are not connected is when the columns vertices from different they complement of clauses.
That
NP-Completeness
352
Figure11.3 An each other, which
assumethat from
of the
cannot
happen
clique reduction u+)>+z)-(i+y+z)-o>+z).
a clique
G contains
column
each
course
of
example
(since
two
in
of size >m.
vertices
from
a consistent The
the expression
truth assignment. Conversely, contain exactly one vertex
must
clique
the same
for
column are never
connected).
We
of 1. If any variables are not in this assign the corresponding variablesa value assigned can be assigned in the clique are manner, they arbitrarily. Since all the vertices connectedto one another, and we made sure that x and x are never connected,this truth is consistent. \342\226\ assignment
11.4.5
3-Coloring
Let G =(V, E) to the vertices
be an
have the
color.
same
The whether
such
coloring of G is an each vertex is assigned one color and no
undirected
that
graph.
Given an Problem G can be coloredwith
\342\226\241 Theorem
A valid
undirected three
graph
G=(V,
assignment
two
adjacent
of colors
vertices
determine \302\243),
colors.
11.8
3-coloring is N?-complete.
Proof: The
3-coloring problem belongs to NP since we can guess a 3-coloring and valid coloring easily in polynomial We reduce 3SAT to the 3time. This is a more complicatedproof for two reasons. First, the two coloring problem. deal with different versus problems (Boolean objects graphs). Second,we expressions cannot one with another vertex, just replace object (e.g., (e.g., clause); we have to edge) deal with the whole structure. The idea is to use building and then to tie them blocks instance of 3SAT. We have to construct G such a graph together. Let E be an arbitrary if and only if G can be 3-colored. First, that E is satisfiable we build the main triangle M.
check
that
it
is a
11.4 Examplesof a triangle,
Since M is
F (for
true),
(for
used only colors
the
with
another
build
vertex
M.
in
assignment
We
We
do
with the
that
six new
introduce
11.5.
(since
vertices
labeled 7, and one to 7 and jc, yy or z new vertices in the
vertex the
that
condition
the \"colors\" T These colors are
connect
and
for each
outer
xy
vertices
yy or
J, and
x is
if
them
is
A
the
same
one variable in each clause has that the clause is (x+y+z).
Assume
to the
z.) Let's
one vertex
call the
(they are labeled
by
as shown
vertices,
existing
is only
there
A, where
one triangles, all sharing colored with the color 7, then x to A), and vice versa. This is
k +1
at least
construct.
following
x, have
connected
both
consistent,so that
(The labels are
is that,
idea
they are
with the meaning of J. now have to impose the
consistent
value 1.
Fig. 11.4). The F
with
M with
in
whose vertices are labeled are k variables, we will
Mx triangle So, if there
be colored
must
label
We
the bottom
A
(see triangle Fig. 11.4). these they are not part of the graph. We will later associate of truth values to the variables of E. For each variable x, we
and
proof;
vertex A (see
common
We
false),
the
for
colors.
at least three
it requires
353
Proofs
NP-Completeness
three
in
new
O in the
whole
the
vertices
figure),
and
in
Fig
graph
connected the
three
vertices (labeledby / in the figure). We claim that this construct guarantees that, if no more than 3 colors are used, then at least one of x> y> of them can be colored A, since or z must be colored7. None they are all connected to A connected to them must If all are colored then the three new vertices F, (see Fig. 11.4). three colors! The then the inner triangle cannot be colored with be colored A, but is given in Fig. to the expression (x+y+~z)-(x+y+z) complete graph corresponding triangle
the inner
11.6. is We have to prove two sides: the proof. (1) if \302\243 complete can be coloredwith three colors; and (2) if G can be colored with three then E is satisfiable. If E is satisfiablethen there is a satisfiable truth colors, assignment. the variables We color the vertices associatedwith (7 according to this truth assignment M is colored with 7, F, and A as indicated. Eachclause must if x = 1, and F otherwise). outer at least one variable whosevalue have is 1. Hence, we can color the corresponding vertex with F, the rest of the outer vertices with A, and the inner triangle accordingly. three if G can be colored with three colors. Thus, G can be colored with Conversely,
We can
satisfiable,
Figure
now
then G
11.4
The
first
part
of the
construction
in
the
reduction
of 3SAT to
3-coloring.
354 NP-Completeness
Figure11.5 The
to the
corresponding
subgraphs
clauses
in
the
of 3SAT to
reduction
3-
coloring. y
y
Figure colors, three
we name colors).
to a consistent variable in polynomial
11.6 The
the
colors
according
corresponding
to the
to (;t
+>>
+F)
coloring of M (which
+z). \342\200\242(*+>>
must
be colored
with
11.4, the colors of the variables correspond triangles truth The that at least one 11.6 guarantees in Fig. construct assignment. in each is colored with clause T. be constructed G can clearly Finally, D time, which completes the proof. Because
of
graph
the
in Fig.
Examples of
11.4
NP-CompletenessProofs
355
11.4.6 General Observations complete. not
to
look very which
It is
briefly
first
The
always).
or similar
that a problem some general methods for proving Q is NP\342\200\224 \342\200\224 that to NP is Q belongs showing usually easy (but that seems related Then, we have to select a known NP-complete problem to define this \"similarity\" since sometimes the problems Q. It is hard goal,
here
discuss
We
different
to reduce
condition
(e.g., the clique problem is sometimes a difficult task,
a good idea to
try
several
reductions
and
SAT).
several
with
the right
Finding
can be
which
learned
until a
problems
problem by
only
from
experience.
successful one is
found.
We
stress
that the reduction is
The
done
from
an
instance
arbitrary
of
the
known
NP-
the complete proofs is to perform reduction to remember the order to that the is ensure NPway right a black-box complete problem can be solved by algorithm for Q. This is a little The natural thing to do when counterintuitive. a problem Q is to try solve it. Here, given however, we try to show that we can solve another (the NP-complete problem problem) the solution of Q. We are not trying to solve Q\\ using are several degrees of freedom that There can be used in the reduction. For in if a can be convenient includes then its value set Q parameter, any way. (In example, in the problem that is reduced to Q, which cannot be fixed!) contrast with the parameter therefore, we can use it in any Again, Q is just a tool to solve the NP-complete problem; to special its cases in other ways, besides fixing way we wish. Q can be restricted a of For we want to use certain may only types input (e.g., regular parameter. example, we have is the fact that flexibility graphs, biconnected graphs) for Q. Another important the efficiency of the reduction is unimportant, as long as the reduction can be done in time. We can ignore not only constants and, for example, double the size of polynomial the problem, but we can also square the size of the problem! We can introduce in a graph by a new large vertex polynomially many new variables, we can replaceeach the on. There to be efficient bounds of a polynomial), and so is no need (within graph,
problem to Q. backward. One
most
error
common
since the reduction is not meant to be converted into found to be equal to NP, if ever). There are some common techniques used in
(again, Garey
and
Johnson
[1979]
provides
problem is a
many
in such
an
the
(at least
algorithm
examples).
until
P is
of the reductions
construction
special caseof
not
The simplest one is
is, then the proof is immediate, since solving Q implies solving the NP-complete problem. For example, the set-cover problem. The input consider to the problem is a collection of subsets of a k. set an The to there and is determine whether Uy Si,S2,...,5\342\200\236 integer problem exists a subset W \302\243 contains at least one element from Uy with at most k elements, which each the vertex-cover set 5,. We can see that problem is a specialcaseof the set-cover to the of set in which U vertices V, and each set 5, correspondsto problem corresponds an edge incident to that edge. if we can solve the and contains the two vertices Thus, we can then solve the vertex-cover for set-cover sets, arbitrary problem. problem this approach. It is not true, in be very careful, however, when We must using a to makes that more difficult. more that problem problem requirements adding general, showing
that
an NP-complete
Q.
If it
NP-Completeness
356
Considerthe cover
vertex-cover
independent
and
problem,
be solved
can
it
extra requirements found
vertex-cover
the
an
independent
In
constraint that the vertex we are looking for a small set at the same time. (An
7.115).
(Exercise
sets to such
this problem is
however,
fact,
time
polynomial
candidate
the
limit
in
words,
not adjacent to one another.) This problem or the independent-set problem, because
are
that
other
In
about more requirements.
to worry
have
of vertices
than either
more difficult
seems we
a set
set is
vertices.
a vertex cover and
that forms
of vertices
set
a
that we add
Suppose
problem.
two adjacent
include
not
must
that
extent
an
It
that the
out
turns
easier
an
can be
the minimum
easily.
Another
object
in
done
is mapped
problem
in a local
involves local
easy technique
relatively
one
manner,
an
into
of
independently
reductions. In
this
an
case,
problem. The mapping object the other objects. The NP-completeness of the other
is proof
We replaced each edge in one graph dominating set problem followedthat pattern. were sufficient to reduce the in the other graph. These local replacements by a triangle in the the in this to define The is best difficulty technique objects way. problem. blocks as we did, for example, The most complicated technique is to use building the The blocks of on in the usually proof 3-coloring problem. NP-completeness depend each one separately is impossible. We have to consider all one another, and designing in order to coordinate the design of the different blocks. the objectives of the problems of the
More NP-Complete Problems
11.4.7 The
some more NP-completeproblems that are useful as a basis in A in Garey the ones the list is given exercises). very large (e.g., than more the for the reduction is sometimes Finding right problem
list contains
following
reductions
other
for and
Johnson
half
the work.
[ 1979J.
Hamiltonian
cycle: A Hamiltonian once. The problem Hamiltonian circuit. The problem is (Reduction from vertex cover.) graphs. each
vertex exactly
Traveling salesman:Let salesman
tour
is
Hamiltonian
path.
The
G contains
integer reduction from clique.)
3-dimensionalmatching: of
triples
(x, y,
z) such
that
problem
problem
X,ye
is
graph
a simple
open
is to determine whether is NP-complete for both
a
directed
that
path
a given
graph and
undirected
cover.) set in an
undirected
graph
The problem is to
set
an independent Let
xe
in a
path
graphs. (Reduction from vertex set: An independent Independent no two of which are connected. whether
NP-complete
A travelinga weighted graph. complete G and a to The is determine, given cycle. problem of its exists a traveling-salesman tour such that the total length reduction from Hamiltonian cycle.)
directed
ky
contains
contains
E) be
G =(V,
Hamiltonian path: A Hamiltonian contains each vertex exactly once. The
vertices
is a simple cycle that whether a given graph for both undirected and
graph
determine
a Hamiltonian
number W, whether there < is W. edges (Straightforward
contains a
in a
cycle is to
A\\ K, and Yy
and
>
k vertices.
E) is
given
G
a set of and
an
(Straightforward
sets of size k. Let Mbea disjoint Z. The problem is to determine whether
Z be z e
with
G =(V,
determine,
set
11.5
a subset of
there exists two-dimensional
with
problem
357
Problems
NP-Complete
exactly once. The
each element is the regular
contains
that
M
matching
(Reduction from
For Dealing
Techniques
corresponding
problem.
matching
bipartite
3SAT.)
The input is a set X such that whether is to determine
Partition:
s(x). The problem
element
each
xe
an associated
has
X
size
it is possible to partition the set into two total size. (Reduction from 3-dimensional Matching.) (Notice that this problem, as well as the next problem, can be solved efficiently by However, since the algorithm Knapsack (Section 5.10) if the sizes are all small integers. of bits required to represent that of the input is the number size such algorithms, input, are in the which are calledpseudopolynomial size of the input.) algorithms, exponential is a setX such that each element xe X has The an size associated Knapsack: input there is a subset B^X whose s(x) and value v(x). The problem is to determine whether total value is > v. (Reduction from partition.) total size is < s and whose is a sequence of numbers a\\,a2, ...,a\342\200\236, Bin packing: The input and two other the set can be partitioned numbers b and k. The problem is to determine whether into k subsets such that the sum of numbers in each subset is < b. (Reduction from partition.)
subsets
with
11.5
the same
exactly
for
Techniques
with NP-Complete
Dealing
Problems notion
The
is a basis for no polynomial algorithm
of NP-completeness for which
problems
an
theory that allows us to identify to exist. But proving that a given
elegant
is likely
problem is NP-complete does not make the problem go away! We still need to solve it. different from the The for solving NP-complete problems are sometimes techniques NPseen. cannot solve an we that have .We (most probably) techniques previously precisely
problem
complete
have to compromise. robustness,
in
different
situations,
that
guarantee a
on
bound
in
degree
compromises
completenessof
the
something. The same algorithm
different
compromises.
lead to
Of particular
the
algorithm. So, we concern the optimality, solution. There are other
a polynomial-time
sacrifice not
may
algorithm.
approximation
which
resulting
An algorithm
common
most
The
of
all
with
completely
efficiency, or
guaranteed
alternatives as well,
and
the
(or
optimal are
interest
of imprecision.
We
may
be used
result is called
precise)
approximation algorithms see three examples will
an
that can
of such
later.
algorithms
In Section
6.II,
The most famous known
to be
primality-testing
such in
P,
but
we discussed
that
ones for primality
testing,
believed to be NP-complete
is not
algorithms,
probabilisticalgorithms
are the
algorithms
approximation
can be used as algorithms.
We
either.
will not
describe
It is they requires knowledge of number theory. cannot be a solved problems by polynomial-time mistakes with low probability for all inputs. Therefore,
because
commonly believed that NP-complete probabilistic algorithm that make are more such algorithms likely to be effectivefor problems Such problemsare to be NP-complete. but are not believed algorithms
may make mistakes. a problem that is not
part
of
other
strategies
that
are not
not
\342\200\224 for
known
common. example,
to be in P Probabilistic as part of
NP-Completeness
358
Another compromise can
We
times.
The problem exclude
try
with for
inputs
from
vertices)
well on
the
example, the
the
Another
definition.
We describeseveral We start
section.
with
These
branch-and-bound.
an
of two
approximation
We
describe involves
is an
example
the
for in
abstract a
which
techniques an
only
the average be useful if the
the right
distribution,
work
that
algorithms
completeness
involving
some
only
polynomial from
problem
the
whose running
For inputs. for bipartite graphs a real-life situation we special
time
is included
input
times
in the abstract
are exponential,
but
they
may be sufficient.
techniques and illustrate and useful techniques general
algorithm or
Backtracking
which
on
isolated
very complicated. of the algorithms; namely,
is usually
which
can
inputs
in designing
these
several examples of approximation 11.5.1
random
to
difficult
is
lower
may
inputs
difficulty
efficiently can be solved
small inputs,
(e.g.,
with
major
is algorithms
example
it
a graph
Finding
them,
problem
example,
assumption.
also compromise to work algorithm
well for
reasonably
A
For
their
when we Therefore, that any extra condition
sure
make
follows
very difficult.
vertex-cover
7.110).
(Exercise
work
designed
time
polynomial
Such trivial average. for certain types of
can
we
we can allow
the
is analyzing
average
Finally,
should
in
of inputs
is usually
however,
in
is defining average. approach which the particular problem is trivial
participating
distribution
problems
NP-complete
worst-case running on the average.
for polynomial
requirement
this
Algorithms
significantly.
actual
the
involves
solve
to
them
with
examples
are similar. They can be usedas a basisfor for small inputs. We then algorithm
optimal
in
called backtracking
this and
either
give
algorithms.
and
Branch-and-Bound
Consider the 3-coloring problem, techniques through an example. under certain to n vertices of a graph. This colors, constraints, assigning of a problem that requires finding optimal values (colors in this case) for n
these
for each parameter parameters. In the 3-coloring example, there are three possiblevalues is 3\", the number of potential solutions Therefore, corresponding to the three colors. which is the number Of of all possible ways of coloring three colors. n vertices with of possible valid solutions will course, unless there are no edges in the graph, the number be quite a bit smaller than on the possible the edges impose constraints 3\", because To all we can start by the of vertices, colorings. explore possible ways coloring an vertices and continue coloring the other assigning arbitrary color to one of the vertices \342\200\224 while maintaining the constraints that adjacent vertices namely, imposed by the edges must be colored colors with different colors. When we color a vertex, we try all possible that are consistent with the previously colored vertices. This process can be performed and branch-andwhich is the essence of the backtracking by a tree-traversal algorithm, bound techniques. To avoid confusion the vertices of the graph and the tree, we between
the vertices of the tree nodes. branch The root of the tree corresponds state of the problem, and each to the initial to a decision colors three one Denote the R(ed), corresponds by concerning parameter. we can pick and G(reen). Initially, color w v and and vertices two B(lue), adjacent any them, say with B and G. Since they will be colored differently in any valid coloring, it is will
call
11.5
not
which
important
is why
we can
we choose
colors with
start
Techniques
For Dealing
with
(we can always
coloring
the final
permute
two vertices instead
of one.
The
359
Problems
NP-Complete
coloring),
which
of these
coloring
two
vertices correspondsto the initial state of the problem, which is associated with the root. The tree is constructed as it is being traversed. At each node t of the tree, we select the next vertex u of the graph or three to color, and add one, two, children to t according to the number of colorsthat can be used to color u. For example,if our first choice (after v has already been colored G), then and w) is w, and if u is adjacent to w (which there are of coloring two possible ways w, B or R, root. We then pick one of these children,
colored, there is less flexibility
of children is likely
in coloring
add
we
and
and
two corresponding
continue
the rest
of the
this process. vertices;
children to a vertex
After
the
is
the number
therefore,
as we go deeperin the tree. If we manage to color all the vertices we are done. of the graph, then More likely, however, we will reach a vertex that cannot be colored (sinceit has three adjacent \342\200\224 we go up vertices already colored with the three colors). At that point, we backtrack other children. An example of a graph and the corresponding 3the tree and explore in tree in 11.7. Notice this once the backtrack is colors of that, case, Fig. coloring given to color the rest of the graph there is only one way is vertices 1 and 2 are fixed, (which in the the rightmost found through tree). path of this tree-traversal based on induction. We can think algorithm as an algorithm of whose We have to strengthen the hypothesis some slightly to include coloring graphs will have to vertices have already been colored.In other words, the induction hypothesis a partial 3-coloring: deal not with coloring graphs from scratch, but with completing to be
smaller
Induction hypothesis:We that
has
know
how
to complete
vertices that are not already cannot be completed.
< k
3-coloring
the
of a
3-coloring
colored, or to determine
graph
that
the
1B,2G
Figure 11.7 backtrack
tree.
An
example
of backtracking
for 3-coloring.
(a) The
graph
(b) The
NP-Completeness
360
Given a
possible colors
k-\\
uncolored
(one
(Gf var
in Fig.
is given
all its
with
now have
(which
problems
find
and
used for
the vertex
color
we
remaining
The algorithm
by induction.
3-coloring
the
one of them
have already been
all colors
If
and solve
a time)
at
vertices)
Algorithm
we pick
colored,
yet it.
3-coloring cannot be completed. Otherwise,
colors
possible
not
to
be assigned
can
that
then the
neighbors, the
that are
k vertices
with
graph
11.8.
U) ;
and U (a set of vertices that have Input: G =(V, E) (an undirected graph), with their colors). { U is initially empty ) already been coloredtogether of one of three colors to each vertex of G. Output: An assignment begin
ifU =
then
V
\"coloring is
print
completed\";halt
else
a vertex
pick
v
;
3 do
if no neighbor
add
in U
not
v
forC := 1 to
to U
ofv is colored with C;
C then
color
with color
3-coloring(G,U)
end Figure11.8Algorithm It is
problem
that
not
common
is quite
to
hard
come
in a
results in
\"good\"
order, we
does
not
specify
have
a degree
point
shortly.
up
tree
an
how
find
the solution
to pick
is a
Since
minimum color similar
the graph rather to the one for
than
just
a yes
3-coloring,
but
3-coloring
for problems Consider
the minimum
involving
be
to this the
finding
coloring
colors required
We
3-coloring.
we
next,
return
the general
number of
of branches may
number
will
can build
quite
large.
a
so far
described
we
be chosen
can
vertex
function.
or no answer for the
The algorithm
any
of backtracking
variation
the
for
traversal
design heuristics. We
use to
(or maximum) of some objective \342\200\224 we are now interested in finding problem
of
of nodes (Exercise11.34).This the tree in Our hope is that, by traversing number
early enough.
can
we
and an order
vertex.
next
the
of freedom that
Branch-and-bound
exponential
algorithms.
backtracking will
a graph
with
with
3-coloring.
to
a tree Each
its can be colored either by one of the colors already used (unless one The 3-coloring or by a new color. neighbors already uses that color), algorithm is thus modified in two ways: (1) the constant number of colors 3 is replaced by the maximal be used so far, and (2) the algorithm does not terminate when V = U, since there may better ways to color the graph. The problem is that this algorithm a leaf is reached (i.e., backtracks only when is V = U), since a new color can always be assigned to the vertex. Thus, the algorithm almost guaranteed to have poor performance We can the graph is very (unless dense).
new vertex
improve
the
of
performance
of this
algorithm
by
the
following
observation,
which is
the
11.5
For Dealing
Techniques
basis of the branch-and-bound leaf and find a valid coloring
method.
Suppose k colors.
with
with
that we
361
Problems
NP-Complete
traverse the
all the
tree
Suppose further that, and reach a vertex path
after
to a
way
backtracking
that color steps up the tree, we traverse another requires know a better solution. number k +1. At this point, we can backtrack, since we already At each node, we compute a lower bound on Thus, k serves as a bound for backtracking. the tree. If that lower bound that can be found farther down the best solution is greater we backtrack. One key to making a branch-and-bound algorithm than a known solution, if we want to maximize the is computing efficient bounds, good lower bounds (or upper Another is a traversal order so that good solutions function). good objective key finding case we can backtrack earlier. are found fast, in which of integer We illustrate this idea through the linear programming problem in Section is also mentioned The problem is similar to linear programming, (which 10.3). with that of the variables but the extra constraints the values are integers. Let be vectors be the vector of variables', ax,a2, \342\200\242\342\200\242\342\200\242,ak of real numbers, ! = (*!,jt2, ..,*\342\200\236) and c i, c2,..., ck be real numbers. The problem is to each of size n\\ and b\\,b2,..;bk maximize the value of the linear objective function
several
the integrality
under
~d\\
* * *
+C2*2+
Z=CXXX
(11.1)
+C\342\200\236Xn
of J
constraints
and the
following
constraints
-X I\342\200\224\342\226\272
(1,5)
a
b
2
\342\226\272 (2,3)
c
3
\342\226\272 (3,2)
4
\342\226\272 (4,2)
5
\342\226\272 (5,1)
f
h
Figure
tree
list
12.4 The representation of the
tree.
\342\226\272 (2,1)
e
\342\226\272 (2,4)
d
Parallel
388
always traverse parallel.
construct
will see
two
perform us to
examples
list
was linear.
A
together
operations
perform such operationson
of
such operations.
But
algorithm
sequential with
first
the the
we
traversal.
can This
tree
efficiently in discuss how to
tour.
Euler's
the
allows
tree
the
the
though
and
tree
the
\"linearization\" of We
as
list almost
the
on
directly
Algorithms
tour of T (with an Euler's It is easy to find edge appearing twice) every the opposite the tree using depth-first search, taking order sequentially. We can traverse in parallel. the search backtracks. We will do a similar thing Let of an edge whenever in the edge following the tour. We claim that NextTour NextTour (/, j) denote (/, j) edge which can be easily computed in parallel: is defined by the following rule, i)
if Next
(j\\
i) is
not
nil
otherwise.
\302\243(/)
{Next(j\\
incident to a vertex is consideredin a cyclic order, such last on /s list, to by \302\243(/), is tne then the first edge on that if list, that, pointed 12.4 (we assume that with the For example, if we start taken. edge a in Fig. edges then the tour consists of the incident to each vertex are ordered clockwise), edges Next (J, i) to follow (/, j) in the tour, we a, d, #, c, /, e, b, h, and back to a. By choosing to j are chosen. guarantee that (/, /) will be chosen only after all other edges incident the subtree rooted at will be traversed we backtrack to /. before Therefore, j completely We leave the proof that this procedure is correctas an exercise. Once the list is constructed, we can choosea starting (r, t) (any edge will do), edge and mark the edge before it in the tour as the end of the list. The vertex r is chosenas the root of the tree. We can now the edges according to their in the list by number position R in Let denote the rank of the list (e.g., (/, j) algorithm (Fig. 12.3). edge (/, j) List_Rank = where n is the of We now show two number /?(r, t) 2(n -1), vertices). examples of \342\200\224 on the tree the vertices in preorder and computing, for traversal, operations ordering all vertices, the number of their descendants. Let (/, j) be an edge in the tour. We call (/, j) a forward if it points away edge from the root, and a backwardedgeotherwise. of the edges allow us to The numbering if between forward and backward edges: An edge distinguish (/, j) is a forward edge R (/, j) > R (/, /). Since the two copies we a of the edge (/, j) are connected pointer, by can easily determine we can make which one of them is the forward Furthermore, edge. this determination for all edges quickly in parallel. We are interested in forward edges because they impose preorder on the vertices. Let (/, j) be the forward edge leading to j In other
words, (/, 0 is
/ is
/s
list
the
of edges
If/(/, j) is the number for j is n
the tree).
of forward
edges following of r, (The preorder number which is the only to it, is 1.) We can now use a forward edge pointing variation of the doubling (/', y), the value of edge algorithm to compute, for each forward We leave the exact implementation of this as an exercise. / 0\\ 7 )\342\200\242 algorithm doubling The secondexample involves for each vertex y\", the number of vertices computing, below j (descendants) in the tree. Let (/, j) be the (unique) forward edge leading to j. Considerthe edges following the edge (/', j) in the list. The number of verticesbelow j in of forward edges that are below j in the tree. We already the tree is equal to the number (namely,
(/, j)
in the
list,
parent
in
the preorder vertex with no
then
number
-f
(/', j).
12.4
know
how
edges
that
to compute are after (/,
edges
/(/,
/) following
of j is equal performed
in
We can
list. the
list.
y),
compute
similarly
easy to see that
It is
is
which
the
of forward of forward of descendants
the
number
the
number
number
be modeled
can to
the
nodes and
two
by
graphs
nodes
389
The be
Networks
for Interconnection
networks
processorscorrespond
(almost are connected
always undirected). The if there is a direct link
and can also processors. Each processorhas local memory, local the memories of other network, Thus, all memory processors. but the cost of accessing a variable on the locations of the depends the variable. A shared-memory access as a local access may be as quick to be in the same processor), or as slow as a traversal of the happens
the corresponding
between
access, through can be
the
shared,
and
processor the
(/, /)
Algorithms
Interconnection
(if
j)
the
value of/(/,
the
parallel in
Networks
Interconnection
0. which can be found again to/(/, by the doubling technique. j)-f(j, the and for of descendants can both for preorder number finding numbering = in time T(n, n) 0 (logn)under the EREW model.
algorithms
12.4
in
for
Algorithms
variable
whole network (in case communicate
Processors
variable
that
is located
is a
simple chain). It is usually somewhere in between. to access a shared by messages. When a processor wants at another processor, it sends a message asking for this variable. the
graph
is routed across the network. networks. The simplest graphs have been suggestedas interconnection ones include linear arrays, rings, trees, stars, and two-dimensional meshes(grids). binary the better the communication becomes. But edges The more edges we add to the graph, increase the for the layout of the wires, which are expensive area required (e.g., edges We have to find the right tradeoff. There is no increases the time for communication). on a certain graph one type of graphs that is good for all purposes. The performance of the particular on the communication patterns algorithm. There are, depends heavily that are very useful. We list some of them below, together several however, properties with of interconnection networks. examples is the maximum The diameter of the graph is of great importance. (The diameter between any two nodes.) It determines the maximum number of all the shortest distances nwhereas an of hops that a message have to take. An/ix/j has diameter In, may grid + node balanced binary tree has diameter 2log2(fl l)-2. A tree can thus deliver a message(in the worst case) much faster than a grid. On the other hand, the tree has a major bottleneck. All traffic from one half of the tree to the other half must pass through the root. The two-dimensional mesh has no such bottlenecks, and it is very symmetric, which is important for algorithms that communicate in a symmetric fashion. A hypercube is a popular topology that combines the benefits of high symmetry, routes alternate between A dsmall two and no bottlenecks. diameter, nodes, many n = 2d processors. of consists dimensional The addresses of the are hypercube processors each address contains d bits. ProcessorPx is integers in the range 0 to 2d 1. Therefore, one bit. The distance to processor Pj if and only if / differs from j by exactly connected is never more than we can go from betweentwo processors d, since Px to Pj by changing A four-dimensional hypercube is shown in Fig. 12.5. d bits (one bit at a time). at most The
message
Numerous
Parallel
390
Algorithms
12.5
Figure
The hypercube any
two
a rich
provides
(e.g., we
processors
combine the
with
hypercube
A four-dimensional
hypercube.
are many
connection, since there
can changethe
bits
appropriate
the mesh architecture
by,
for
of the
faces
12.4.1 We
Sorting with
start
on an
We
order).
embedding
example,
include hypercube. Other suggestednetworks connectedcycles,quad and octal trees, mesh of trees, butterfly, the
any
between
routes
different
in
can also meshes in
shuffle, cube-
the perfect more.
and
Array
the relatively simple problem of sorting P X,P2, i>,Pn> and n inputs X\\,x2,
on
of processors.
an array
There
one smallest the such that the goal input among processors we may want to assign input is in P x, the second smallest is in P2, and so on. In general, more than one input to each processor. We will see that the same algorithm be can to Each this case as well. The processors are connected in a linear fashion. adapted processorP{ is connected to Pi+l, 1 < / < n. are
input.
n
processors
is to
The
Since each comparisons the
array.
and
distribute
processor possible
Each ...,*\342\200\236.
processor
holds
the
can exchanges
communicate can be
only
done
In particular, in the worst case, the it takes for an input to move from
with
must
algorithm
its
with
neighbors, that
elements
allow
n
the
only
are consecutive
in
- 1 steps, which
is
The array of is number to the as its number follows. Each algorithm basically processorcompares one of its neighbors, exchanges the numbers if they are in the wrong order, and then does the same with the second neighbor. (We must because otherwise the alternate neighbors, same numbers all numbers will be compared again.) The same process continues until are in the correct order. The steps are divided into odd steps and even steps. In the odd with those of their numbers their right steps, the odd-numbered processors compare in the even steps, the even-numbered neighbors; compare their numbers with processors those of their right neighbors (see Fig. 12.6). This are synchronized all processors way, and a comparison If a processor does not have always involved the correct processors. the
time
one
end
of the
to
the
other.
12.4
X2
Xt,
X$
X$
X(y
X-j
;*8
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
corresponding
this step.
through in Fig.
12.7.
sort
be
very
algorithm
A
Odd-even
12.6
neighbor (e.g.,
the
This algorithm
is called
numeric
first
of the
example
sort.
transposition
second step), it remains idle transposition sort. It is given in Fig. 12.8. Notice that presented
in the
processor
the odd-even is
algorithm
example is completeafter only six hard to detect in a network. Therefore, run to its worst-case completion.
in the
the
Networks 391
X i
Figure the
for Interconnection
Algorithms
However, early termination it is better to let cases many
steps. in
can the
seems of natural and clear, but its Algorithm proof Sorting_on_an_Array correctness is far from obvious. For one thing, an element from its final may move away destination. For example,in Fig. 12.8, 5 moves to the left for two steps before it starts to the right, and 3 moves to the leftmost and stays there for three steps moving position before it moves back to the right. is the correctness of parallel algorithms Proving of because the the of different actions the difficult, interdependencies among processors.
Algorithm Input:
Output:
(x, n)
Sorting_on_an_Array
x (an array
x (the
the
in
range
az,
that xt
such
order, such
in sorted
array
1 to
the
that
resides
at
Pt).
/th smallest
element is
begin
do in parallel
\\
n/2\\
times
/>2/-i and
elements and exchange them such that \\ b.
We present a proof of congruent to k modulo 3, written by 3. (k can be any one of 0,
theorem.
this
an integer
when
In
+ bn(a-b) = (b-a)(an-bn).
= an(b-a)
then the expression above is equal to 0. Otherwise, factor is negative, and vice versa positive but the right no is more the proof is complete. than and 0, expression
theorem.
most 0.
expression becomes
it the
without
a = b,
There
is at
term
right
that
ignored, and
-bn+l
-an+l
is
so it
+ abn)
+ban
bn+l
bn+l) < 0.
-
an+l
can be
factor
+
= 2n~l(an+l
+ 2n~\\ban+abn-an+l
2n~\\b an+abn-
2.9
is true
the desired
to
If
that the claim
Assume
+ b)(a + b)n(a + b)n+l.
2n(an+l We
Exercises
Selected
then the left factor In either case,
is the
a slightly more general n =k (modulo 3), if k
1, or
2.)
We prove the
following stronger theorem. D
A.l
Theorem
A number n, given in its decimal representation, only if the sum of its digits is congruent to k Proof: The claim
is congruent
checked for
to k modulo 3 if
and
3.
modulo
Assume that it is true for representations of n 1 and n is in the last -1 not 9, then of n is only digit, unless the last digit of n 1 is 9. If the last digit when we go from n -1 to \302\253, we increment both the number and the sum of its digits; thus, the congruence remains the same. If the last digit of n -1 is 9, then it changes to 0 and 1 is added to the second digit. The change of 9 to 0 does not change the congruence of the sum of digits (since both are divisible by 3), and the addition of 1 to the second digit has the same effect on the congruence of the sum of digits as that of the addition of 1 to the first to 0). digit (e.g., it may cause another 9 to change
n -
2.15
1, and
The subtle
consider
hypothesis
is that,
point
of the number
in
be easily
can
difference between
n. The
we subtract
when
question
we \302\253,
on n-at cannot be used But the requirement was
2.26 The only that
tree with two the theorem is true
vertices for
n -
a single
is 1
vertices,
sure
numbers.
decimal
an element,
say
ah
from the list
to reduce the
size
2cn
c(2n)(\\og2(2n))2
to verify (\\og2n)2
(see Section +4n
log2(2\302\253),
(2) to
> 2cn (\\og2n)2
+ 4n.
+ 4\302\253 \\og2n
c >2.
We have \302\243/Mog2i
3.15
recursion:
= T(n-\\) + n/2
1+2/2
3.14
w.
in the proof: We took it for granted that a maximum a subtle implicit added assumption then there are finitely many possible sets always exists. If all the x(s are positive integers, if some of the exists. whose sum is S, and so a maximum However, jc,s are real always then a maximum may not exist. Hence, the proof, as is, is valid only for integers. numbers,
T(n)
3.8
v and
We
Chapter
3.6
is clearly
vertices
<
The functions f
J
xk\\o%2xdx =
(n) = 3n,
g(n)
jf]-[
= 2n, and
\302\260gX~
s(n) = r(n) = n
\\\\
form
=0(nk+l\\ogn).
a counterexample
to the
claim.
SelectedExercises
of Solutions to
420
Sketches
3.17
We
can
and
the other
the two
construct
one grow
functions
by
one of
letting
odd numbers.
for the
fast
We
them
/ 1. So,we can remove dk from the list and subtract 1 from dj. This is a valid problem of size n-\\ (everything is still positive and the sum is 2\302\253-4), which we solve it by induction. We find a tree T with the corresponding degrees. We need only to add a leaf to the vertex to dj in T to find the tree that solves the original problem. corresponding use
We
the KMP pattern-matching algorithm (Section 6.7) in combination with a DFS-like traversal of the tree. During the traversal, we maintain the status of each vertex
(preorder) in the
to current potential matching.
regard maximal
prefix in the pattern
that
That is, we
matches
the
define
suffix
v.match
to be
of the path from
equal
to the
the root to
v.
size of (This
is exactly the same information that is maintained by the KMP algorithm.) preWORK for v is defined by including v's character as the next step in the KMP algorithm, and computing \342\200\224 v the status of that vmatch. postWORK is defined is, by restoring by continuing the in the number time of this algorithm is linear matching from v using v.match. The running
of vertices in 7.35
Without the cycle, the graph would have been a tree for which the problem is easy (pick a root and direct all edges away from it). We can discoverthe cycle the first by DFS when is found. to a previously visited vertex The cycle can be directed so (and only) edge leading 1. All other edges should that it becomes a directed and all its vertices have indegree cycle another DFS from any be directed away from the cycle. This can be achieved by starting all edges in the direction they are visited the search. vertex on the cycle and directing during The single back edge will point to the root of the search which will have indegree 1 (as will all other
7.37
the tree.
vertices).
DFS on the graph, and build a DFS tree T. Sinceall the other edges in the graph of T to point away from the root and all are back edges relative to 7\\ we can direct all edges vertices that are closerto the root (i.e., toward the ancestors). other edges to point toward
Perform
7.40
of indegree 0 are removed together so that all vertices Modify the topological sort algorithm handles from the queue and are put in one group. The algorithm these vertices as before the corresponding indegrees), and (i.e., it removes their emanating edges by decrementing then 0 are removed. Prove that the groups satisfy the again all vertices of indegree conditions of the problem, and that there are no more than k + 1 of them.
7.42
Use BFS starting at v. Each edge (jc, y) encountered through the search, such that y is a new the count of vertex, is part of a shortest path from v to y. A common error is to increment there (jc, y) is encountered. However, may already be several and one of a shortest to each of them is a to y (ending with jc, path part paths to x is to add the number of shortest the edge (jc, y)). Hence, the correct solution paths to 0. to the count of shortest known) (which is already paths to y. All counts are initialized shortest
known
paths
shortest
to y
when
428
Sketches of
7.43
The
no heap
time (since
Add to compare
possible small
be
7.48
V \\)
find a new
we
Instead,
but now the
time,
minimum
update for
each
path by looking at all edge takes only constant
is maintained).
Use the same algorithm, except edge a weight of \\u where u stands for a unit. The units are used to plus the sum of units. length of a path is its regular length artificial two paths with the same regular weight. Instead of using units, it may be to add a very small number x to the weight of each edge. The value of x must be
each
the
that
O (|
takes
This
Exercises
Selected
to use a heap.
is not
idea
main
the vertices.
7.45
Solutionsto
that it
enough taken
as m/( |
then
m -
1).
We
use
induction
does not
on k,
paths with
affect
V | + 1), where
find
and
different
the precision of
m is
not only the
we discuss vertices. For simplicity, of k = 1 themselves.The case paths
For example, x can
weight.
if they
(e.g.,
weights
are
integers,
to w, but also the paths to all other the lengths of the paths, rather than the Denote by w.SPk the length of the shortest
path
only
finding
regular the
is obvious.
that we know the shortest paths, path from v to w which contains exactly k edges. Assume That is, we know the values which contain exactly k - 1 edges, from v to all other vertices.
of w.SPk.l for all to x.SPk_x + length 0 i. For each such edge (/', j), we check whether edges same component the and, if not, we join the two corresponding (using the find operation), We stop when one component contains more than components (using the union operation).
The
main
problem
idea is to solve the problem is stated, we add vertices
vertices.
nil
7.71
Exercises 429
costs
the
change the
7.70
to Selected
an MCST, because only the relative order costs by a constant does not change is important for the algorithm. Whenever costs are used in the algorithm, they are compared to each other with a simple comparison ( \"\.") They are not used (or a constant to all costs does not manipulated) in any other way. Therefore, since adding
Increasing
of
7.68
from another
of Solutions
Since every E-F cannot
cycle
in G
an must contain We want The largest
a cycle. the size of E-F. contain
edge from the feedback-edge the size of F, to minimize
set F, the which
set of
is the
edges same as
no cycle in a graph corresponds set of edges with if the graph is not connected). tree (or spanning Hence, the minimumforest, tree of the size feedback-edge set in an undirected graph is a complement of a spanning that the graph is connected). The graph, and its size is | E \\ - \\ V | + 1 (assuming for directed graphs is NP-complete (see Chapter 11). problem corresponding maximizing
to
7.74
a spanning
Check w.SP
all the edges not in (where Jt.SP is the
obviously not a shortest-path tree is indeed a shortest-path
the
tree. If there is an edge (v, w) such that v.SP + cost(v, w) < of the shortest path to x via the tree), then the tree is tree. It remains to prove that, if no such edge exists, then the
length
tree.
430
7.79
connected in the the other hand, vertices according 7.84
order, and
a topological
Find
We proved
to their
in
that
labels, if they are a Hamiltonian path. On will sort the then any topological order
path,
run
the biconnected
will
a slightly
prove
the claim
path.
stronger theorem
(and
leave it
There is
Proof:
is not
one
components
to show
reader
are
in
graph and to to the same
it implies
that
A.2
and w are two vertices of the same biconnected Ifv path between v and w is contained in B.
that
to the
if they
the exercise).
in
D Theorem
from
gives
7.9 that two edges are contained in a cycle if and only component. Hence,we need only to remove c from the a and b belong whether components algorithm to determine in the graph without c.
same biconnected
path
consecutive
in Section
the
We
place
vertices with topological order
for all
check,
then
graph. If they are, then the if there exists a Hamiltonian
biconnected component
7.85
to SelectedExercises
of Solutions
Sketches
then \302\243,
in
and to component can be connected
the existence of a
cycle
one simple
at least
obviously
contained
it has
come back only
involving
more
path
v and
between
to leaveB and through
through
component B, then
to come
another
(since
one articulation than
w
back. is
it
point). one biconnected
But
simple
any
is B and two
in B.
If there
has
to leave
It
simple, would
that
imply
which
component,
is
impossible.
7.87
Pick
articulation point v. Remove v and its incident edges from the graph and connected components of the resulting graph. Then put back the removed Consider now each component separately, and edges into the appropriate components. and so on. perform the same procedure by picking another articulation point, find
7.89
an
the
arbitrary
different
to may be to try to modify the strongly connected component algorithm the values more For instead of DFS we numbers, update High accurately. example, using can use the currently known High values. In other words, when an edge (v, w) is considered for the purpose of computing we use w.High instead of w.DFSin the v.High, However, w.High may be updated later (e.g., one of its other children may computation. have a path to a higher vertex). Updating of all the vertices that used the previous value A
first
w.High Denote
same
approach
may be too costly. We use another the largest
concerned. That Hence,
it is
connected
DFS number
connected
strongly
is,
sufficient
component.
of
from v by v.RealHigh. have the same characteristics can reach a vertex w, then all
reachable
component
if any
approach.
them
for v.RealHigh compute We first use the strongly
to
All
in the reachability is them can reach w. the
vertices
as far as of
one representative connected component
of
each
strongly
algorithm
to
of the root connected (this is the vertex with the highest DFS component number in the component). SG must be acyclic, so we have reduced the problem to that of the of this reduction is that it The advantage values of acyclic graphs. finding RealHigh solves the problem with repeated SGis acyclic, there Since we before. had that updating are no back edges. Therefore, at the time the edge (v, w) (in SG) is used to update the value of w.RealHigh We leave it to the reader is the correct final value. v.RealHigh, for example, to prove this fact \342\200\224 induction is not trivial). by (the proof
construct the SCC graph SG. of the corresponding strongly
Each node
in
SG
is marked
with
the
DFS number
Sketches of
7.92
Solutionsto
431
Exercises
Selected
to a vertex basis B. If v is not on a cycle and v has a nonzero a vertex w such that (w, v) is an edge and w is not reachable w belongs from v. For B to be a vertex basis, either to B or there is a path from some vertex in B to w. In either case, however, there is also a path from a vertex in B to v, and v can be removed from B. This contradicts the minimality of B.
a. Supposethat
v belongs
indegree, then
b. Sinceno
exists
there
is on a
cycle in an acyclic graph, it follows from part a that only the 0 can be in a vertex basis. But, it is also true that each vertex of indegree 0 must be in any vertex basis, since there is no path leading to it. Hence, the 0 (we proved in unique vertex basis of an acyclic graph is the set of vertices of indegree Section 7.3 that this set is not empty). vertices
vertex
of indegree
find the SCC graph (see Section Any vertex in a strongly connected all other vertices in this component. Hence,it is sufficient to choose in each component. Thus, a vertex basis consists of one vertex (any to a node of indegree 0 in the SCC graph. component that corresponds Given
a general graph
the SCC
shown,
7.95
now,
we
first
is acyclic.
tree with an the tree as a rooted both from w. Remove parent
Consider
it to
graph
arbitrary
root.
7.9). As we have component covers one of the vertices
leaf
Take an arbitrary *
from each
vertex)
v
and
match
become (and disconnected). Solve the resulting problem by induction. We need to prove that the edge If M does Let M be a maximum (v, w) indeed belongs to a maximum matching. matching. to w). If (w, w) is in the not contain (v, w), then v is not matched (it is connected only then simply replace it with (v, w). It is still a matching, and it has the maximum matching, same cardinality. 7.99
its
a. A graph
must not
b.
include
we
vertex,
arbitrary
may
a vertex
tree
of degree0 to have component that
up with a
end
the
all
other
'brothers\"
such a cover. consists
that
When we remove an
of only
one vertex.
the graph contains only vertices of degree > 0, and we make sure Only vertices of degree 1 pose any graph satisfies this condition. one such vertex problem. So, if the graph contains any vertex of degree 1, we remove one to the cover. with its edge and add this edge (which is a star graph itself) Only by is reduced by 1. It is easy to see that this is now a valid degree of one other vertex reduction.
assume
We now
that
reduced
the
that
in combination with the maximum-matching search algorithm described in Let's solve first the problem of verifying whether there exists a maximum such that all of its edges have weights x and applying the maximumin the remaining has the to see whether the maximum matching graph matching algorithm in the original graph. The cost of this of edges as the maximum same number matching is is dominated by the cost of the maximum-matching verification algorithm, which for weights. 0(^fn m). There are m edges in G, so there are at most m different possibilities We can now use binary search to find the smallest x such that x is a weight of some edge and is a maximum matching there such that all of its edges have weights r,. in the set (except possibly the with /,); hence, /, can replace /,, and any other interval follows set. The algorithm modified set is still maximum independent directly by induction, because we know how to handle /1.
Each
interval
intervals
/, is
according
Sketchesof
434
7.119 a.
Let
v,,
matrix
Exercises
to Selected
Solutions
4 (in that order). Considerthe adjacency v2, v3, and v4 be a cycle of length is 0 (i.e., a vertex is not that the main of the graph, and assume diagonal
Is to itself). The rows corresponding to v, and v3 both contain some a of is characterization a of to and This sense, is, v4. v2 cycle corresponding is symmetric since the graph is undirected.) We can 4. (Notice that the matrix length look at all pairs of rows, and for each pair check its intersection. In other words, for each There pair of vertices, we check all the other vertices that are adjacent to both of them. contains at least two vertices. Any row is a square if and only if any such intersection a row and operation) can be performed in linear time. intersection (which is basically There are O (| V \\2) pairs of vertices, so the overall running time is O (| V |3).
considered
b.
adjacent
representation. First sort all the edges in all the lists. Then between two rows described in part a can be performed in number of edges incident to the two corresponding vertices.
Use an adjacency-list the intersection
that
proportional to the
7.120 Compute vertices,
squares
the
and
maximum
number
show
it may
may take more than
Chapter
8
8.6
counterexample
The
that
scan even though
O
is given it
is on
the
of squares that be more than \342\200\242 (| V | | E |) time.
in
Fig.
A.2.
be contained
can 0(\\
V
The circled
\\
\342\200\242
point
\\E |).
will
in
show
time
with | V
a graph
Therefore, just
listing
be removed
Graham's
by
hull.
vertices of the polygon in a cyclic order according to the angles of the line segment they make with q (and a fixed line). Then scan these vertices in that order. Start with an of edges of the polygon that intersect with the arbitrary vertex v, and determine the number half-line 8.2. Then, passing through q and v using the algorithm presented in Section whenever a vertex w is visited, one can in constant time determine whether the line through q and w intersects one more edge,one lessedge, or the same number of edges.
8.11
Sort the
8.16
Divide the plane into vertical columns of width d starting among the points and ending with the largest x coordinate.
Figure A.2
A
counter
example
with the Find
for Exercise
8.6.
x coordinate
minimal
the points
with
\\
all
the minimal
Sketches of Solutionsto
435
Exercises
Selected
in each of the columns. This can be done in time 0(n +Xld) contains a given point can be found in constant time. Apply a similar to Graham's scan to all the minimal and then to all the maximal points procedure can easily be connected at both ends to in two convex paths, which points. This results form a convex polygon the convex hull of the points, C. C is not necessarily but it is easy to show that the distance of any point from the set outside the hull to the hull is at most d.
and
y coordinates
maximal
since
column
the
8.17 The gift-wrapping
can be
algorithm
observation
The necessary
points.
that
the
Thus,
here
requires
algorithm
comparisons to
applied to this problem without each point is compared against
additional
is that
n
-
1
to find the
comparisons
all first
complexity.
other
vertex
remaining of the
hull,
and so on. After the hull is discovered, all its vertices are marked with the appropriate depth, and removed. The same algorithm continues. The first vertex on it requires n-k comparisons (where k - 1 is the number of vertices removed so far), and so on. Overall, O (n2) steps are required. n-2
8.22
8.26
the
find
The only difficult the segments with
case is
We strengthen
induction
second,
intersections between all vertical line segments and or can be handled (the other casesare either symmetric of 8.6). We use the same approach as the intersection algorithm by the algorithm in Section Section 8.6. The segments are sorted according to the x coordinates of their endpoints. The is performed in the same way. A segment with 45-degree angle is line-sweep algorithm is meeting the sweeping line, and is deleted when inserted when its left endpoint its right line. We now have to find intersections between a new the sweeping endpoint is meeting vertical line and several candidate with 45-degree angles. This is done as follows. segments For each segment of the full line with 45-degree angle, we compute the intersection the segment with the x axis. We use this value when we perform the range overlapping is the intersection between The range is set as follows. The left point of the range queries. the line with 45-degree angle that contains the top endpoint of the vertical line and the x between the line with 45-degree axis. The right point of the range is the intersection angle contains line and the x axis. We have converted the that the bottom endpoint of the vertical the same. problem to a one-dimensional range query, and the complexity remains the
of finding
that
angle
45-degree
hypothesis a
We know how to
Induction other
hypothesis: intervals among
endpoint among
a set of < n
intervals,
the intervals
all
mark
and how
to find
contained
the largest
in
right
them.
interval;
left endpoints. Assume that we solved the their If its right intervals, and consider the nX\\\\ interval. it is not contained in any is larger than the largest right endpoint so far, then we update the largest Otherwise, it is contained in another right endpoint.
interval;
we mark
We first sort the intervals
problem endpoint
8.28
little.
The
for
the
according
first (leftmost)
n
to
-1
it.
of two rectangles whose edges are parallel time. Furthermore, either this intersection is empty solve the problem in linear time by intersecting one
intersection
constant can
to the or it rectangle
in axes can be computed is a rectangle. Hence,we after the other in any
order.
8.31
We give slabs and
only a rough sketch of a solution. to intersect the slabs separately.
idea is
each polygon into according to their x This sorting can be done in linear time, because the cyclic order of the vertices coordinates. in each polygon is known. We associatewith each vertex a vertical line, and we use these The
First,
basic
we sort
all
to
divide
the vertices
of Solutions to
Sketches
436
slabs. An because each slab into
intersections
slabs, as in shown
in
Fig.
a polygon
in
Since we know
A.3.
only to
compute intersections two slabs can be computed intersection between then assemble has at most four edges. We can need
between
disjoint in constant time,
the corresponding
time.
linear
two triangles in constant is a convex polygon. We can intersect An intersection of triangles in linear time time, and we can intersect two convex (Exercise 8.31). Thus, a polygons that divides the set of triangles into two sets, computes the divide-and-conquer algorithm intersects the two resulting of all triangles in each set recursively, and then intersection has a running time of O (n log n). convex polygons
8.34
9
Chapter
One example
The method
is w = 15.
(jc15 = (((x2-x)2)-x)2-x). (jc15 =
9.10
into
order of the vertical
sorted
pairs of
9.2
two polygons lines, we
to divide the
lines
vertical the
Selected Exercises
It
((jcjc)2jc)3). See
We use the
notation
the following
types of
of Section which
introduced
presented
in
Section
to
compute
x]5
possible
p. 443]
1981,
[Knuth
definitions,
products
is
for a detailed
9.2 requires 6 with only 5
discussion of this
multiplications multiplications
issue.
(B +C +D + F +G)X. We make in the four to the seven multiplications correspond directly in Section 9.5.2: z, = b(e+f\\ z2 = c(g+h), z3 =
9.5.2. We need to
compute
We z4 = (a-h)e, z5 = (a -c)(g -e\\ z6 = (d-c)h, and z7 = (d-b)(f-h). now B contributes [z,, z,, 0, 0] (we write it look at the contribution of each of the matrices. as a row matrix instead of as a column matrix for convenience); C contributes [0, 0, z2, z2]; D contributes [0, z3, -z3, 0];F contributes [0, z6 + z7, [z4, 0, z4 +z5, 0]; and G contributes = Z\\ + we have p = Cu = z, +z4, r = C2J 0, z6]. So,overall, z3+z6 + z1, s = C,i2 = + Z5, and t = C2,2 =z2 + z6. There are 18 additions and 7 multiplications. Z2+Z3+Z4 (c-b)(e+h),
9.16 The hard that
part
it does
of this
problem is proving
by induction.
It
is easy
to
that verify
such a representation always exists. We prove the base case. Let n > 2 be an integer, and let
not larger than n (i.e.,F(k)n/2, since otherwise F(k+ \\) = F(k)+F(kcan be as a sum of at most hypothesis, n-F(k) represented
Figure
A.3
Intersection
of two
convex polygons
and F(k +
1) n).
the
We
induction
Fibonacci
Sketches of
numbers. To that
9.19
since
But,
the numbers
keeps find
the
We denote
furthermore,
parts of the
four
the
437
Exercises
of n-F(k) still adding F(k) to the representation - F (k)) + 1 < it is easy to see that \\og2(n log2w. all Fibonacci numbers until we reach one we need, we compute to n\\ we can then follow the proof given here.
than or equal
second
the
Selected
n-F(k)
