Introduction to Algorithms: A Creative Approach

INTRODUCTION TO ALGORITHMS A Creative Approach

UDIMANBER University ofArizona

• • •

ADDISON-WESLEY PUBLISHING COMPANY Reading, Massachusetts • Menlo Park, California • New York Don Mills, Ontario • Wokingham, England • Amsterdam Bonn • Sydney • Singapore • Tokyo • Madrid • San Juan

CONTENTS Chapter 1 Introduction

l

Chapter 2 Mathematical Induction

9

2.1 2.2 2.3 2.4 2.5 2.6 2.7 2.8 2.9 2.10 2.11 2.12 2.13 2.14

Introduction Three Simple Examples Counting Regions in the Plane A Simple Coloring Problem A More Complicated Summation Problem A Simple Inequality Euler's Formula A Problem in Graph Theory Gray Codes Finding Edge-Disjoint Paths in a Graph Arithmetic versus Geometrie Mean Theorem Loop Invariants: Converting a Decimal Number to Binary Common Errors Summary Bibliographie Notes and Further Reading Exercises

Chapter 3 Analysis of Algorithms 3.1 3.2 3.3 3.4 3.5

3.6 3.7

Introduction The O Notation Time and Space Complexity Summations Recurrence Relations 3.5.1 Intelligent Guesses 3.5.2 Divide and Conquer Relations 3.5.3 Recurrence Relations with Füll History Useful Facts Summary Bibliographie Notes and Further Reading Exercises

9 11 13 14 15 16 17 18 20 23 24 26 28 29 30 31 37 37 39 42 43 46 47 50 51 53 55 55 56

IX

x

Contents

Chapter 4 Data Structures 4.1 4.2

4.3

4.4 4.5 4.6 4.7

Introduction Elementary Data Structures 4.2.1 Elements 4.2.2 Arrays 4.2.3 Records 4.2.4 Linked Lists Trees 4.3.1 Representation of Trees 4.3.2 Heaps 4.3.3 Binary Search Trees 4.3.4 AVL Trees Hashing The Union-Find Problem Graph s Summary Bibliographie Notes and Further Reading Exercises

Chapter 5 Design of Algorithms by Induction 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 5.9 5.10 5.11 5.12

61 62 62 63 63 64 66 67 68 71 75 78 80 83 84 85 86 91

Introduction Evaluating Polynomials Maximal Induced Subgraph Finding One-to-One Mappings The Celebrity Problem A Divide-and-Conquer Algorithm: The Skyline Problem Computing Balance Factors in Binary Trees Finding the Maximum Consecutive Subsequence Strengthening the Induction Hypothesis Dynamic Programming: The Knapsack Problem Common Errors Summary Bibliographie Notes and Further Reading Exercises

Chapter 6 Algorithms Involving Sequences and Sets 6.1 6.2 6.3 6.4

61

Introduction Binary Search and Variations Interpolation Search Sorting 6.4.1 Bücket Sort and Radix Sort 6.4.2 Insertion Sort and Selection Sort 6.4.3 Mergesort

91 92 95 96 98 102 104 106 107 108 111 112 113 114 119 119 120 125 127 127 130 130

Contents

6.4.4 Quicksort 6.4.5 Heapsort 6.4.6 A Lower Bound for Sorting 6.5 Order Statistics 6.5.1 Maximum and Minimum Elements 6.5.2 Finding the M)-Smallest Element 6.6 Data Compression 6.7 String Matching 6.8 Sequence Comparisons 6.9 Probabilistic Algorithms 6.9.1 Random Numbers 6.9.2 A Coloring Problem 6.9.3 A Technique for Transforming Probabilistic Algorithms into Deterministic Algorithms 6.10 Finding a Majority 6.11 Three Problems Exhibiting Interesting Proof Techniques 6.11.1 Longest Increasing Subsequence 6.11.2 Finding the Two Largest Elements in a Set 6.11.3 Computing the Mode of a Multiset 6.12 Summary Bibliographie Notes and Further Reading Exercises Chapter7 Graph Algorithms 7.1 7.2 7.3

Introduction Eulerian Graphs Graph Traversais 7.3.1 Depth-First Search 7.3.2 Breadth-First Search 7.4 Topological Sorting 7.5 Single-Source Shortest Paths 7.6 Minimum-Cost Spanning Trees 7.7 All Shortest Paths 7.8 Transitive Closure 7.9 Decompositions of Graphs 7.9.1 Biconnected Components 7.9.2 Strongly Connected Components 7.9.3 Examples of the Use of Graph Decomposition 7.10 Matching 7.10.1 Perfect Matching in Very Dense Graphs 7.10.2 Bipartite Matching 7.11 Network Flows 7.12 Hamiltonian Tours 7.12.1 Reversed Induction

xi

131 137 141 143 143 144 145 148 155 158 160 161 161 164 167 167 169 171 173 173 175 185 185 187 189 190 198 199 201 208 212 214 217 217 226 230 234 ,234 235 238 243 244

xii

Contents

7.13

7.12.2 Finding Hamiltonian Cycles in Very Dense Graphs Summary Bibliographie Notes and Further Reading Exercises

Chapter 8 Geometrie Algorithms 8.1 8.2 8.3 8.4

8.5 8.6 8.7

9.6 9.7

265

Introduction Determining Whether a Point Is Inside a Polygon Constructing Simple Polygons Convex Hulls 8.4.1 A Straightforward Approach 8.4.2 Gift Wrapping 8.4.3 Graham'sScan Closest Pair Intersections of Horizontal and Vertical Line Segments Summary Bibliographie Notes and Further Reading Exercises

Chapter 9 Algebraic and Numeric Algorithms 9.1 9.2 9.3 9.4 9.5

244 246 247 248

265 266 270 273 273 274 275 278 281 285 286 287 293

Introduction Exponentiation Euclid's Algorithm Polynomial Multiplication Matrix Multiplication 9.5.1 Winograd's Algorithm 9.5.2 Strassen's Algorithm 9.5.3 Boolean Matrices The Fast Fourier Transform Summary Bibliographie Notes and Further Reading Exercises

Chapter 10 Reductions 10.1 Introduction 10.2 Examples of Reductions 10.2.1 A Simple String-Matching Problem 10.2.2 Systems of Distinct Representatives 10.2.3 A Reduction Involving Sequence Comparisons 10.2.4 Finding a Triangle in Undirected Graphs 10.3 Reductions Involving Linear Programming 10.3.1 Introduction and Definitions 10.3.2 Examples of Reductions to Linear Programming

293 294 297 298 301 301 301 304 309 316 316 317

321 321 323 323 323 324 325 327 327 329

Contents

10.4 Reductions for Lower Bounds 10.4.1 A Lower Bound for Finding Simple Polygons 10.4.2 Simple Reductions Involving Matrices 10.5 Common Errors 10.6 Summary Bibliographie Notes and Further Reading Exercises ChapterU

NP-Completeness

331 331 333 334 336 336 337 341

11.1 11.2 11.3 11.4

Introduction Polynomial-Time Reductions Nondeterminism and Cook's Theorem Examples of NP-Completeness Proofs 11.4.1 Vertex Cover 11.4.2 Dominating Set 11.4.3 3SAT 11.4.4 Clique 11.4.5 3-Coloring 11.4.6 General Observations 11.4.7 More NP-Complete Problems 11.5 Techniques For Dealing with NP-Complete Problems 11.5.1 Backtracking and Branch-and-Bound 11.5.2 Approximation Algorithms with Guaranteed Performance 11.6 Summary Bibliographie Notes and Further Reading Exercises Chapter12 Parallel Algorithms 12.1 Introduction 12.2 Models of Parallel Computation 12.3 Algorithms for Shared-Memory Machines 12.3.1 Parallel Addition 12.3.2 Maximum-Finding Algorithms 12.3.3 The Parallel-Prefix Problem 12.3.4 Finding Ranks in Linked Lists 12.3.5 The Euler's Tour Technique 12.4 Algorithms for Interconnection Networks 12.4.1 Sorting on an Array 12.4.2 Sorting Networks 12.4.3 Finding the Äth-Smallest Element on a Tree 12.4.4 Matrix Multiplication on the Mesh 12.4.5 Routing in a Hypercube

xiii

341 342 344 347 348 348 350 351 352 355 356 357 358 363 368 368 370 375 375 376 378 379 380 382 385 387 389 390 393 396 398 401

xiv

Contents

12.5 Systolic Computation 12.5.1 Matrix-Vector Multiplication 12.5.2 The Convolution Problem 12.5.3 Sequence Comparisons 12.6 Summary Bibliographie Notes and Further Reading Exercises

tions to Selected Exercises

404 404 405 407 409 409 411 417

References

445

Index

465