Algorithms Sequential and Parallel: A Unified Approach - Hardcover

A major thrust of computer science is the design, analysis, implementation, and scientific evaluation of algorithms to solve critical problems. In addition, new challenges are being offered to computer scientists in the field of computational science and engineering, which includes challenging problems in computational biology, computational fluid dynamics, and computational chemistry, to name a few. As parallel computing continues to merge into the mainstream of computing, it becomes more and more important for students and scientists to understand the application and analysis of algorithmic paradigms to both the (traditional) sequential model of computing and to a variety of parallel models.

Many computer science departments offer courses in "Analysis of Algorithms," "Algorithms," "An Introduction to Algorithms," or "Data Structures and their Algorithms" at the junior or senior level. In addition, a course in "Analysis of Algorithms" is required of most graduate students pursuing a degree in computer science. Throughout the 1980s, the vast majority of these course offerings focused on algorithms for sequential (von Neumann) computers. In fact, not until the late-1980's did courses covering an introduction to parallel algorithms begin to appear in research-oriented departments. Furthermore, these courses in parallel algorithms were typically presented to advanced graduate students. However, by the early 1990s, courses in parallel computing began to emerge at the undergraduate level, especially at progressive 4-year colleges.

It is interesting to note that throughout much of the 1990's, traditional algorithms-based courses changed very little. Gradually, such courses began to incorporate a component of parallel algorithms, typically one to three weeks near the end of the semester. During the later part of the 1990s, however, it was not uncommon to find algorithms courses that contained as much as 1/3 of the material devoted to parallel algorithms.

In this book, we take a very different approach to a traditional algorithms-based course. Parallel computing has become more mainstream, with small multiprocessor machines (which can be ordered by mail from your favorite catalog vendor) flooding the marketplace and with distributed computing systems being efficiently exploited. Therefore, we believe the time is right to teach a fundamental course in algorithms that covers paradigms for both the sequential and parallel models. In fact, the approach we take is to integrate the coverage of parallel and sequential algorithms throughout the course.

The philosophy taken in this book is to cover a paradigm, such as divide-and-conquer, and then cover implementation issues for both the sequential and parallel models. Due to the fact that we present design and analysis of paradigms for sequential and parallel models, the reader might notice that the number of paradigms we can treat within a semester is limited.

Several offerings of a course based on a preliminary version of this book have been taught successfully at both the undergraduate and graduate levels at the State University of New York at Buffalo.

Prerequisites: We assume that the reader has a basic knowledge of data structures. That is, the reader should be comfortable with the notion of a stack, queue, list, and binary tree, at a level that is typically taught in a CS2 course. The reader should also be familiar with fundamentals of discrete mathematics and Calculus. Specifically, the reader should be comfortable with limits, summations, and integrals.

OVERVIEW OF CHAPTERS

Background material for the course is presented in Chapters 1, 2, & 3. Chapter 1 introduces the concept of asymptotic analysis. While the reader might have seen some of this material in a course on data structures, we present this material in a fair amount of detail. The reader who is uncomfortable with some of the fundamental material from a Freshman-level Calculus sequence might want to brush up on notions such as limits, summations and integrals, and derivatives, as they naturally arise in the presentation and application of asymptotic analysis. Chapter 2 focuses on fundamentals of induction and recursion. While many students have seen this material in previous courses in computer science and/or mathematics, we have found it important to review this material briefly and to provide the students with a reference for performing the necessary review. In Chapter 3, we present the Master Method, a very useful cookbook-type of system for evaluating recurrence equations that are common in an algorithms-based setting.

Chapter 4 presents an overview of combinational circuits and sorting networks. This work is used to motivate the natural use of parallel models and to demonstrate the blending of architectural and algorithmic approaches. In Chapter 5, we introduce fundamental models of computation, including the RAM (a formal sequential architecture) and a variety of parallel models of computation. The parallel models introduced include the PRAM, mesh, and hypercube, to name a few. In addition, Chapter 5 introduces terminology such as shared-memory and distributed memory.

The focus of Chapter 6 is the important problem of matrix multiplication, which is considered for a variety of models of computation. In Chapter 7, we introduce the parallel prefix operation. This is a very powerful operation with a wide variety of applications. We discuss implementations and analysis for a number of the models presented in Chapter 5 and give sample applications. In Chapter 8, we introduce pointer jumping techniques and show how some list-based algorithms can be efficiently implemented in parallel.

In Chapter 9, we introduce the powerful divide-and-conquer paradigm. We discuss applications of divide-and-conquer to problems involving data movement, including sorting, concurrent reads/writes, and so forth. Algorithms and their analysis are presented for a variety of models.

Chapters 10 & 11 focus on two important application areas, namely, computational geometry and image processing. In these chapters, we focus on interesting problems chosen from these important domains as a way of solidifying the approach of this book in terms of developing machine independent solution strategies, which can then be tailored for specific models, as required.

Chapter 12 focuses on fundamental graph theoretic problems. Initially, we present standard traversal techniques, including breadth-first search, depth-first search, and pointer jumping. We then discuss fundamental problems, including tree contraction and transitive closure. Finally, we couple these techniques with greedy algorithms to solve problems, such as labeling the connected components of a graph, determining a minimal spanning forest of a graph, and problems involving shortest or minimal-weight paths in a graph.

Chapter 13 is an optional chapter concerned with some fundamental numerical problems. The focus of the chapter is on sequential algorithms for polynomial evaluation and approximations of definite integrals.

RECOMMENDED USE

This book has been used in both a junior/senior-level elective and a required first year graduate level course in the Department of Computer Science and Engineering at the State University of New York at Buffalo (SUNY Buffalo). The course was presented in 14 weeks, consisting of twenty-four (24) 75-minute lectures, two review classes, and two exams. Recitations were conducted by an advanced graduate student and were used to help the students with homework sets and an understanding of the material. The recitations were especially important early in the semester when mathematically-based background material was being presented. This course is tailored towards students who are not advanced in a mathematical sense, but have a basic, fundamental, background.

CORRESPONDENCE

Please feel free to contact the authors directly with any comments or criticisms (constructive or otherwise) of this book. Russ Miller may be reached at miller@cse.buffalo and Laurence Boxer may be reached at boxer@niagara.This Web site contains information related to the book, including pointers to education-based pages, relevant parallel computing links, and errata.

• The text offers a unified approach that relates sequential and parallel algorithms where appropriate and contrasts where appropriate
.
• Mathematical tools are developed in early chapters
.
• A variety of examples are worked out in great detail with multiple methods of solutions
.
• Sequential and parallel examples and exercises are featured
.
• Supplemental material is available for instructors
.
• A Prentice Hall Companion Website with additional material is available at http://www.prenhall.com/millerboxer.