第1页 / 共219页
第2页 / 共219页
第3页 / 共219页
第4页 / 共219页
第5页 / 共219页
第6页 / 共219页
第7页 / 共219页
第8页 / 共219页
Designing Distributed Systems.pdf
Preface
Who Should Read This Book
Why I Wrote This Book
The World of Distributed Systems Today
Navigating This Book
Conventions Used in This Book
Online Resources
Using Code Examples
O’Reilly Safari
How to Contact Us
Acknowledgments
1. Introduction
A Brief History of Systems Development
A Brief History of Patterns in Software Development
Formalization of Algorithmic Programming
Patterns for Object-Oriented Programming
The Rise of Open Source Software
The Value of Patterns, Practices, and Components
Standing on the Shoulders of Giants
A Shared Language for Discussing Our Practice
Shared Components for Easy Reuse
Summary
I. Single-Node Patterns
Motivations
Summary
2. The Sidecar Pattern
An Example Sidecar: Adding HTTPS to a Legacy Service
Dynamic Configuration with Sidecars
Modular Application Containers
Hands On: Deploying the topz Container
Building a Simple PaaS with Sidecars
Designing Sidecars for Modularity and Reusability
Parameterized Containers
Define Each Container’s API
Documenting Your Containers
Summary
3. Ambassadors
Using an Ambassador to Shard a Service
Hands On: Implementing a Sharded Redis
Using an Ambassador for Service Brokering
Using an Ambassador to Do Experimentation or Request Splitting
Hands On: Implementing 10% Experiments
4. Adapters
Monitoring
Hands On: Using Prometheus for Monitoring
Logging
Hands On: Normalizing Different Logging Formats with Fluentd
Adding a Health Monitor
Hands On: Adding Rich Health Monitoring for MySQL
II. Serving Patterns
Introduction to Microservices
5. Replicated Load-Balanced Services
Stateless Services
Readiness Probes for Load Balancing
Hands On: Creating a Replicated Service in Kubernetes
Session Tracked Services
Application-Layer Replicated Services
Introducing a Caching Layer
Deploying Your Cache
Hands On: Deploying the Caching Layer
Expanding the Caching Layer
Rate Limiting and Denial-of-Service Defense
SSL Termination
Hands On: Deploying nginx and SSL Termination
Summary
6. Sharded Services
Sharded Caching
Why You Might Need a Sharded Cache
The Role of the Cache in System Performance
Replicated, Sharded Caches
Hands On: Deploying an Ambassador and Memcache for a Sharded Cache
An Examination of Sharding Functions
Selecting a Key
Consistent Hashing Functions
Hands On: Building a Consistent HTTP Sharding Proxy
Sharded, Replicated Serving
Hot Sharding Systems
7. Scatter/Gather
Scatter/Gather with Root Distribution
Hands On: Distributed Document Search
Scatter/Gather with Leaf Sharding
Hands On: Sharded Document Search
Choosing the Right Number of Leaves
Scaling Scatter/Gather for Reliability and Scale
8. Functions and Event-Driven Processing
Determining When FaaS Makes Sense
The Benefits of FaaS
The Challenges of FaaS
The Need for Background Processing
The Need to Hold Data in Memory
The Costs of Sustained Request-Based Processing
Patterns for FaaS
The Decorator Pattern: Request or Response Transformation
Hands On: Adding Request Defaulting Prior to Request Processing
Handling Events
Hands On: Implementing Two-Factor Authentication
Event-Based Pipelines
Hands On: Implementing a Pipeline for New-User Signup
9. Ownership Election
Determining If You Even Need Master Election
The Basics of Master Election
Hands On: Deploying etcd
Implementing Locks
Hands On: Implementing Locks in etcd
Implementing Ownership
Hands On: Implementing Leases in etcd
Handling Concurrent Data Manipulation
III. Batch Computational Patterns
10. Work Queue Systems
A Generic Work Queue System
The Source Container Interface
The Worker Container Interface
The Shared Work Queue Infrastructure
Hands On: Implementing a Video Thumbnailer
Dynamic Scaling of the Workers
The Multi-Worker Pattern
11. Event-Driven Batch Processing
Patterns of Event-Driven Processing
Copier
Filter
Splitter
Sharder
Merger
Hands On: Building an Event-Driven Flow for New User Sign-Up
Publisher/Subscriber Infrastructure
Hands On: Deploying Kafka
12. Coordinated Batch Processing
Join (or Barrier Synchronization)
Reduce
Hands On: Count
Sum
Histogram
Hands On: An Image Tagging and Processing Pipeline
13. Conclusion: A New Beginning?
Index
Download from finelybook www.finelybook.com
Designing Distributed Systems
Patterns and Paradigms for Scalable, Reliable Services
Brendan Burns
2
Download from finelybook www.finelybook.com
Designing Distributed Systems
by Brendan Burns
Copyright © 2018 Brendan Burns. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North,
Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales
promotional use. Online editions are also available for most titles
(http://oreilly.com/safari). For more information, contact our
corporate/institutional sales department: 800-998-9938 or
corporate@oreilly.com.
Editor: Angela Rufino
Production Editor: Colleen Cole
Copyeditor: Gillian McGarvey
Proofreader: Christina Edwards
Indexer: WordCo Indexing Services, Inc.
Interior Designer: David Futato
Cover Designer: Randy Comer
Illustrator: Rebecca Demarest
February 2018: First Edition
Revision History for the First Edition
2018-02-20: First Release
3
Download from finelybook www.finelybook.com
See http://oreilly.com/catalog/errata.csp?isbn=9781491983645 for release
details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc.
Designing Distributed Systems, the cover image, and related trade dress are
trademarks of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that
the information and instructions contained in this work are accurate, the
publisher and the author disclaim all responsibility for errors or omissions,
including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained
in this work is at your own risk. If any code samples or other technology this
work contains or describes is subject to open source licenses or the
intellectual property rights of others, it is your responsibility to ensure that
your use thereof complies with such licenses and/or rights.
978-1-491-98364-5
[LSI]
4
Download from finelybook www.finelybook.com
Preface
Who Should Read This Book
At this point, nearly every developer is a developer or consumer (or both) of
distributed systems. Even relatively simple mobile applications are backed
with cloud APIs so that their data can be present on whatever device the
customer happens to be using. Whether you are new to developing distributed
systems or an expert with scars on your hands to prove it, the patterns and
components described in this book can transform your development of
distributed systems from art to science. Reusable components and patterns for
distributed systems will enable you to focus on the core details of your
application. This book will help any developer become better, faster, and
more efficient at building distributed systems.
Why I Wrote This Book
Throughout my career as a developer of a variety of software systems from
web search to the cloud, I have built a large number of scalable, reliable
distributed systems. Each of these systems was, by and large, built from
scratch. In general, this is true of all distributed applications. Despite having
many of the same concepts and even at times nearly identical logic, the
ability to apply patterns or reuse components is often very, very challenging.
This forced me to waste time reimplementing systems, and each system
ended up less polished than it might have otherwise been.
The recent introduction of containers and container orchestrators
fundamentally changed the landscape of distributed system development.
Suddenly we have an object and interface for expressing core distributed
system patterns and building reusable containerized components. I wrote this
book to bring together all of the practitioners of distributed systems, giving us
a shared language and common standard library so that we can all build better
5
Download from finelybook www.finelybook.com
systems more quickly.
The World of Distributed Systems Today
Once upon a time, people wrote programs that ran on one machine and were
also accessed from that machine. The world has changed. Now, nearly every
application is a distributed system running on multiple machines and
accessed by multiple users from all over the world. Despite their prevalence,
the design and development of these systems is often a black art practiced by
a select group of wizards. But as with everything in technology, the world of
distributed systems is advancing, regularizing, and abstracting. In this book I
capture a collection of repeatable, generic patterns that can make the
development of reliable distributed systems more approachable and efficient.
The adoption of patterns and reusable components frees developers from
reimplementing the same systems over and over again. This time is then freed
to focus on building the core application itself.
Navigating This Book
This book is organized into a 4 parts as follows:
Chapter 1, Introduction
Introduces distributed systems and explains why patterns and reusable
components can make such a difference in the rapid development of
reliable distributed systems.
Part I, Single-Node Patterns
Chapters 2 through 4 discuss reusable patterns and components that occur
on individual nodes within a distributed system. It covers the side-car,
adapter, and ambassador single-node patterns.
Part II, Serving Patterns
Chapters 8 and 9 cover multi-node distributed patterns for long-running
serving systems like web applications. Patterns for replicating, scaling,
6
Download from finelybook www.finelybook.com
and master election are discussed.
Part III, Batch Computational Patterns
Chapters 10 through 12 cover distributed system patterns for large-scale
batch data processing covering work queues, event-based processing, and
coordinated workflows.
If you are an experienced distributed systems engineer, you can likely skip
the first couple of chapters, though you may want to skim them to understand
how we expect these patterns to be applied and why we think the general
notion of distributed system patterns is so important.
Everyone will likely find utility in the single-node patterns as they are the
most generic and most reusable patterns in the book.
Depending on your goals and the systems you are interested in developing,
you can choose to focus on either large-scale big data patterns, or patterns for
long-running servers (or both). The two parts are largely independent from
each other and can be read in any order.
Likewise, if you have extensive distributed system experience, you may find
that some of the early patterns chapters (e.g., Part II on naming, discovery,
and load balancing) are redundant with what you already know, so feel free
to skim through to gain the high-level insights—but don’t forget to look at all
of the pretty pictures!
Conventions Used in This Book
The following typographical conventions are used in this book:
Italic
Indicates new terms, URLs, email addresses, filenames, and file
extensions.
Constant width
Used for program listings, as well as within paragraphs to refer to
program elements such as variable or function names, databases, data
7
Download from finelybook www.finelybook.com
types, environment variables, statements, and keywords.
Constant width bold
Shows commands or other text that should be typed literally by the user.
Constant width italic
Shows text that should be replaced with user-supplied values or by values
determined by context.
This icon signifies a tip, suggestion, or general note.
TIP
This icon indicates a warning or caution.
WARNING
Online Resources
Though this book describes generally applicable distributed system patterns,
it expects that readers are familiar with containers and container orchestration
systems. If you don’t have a lot of pre-existing knowledge about these things,
we recommend the following resources:
https://docker.io
https://kubernetes.io
https://dcos.io
Using Code Examples
Supplemental material (code examples, exercises, etc.) is available for
8
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
