第1页 / 共341页
第2页 / 共341页
第3页 / 共341页
第4页 / 共341页
第5页 / 共341页
第6页 / 共341页
第7页 / 共341页
第8页 / 共341页
【2018新书】Deep Learning Cookbook.pdf
Preface
A Brief History of Deep Learning
Why Now?
What Do You Need to Know?
How This Book Is Structured
Conventions Used in This Book
Accompanying Code
O’Reilly Safari
How to Contact Us
Acknowledgments
Tools and Techniques
1.1. Types of Neural Networks
1.2. Acquiring Data
1.3. Preprocessing Data
Getting Unstuck
2.1. Determining That You Are Stuck
2.2. Solving Runtime Errors
2.3. Checking Intermediate Results
2.4. Picking the Right Activation Function (for Your Final Layer)
2.5. Regularization and Dropout
2.6. Network Structure, Batch Size, and Learning Rate
Calculating Text Similarity Using Word Embeddings
3.1. Using Pretrained Word Embeddings to Find Word Similarity
3.2. Word2vec Math
3.3. Visualizing Word Embeddings
3.4. Finding Entity Classes in Embeddings
3.5. Calculating Semantic Distances Inside a Class
3.6. Visualizing Country Data on a Map
Building a Recommender System Based on Outgoing Wikipedia Links
4.1. Collecting the Data
4.2. Training Movie Embeddings
4.3. Building a Movie Recommender
4.4. Predicting Simple Movie Properties
Generating Text in the Style of an Example Text
5.1. Acquiring the Text of Public Domain Books
5.2. Generating Shakespeare-Like Texts
5.3. Writing Code Using RNNs
5.4. Controlling the Temperature of the Output
5.5. Visualizing Recurrent Network Activations
Question Matching
6.1. Acquiring Data from Stack Exchange
6.2. Exploring Data Using Pandas
6.3. Using Keras to Featurize Text
6.4. Building a Question/Answer Model
6.5. Training a Model with Pandas
6.6. Checking Similarities
Suggesting Emojis
7.1. Building a Simple Sentiment Classifier
7.2. Inspecting a Simple Classifier
7.3. Using a Convolutional Network for Sentiment Analysis
7.4. Collecting Twitter Data
7.5. A Simple Emoji Predictor
7.6. Dropout and Multiple Windows
7.7. Building a Word-Level Model
7.8. Constructing Your Own Embeddings
7.9. Using a Recurrent Neural Network for Classification
7.10. Visualizing (Dis)Agreement
7.11. Combining Models
Sequence-to-Sequence Mapping
8.1. Training a Simple Sequence-to-Sequence Model
8.2. Extracting Dialogue from Texts
8.3. Handling an Open Vocabulary
8.4. Training a seq2seq Chatbot
Reusing a Pretrained Image Recognition Network
9.1. Loading a Pretrained Network
9.2. Preprocessing Images
9.3. Running Inference on Images
9.4. Using the Flickr API to Collect a Set of Labeled Images
9.5. Building a Classifier That Can Tell Cats from Dogs
9.6. Improving Search Results
9.7. Retraining Image Recognition Networks
Building an Inverse Image Search Service
10.1. Acquiring Images from Wikipedia
10.2. Projecting Images into an N-Dimensional Space
10.3. Finding Nearest Neighbors in High-Dimensional Spaces
10.4. Exploring Local Neighborhoods in Embeddings
Detecting Multiple Images
11.1. Detecting Multiple Images Using a Pretrained Classifier
11.2. Using Faster RCNN for Object Detection
11.3. Running Faster RCNN over Our Own Images
Image Style
12.1. Visualizing CNN Activations
12.2. Octaves and Scaling
12.3. Visualizing What a Neural Network Almost Sees
12.4. Capturing the Style of an Image
12.5. Improving the Loss Function to Increase Image Coherence
12.6. Transferring the Style to a Different Image
12.7. Style Interpolation
Generating Images with Autoencoders
13.1. Importing Drawings from Google Quick Draw
13.2. Creating an Autoencoder for Images
13.3. Visualizing Autoencoder Results
13.4. Sampling Images from a Correct Distribution
13.5. Visualizing a Variational Autoencoder Space
13.6. Conditional Variational Autoencoders
Generating Icons Using Deep Nets
14.1. Acquiring Icons for Training
14.2. Converting the Icons to a Tensor Representation
14.3. Using a Variational Autoencoder to Generate Icons
14.4. Using Data Augmentation to Improve the Autoencoder’s Performance
14.5. Building a Generative Adversarial Network
14.6. Training Generative Adversarial Networks
14.7. Showing the Icons the GAN Produces
14.8. Encoding Icons as Drawing Instructions
14.9. Training an RNN to Draw Icons
14.10. Generating Icons Using an RNN
Music and Deep Learning
15.1. Creating a Training Set for Music Classification
15.2. Training a Music Genre Detector
15.3. Visualizing Confusion
15.4. Indexing Existing Music
15.5. Setting Up Spotify API Access
15.6. Collecting Playlists and Songs from Spotify
15.7. Training a Music Recommender
15.8. Recommending Songs Using a Word2vec Model
Productionizing Machine Learning Systems
16.1. Using Scikit-Learn’s Nearest Neighbors for Embeddings
16.2. Use Postgres to Store Embeddings
16.3. Populating and Querying Embeddings Stored in Postgres
16.4. Storing High-Dimensional Models in Postgres
16.5. Writing Microservices in Python
16.6. Deploying a Keras Model Using a Microservice
16.7. Calling a Microservice from a Web Framework
16.8. TensorFlow seq2seq models
16.9. Running Deep Learning Models in the Browser
16.10. Running a Keras Model Using TensorFlow Serving
16.11. Using a Keras Model from iOS
Index
Deep Learning Cookbook
Practical Recipes to Get Started Quickly
Douwe Osinga
Deep Learning Cookbook
by Douwe Osinga
Copyright © 2018 Douwe Osinga. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North,
Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales
promotional use. Online editions are also available for most titles
(http://oreilly.com/safari). For more information, contact our
corporate/institutional sales department: 800-998-9938 or
corporate@oreilly.com.
Editors: Rachel Roumeliotis and Jeff Bleiel
Production Editor: Colleen Cole
Copyeditor: Rachel Head
Proofreader: Charles Roumeliotis
Indexer: WordCo Indexing Services, Inc.
Interior Designer: David Futato
Cover Designer: Randy Comer
Illustrator: Rebecca Demarest
June 2018: First Edition
Revision History for the First Edition
2018-05-23: First Release
See http://oreilly.com/catalog/errata.csp?isbn=9781491995846 for release
details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc. Deep
Learning Cookbook, the cover image, and related trade dress are trademarks
of O’Reilly Media, Inc.
While the publisher and the author have used good faith efforts to ensure that
the information and instructions contained in this work are accurate, the
publisher and the author disclaim all responsibility for errors or omissions,
including without limitation responsibility for damages resulting from the use
of or reliance on this work. Use of the information and instructions contained
in this work is at your own risk. If any code samples or other technology this
work contains or describes is subject to open source licenses or the
intellectual property rights of others, it is your responsibility to ensure that
your use thereof complies with such licenses and/or rights.
978-1-491-99584-6
[LSI]
Preface
A Brief History of Deep Learning
The roots of the current deep learning boom go surprisingly far back, to the
1950s. While vague ideas of “intelligent machines” can be found further back
in fiction and speculation, the 1950s and ’60s saw the introduction of the first
“artificial neural networks,” based on a dramatically simplified model of
biological neurons. Amongst these models, the Perceptron system articulated
by Frank Rosenblatt garnered particular interest (and hype). Connected to a
simple “camera” circuit, it could learn to distinguish different types of
objects. Although the first version ran as software on an IBM computer,
subsequent versions were done in pure hardware.
Interest in the multilayer perceptron (MLP) model continued through the
’60s. This changed when, in 1969, Marvin Minksy and Seymour Papert
published their book Perceptrons (MIT Press). The book contained a proof
showing that linear perceptrons could not classify the behavior of a nonlinear
function (XOR). Despite the limitations of the proof (nonlinear perceptron
models existed at the time of the book’s publication, and are even noted by
the authors), its publication heralded the plummeting of funding for neural
network models. Research would not recover until the 1980s, with the rise of
a new generation of researchers.
The increase in computing power together with the development of the back-
propagation technique (known in various forms since the ’60s, but not
applied in general until the ’80s) prompted a resurgence of interest in neural
networks. Not only did computers have the power to train larger networks,
but we also had the techniques to train deeper networks efficiently. The first
convolutional neural networks combined these insights with a model of
visual recognition from mammalian brains, yielding for the first time
networks that could efficiently recognize complex images such as
handwritten digits and faces. Convolutional networks do this by applying the
same “subnetwork” to different locations of the image and aggregating the
results of these into higher-level features. In Chapter 12 we look at how this
works in more detail.
In the ’90s and early 2000s interest in neural networks declined again as more
“understandable” models like support vector machines (SVMs) and decision
trees became popular. SVMs proved to be excellent classifiers for many data
sources of the time, especially when coupled with human-engineered
features. In computer vision, “feature engineering” became popular. This
involves building feature detectors for small elements in a picture and
combining them by hand into something that recognizes more complex
forms. It later turned out that deep learning nets learn to recognize very
similar features and learn to combine them in a very similar way. In
Chapter 12 we explore some of the inner workings of these networks and
visualize what they learn.
With the advent of general-purpose programming on graphics processing
units (GPUs) in the late 2000s, neural network architectures were able to
make great strides over the competition. GPUs contain thousands of small
processors that can do trillions of operations per second in parallel. Originally
developed for computer gaming, where this is needed to render complex 3D
scenes in real time, it turned out that the same hardware can be used to train
neural networks in parallel, achieving speed improvements of a factor of 10
or higher.
The other thing that happened was that the internet made very large training
sets available. Where researchers had been training classifiers with thousands
of images before, now they had access to tens if not hundreds of millions of
images. Combined with larger networks, neural networks had their chance to
shine. This dominance has only continued in the succeeding years, with
improved techniques and applications of neural networks to areas outside of
image recognition, including translation, speech recognition, and image
synthesis.
Why Now?
While the boom in computational power and better techniques led to an
increase in interest in neural networks, we have also seen huge strides in
usability. In particular, deep learning frameworks like TensorFlow, Theano,
and Torch allow nonexperts to construct complex neural networks to solve
their own machine learning problems. This has turned a task that used to
require months or years of handcoding and head-on-table-banging effort
(writing efficient GPU kernels is hard!) into something that anyone can do in
an afternoon (or really a few days in practice). Increased usability has greatly
increased the number of researchers who can work on deep learning
problems. Frameworks like Keras with an even higher level of abstraction
make it possible for anyone with a working knowledge of Python and some
tools to run some interesting experiments, as this book will show.
A second important factor for “why now” is that large datasets have become
available for everybody. Yes, Facebook and Google might still have the
upper hand with access to billions of pictures, user comments, and what have
you, but datasets with millions of items can be had from a variety of sources.
In Chapter 1 we’ll look at a variety of options, and throughout the book the
example code for each chapter will usually show in the first recipe how to get
the needed training data.
At the same time, private companies have started to produce and collect
orders of magnitude more data, which has made the whole area of deep
learning suddenly commercially very interesting. A model that can tell the
difference between a cat and a dog is all very well, but a model that increases
sales by 15% by taking all historic sales data into account can be the
difference between life and death for a company.
What Do You Need to Know?
These days there is a wide choice of platforms, technologies, and
programming languages for deep learning. In this book all the examples are
in Python and most of the code relies on the excellent Keras framework. The
example code is available on GitHub as a set of Python notebooks, one per
chapter. So, having a working knowledge of the following will help:
Python
Python 3 is preferred, but Python 2.7 should also work. We use a variety
of helper libraries that all can easily be installed using pip. The code is
generally straightforward so even a relative novice should be able to
follow the action.
Keras
The heavy lifting for machine learning is done almost completely by
Keras. Keras is an abstraction over either TensorFlow or Theano, both
deep learning frameworks. Keras makes it easy to define neural networks
in a very readable way. All code is tested against TensorFlow but should
also work with Theano.
NumPy, SciPy, scikit-learn
These useful and extensive libraries are casually used in many recipes.
Most of the time it should be clear what is happening from the context,
but a quick read-up on them won’t hurt.
Jupyter Notebook
Notebooks are a very nice way to share code; they allow for a mixture of
code, output of code, and comments, all viewable in the browser.
Each chapter has a corresponding notebook that contains working code. The
code in the book often leaves out details like imports, so it is a good idea to
get the code from Git and launch a local notebook. First check out the code
and enter the new directory:
git clone https://github.com/DOsinga/deep_learning_cookbook.git
cd deep_learning_cookbook
Then set up a virtual environment for the project:
python3 -m venv venv3
source venv3/bin/activate
And install the dependencies:
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
