第1页 / 共261页
第2页 / 共261页
第3页 / 共261页
第4页 / 共261页
第5页 / 共261页
第6页 / 共261页
第7页 / 共261页
第8页 / 共261页
Deep Learning Quick Reference.pdf
Cover
Copyright and Credits
Dedication
Packt Upsell
Foreword
Contributors
Table of Contents
Preface
Chapter 1: The Building Blocks of Deep Learning
The deep neural network architectures
Neurons
The neuron linear function
Neuron activation functions
The loss and cost functions in deep learning
The forward propagation process
The back propagation function
Stochastic and minibatch gradient descents
Optimization algorithms for deep learning
Using momentum with gradient descent
The RMSProp algorithm
The Adam optimizer
Deep learning frameworks
What is TensorFlow?
What is Keras?
Popular alternatives to TensorFlow
GPU requirements for TensorFlow and Keras
Installing Nvidia CUDA Toolkit and cuDNN
Installing Python
Installing TensorFlow and Keras
Building datasets for deep learning
Bias and variance errors in deep learning
The train, val, and test datasets
Managing bias and variance in deep neural networks
K-Fold cross-validation
Summary
Chapter 2: Using Deep Learning to Solve Regression Problems
Regression analysis and deep neural networks
Benefits of using a neural network for regression
Drawbacks to consider when using a neural network for regression
Using deep neural networks for regression
How to plan a machine learning problem
Defining our example problem
Loading the dataset
Defining our cost function
Building an MLP in Keras
Input layer shape
Hidden layer shape
Output layer shape
Neural network architecture
Training the Keras model
Measuring the performance of our model
Building a deep neural network in Keras
Measuring the deep neural network performance
Tuning the model hyperparameters
Saving and loading a trained Keras model
Summary
Chapter 3: Monitoring Network Training Using TensorBoard
A brief overview of TensorBoard
Setting up TensorBoard
Installing TensorBoard
How TensorBoard talks to Keras/TensorFlow
Running TensorBoard
Connecting Keras to TensorBoard
Introducing Keras callbacks
Creating a TensorBoard callback
Using TensorBoard
Visualizing training
Visualizing network graphs
Visualizing a broken network
Summary
Chapter 4: Using Deep Learning to Solve Binary Classification Problems
Binary classification and deep neural networks
Benefits of deep neural networks
Drawbacks of deep neural networks
Case study – epileptic seizure recognition
Defining our dataset
Loading data
Model inputs and outputs
The cost function
Using metrics to assess the performance
Building a binary classifier in Keras
The input layer
The hidden layers
What happens if we use too many neurons?
What happens if we use too few neurons?
Choosing a hidden layer architecture
Coding the hidden layers for our example
The output layer
Putting it all together
Training our model
Using the checkpoint callback in Keras
Measuring ROC AUC in a custom callback
Measuring precision, recall, and f1-score
Summary
Chapter 5: Using Keras to Solve Multiclass Classification Problems
Multiclass classification and deep neural networks
Benefits
Drawbacks
Case study - handwritten digit classification
Problem definition
Model inputs and outputs
Flattening inputs
Categorical outputs
Cost function
Metrics
Building a multiclass classifier in Keras
Loading MNIST
Input layer
Hidden layers
Output layer
Softmax activation
Putting it all together
Training
Using scikit-learn metrics with multiclass models
Controlling variance with dropout
Controlling variance with regularization
Summary
Chapter 6: Hyperparameter Optimization
Should network architecture be considered a hyperparameter?
Finding a giant and then standing on his shoulders
Adding until you overfit, then regularizing
Practical advice
Which hyperparameters should we optimize?
Hyperparameter optimization strategies
Common strategies
Using random search with scikit-learn
Hyperband
Summary
Chapter 7: Training a CNN from Scratch
Introducing convolutions
How do convolutional layers work?
Convolutions in three dimensions
A layer of convolutions
Benefits of convolutional layers
Parameter sharing
Local connectivity
Pooling layers
Batch normalization
Training a convolutional neural network in Keras
Input
Output
Cost function and metrics
Convolutional layers
Fully connected layers
Multi-GPU models in Keras
Training
Using data augmentation
The Keras ImageDataGenerator
Training with a generator
Summary
Chapter 8: Transfer Learning with Pretrained CNNs
Overview of transfer learning
When transfer learning should be used
Limited data
Common problem domains
The impact of source/target volume and similarity
More data is always beneficial
Source/target domain similarity
Transfer learning in Keras
Target domain overview
Source domain overview
Source network architecture
Transfer network architecture
Data preparation
Data input
Training (feature extraction)
Training (fine-tuning)
Summary
Chapter 9: Training an RNN from scratch
Introducing recurrent neural networks
What makes a neuron recurrent?
Long Short Term Memory Networks
Backpropagation through time
A refresher on time series problems
Stock and flow
ARIMA and ARIMAX forecasting
Using an LSTM for time series prediction
Data preparation
Loading the dataset
Slicing train and test by date
Differencing a time series
Scaling a time series
Creating a lagged training set
Input shape
Data preparation glue
Network output
Network architecture
Stateful versus stateless LSTMs
Training
Measuring performance
Summary
Chapter 10: Training LSTMs with Word Embeddings from Scratch
An introduction to natural language processing
Semantic analysis
Document classification
Vectorizing text
NLP terminology
Bag of Word models
Stemming, lemmatization, and stopwords
Count and TF-IDF vectorization
Word embedding
A quick example
Learning word embeddings with prediction
Learning word embeddings with counting
Getting from words to documents
Keras embedding layer
1D CNNs for natural language processing
Case studies for document classifications
Sentiment analysis with Keras embedding layers and LSTMs
Preparing the data
Input and embedding layer architecture
LSTM layer
Output layer
Putting it all together
Training the network
Performance
Document classification with and without GloVe
Preparing the data
Loading pretrained word vectors
Input and embedding layer architecture
Without GloVe vectors
With GloVe vectors
Convolution layers
Output layer
Putting it all together
Training
Performance
Summary
Chapter 11: Training Seq2Seq Models
Sequence-to-sequence models
Sequence-to-sequence model applications
Sequence-to-sequence model architecture
Encoders and decoders
Characters versus words
Teacher forcing
Attention
Translation metrics
Machine translation
Understanding the data
Loading data
One hot encoding
Training network architecture
Network architecture (for inference)
Putting it all together
Training
Inference
Loading data
Creating reverse indices
Loading models
Translating a sequence
Decoding a sequence
Example translations
Summary
Chapter 12: Using Deep Reinforcement Learning
Reinforcement learning overview
Markov Decision Processes
Q Learning
Infinite state space
Deep Q networks
Online learning
Memory and experience replay
Exploitation versus exploration
DeepMind
The Keras reinforcement learning framework
Installing Keras-RL
Installing OpenAI gym
Using OpenAI gym
Building a reinforcement learning agent in Keras
CartPole
CartPole neural network architecture
Memory
Policy
Agent
Training
Results
Lunar Lander
Lunar Lander network architecture
Memory and policy
Agent
Training
Results
Summary
Chapter 13: Generative Adversarial Networks
An overview of the GAN
Deep Convolutional GAN architecture
Adversarial training architecture
Generator architecture
Discriminator architecture
Stacked training
Step 1 – train the discriminator
Step 2 – train the stack
How GANs can fail
Stability
Mode collapse
Safe choices for GAN
Generating MNIST images using a Keras GAN
Loading the dataset
Building the generator
Building the discriminator
Building the stacked model
The training loop
Model evaluation
Generating CIFAR-10 images using a Keras GAN
Loading CIFAR-10
Building the generator
Building the discriminator
The training loop
Model evaluation
Summary
Other Books You May Enjoy
Index
Deep Learning Quick
Reference
Useful hacks for training and optimizing deep neural networks
with TensorFlow and Keras
Mike Bernico
BIRMINGHAM - MUMBAI
Deep Learning Quick Reference
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, without the prior written permission of the publisher, except in the case of brief quotations
embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented.
However, the information contained in this book is sold without warranty, either express or implied. Neither the
author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to
have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products
mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy
of this information.
Commissioning Editor: Amey Varangaonkar
Acquisition Editor: Viraj Madhav
Content Development Editor: Varun Sony
Technical Editor: Dharmendra Yadav
Copy Editors: Safis Editing
Project Coordinator: Manthan Patel
Proofreader: Safis Editing
Indexer: Pratik Shirodkar
Graphics: Tania Dutta
Production Coordinator: Deepika Naik
First published: March 2018
Production reference: 1070318
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78883-799-6
www.packtpub.com
To my wife, Lana, whose love and support define the best epoch of my life
To my son, William, who is likely disappointed that this book doesn't have more dragons in
it
To my mother, Sharon, and to the memory of my father, Bob, who taught me that
determination and resilience matter more than intelligence
mapt.io
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as
well as industry leading tools to help you plan your personal development and advance
your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos
from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at www.PacktPub.com and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at service@packtpub.com for more details.
At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a
range of free newsletters, and receive exclusive discounts and offers on Packt books and
eBooks.
Foreword
I first met Mike Bernico when we were two of the founding members of a new data science
team at a Fortune 50 company. Then, it was a heady time; there wasn't such a thing as
formal data science education, so we were all self-taught. We were a collection of
adventurous people from diverse backgrounds, who identified and learned data science
techniques because we needed them to solve the problems that we were interested in. We
built a team with an optimistic hacker approach—the belief that we could find and apply
techniques "from the wild" to build interesting, useful things.
It is in this practical, scrappy spirit that Mike wrote Deep Learning Quick Reference book.
Deep learning is frequently made out to be mysterious and difficult; however, in this guide,
Mike breaks down major deep learning techniques, making them approachable and
applicable. With this book, you (yes, you!) can quickly get started with using deep learning
for your own projects in a variety of different modalities.
Mike has been practising data science since before the discipline was named, and he has
been specifically teaching the topic to university students for 3 years. Prior to this, he spent
many years as a working computer scientist with a specialization in networks and security,
and he also has a knack for engaging with people and communicating with nonspecialists.
He is currently the Lead Data Scientist for a large financial services company, where he
designs systems for data science, builds machine learning models with direct applications
and for research publications, mentors junior data scientists, and teaches stakeholders about
data science. He knows his stuff!
With Deep Learning Quick Reference book, you'll benefit from Mike's deep experience, humor,
and down-to-earth manner as you build example networks alongside him. After you
complete Mike's book, you'll have the confidence and knowledge to understand and apply
deep learning to the problems of your own devising, for both fun and function.
Bon voyage, and good hacking!
- J. Malia Andrus, Ph.D.
Data Scientist
Seattle Washington
Contributors
About the author
Mike Bernico is a Lead Data Scientist at State Farm Mutual Insurance Companies. He also
works as an adjunct for the University of Illinois at Springfield, where he teaches Essentials
of Data Science, and Advanced Neural Networks and Deep Learning. Mike earned his
MSCS from the University of Illinois at Springfield. He's an advocate for open source
software and the good it can bring to the world. As a lifelong learner with umpteen hobbies,
Mike also enjoys cycling, travel photography, and wine making.
I'd like to thank the very talented State Farm Data Scientists, current and past, for their
friendship, expertise, and encouragement.
Thanks to my technical reviewers for providing insight and assistance with this book.
Most importantly, I’d like to thank my wife, Lana, and my son, Will, for making time for
this in our lives.
About the reviewer
Vitor Bianchi Lanzetta has a master’s degree in Applied Economics from the University of
São Paulo, one of the most reputable universities in Latin America. He has done a lot of
research in economics using neural networks. He has also authored R Data Visualization
Recipes, Packt Publishing. Vitor is very passionate about data science in general, and he
walks the earth with a personal belief that he is just as cool as he is geek. He thinks that you
will learn a lot from this book, and that TensorFlow may be the greatest deep learning tool
currently available.
Packt is searching for authors like you
If you're interested in becoming an author for Packt, please visit authors.packtpub.com and
apply today. We have worked with thousands of developers and tech professionals, just
like you, to help them share their insight with the global tech community. You can make a
general application, apply for a specific hot topic that we are recruiting an author for, or
submit your own idea.
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
