第1页 / 共255页
第2页 / 共255页
第3页 / 共255页
第4页 / 共255页
第5页 / 共255页
第6页 / 共255页
第7页 / 共255页
第8页 / 共255页
DeepLearningWithPyTorch.pdf
Cover
Copyright and Credits
Dedication
Packt Upsell
Foreword
Contributors
Table of Contents
Preface
Chapter 1: Getting Started with Deep Learning Using PyTorch
Artificial intelligence
The history of AI
Machine learning
Examples of machine learning in real life
Deep learning
Applications of deep learning
Hype associated with deep learning
The history of deep learning
Why now?
Hardware availability
Data and algorithms
Deep learning frameworks
PyTorch
Summary
Chapter 2: Building Blocks of Neural Networks
Installing PyTorch
Our first neural network
Data preparation
Scalar (0-D tensors)
Vectors (1-D tensors)
Matrix (2-D tensors)
3-D tensors
Slicing tensors
4-D tensors
5-D tensors
Tensors on GPU
Variables
Creating data for our neural network
Creating learnable parameters
Neural network model
Network implementation
Loss function
Optimize the neural network
Loading data
Dataset class
DataLoader class
Summary
Chapter 3: Diving Deep into Neural Networks
Deep dive into the building blocks of neural networks
Layers – fundamental blocks of neural networks
Non-linear activations
Sigmoid
Tanh
ReLU
Leaky ReLU
PyTorch non-linear activations
The PyTorch way of building deep learning algorithms
Model architecture for different machine learning problems
Loss functions
Optimizing network architecture
Image classification using deep learning
Loading data into PyTorch tensors
Loading PyTorch tensors as batches
Building the network architecture
Training the model
Summary
Chapter 4: Fundamentals of Machine Learning
Three kinds of machine learning problems
Supervised learning
Unsupervised learning
Reinforcement learning
Machine learning glossary
Evaluating machine learning models
Training, validation, and test split
Simple holdout validation
K-fold validation
K-fold validation with shuffling
Data representativeness
Time sensitivity
Data redundancy
Data preprocessing and feature engineering
Vectorization
Value normalization
Handling missing values
Feature engineering
Overfitting and underfitting
Getting more data
Reducing the size of the network
Applying weight regularization
Dropout
Underfitting
Workflow of a machine learning project
Problem definition and dataset creation
Measure of success
Evaluation protocol
Prepare your data
Baseline model
Large model enough to overfit
Applying regularization
Learning rate picking strategies
Summary
Chapter 5: Deep Learning for Computer Vision
Introduction to neural networks
MNIST – getting data
Building a CNN model from scratch
Conv2d
Pooling
Nonlinear activation – ReLU
View
Linear layer
Training the model
Classifying dogs and cats – CNN from scratch
Classifying dogs and cats using transfer learning
Creating and exploring a VGG16 model
Freezing the layers
Fine-tuning VGG16
Training the VGG16 model
Calculating pre-convoluted features
Understanding what a CNN model learns
Visualizing outputs from intermediate layers
Visualizing weights of the CNN layer
Summary
Chapter 6: Deep Learning with Sequence Data and Text
Working with text data
Tokenization
Converting text into characters
Converting text into words
N-gram representation
Vectorization
One-hot encoding
Word embedding
Training word embedding by building a sentiment classifier
Downloading IMDB data and performing text tokenization
torchtext.data
torchtext.datasets
Building vocabulary
Generate batches of vectors
Creating a network model with embedding
Training the model
Using pretrained word embeddings
Downloading the embeddings
Loading the embeddings in the model
Freeze the embedding layer weights
Recursive neural networks
Understanding how RNN works with an example
LSTM
Long-term dependency
LSTM networks
Preparing the data
Creating batches
Creating the network
Training the model
Convolutional network on sequence data
Understanding one-dimensional convolution for sequence data
Creating the network
Training the model
Summary
Chapter 7: Generative Networks
Neural style transfer
Loading the data
Creating the VGG model
Content loss
Style loss
Extracting the losses
Creating loss function for each layers
Creating the optimizer
Training
Generative adversarial networks
Deep convolutional GAN
Defining the generator network
Transposed convolutions
Batch normalization
Generator
Defining the discriminator network
Defining loss and optimizer
Training the discriminator
Training the discriminator with real images
Training the discriminator with fake images
Training the generator network
Training the complete network
Inspecting the generated images
Language modeling
Preparing the data
Generating the batches
Batches
Backpropagation through time
Defining a model based on LSTM
Defining the train and evaluate functions
Training the model
Summary
Chapter 8: Modern Network Architectures
Modern network architectures
ResNet
Creating PyTorch datasets
Creating loaders for training and validation
Creating a ResNet model
Extracting convolutional features
Creating a custom PyTorch dataset class for the pre-convoluted features and loader
Creating a simple linear model
Training and validating the model
Inception
Creating an Inception model
Extracting convolutional features using register_forward_hook
Creating a new dataset for the convoluted features
Creating a fully connected model
Training and validating the model
Densely connected convolutional networks – DenseNet
DenseBlock
DenseLayer
Creating a DenseNet model
Extracting DenseNet features
Creating a dataset and loaders
Creating a fully connected model and train
Model ensembling
Creating models
Extracting the image features
Creating a custom dataset along with data loaders
Creating an ensembling model
Training and validating the model
Encoder-decoder architecture
Encoder
Decoder
Summary
Chapter 9: What Next?
What next?
Overview
Interesting ideas to explore
Object detection
Image segmentation
OpenNMT in PyTorch
Alien NLP
fast.ai – making neural nets uncool again
Open Neural Network Exchange
How to keep yourself updated
Summary
Other Books You May Enjoy
Index
Vishnu Subramanian
Foreword by:
Manas Agarwal
CEO, Co-Founder of Affi ne Analytics, Bengaluru, India
Deep Learning
with PyTorch
A practical approach to building neural network models
using PyTorch
Deep Learning with PyTorch
Vishnu Subramanian
BIRMINGHAM - MUMBAI
Deep Learning with PyTorch
Copyright 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form
or by any means, without the prior written permission of the publisher, except in the case of brief quotations
embedded in critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy of the information presented.
However, the information contained in this book is sold without warranty, either express or implied. Neither the
author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to
have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the companies and products
mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy
of this information.
Commissioning Editor: Veena Pagare
Acquisition Editor: Aman Singh
Content Development Editor: Snehal Kolte
Technical Editor: Sayli Nikalje
Copy Editor: Safis Editing
Project Coordinator: Manthan Patel
Proofreader: Safis Editing
Indexer: Pratik Shirodkar
Graphics: Tania Dutta
Production Coordinator: Deepika Naik
First published: February 2018
Production reference: 1210218
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham
B3 2PB, UK.
ISBN 978-1-78862-433-6
To Jeremy Howard and Rachel Thomas for inspiring me to write this book,
and to my family for their love.
Vishnu Subramanian
Mapt is an online digital library that gives you full access to over 5,000 books and videos, as
well as industry leading tools to help you plan your personal development and advance
your career. For more information, please visit our website.
Why subscribe?
Spend less time learning and more time coding with practical eBooks and Videos
from over 4,000 industry professionals
Improve your learning with Skill Plans built especially for you
Get a free eBook or video every month
Mapt is fully searchable
Copy and paste, print, and bookmark content
PacktPub.com
Did you know that Packt offers eBook versions of every book published, with PDF and
ePub files available? You can upgrade to the eBook version at and as a
print book customer, you are entitled to a discount on the eBook copy. Get in touch with us
at for more details.
At , you can also read a collection of free technical articles, sign up for a
range of free newsletters, and receive exclusive discounts and offers on Packt books and
eBooks.
Foreword
I have been working with Vishnu Subramanian for the last few years. Vishnu comes across
as a passionate techno-analytical expert who has the rigor one requires to achieve
excellence. His points of view on big data/machine learning/AI are well informed and carry
his own analysis and appreciation of the landscape of problems and solutions. Having
known him closely, I'm glad to be writing this foreword in my capacity as the CEO of
Affine.
Increased success through deep learning solutions for our Fortune 500 clients clearly
necessitates quick prototyping. PyTorch (a year-old deep learning framework) allows rapid
prototyping for analytical projects without worrying too much about the complexity of the
framework. This leads to an augmentation of the best of human capabilities with
frameworks that can help deliver solutions faster. As an entrepreneur delivering advanced
analytical solutions, building this capability in my teams happens to be the primary
objective for me. In this book, Vishnu takes you through the fundamentals of building deep
learning solutions using PyTorch while helping you build a mindset geared towards
modern deep learning techniques.
The first half of the book introduces several fundamental building blocks of deep learning
and PyTorch. It also covers key concepts such as overfitting, underfitting, and techniques
that helps us deal with them.
In the second half of the book, Vishnu covers advanced concepts such as CNN, RNN, and
LSTM transfer learning using pre-convoluted features, and one-dimensional convolutions,
along with real-world examples of how these techniques can be applied. The last two
chapters introduce you to modern deep learning architectures such as Inception, ResNet,
DenseNet model and ensembling, and generative networks such as style transfer, GAN,
and language modeling.
With all the practical examples covered and with solid explanations, this is one of the best
books for readers who want to become proficient in deep learning. The rate at which
technology evolves is unparalleled today. To a reader looking forward towards developing
mature deep learning solutions, I would like to point that the right framework also drives
the right mindset.
To all those reading through this book, happy exploring new horizons!
Wishing Vishnu and this book a roaring success, which they both deserve.
Manas Agarwal
CEO, Co-Founder of Affine Analytics,
Bengaluru, India
Contributors
About the author
Vishnu Subramanian has experience in leading, architecting, and implementing several big
data analytical projects (artificial intelligence, machine learning, and deep learning). He
specializes in machine learning, deep learning, distributed machine learning, and
visualization. He has experience in retail, finance, and travel. He is good at understanding
and coordinating between businesses, AI, and engineering teams.
This book would not have been possible without the inspiration and MOOC by Jeremy
Howard and Rachel Thomas of fast.ai. Thanks to them for the important role they are
playing in democratizing AI/deep learning.
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
