Cover
Copyright
Contents
Preface
1 Introduction
What is Computer Vision?
Applications
Image Processing vs. Computer Vision
What is Machine Learning?
Why Deep Learning?
Book Overview
2 Features and Classifiers
Importance of Features and Classifiers
Features
Classifiers
Traditional Feature Descriptors
Histogram of Oriented Gradients (HOG)
Scale-invariant Feature Transform (SIFT)
Speeded-up Robust Features (SURF)
Limitations of Traditional Hand-engineered Features
Machine Learning Classifiers
Support Vector Machine (SVM)
Random Decision Forest
Conclusion
3 Neural Networks Basics
Introduction
Multi-layer Perceptron
Architecture Basics
Parameter Learning
Recurrent Neural Networks
Architecture Basics
Parameter Learning
Link with Biological Vision
Biological Neuron
Computational Model of a Neuron
Artificial vs. Biological Neuron
4 Convolutional Neural Network
Introduction
Network Layers
Pre-processing
Convolutional Layers
Pooling Layers
Nonlinearity
Fully Connected Layers
Transposed Convolution Layer
Region of Interest Pooling
Spatial Pyramid Pooling Layer
Vector of Locally Aggregated Descriptors Layer
Spatial Transformer Layer
CNN Loss Functions
Cross-entropy Loss
SVM Hinge Loss
Squared Hinge Loss
Euclidean Loss
The 1 Error
Contrastive Loss
Expectation Loss
Structural Similarity Measure
5 CNN Learning
Weight Initialization
Gaussian Random Initialization
Uniform Random Initialization
Orthogonal Random Initialization
Unsupervised Pre-training
Xavier Initialization
ReLU Aware Scaled Initialization
Layer-sequential Unit Variance
Supervised Pre-training
Regularization of CNN
Data Augmentation
Dropout
Drop-connect
Batch Normalization
Ensemble Model Averaging
The 2 Regularization
The 1 Regularization
Elastic Net Regularization
Max-norm Constraints
Early Stopping
Gradient-based CNN Learning
Batch Gradient Descent
Stochastic Gradient Descent
Mini-batch Gradient Descent
Neural Network Optimizers
Momentum
Nesterov Momentum
Adaptive Gradient
Adaptive Delta
RMSprop
Adaptive Moment Estimation
Gradient Computation in CNNs
Analytical Differentiation
Numerical Differentiation
Symbolic Differentiation
Automatic Differentiation
Understanding CNN through Visualization
Visualizing Learned Weights
Visualizing Activations
Visualizations based on Gradients
6 Examples of CNN Architectures
LeNet
AlexNet
Network in Network
VGGnet
GoogleNet
ResNet
ResNeXt
FractalNet
DenseNet
7 Applications of CNNs in Computer Vision
Image Classification
PointNet
Object Detection and Localization
Region-based CNN
Fast R-CNN
Regional Proposal Network (RPN)
Semantic Segmentation
Fully Convolutional Network (FCN)
Deep Deconvolution Network (DDN)
DeepLab
Scene Understanding
DeepContext
Learning Rich Features from RGB-D Images
PointNet for Scene Understanding
Image Generation
Generative Adversarial Networks (GANs)
Deep Convolutional Generative Adversarial Networks (DCGANs)
Super Resolution Generative Adversarial Network (SRGAN)
Video-based Action Recognition
Action Recognition From Still Video Frames
Two-stream CNNs
Long-term Recurrent Convolutional Network (LRCN)
8 Deep Learning Tools and Libraries
Caffe
TensorFlow
MatConvNet
Torch7
Theano
Keras
Lasagne
Marvin
Chainer
PyTorch
Conclusion
Bibliography
Authors' Biographies
Blank Page