第1页 / 共171页
第2页 / 共171页
第3页 / 共171页
第4页 / 共171页
第5页 / 共171页
第6页 / 共171页
第7页 / 共171页
第8页 / 共171页
深度学习Theano教程，python语言.pdf
LICENSE
Deep Learning Tutorials
Getting Started
Download
Datasets
Notation
A Primer on Supervised Optimization for Deep Learning
Theano/Python Tips
Classifying MNIST digits using Logistic Regression
The Model
Defining a Loss Function
Creating a LogisticRegression class
Learning the Model
Testing the model
Putting it All Together
Multilayer Perceptron
The Model
Going from logistic regression to MLP
Putting it All Together
Tips and Tricks for training MLPs
Convolutional Neural Networks (LeNet)
Motivation
Sparse Connectivity
Shared Weights
Details and Notation
The Convolution Operator
MaxPooling
The Full Model: LeNet
Putting it All Together
Running the Code
Tips and Tricks
Denoising Autoencoders (dA)
Autoencoders
Denoising Autoencoders
Putting it All Together
Running the Code
Stacked Denoising Autoencoders (SdA)
Stacked Autoencoders
Putting it all together
Running the Code
Tips and Tricks
Restricted Boltzmann Machines (RBM)
Energy-Based Models (EBM)
Restricted Boltzmann Machines (RBM)
Sampling in an RBM
Implementation
Results
Deep Belief Networks
Deep Belief Networks
Justifying Greedy-Layer Wise Pre-Training
Implementation
Putting it all together
Running the Code
Tips and Tricks
Hybrid Monte-Carlo Sampling
Theory
Implementing HMC Using Theano
Testing our Sampler
References
Recurrent Neural Networks with Word Embeddings
Summary
Code - Citations - Contact
Task
Dataset
Recurrent Neural Network Model
Evaluation
Training
Running the Code
LSTM Networks for Sentiment Analysis
Summary
Data
Model
Code - Citations - Contact
References
Modeling and generating sequences of polyphonic music with the RNN-RBM
The RNN-RBM
Implementation
Results
How to improve this code
Miscellaneous
Plotting Samples and Filters
References
Bibliography
Index
Deep Learning Tutorial
Release 0.1
LISA lab, University of Montreal
February 13, 2015
1 LICENSE
2 Deep Learning Tutorials
CONTENTS
1
3
3 Getting Started
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1 Download .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2 Datasets .
.
3.3 Notation .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4 A Primer on Supervised Optimization for Deep Learning . . . . . . . . . . . . . . . . . . .
3.5
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Theano/Python Tips
5
5
5
7
8
. 14
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
The Model
4 Classifying MNIST digits using Logistic Regression
Creating a LogisticRegression class . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
Learning the Model
Testing the model
.
Putting it All Together .
17
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
. 19
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
.
. 23
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
4.1
.
4.2 Defining a Loss Function .
4.3
4.4
4.5
4.6
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
.
.
.
.
.
.
.
.
.
.
The Model
5 Multilayer Perceptron
.
35
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
. 36
Putting it All Together .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
Tips and Tricks for training MLPs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
5.1
5.2 Going from logistic regression to MLP . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.3
5.4
.
.
.
.
.
.
.
.
.
.
6 Convolutional Neural Networks (LeNet)
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
6.1 Motivation .
Sparse Connectivity .
6.2
6.3
Shared Weights .
.
6.4 Details and Notation .
6.5
6.6 MaxPooling .
6.7
6.8
6.9
6.10 Tips and Tricks .
.
.
.
.
The Convolution Operator .
.
.
.
.
.
.
The Full Model: LeNet .
Putting it All Together .
.
Running the Code .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
51
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
.
. 52
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
.
. 53
.
. 54
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
.
. 57
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
.
i
7 Denoising Autoencoders (dA)
.
7.1 Autoencoders .
.
7.2 Denoising Autoencoders .
.
7.3
7.4
.
Putting it All Together .
Running the Code .
.
.
.
.
.
.
65
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76
.
.
.
.
.
.
.
.
8 Stacked Denoising Autoencoders (SdA)
8.1
8.2
8.3
8.4
Stacked Autoencoders .
Putting it all together .
.
.
.
Running the Code .
Tips and Tricks .
.
.
.
.
.
.
.
.
.
.
.
.
79
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
.
.
9 Restricted Boltzmann Machines (RBM)
89
Energy-Based Models (EBM)
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
Restricted Boltzmann Machines (RBM) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
Sampling in an RBM .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
.
.
Implementation .
Results .
.
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
9.1
9.2
9.3
9.4
9.5
.
10 Deep Belief Networks
107
10.1 Deep Belief Networks .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
10.2 Justifying Greedy-Layer Wise Pre-Training . . . . . . . . . . . . . . . . . . . . . . . . . . 108
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
10.3 Implementation .
.
.
10.4 Putting it all together .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
.
.
.
10.5 Running the Code .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
10.6 Tips and Tricks .
.
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
11 Hybrid Monte-Carlo Sampling
.
117
11.1 Theory .
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
11.2 Implementing HMC Using Theano . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
11.3 Testing our Sampler
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
.
11.4 References .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
12 Recurrent Neural Networks with Word Embeddings
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
12.1 Summary .
12.2 Code - Citations - Contact
.
.
12.3 Task .
.
12.4 Dataset
.
.
12.5 Recurrent Neural Network Model
.
12.6 Evaluation .
12.7 Training .
.
.
12.8 Running the Code .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 137
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
.
.
.
.
.
.
13 LSTM Networks for Sentiment Analysis
.
.
13.1 Summary .
.
.
.
13.2 Data .
.
13.3 Model .
.
.
.
13.4 Code - Citations - Contact
13.5 References .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
ii
141
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
.
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
.
14 Modeling and generating sequences of polyphonic music with the RNN-RBM
147
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
.
.
.
.
.
.
.
.
.
14.1 The RNN-RBM .
.
.
14.2 Implementation .
14.3 Results .
.
.
.
14.4 How to improve this code .
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
15 Miscellaneous
157
15.1 Plotting Samples and Filters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
16 References
Bibliography
Index
161
163
165
iii
iv
CHAPTER
ONE
LICENSE
Copyright (c) 2008–2013, Theano Development Team All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that
the following conditions are met:
• Redistributions of source code must retain the above copyright notice, this list of conditions and the
following disclaimer.
• Redistributions in binary form must reproduce the above copyright notice, this list of conditions and
the following disclaimer in the documentation and/or other materials provided with the distribution.
• Neither the name of Theano nor the names of its contributors may be used to endorse or promote
products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ‘’AS IS” AND ANY EXPRESS
OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
NO EVENT SHALL THE COPYRIGHT HOLDERS BE LIABLE FOR ANY DIRECT, INDIRECT, IN-
CIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIA-
BILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR
OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED
OF THE POSSIBILITY OF SUCH DAMAGE.
1
Deep Learning Tutorial, Release 0.1
2
Chapter 1. LICENSE
相关推荐
2023年江西萍乡中考道德与法治真题及答案.doc
2012年重庆南川中考生物真题及答案.doc
2013年江西师范大学地理学综合及文艺理论基础考研真题.doc
2020年四川甘孜小升初语文真题及答案I卷.doc
2020年注册岩土工程师专业基础考试真题及答案.doc
2023-2024学年福建省厦门市九年级上学期数学月考试题及答案.doc
2021-2022学年辽宁省沈阳市大东区九年级上学期语文期末试题及答案.doc
2022-2023学年北京东城区初三第一学期物理期末试卷及答案.doc
2018上半年江西教师资格初中地理学科知识与教学能力真题及答案.doc
2012年河北国家公务员申论考试真题及答案-省级.doc
2020-2021学年江苏省扬州市江都区邵樊片九年级上学期数学第一次质量检测试题及答案.doc
2022下半年黑龙江教师资格证中学综合素质真题及答案.doc
