Table of Contents
About the Author
About the Technical Reviewer
Introduction
Chapter 1: Embedded Class Labels
Code for Learning Embedded Labels
Cross Entropy Reconstruction Error
Fast vs. Slow Reconstruction Error Computation
Classifying Cases
Class-Conditional Generative Sampling
Chapter 2: Signal Preprocessing
Simple, Minimal Transformation
Logs and Differences
Windows and Shifting
Pseudocode for Simple Series Processing
Tail Trimming
Example of Simple Series Creation
Displaying Differenced Generative Samples
Path of a Function
Pseudocode for Series Path Computation
Example of Path Series Creation
Fourier Coefficients in a Moving Window
Pseudocode for Fourier Series Processing
Example of Fourier Series Generation
Length=10 with No Centering
Length=10 with Centering
Length=11 with No Centering
Length=11 with Centering
Morlet Wavelets
Period, Width, and Lag
Code for Morlet Wavelets
Example of Morlet Wavelet Series Generation
Path in an XY Plane
Normalization for Invariance
Pseudocode for XY Plane Series Processing
Example of XY Plane Series Processing
Chapter 3: Image Preprocessing
The Fourier Transform in Two Dimensions
Data Windows in Two Dimensions
Code for the Fourier Transform of an Image
Displaying Generative Samples of Fourier Transforms
Chapter 4: Autoencoding
Basic Mathematics of Feedforward Networks
Greedy Training with Autoencoders
Review of Complex Numbers
Fast Dot Product Computation in the Complex Domain
Singular Value Decomposition in the Complex Domain
Activation in the Complex Domain
Derivatives of the Activation Function
The Logistic Activation Function and Its Derivative
Computing the Gradient
Pure Real and SoftMax Output Errors
Gradient of the Hidden-Layer Weights
Code for Gradient Computation
Evaluating the Entire Network and Derivatives
Computing the Gradient
Multithreading Gradient Computation
CUDA Gradient Computation
The Overall Algorithm
Device Initialization
Copying Weights from Host to Device
Activation and Its Derivatives
Output Activation
SoftMax Modification of Outputs
Output Delta
Delta for SoftMax Outputs
Output Gradient
Gradient of the First Hidden Layer
Gradient of a Subsequent Hidden Layer
Mean Squared Error
The Log Likelihood Criterion for Classification
An Analysis
Chapter 5: DEEP Operating Manual
Menu Options
File Menu Options
Test Menu Options
Display Menu Options
Read a Database
Read a Series (Simple)
Read a Series (Trend Path)
Read a Series (Fourier)
Read a Series (Morlet)
Read XY Points
Read MNIST Image
Read MNIST Image (Fourier)
Read MNIST Labels
Write Activation File
Clear All Data
Model Architecture
Database Inputs and Targets
RBM Training Params
Supervised Training Params
Autoencoding Training Params
Train
Test
Cross Validate
Analyze
Receptive Field
Generative Sample
Samples from an Embedded Model
Samples from a Path Series
The DEEP.LOG File
Predictive Performance Measures
Index