Introduction to Statistical Machine Learning
Cheng Soon Ong & Christian Walder
Machine Learning Research Group
College of Engineering and Computer Science
The Australian National University
Data61 | CSIRO
and
Canberra
February – June 2018
(Many figures from C. M. Bishop, "Pattern Recognition and Machine Learning")
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
1of 826
Overview
1 Administration
2 Examples
3 What is common to these examples?
4 Definition
5 Related Fields
6 Notation for Learning
7 Python
8 Human Learning
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
2of 826
Introduction
9 Polynomial Curve Fitting
10 Probability Theory
11 Probability Densities
12 Expectations and Covariances
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
3of 826
Linear Algebra
13 Motivation
14 Basic Concepts
15 Linear Transformations
16 Trace
17 Inner Product
18 Projection
19 Rank, Determinant, Trace
20 Matrix Inverse
21 Eigenvectors
22 Singular Value Decomposition
23 Gradient
24 Books
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
4of 826
Probability
25 Motivation
26 Boxes with Apples and Oranges
27 Bayes’ Theorem
28 Bayes’ Probabilities
29 Probability Distributions
30 Gaussian Distribution over a Vector
31 Decision Theory
32 Model Selection - Key Ideas
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
5of 826
Linear Regression 1
33 Review
34 Linear Basis Function Models
35 Maximum Likelihood and Least Squares
36 Geometry of Least Squares
37 Sequential Learning
38 Regularized Least Squares
39 Multiple Outputs
40 Loss Function for Regression
41 The Bias-Variance Decomposition
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
6of 826
Linear Regression 2
42 Review
43 Bayesian Regression
44 Sequential Update of the Posterior
45 Predictive Distribution
46 Proof of the Predictive Distribution
47 Predictive Distribution with Simplified Prior
48 Limitations of Linear Basis Function Models
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
7of 826
Linear Classification 1
49 Classification
50 Generalised Linear Model
51 Inference and Decision
52 Discriminant Functions
Introduction to Statistical
Machine Learning
c2018
Ong & Walder & Webers
Data61 | CSIRO
The Australian National
University
Outlines
Overview
Introduction
Linear Algebra
Probability
Linear Regression 1
Linear Regression 2
Linear Classification 1
Linear Classification 2
Kernel Methods
Sparse Kernel Methods
Mixture Models and EM 1
Mixture Models and EM 2
Neural Networks 1
Neural Networks 2
Principal Component Analysis
Autoencoders
Graphical Models 1
Graphical Models 2
Graphical Models 3
Sampling
Sequential Data 1
Sequential Data 2
8of 826