Advanced Deep Learning
with Keras
Apply deep learning techniques, autoencoders, GANs,
variational autoencoders, deep reinforcement learning,
policy gradients, and more
Rowel Atienza
BIRMINGHAM - MUMBAI
Advanced Deep Learning with Keras
Copyright © 2018 Packt Publishing
All rights reserved. No part of this book may be reproduced, stored in a retrieval
system, or transmitted in any form or by any means, without the prior written
permission of the publisher, except in the case of brief quotations embedded in
critical articles or reviews.
Every effort has been made in the preparation of this book to ensure the accuracy
of the information presented. However, the information contained in this book
is sold without warranty, either express or implied. Neither the author, nor Packt
Publishing or its dealers and distributors, will be held liable for any damages
caused or alleged to have been caused directly or indirectly by this book.
Packt Publishing has endeavored to provide trademark information about all of the
companies and products mentioned in this book by the appropriate use of capitals.
However, Packt Publishing cannot guarantee the accuracy of this information.
Acquisition Editor: Frank Pohlmann, Andrew Waldron, Suresh Jain
Content Development Editor: Alex Sorrentino
Technical Editor: Gaurav Gavas
Project Editor: Kishor Rit
Proofreader: Safis Editing
Indexers: Aishwarya Gangawane
Graphics: Tom Scaria
Production Coordinator: Sandip Tadge
First published: October 2018
Production reference: 1311018
Published by Packt Publishing Ltd.
Livery Place
35 Livery Street
Birmingham B3 2PB, UK.
ISBN 978-1-78862-941-6
www.packtpub.com
mapt.io
Mapt is an online digital library that gives you full access to over 5,000 books
and videos, as well as industry leading tools to help you plan your personal
development and advance your career. For more information, please visit
our website.
Why subscribe?
• Spend less time learning and more time coding with practical eBooks
and Videos from over 4,000 industry professionals
• Learn better with Skill Plans built especially for you
• Get a free eBook or video every month
• Mapt is fully searchable
• Copy and paste, print, and bookmark content
Packt.com
Did you know that Packt offers eBook versions of every book published, with
PDF and ePub files available? You can upgrade to the eBook version at www.Packt.
com and as a print book customer, you are entitled to a discount on the eBook copy.
Get in touch with us at customercare@packtpub.com for more details.
At www.Packt.com, you can also read a collection of free technical articles, sign up
for a range of free newsletters, and receive exclusive discounts and offers on Packt
books and eBooks.
Contributors
About the author
Rowel Atienza is an Associate Professor at the Electrical and Electronics
Engineering Institute of the University of the Philippines, Diliman. He
holds the Dado and Maria Banatao Institute Professorial Chair in Artificial
Intelligence. Rowel has been fascinated with intelligent robots since he graduated
from the University of the Philippines. He received his MEng from the National
University of Singapore for his work on an AI-enhanced four-legged robot. He
finished his Ph.D. at The Australian National University for his contribution on the
field of active gaze tracking for human-robot interaction. Rowel's current research
work focuses on AI and computer vision. He dreams on building useful machines
that can perceive, understand, and reason. To help make his dreams become
real, Rowel has been supported by grants from the Department of Science and
Technology (DOST), Samsung Research Philippines, and Commission on Higher
Education-Philippine California Advanced Research Institutes (CHED-PCARI).
I would like to thank my family, Che, Diwa, and Jacob. They never
cease to support my work.
I would like to thank my mother who instilled into me the value
of education.
I would like to express my gratitude to the people of Packt and this
book's technical reviewer, Frank, Kishor, Alex, and Valerio. They
are inspiring and easy to work with.
I would like to thank the institutions who always support my
teaching and research agenda, University of the Philippines,
DOST, Samsung Research PH, and CHED-PCARI.
I would like to acknowledge my students. They have been patient
as I develop my courses in AI.
About the reviewer
Valerio Maggio is currently a Post-Doc Data Scientist at Fondazione Bruno
Kessler (FBK) in Trento, Italy, responsible for Machine Learning and Deep Learning
in the MPBA lab (Predictive Models for Biomedicine and Environment). Valerio has
a Ph.D. in Computational Science from the University of Naples "Federico II." His
research interests are focused on Machine Learning and Deep Learning applied to
Software Maintenance and Computational Biology. Valerio is very much involved
in the scientific Python community, and he is an active speaker at many Python
conference.
He is also the lead organiser of PyCon Italy/PyData Florence, and EuroSciPy.
He uses Python as the mainstream language for his deep/machine learning code,
making an intensive use of Python to analyse, visualise, and learn from data. In the
context of Deep Learning, Valerio is the author of a quite popular Keras/TensorFlow
tutorial, publicly available on his GitHub Profile – github.com/leriomaggio/deep-
learning-keras-tensorflow – and presented in many conferences (EuroSciPy,
PyData London, PySS) and University courses. Valerio is also passionate about
(black) tea, and an "old-school" Magic The Gathering (MTG) player, who enjoys
playing and teaching MTG to newbies.
Packt is Searching for Authors Like You
If you're interested in becoming an author for Packt, please visit authors.packtpub.
com and apply today. We have worked with thousands of developers and tech
professionals, just like you, to help them share their insight with the global tech
community. You can make a general application, apply for a specific hot topic
that we are recruiting an author for, or submit your own idea.
Table of Contents
Preface
Chapter 1: Introducing Advanced Deep Learning with Keras
Why is Keras the perfect deep learning library?
Installing Keras and TensorFlow
Implementing the core deep learning models - MLPs,
CNNs and RNNs
The difference between MLPs, CNNs, and RNNs
Multilayer perceptrons (MLPs)
MNIST dataset
MNIST digits classifier model
Building a model using MLPs and Keras
Regularization
Output activation and loss function
Optimization
Performance evaluation
Model summary
Convolutional neural networks (CNNs)
Convolution
Pooling operations
Performance evaluation and model summary
Recurrent neural networks (RNNs)
Conclusion
Chapter 2: Deep Neural Networks
Functional API
Creating a two-input and one-output model
V
1
2
3
4
5
6
6
8
12
14
15
17
20
21
23
26
27
28
31
37
39
40
43
[ i ]