Neural Network Toolbox™
Getting Started Guide
Mark Hudson Beale
Martin T. Hagan
Howard B. Demuth
R2016a
How to Contact MathWorks
Latest news:
Sales and services:
User community:
Technical support:
Phone:
www.mathworks.com
www.mathworks.com/sales_and_services
www.mathworks.com/matlabcentral
www.mathworks.com/support/contact_us
508-647-7000
The MathWorks, Inc.
3 Apple Hill Drive
Natick, MA 01760-2098
Neural Network Toolbox™ Getting Started Guide
© COPYRIGHT 1992–2016 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used
or copied only under the terms of the license agreement. No part of this manual may be photocopied or
reproduced in any form without prior written consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation
by, for, or through the federal government of the United States. By accepting delivery of the Program
or Documentation, the government hereby agrees that this software or documentation qualifies as
commercial computer software or commercial computer software documentation as such terms are used
or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and
conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and
govern the use, modification, reproduction, release, performance, display, and disclosure of the Program
and Documentation by the federal government (or other entity acquiring for or through the federal
government) and shall supersede any conflicting contractual terms or conditions. If this License fails
to meet the government's needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.
Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See
www.mathworks.com/trademarks for a list of additional trademarks. Other product or brand
names may be trademarks or registered trademarks of their respective holders.
Patents
MathWorks products are protected by one or more U.S. patents. Please see
www.mathworks.com/patents for more information.
First printing
Second printing
Third printing
Fourth printing
Fifth printing
Sixth printing
Seventh printing
Online only
Online only
Online only
Online only
Eighth printing
Online only
Online only
Revision History
June 1992
April 1993
January 1997
July 1997
January 1998
September 2000
June 2001
July 2002
January 2003
June 2004
October 2004
October 2004
March 2005
March 2006
September 2006 Ninth printing
Online only
March 2007
September 2007 Online only
Online only
March 2008
Online only
October 2008
March 2009
Online only
September 2009 Online only
March 2010
Online only
September 2010 Tenth printing
April 2011
Online only
September 2011 Online only
March 2012
Online only
September 2012 Online only
March 2013
Online only
September 2013 Online only
Online only
March 2014
October 2014
Online only
March 2015
Online only
September 2015 Online only
March 2016
Online only
Revised for Version 3 (Release 11)
Revised for Version 4 (Release 12)
Minor revisions (Release 12.1)
Minor revisions (Release 13)
Minor revisions (Release 13SP1)
Revised for Version 4.0.3 (Release 14)
Revised for Version 4.0.4 (Release 14SP1)
Revised for Version 4.0.4
Revised for Version 4.0.5 (Release 14SP2)
Revised for Version 5.0 (Release 2006a)
Minor revisions (Release 2006b)
Minor revisions (Release 2007a)
Revised for Version 5.1 (Release 2007b)
Revised for Version 6.0 (Release 2008a)
Revised for Version 6.0.1 (Release 2008b)
Revised for Version 6.0.2 (Release 2009a)
Revised for Version 6.0.3 (Release 2009b)
Revised for Version 6.0.4 (Release 2010a)
Revised for Version 7.0 (Release 2010b)
Revised for Version 7.0.1 (Release 2011a)
Revised for Version 7.0.2 (Release 2011b)
Revised for Version 7.0.3 (Release 2012a)
Revised for Version 8.0 (Release 2012b)
Revised for Version 8.0.1 (Release 2013a)
Revised for Version 8.1 (Release 2013b)
Revised for Version 8.2 (Release 2014a)
Revised for Version 8.2.1 (Release 2014b)
Revised for Version 8.3 (Release 2015a)
Revised for Version 8.4 (Release 2015b)
Revised for Version 9.0 (Release 2016a)
1
Contents
Acknowledgments
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
viii
Getting Started
Neural Network Toolbox Product Description . . . . . . . . . . .
Key Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Neural Networks Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using Neural Network Toolbox . . . . . . . . . . . . . . . . . . . . . . . .
Automatic Script Generation . . . . . . . . . . . . . . . . . . . . . . . . .
Neural Network Toolbox Applications . . . . . . . . . . . . . . . . . .
Neural Network Design Steps . . . . . . . . . . . . . . . . . . . . . . . . .
Fit Data with a Neural Network . . . . . . . . . . . . . . . . . . . . . .
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using the Neural Network Fitting Tool . . . . . . . . . . . . . . . .
Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . .
Classify Patterns with a Neural Network . . . . . . . . . . . . . . .
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using the Neural Network Pattern Recognition Tool . . . . . .
Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . .
Cluster Data with a Self-Organizing Map . . . . . . . . . . . . . . .
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1-2
1-2
1-3
1-5
1-5
1-7
1-9
1-10
1-10
1-11
1-22
1-31
1-31
1-33
1-45
1-53
1-53
v
Using the Neural Network Clustering Tool . . . . . . . . . . . . .
Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . .
Neural Network Time Series Prediction and Modeling . . .
Defining a Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Using the Neural Network Time Series Tool . . . . . . . . . . . .
Using Command-Line Functions . . . . . . . . . . . . . . . . . . . . .
Parallel Computing on CPUs and GPUs . . . . . . . . . . . . . . . .
Parallel Computing Toolbox . . . . . . . . . . . . . . . . . . . . . . . .
Parallel CPU Workers . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
GPU Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Multiple GPU/CPU Computing . . . . . . . . . . . . . . . . . . . . . .
Cluster Computing with MATLAB Distributed Computing
Server . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Load Balancing, Large Problems, and Beyond . . . . . . . . . . .
Neural Network Toolbox Sample Data Sets . . . . . . . . . . . . .
1-54
1-63
1-69
1-69
1-70
1-82
1-93
1-93
1-93
1-94
1-94
1-95
1-95
1-96
Glossary
vi
Contents
Acknowledgments
vii
Acknowledgments
Acknowledgments
The authors would like to thank the following people:
Joe Hicklin of MathWorks for getting Howard into neural network research years ago
at the University of Idaho, for encouraging Howard and Mark to write the toolbox, for
providing crucial help in getting the first toolbox Version 1.0 out the door, for continuing
to help with the toolbox in many ways, and for being such a good friend.
Roy Lurie of MathWorks for his continued enthusiasm for the possibilities for Neural
Network Toolbox™ software.
Mary Ann Freeman of MathWorks for general support and for her leadership of a great
team of people we enjoy working with.
Rakesh Kumar of MathWorks for cheerfully providing technical and practical help,
encouragement, ideas and always going the extra mile for us.
Alan LaFleur of MathWorks for facilitating our documentation work.
Stephen Vanreusel of MathWorks for help with testing.
Dan Doherty of MathWorks for marketing support and ideas.
Orlando De Jesús of Oklahoma State University for his excellent work in developing
and programming the dynamic training algorithms described in “Time Series and
Dynamic Systems” and in programming the neural network controllers described in
“Neural Network Control Systems” in the Neural Network Toolbox User's Guide.
Martin T. Hagan, Howard B. Demuth, and Mark Hudson Beale for permission to
include various problems, examples, and other material from Neural Network Design,
January, 1996.
viii