Engineering Optimization
Engineering Optimization: Theory and Practice, Fourth Edition
Copyright © 2009 by John Wiley & Sons, Inc.
Singiresu S. Rao
Engineering Optimization
Theory and Practice
Fourth Edition
Singiresu S. Rao
JOHN WILEY & SONS, INC.
This book is printed on acid-free paper.
Copyright c 2009 by John Wiley & Sons, Inc. All rights reserved
Published by John Wiley & Sons, Inc., Hoboken, New Jersey
Published simultaneously in Canada
No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or
by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted
under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission
of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance
Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750– 8400, fax (978) 646– 8600, or on the web
at www.copyright.com. Requests to the Publisher for permission should be addressed to the Permissions
Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748– 6011, fax (201)
748– 6008, or online at www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: While the publisher and the author have used their best efforts in
preparing this book, they make no representations or warranties with respect to the accuracy or completeness
of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness
for a particular purpose. No warranty may be created or extended by sales representatives or written sales
materials. The advice and strategies contained herein may not be suitable for your situation. You should
consult with a professional where appropriate. Neither the publisher nor the author shall be liable for any loss
of profit or any other commercial damages, including but not limited to special, incidental, consequential,
or other damages.
For general information about our other products and services, please contact our Customer Care Department
within the United States at (800) 762– 2974, outside the United States at (317) 572– 3993 or fax (317)
572– 4002.
Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may
not be available in electronic books. For more information about Wiley products, visit our web site at
www.wiley.com.
Library of Congress Cataloging-in-Publication Data:
Rao, S. S.
Engineering optimization : theory and practice / Singiresu S. Rao.– 4th ed.
p. cm.
Includes index.
ISBN 978-0-470-18352-6 (cloth)
1.
Engineering— Mathematical models. 2. Mathematical optimization. I. Title.
TA342.R36 2009
620.001′5196— dc22
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
2009018559
Contents
Preface
xvii
1
Introduction to Optimization
1
1.1
1.2
1.3
1.4
Introduction
1
Historical Development
3
Engineering Applications of Optimization
5
Statement of an Optimization Problem
6
1.4.1 Design Vector
6
1.4.2 Design Constraints
1.4.3 Constraint Surface
1.4.4 Objective Function
7
8
9
1.4.5 Objective Function Surfaces
9
1.5
Classification of Optimization Problems
14
1.5.1 Classification Based on the Existence of Constraints
14
1.5.2 Classification Based on the Nature of the Design Variables
15
1.5.3 Classification Based on the Physical Structure of the Problem
1.5.4 Classification Based on the Nature of the Equations Involved
16
19
1.5.5 Classification Based on the Permissible Values of the Design Variables
28
1.5.6 Classification Based on the Deterministic Nature of the Variables
29
1.5.7 Classification Based on the Separability of the Functions
30
1.5.8 Classification Based on the Number of Objective Functions
32
1.6
1.7
1.8
Optimization Techniques
35
Engineering Optimization Literature
35
Solution of Optimization Problems Using MATLAB
36
References and Bibliography
39
Review Questions
45
Problems
46
2 Classical Optimization Techniques
63
2.1
2.2
Introduction
63
Single-Variable Optimization
63
2.3 Multivariable Optimization with No Constraints
68
2.3.1
2.3.2
Semidefinite Case
73
Saddle Point
73
2.4 Multivariable Optimization with Equality Constraints
75
2.4.1
2.4.2
2.4.3
Solution by Direct Substitution
76
Solution by the Method of Constrained Variation
Solution by the Method of Lagrange Multipliers
77
85
vii
viii Contents
2.5 Multivariable Optimization with Inequality Constraints
93
2.5.1 Kuhn – Tucker Conditions
98
2.5.2 Constraint Qualification
98
2.6
Convex Programming Problem
104
References and Bibliography
105
Review Questions
105
Problems
106
3 Linear Programming I: Simplex Method
119
3.1
3.2
3.3
3.4
3.5
3.6
3.7
Introduction
119
Applications of Linear Programming
120
Standard Form of a Linear Programming Problem
122
Geometry of Linear Programming Problems
124
Definitions and Theorems
127
Solution of a System of Linear Simultaneous Equations
133
Pivotal Reduction of a General System of Equations
135
3.8 Motivation of the Simplex Method
138
3.9
Simplex Algorithm
139
3.9.1
3.9.2
Identifying an Optimal Point
140
Improving a Nonoptimal Basic Feasible Solution
141
3.10 Two Phases of the Simplex Method
3.11 MATLAB Solution of LP Problems
150
156
References and Bibliography
158
Review Questions
158
Problems
160
4 Linear Programming II: Additional Topics and Extensions
177
4.1
4.2
4.3
4.4
4.5
Introduction
177
Revised Simplex Method
177
Duality in Linear Programming
192
4.3.1
Symmetric Primal – Dual Relations
192
4.3.2 General Primal – Dual Relations
193
4.3.3
Primal – Dual Relations When the Primal Is in Standard Form
193
4.3.4 Duality Theorems
195
4.3.5 Dual Simplex Method
195
Decomposition Principle
200
Sensitivity or Postoptimality Analysis
4.5.1 Changes in the Right-Hand-Side Constants bi
4.5.2 Changes in the Cost Coefficients cj
4.5.3 Addition of New Variables
214
4.5.4 Changes in the Constraint Coefficients aij
4.5.5 Addition of Constraints
212
207
218
208
215
4.6
Transportation Problem
220
4.7
Karmarkar’s Interior Method
222
4.7.1
Statement of the Problem
223
4.7.2 Conversion of an LP Problem into the Required Form
224
Contents
ix
4.7.3 Algorithm
226
4.8
Quadratic Programming
229
4.9 MATLAB Solutions
235
References and Bibliography
237
Review Questions
239
Problems
239
5 Nonlinear Programming I: One-Dimensional Minimization Methods
248
5.1
5.2
Introduction
248
Unimodal Function
253
ELIMINATION METHODS
254
5.3
Unrestricted Search
254
5.3.1
Search with Fixed Step Size
254
5.3.2
Search with Accelerated Step Size
255
5.4
5.5
5.6
5.7
5.8
5.9
Exhaustive Search
256
Dichotomous Search
257
Interval Halving Method
260
Fibonacci Method
263
Golden Section Method
267
Comparison of Elimination Methods
271
INTERPOLATION METHODS
271
5.10 Quadratic Interpolation Method
273
5.11 Cubic Interpolation Method
280
5.12 Direct Root Methods
286
5.12.1 Newton Method
286
5.12.2 Quasi-Newton Method
288
5.12.3 Secant Method
5.13 Practical Considerations
290
293
5.13.1 How to Make the Methods Efficient and More Reliable
5.13.2 Implementation in Multivariable Optimization Problems
293
293
5.13.3 Comparison of Methods
294
5.14 MATLAB Solution of One-Dimensional Minimization Problems
294
References and Bibliography
295
Review Questions
295
Problems
296
x Contents
6 Nonlinear Programming II: Unconstrained Optimization Techniques
301
6.1
Introduction
301
6.1.1 Classification of Unconstrained Minimization Methods
304
6.1.2 General Approach
305
6.1.3 Rate of Convergence
305
6.1.4
Scaling of Design Variables
305
DIRECT SEARCH METHODS
309
6.2
Random Search Methods
309
6.2.1 Random Jumping Method
311
6.2.2 Random Walk Method
312
6.2.3 Random Walk Method with Direction Exploitation
313
6.2.4 Advantages of Random Search Methods
314
6.3
6.4
6.5
6.6
Grid Search Method
314
Univariate Method
Pattern Directions
315
318
Powell’s Method
319
6.6.1 Conjugate Directions
319
6.6.2 Algorithm
323
6.7
Simplex Method
328
6.7.1 Reflection
6.7.2
Expansion
328
331
6.7.3 Contraction
332
INDIRECT SEARCH (DESCENT) METHODS
335
6.8
Gradient of a Function
335
6.8.1
Evaluation of the Gradient
337
6.8.2 Rate of Change of a Function along a Direction
338
6.9
Steepest Descent (Cauchy) Method
339
6.10 Conjugate Gradient (Fletcher– Reeves) Method
341
6.10.1 Development of the Fletcher– Reeves Method
342
6.10.2 Fletcher– Reeves Method
343
6.11 Newton’s Method
6.12 Marquardt Method
345
348
6.13 Quasi-Newton Methods
350
6.13.1 Rank 1 Updates
6.13.2 Rank 2 Updates
351
352
6.14 Davidon – Fletcher– Powell Method
354
6.15 Broyden – Fletcher– Goldfarb – Shanno Method
360
6.16 Test Functions
363
6.17 MATLAB Solution of Unconstrained Optimization Problems
365
References and Bibliography
366
Review Questions
368
Problems
370