Introduction to the Theory
of Nonhnear Optimization
Johannes Jahn
Introduction
to the Theory
of NonHnear
Optimization
Third Edition
With 31 Figures
Sprin g er
Prof. Dr. Johannes Jahn
Universitat Erlangen-Niirnberg
Institut fur Angewandte Mathematik
Martensstr. 3
91058 Erlangen
Germany
jahn@am.uni-erlangen.de
Library of Congress Control Number: 2006938674
ISBN 978-3-540-49378-5 Springer Berlin Heidelberg New York
ISBN 978-3-540-61407-4 Second Edition Springer Berlin Heidelberg New York
This work is subject to copyright. All rights are reserved, whether the whole or part of the material is
concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broad
casting, reproduction on microfilm or in any other way, and storage in data banks. Duplication of
this publication or parts thereof is permitted only under the provisions of the German Copyright
Law of September 9,1965, in its current version, and permission for use must always be obtained
from Springer. Violations are liable to prosecution under the German Copyright Law.
Springer is part of Springer Science+Business Media
springer.com
© Springer-Verlag Berlin Heidelberg 1994,1996,2007
The use of general descriptive names, registered names, trademarks, etc. in this publication does
not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
Production: LE-TgX Jelonek, Schmidt & Vockler GbR, Leipzig
Cover-design: Erich Kirchner, Heidelberg
SPIN 11932048
42/3100YL - 5 4 3 2 10
Printed on acid-free paper
To Claudia and Martin
Preface
This book presents an application-oriented introduction to the the
ory of nonhnear optimization. It describes basic notions and concep
tions of optimization in the setting of normed or even Banach spaces.
Various theorems are appHed to problems in related mathematical
areas. For instance, the Euler-Lagrange equation in the calculus of
variations, the generahzed Kolmogorov condition and the alternation
theorem in approximation theory as well as the Pontryagin maximum
principle in optimal control theory are derived from general results of
optimization.
Because of the introductory character of this text it is not intended
to give a complete description of all approaches in optimization. For
instance, investigations on conjugate duality, sensitivity, stability, re
cession cones and other concepts are not included in the book.
The bibliography gives a survey of books in the area of nonlinear
optimization and related areas like approximation theory and optimal
control theory. Important papers are cited as footnotes in the text.
This third edition is an enlarged and revised version containing
an additional chapter on extended semidefinite optimization and an
updated bibliography.
I am grateful to S. GeuB, S. Gmeiner, S. Keck, Prof. Dr. E.W.
Sachs and H. Winkler for their support, and I am especially indebted
to D.G. Cunningham, Dr. G. Eichfelder, Dr. F. Hettlich, Dr. J. Klose,
Prof. Dr. E.W. Sachs, Dr. T. Staib and Dr. M. Stingl for fruitful
discussions.
Erlangen, September 2006
Johannes Jahn
Contents
Preface
1 Introduction and Problem Formulation
2 Existence Theorems for Minimal Points
2.1 Problem Formulation
2.2 Existence Theorems
2.3 Set of Minimal Points
2.4 Application to Approximation Problems
2.5 Application to Optimal Control Problems
Exercises
3 Generalized Derivatives
3.1 Directional Derivative
3.2 Gateaux and Frechet Derivatives
3.3 Subdifferential
3.4 Quasidifferential
3.5 Clarke Derivative
Exercises
4 Tangent Cones
4.1 Definition and Properties
4.2 Optimality Conditions
4.3 A Lyusternik Theorem
Exercises
5 Generalized Lagrange Multiplier Rule
5.1 Problem Formulation
vii
1
7
7
8
18
19
23
29
31
31
37
49
57
67
75
79
79
88
95
103
105
105
X
Contents
5.2 Necessary Optimality Conditions
5.3 Sufficient Optimality Conditions
5.4 Application to Optimal Control Problems
Exercises
6 Duality
6.1 Problem Formulation
6.2 Duality Theorems
6.3 Saddle Point Theorems
6.4 Linear Problems
6.5 Application to Approximation Problems
Exercises
108
126
136
156
159
159
164
168
172
175
184
7 Application to Extended Semidefinite Optimization 187
187
202
207
210
7.1 Lowner Ordering Cone and Extensions
7.2 Optimality Conditions
7.3 Duality
Exercises
8 Direct Treatment of Special Optimization Problems 213
213
221
238
8.1 Linear Quadratic Optimal Control Problems
8.2 Time Minimal Control Problems
Exercises
A Weak Convergence
B Reflexivity of Banach Spaces
C Hahn-Banach Theorem
D Partially Ordered Linear Spaces
Bibliography
Answers to the Exercises
Index
241
243
245
249
253
275
289
Chapter 1
Introduction and Problem
Formulation
In optimization one investigates problems of the determination of a
minimal point of a functional on a nonempty subset of a real linear
space. To be more specific this means: Let X be a real linear space,
let S' be a nonempty subset of X, and let /
: iS —> R be a given
functional. We ask for the minimal points of / on S. An element
X E S is called a minimal point offonS
if
f{x) < f{x) for all
xeS.
The set S is also called constraint set^ and the functional / is called
objective functional
In order to introduce optimization we present various typical op
timization problems from Applied Mathematics. First we discuss a
design problem from structural engineering.
Example 1.1. As a simple example consider the design of a beam
with a rectangular cross-section and a given length I (see Fig. 1.1 and
1.2). The height xi and the width X2 have to be determined.
The design variables Xi and X2 have to be chosen in an area which
makes sense in practice. A certain stress condition must be satisfied,
i.e. the arising stresses cannot exceed a feasible stress. This leads to
the inequality
2000 < x\x2.
(1.1)