# Optimization Theory and Methods: Nonlinear Programming

Springer Science & Business Media, Aug 6, 2006 - Mathematics - 688 pages

Optimization Theory and Methods can be used as a textbook for an optimization course for graduates and senior undergraduates. It is the result of the author's teaching and research over the past decade. It describes optimization theory and several powerful methods. For most methods, the book discusses an idea’s motivation, studies the derivation, establishes the global and local convergence, describes algorithmic steps, and discusses the numerical performance.

### What people are saying -Write a review

We haven't found any reviews in the usual places.

### Contents

 Introduction 1 12 Mathematics Foundations 2 121 Norm 3 122 Inverse and Generalized Inverse of a Matrix 9 123 Properties of Eigenvalues 12 124 RankOne Update 17 125 Function and Differential 22 13 Convex Sets and Convex Functions 31
 612 Convergence of TrustRegion Methods 308 613 Solving A TrustRegion Subproblem 316 62 Conic Model and Collinear Scaling Algorithm 324 622 Generalized QuasiNewton Equation 326 623 Updates that Preserve Past Information 330 624 Collinear Scaling BFGS Algorithm 334 63 Tensor Methods 337 632 Tensor Methods for Unconstrained Optimization 341

 131 Convex Sets 32 132 Convex Functions 36 133 Separation and Support of Convex Sets 50 14 Optimality Conditions for Unconstrained Optimization 57 15 Structure of Optimization Methods 63 Exercises 68 Line Search 71 22 Convergence Theory for Exact Line Search 74 23 The Golden Section Method and the Fibonacci Method 84 232 The Fibonacci Method 87 24 Interpolation Method 89 242 Cubic Interpolation Method 98 25 Inexact Line Search Techniques 102 251 Armijo and Goldstein Rule 103 252 WolfePowell Rule 104 253 Goldstein Algorithm and WolfePowell Algorithm 106 254 Backtracking Line Search 108 255 Convergence Theorems of Inexact Line Search 109 Exercises 116 Newtons Methods 119 312 Convergence of the Steepest Descent Method 120 313 Barzilai and Borwein Gradient Method 126 Kantorovich Inequality 129 32 Newtons Method 130 33 Modiﬁed Newtons Method 135 34 FiniteDifference Newtons Method 140 35 Negative Curvature Direction Method 147 351 GillMurray Stable Newtons Method 148 352 FiaccoMcCormick Method 151 353 FletcherFreeman Method 152 354 SecondOrder Step Rules 155 36 Inexact Newtons Method 163 Exercises 172 Conjugate Gradient Method 174 42 Conjugate Gradient Method 178 422 Beales ThreeTerm Conjugate Gradient Method 185 423 Preconditioned Conjugate Gradient Method 188 43 Convergence of Conjugate Gradient Methods 191 432 Convergence Rate of Conjugate Gradient Methods 198 Exercises 200 QuasiNewton Methods 203 511 QuasiNewton Equation 204 512 Symmetric RankOne SR1 Update 207 513 DFP Update 210 514 BFGS Update and PSB Update 217 515 The Least Change Secant Update 223 52 The Broyden Class 225 53 Global Convergence of QuasiNewton Methods 231 531 Global Convergence under Exact Line Search 232 532 Global Convergence under Inexact Line Search 238 54 Local Convergence of QuasiNewton Methods 240 541 Superlinear Convergence of General QuasiNewton Methods 241 542 Linear Convergence of General QuasiNewton Methods 250 543 Local Convergence of Broydens RankOne Update 255 544 Local and Linear Convergence of DFP Method 258 545 Superlinear Convergence of BFGS Method 261 546 Superlinear Convergence of DFP Method 265 547 Local Convergence of Broydens Class Methods 271 55 SelfScaling Variable Metric SSVM Methods 273 552 SelfScaling Variable Metric SSVM Method 277 553 Choices of the Scaling Factor 279 56 Sparse QuasiNewton Methods 282 57 Limited Memory BFGS Method 292 Exercises 301 TrustRegion Methods and Conic Model Methods 302
 Exercises 349 Solving Nonlinear LeastSquares Problems 353 72 GaussNewton Method 355 73 LevenbergMarquardt Method 362 732 Convergence of LevenbergMarquardt Method 367 74 Implementation of LM Method 372 75 QuasiNewton Method 379 Exercises 382 Theory of Constrained Optimization 384 82 FirstOrder Optimality Conditions 388 83 SecondOrder Optimality Conditions 401 84 Duality 406 Exercises 409 Quadratic Programming 411 92 Duality for Quadratic Programming 413 93 EqualityConstrained Quadratic Programming 419 94 Active Set Methods 427 95 Dual Method 435 96 Interior Ellipsoid Method 441 97 PrimalDual InteriorPoint Methods 445 Exercises 451 Penalty Function Methods 455 102 The Simple Penalty Function Method 461 103 Interior Point Penalty Functions 466 104 Augmented Lagrangian Method 474 105 Smooth Exact Penalty Functions 480 106 Nonsmooth Exact Penalty Functions 482 Exercises 490 Feasible Direction Methods 493 112 Generalized Elimination 502 113 Generalized Reduced Gradient Method 509 114 Projected Gradient Method 512 115 Linearly Constrained Problems 515 Exercises 520 Sequential Quadratic Programming 522 122 WilsonHanPowell Method 530 123 Superlinear Convergence of SQP Step 537 124 Maratos Effect 541 125 Watchdog Technique 543 126 SecondOrder Correction Step 545 127 Smooth Exact Penalty Functions 550 128 Reduced Hessian Matrix Method 554 Exercises 558 TrustRegion Methods for Constrained Problems 561 132 Linear Constraints 563 133 TrustRegion Subproblems 568 134 Null Space Method 571 135 CDT Subproblem 580 136 PowellYuan Algorithm 585 Exercises 594 Nonsmooth Optimization 597 142 Nonsmooth Optimization Problem 607 143 The Subgradient Method 609 144 Cutting Plane Method 615 145 The Bundle Methods 617 146 Basic Property of a Composite Nonsmooth Function 620 147 Trust Region Method for Composite Nonsmooth Optimization 623 148 Nonsmooth Newtons Method 628 Exercises 634 Test Functions 636 2 Test Functions for Constrained Optimization Problems 638 Bibliography 649 Index 682 Copyright