Optimization Toolbox

Standard Algorithms

Standard Algorithms provides an introduction to the different optimization problem formulations, and describes the medium-scale (i.e., standard) algorithms used in the toolbox functions. These algorithms have been chosen for their robustness and iterative efficiency. The choice of problem formulation (e.g., unconstrained, least-squares, constrained, minimax, multiobjective, or goal attainment) depends on the problem being considered and the required execution efficiency.
This chapter consists of these sections:

Optimization Overview	Introduces optimization as a way of finding a set of parameters that can in some way be defined as optimal. These parameters are obtained by minimizing or maximizing an objective function, subject to equality or inequality constraints and/or parameter bounds.
Unconstrained Optimization	Discusses the use of quasi-Newton and line search methods for unconstrained optimization. Also provides implementation details for the Hessian update and line search phases of the quasi-Newton algorithm used in `fminunc`.
Least-Squares Optimization	Discusses the use of the Gauss-Newton and Levenberg-Marquardt methods for nonlinear least-squaresleast-squares (LS) optimization. Also provides implementation details for the Gauss-Newton and Levenberg-Marquardt methods used in the nonlinear least-squares optimization routines, `lsqnonlin` and `lsqcurvefit`.
Nonlinear Systems of Equations	Discusses the use of Gauss-Newton, Newton's, and trust-region dogleg methods for the solution of nonlinear systems of equations. Also provides implementation details for the Gauss-Newton and trust-region dogleg methods used by the `fsolve` function.
Constrained Optimization	Discusses the use of the Kuhn-Tucker (KT) equations as the basis for Sequential Quadratic Programming (SQP) methods. Also provides implementation details for the Hessian matrix update, quadratic programming problem solution, and line search and merit function calculation phases of the SQP algorithm used in `fmincon`, `fminimax`, `fgoalattain`, and `fseminf`.
Multiobjective Optimization	Introduces multiobjective optimization and discusses strategies for dealing with competing objectives. It discusses in detail the use of the Goal Attainment method, and suggests improvements to the SQP method for use with the Goal Attainment method.
Selected Bibliography	Lists published materials that support concepts implemented in the medium-scale algorithms.

Note Medium-scale is not a standard term and is used here only to differentiate these algorithms from the large-scale algorithms described in Large-Scale Algorithms.

Selected Bibliography Optimization Overview