Continuous optimization

Understand and master optimization methods dedicated to solving continuous nonlinear problems.

This course provides a brief overview of the state of the art in optimization methods for solving nonlinear and continuous problems. The various topics covered in the different chapters include:

- Formulating a single-objective optimization problem (without constraints), optimality conditions, and classification of the different methods used to solve a problem;

- One-dimensional optimization methods: interval subdivisions, quadratic interpolation, zero crossing of the objective function derivative;

- Gradient-based optimization methods: steepest descent, accelerated or conjugate gradient, Gauss-Newton or Quasi-Newton techniques;

- Geometric optimization methods: fixed or adapted directions during the search (Gauss-Seidel, Powell, Hooke & Jeeves), Nelder & Mead Simplex;

- Stochastic optimization methods: simulated annealing, evolutionary algorithms and nesting methods, particle swarms;

- Formulation of an optimization problem under inequality constraints: Lagrangian concept, KKT optimality conditions, penalization methods;

- Formulating a multi-objective problem: Pareto optimality, a priori decision on objectives (weighted objectives, epsilon constraint methods, global criterion method), a posteriori decision on objectives (using population-based algorithms), progressive or sequential techniques (lexicographic method).

Basic mathematical concepts related to functions with multiple variables, vector and matrix operators, and geometry.