Optimisation 2

Objectives

Machine learning application often lead to optimisation problems of a composite nature: a typical fit-to-data term is penalized so as to enforce some geometrical properties in the solution. Typical properties include sparcity, low rank in matrices. Such problems are often non-differentiable but convex. We review the most popular sub-gradient based methods for solving such problems, insisting on the convergence properties and the complexity of such methods. We will also focus on efficient implementation of such methods on image processing applications. Finally, we will develop in the SPARK software a movie recommendation system.

Description

1) Machine learning in artificial intelligence

2) First order methods in the differentiable case: stochastic gradient, mini-batch, ADAM

3) Computation of a subgradient. Subgradient methods and proximal methods

4) Modeling parsimony by convex relaxation (practical)

5) Complexity analysis

6) Development of a film sub-spark recommendation system

Targeted skills

Know the different first order methods for optimization

Know how to compute the complexity of an optimization algorithm

Know how to compute the subdifferential of a convex function, and if necessary a subgradient

Know how to use Julia and Jupiter Notebook

Know how to build a Spark-based recommendation system

Bibliography

First order methods in optimization, Amir Beck

Convex Optimization: Algorithms and Complexity, Sebastian Bubeck

Pre-requisites

Basic course on linear algebra, Basic algorithms for unconstrained optimisation

Session 1 ou session unique - Contrôle des connaissances

Session 2 - Contrôle des connaissances