Optimization for data sciences

ECTS : 3

Volume horaire : 24

Description du contenu de l'enseignement :

Optimization is a framework to formulate a concrete problem into a mathematical object, which enables its theoretical analysis and its practical solve via numerical algorithms. Although applications range from power systems to finance, the interest for optimization techniques has surged following the advent of data science and deep learning in particular. This course will provide an overview of optimization problems and algorithms that are relevant in a data science. We will describe various problem formulations arising in machine and deep learning, using them to motivate the key mathematical concepts that are useful in that space (linear algebra, convexity,  smoothness). We will then move to optimization algorithms, with a focus on the most popular schemes in data science (gradient descent and its variants), and we will discuss both theoretical motivations and implementation challenges.

Compétence à acquérir :

• Understand the main classical optimization formulations of data science tasks.
• Given an optimization problem, select an algorithm well suited for solving the problem based on theoretical and practical concerns.
• Identify specific challenges posed by optimization in a data science context, and ways to address these challenges.

Bibliographie, lectures recommandées :

• S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
• S. J. Wright and B. Recht. Optimization for Data Analysis. Cambridge University Press, 2022.