|
|
| Academic Year |
2024/2025 |
| Name |
Dott. - MI (1380) Ingegneria dell'Informazione / Information Technology |
| Programme Year |
1 |
| ID Code |
062757 |
| Course Title |
LARGE SCALE OPTIMIZATION |
| Course Type |
MONO-DISCIPLINARY COURSE |
| Credits (CFU / ECTS) |
5.0 |
| Course Description |
Optimization problems are ubiquitous in many engineering and science disciplines, such as machine learning, signal processing, data science,
communications, control and robotics. Optimization methods have to cope well with the demanding requirements of modern applications. These
include handling large numbers of variables and constraints, being amenable to distributed or even parallel implementation or being simple
enough to be embedded into hardware devices with limited storing and computational capabilities. MISSION AND GOALS: The purpose of the
course is to introduce, derive and analyze a wide range of classical and modern optimization algorithms for convex and nonconvex structured,
nonsmooth optimization. After a review on fundamental topics of convex analysis and duality, the course aims at presenting first-order algorithms
under the unifying framework of monotone operators and fixed point theory.
The students will familiarize with essential algorithms such as the proximal gradient, forward-backward and Douglas-Rachford splitting, proximal point,
augmented Lagrangian, ADMM and several primal-dual proximal algorithms. Examples of problem formulations and templates from various disciplines of engineering will be demonstrated. Extensions of the algorithms to the nonconvex setting will also be presented. The course will also cover blockcoordinate and incremental algorithms that are nowadays very popular for optimization problems arising in machine learning and data science.
PROGRAMME OF THE COURSE:
- Basics of convex and variational analysis: preliminaries, convex sets and functions, closed functions, conjugate functions, proximal mapping, separation theorems.
- Subgradient methods: subdifferentials, optimality conditions, projected subgradient methods
- Proximal gradient methods: proximal mapping, (accelerated) proximal gradient method, smoothing and Moreau envelope
- Conjugate duality: Fenchel conjugate, Fenchel-Young inequality, conjugate subgradient theorem, moreau decomposition, abstract duality, Fenchel-
Rockafellar duality, Lagrangian duality
- Dual methods: Alternating minimization algorithm, Alternating Direction Method of Multipliers (ADMM), distributed optimization
- Fixed point iterations: Firmly nonexpansive operators, averaged operators, Douglas-Rachford splitting, multiblock, proximal ADMM, consensus and sharing problems.
- Block-coordinate and finite sum algorithms: Alternating minimization, block-coordinate proximal algorithms, stochastic gradient descent, variance reduced algorithms (SAG, SAGA, Finito, SVRG) |
| Scientific-Disciplinary Sector (SSD)
|
|
SSD Code
|
SSD Description
|
CFU
|
|
ING-INF/04
|
SYSTEMS AND CONTROL ENGINEERING
|
5.0
|
|
|
Alphabetical group
|
Name
|
Teaching Assignment Details
|
|
From (included)
|
To (excluded)
|
|
A
|
ZZZZ
|
Fagiano Lorenzo Mario, Patrinos Panagiotis
|
|
|
