|
|
| Academic Year |
2025/2026 |
| Name |
Dott. - MI (1380) Ingegneria dell'Informazione / Information Technology |
| Programme Year |
1 |
| ID Code |
063914 |
| Course Title |
LEARNING THEORY |
| Course Type |
MONO-DISCIPLINARY COURSE |
| Credits (CFU / ECTS) |
5.0 |
| Course Description |
This course introduces probabilistic and statistical foundations, essential for modern machine learning and decision-making analysis. It begins with an overview of probability theory and stochastic processes, followed by a study of the most important concentration results, crucial for deriving theoretical guarantees in learning algorithms. The course then focuses on supervised learning, covering fundamental risk minimization techniques and generalization bounds. Sequential decision-making is explored by studying multi-armed bandits, emphasizing regret minimization, algorithmic strategies, and the corresponding analysis. The last part of the course concentrates on linear models in both the scenarios of supervised learning and sequential decision-making.
Program of the course:
1. Foundations of probability and stochastic processes (3h)
1.1. Elements on probability: Random variables, Expectation, Moments and moment generating function, Boole's inequality
1.2. Stochastic processes: Sigma algebras, Conditional expectation, Filtrations, Martingales
2. Concentration of measures (2h): Markov inequality, Hoeffding inequality, Chernoff inequality, Bernstein inequality
3. Supervised learning (6h)
3.1. Supervised learning setting and risk definition
3.2. Empirical risk minimization and population risk minimization
3.3. Uniform bounds: Finite case, Covering numbers, Rademacher averages
4. Sequential decision-making (6h)
4.1. Interaction model: Expert feedback, Bandit feedback
4.2. Multi-armed bandits: Notion of regret, Optimism principle and rationale, UCB1 algorithm, and related analysis
4.3. Online to batch conversion
5. Linear models (3h)
5.1. Linear regression: regularization and related analysis
5.2. Linear bandits: framework, the LinUCB algorithm, and related analysis
5.3. Elements of non-linear models
6. Practical sessions (5h)
6.1. Overview of Python: Libraries for scientific programming, Visualization of the results
6.2. Hands-on with supervised learning: Efficient linear regression, Elements of convex optimization for regression
6.3. Hands-on with multi-armed bandits: UCB1 (implementation, computation of the regret, and comparison between the theoretical and empirical results) and LinUCB (implementation, discussion, and computation of the regret) |
| Scientific-Disciplinary Sector (SSD)
|
|
SSD Code
|
SSD Description
|
CFU
|
|
ING-INF/05
|
INFORMATION PROCESSING SYSTEMS
|
5.0
|
|
|
Alphabetical group
|
Name
|
Teaching Assignment Details
|
|
From (included)
|
To (excluded)
|
|
A
|
ZZZZ
|
Metelli Alberto Maria, Mussi Marco
|
|
|
