logo-polimi
Loading...
Risorse bibliografiche
Risorsa bibliografica obbligatoria
Risorsa bibliografica facoltativa
Scheda Riassuntiva
Anno Accademico 2020/2021
Scuola Scuola di Ingegneria Industriale e dell'Informazione
Insegnamento 055697 - NUMERICAL ANALYSIS FOR MACHINE LEARNING
Docente Miglio Edie
Cfu 10.00 Tipo insegnamento Monodisciplinare
Didattica innovativa L'insegnamento prevede  2.0  CFU erogati con Didattica Innovativa come segue:
  • Blended Learning & Flipped Classroom

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (487) MATHEMATICAL ENGINEERING - INGEGNERIA MATEMATICA*AZZZZ055697 - NUMERICAL ANALYSIS FOR MACHINE LEARNING
055694 - NUMERICAL ANALYSIS FOR MACHINE LEARNING

Obiettivi dell'insegnamento

Numerical methods are ubiquitous in Machine Learning. The aim of this course is twofold:
on the one hand, to systematically address the mathematical and numerical background at
the heart of machine learning, with particular attention to applications in Science and
Engineering; on the other hand, to sustain the theoretical analysis with laboratory sessions.
Extensive hands-on sessions are planned.


Risultati di apprendimento attesi

Students are expected to
- know, understand and be able to implement numerical methods for machine learning;
- choose the most appropriate numerical scheme for solving a given problem;
- critically reason and interpret the results;
- model and numerically simulate problems stemming from applications in Science and
Engineering.


Argomenti trattati

1. Mathematical Representation of Neural Networks
- Multilayer Perceptron, Convolutional Neural Networks, Deep Auto Encoder,
Recursive Neural Networks.

2. Numerical Linear Algebra Tools
- Singular value decomposition, principal components and best low rank matrix.
- Iterative methods for linear system of equations and preconditioning techniques
(multilevel and multigrid methods) and related convergence analysis.
- Iterative methods for computing eigenvalues: QR factorization and least-square.
- Toeplitz matrices and shift invariant filters.
- Stochastic rank reduction.

3. Approximation properties of Neural Networks
- Stone-Weierstrass theorem and universality results for shallow neural networks
- Approximation theory of deep neural networks: error bounds, optimality
properties and regularity classes (e.g, Sobolev, Besov).

4. Numerical Optimization and Training of Neural Networks
- Loss functions, back propagation and automatic differentiation: theory and
numerical aspects.
- Stochastic gradient methods, accelerated gradient methods, coordinate descent
methods.
- Second order methods (Hessian-Free Inexact Newton Methods, Stochastic Quasi-
Newton Methods, Gauss-Newton Methods, Natural Gradient Method, Methods
that employ Diagonal Scaling).
- Methods for regularized models: first-Order methods for generic convex
regularizers (iterative soft-thresholding algorithms, bound-constrained methods
for l1-norm regularized problems), second-order methods (proximal Newton
methods, orthant-based methods).

5. Sparse hints to Artificial Intelligence/Machine Learning applications.


Prerequisiti

The course primarily targets graduate students. Students are requested to have a solid
background in multivariate calculus and linear algebra and some programming experience.
Further prerequisites are: foundations of numerical methods, statistics and probability.


Modalità di valutazione

The exam consists of a written test and an oral colloquium. The written test takes place in
the computer room. The written exam covers all the theoretical and practical arguments
considered during the lectures and lab sessions. The problems mainly focus on definition,
application of important lemmas and theorems, and important examples. Light calculations
may be needed. It is not allowed to use any form of course material. The questions will be
answered without books, notes, preparations, etc. The maximum grade of the written test is
28/30. To get a grade >= 28, students are required to an oral exam. The assessment of the
oral exam will be based on subject knowledge matters and quality of the oral delivery.


Bibliografia
Risorsa bibliografica facoltativaG. Strang, Linear Algebra and Learning from Data , Editore: Wellesley-Cambridge Press, Anno edizione: 2019, ISBN: 978-0692196380
Risorsa bibliografica facoltativaJ. Nocedal, S. J. Wright, Numerical Optimization, Editore: Springer Nature, Anno edizione: 2006, ISBN: 978-0387303031
Risorsa bibliografica facoltativaA. Griewank, A. Walther, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Editore: SIAM, Anno edizione: 2008, ISBN: 978-0898716597
Risorsa bibliografica facoltativaS. Raschka, V. Mirjalili, Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2, 3rd Edition, Editore: Packt Publishing, Anno edizione: 2019, ISBN: 978-1789955750
Risorsa bibliografica facoltativaI. Goodfellow, Y. Bengio, A. Courville, Deep Learning, Editore: Mit Press, Anno edizione: 2017, ISBN: 978-0262035613 http://www.deeplearningbook.org/

Forme didattiche
Tipo Forma Didattica Ore di attività svolte in aula
(hh:mm)
Ore di studio autonome
(hh:mm)
Lezione
52:00
110:00
Esercitazione
28:00
60:00
Laboratorio Informatico
0:00
0:00
Laboratorio Sperimentale
0:00
0:00
Laboratorio Di Progetto
0:00
0:00
Totale 80:00 170:00

Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua Inglese
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
schedaincarico v. 1.6.5 / 1.6.5
Area Servizi ICT
27/09/2020