Risorse bibliografiche
 Risorsa bibliografica obbligatoria Risorsa bibliografica facoltativa
 Scheda Riassuntiva
 Anno Accademico 2020/2021 Scuola Scuola di Ingegneria Industriale e dell'Informazione Insegnamento 055697 - NUMERICAL ANALYSIS FOR MACHINE LEARNING Docente Miglio Edie Cfu 10.00 Tipo insegnamento Monodisciplinare Didattica innovativa L'insegnamento prevede  2.0  CFU erogati con Didattica Innovativa come segue: Blended Learning & Flipped Classroom

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (487) MATHEMATICAL ENGINEERING - INGEGNERIA MATEMATICA*AZZZZ055697 - NUMERICAL ANALYSIS FOR MACHINE LEARNING
055694 - NUMERICAL ANALYSIS FOR MACHINE LEARNING

 Obiettivi dell'insegnamento
 Numerical methods are ubiquitous in Machine Learning. The aim of this course is twofold:on the one hand, to systematically address the mathematical and numerical background atthe heart of machine learning, with particular attention to applications in Science andEngineering; on the other hand, to sustain the theoretical analysis with laboratory sessions.Extensive hands-on sessions are planned.

 Risultati di apprendimento attesi
 Students are expected to- know, understand and be able to implement numerical methods for machine learning;- choose the most appropriate numerical scheme for solving a given problem;- critically reason and interpret the results;- model and numerically simulate problems stemming from applications in Science andEngineering.

 Argomenti trattati
 1. Mathematical Representation of Neural Networks- Multilayer Perceptron, Convolutional Neural Networks, Deep Auto Encoder,Recursive Neural Networks. 2. Numerical Linear Algebra Tools- Singular value decomposition, principal components and best low rank matrix.- Iterative methods for linear system of equations and preconditioning techniques(multilevel and multigrid methods) and related convergence analysis.- Iterative methods for computing eigenvalues: QR factorization and least-square.- Toeplitz matrices and shift invariant filters.- Stochastic rank reduction. 3. Approximation properties of Neural Networks- Stone-Weierstrass theorem and universality results for shallow neural networks- Approximation theory of deep neural networks: error bounds, optimalityproperties and regularity classes (e.g, Sobolev, Besov). 4. Numerical Optimization and Training of Neural Networks- Loss functions, back propagation and automatic differentiation: theory andnumerical aspects.- Stochastic gradient methods, accelerated gradient methods, coordinate descentmethods.- Second order methods (Hessian-Free Inexact Newton Methods, Stochastic Quasi-Newton Methods, Gauss-Newton Methods, Natural Gradient Method, Methodsthat employ Diagonal Scaling).- Methods for regularized models: first-Order methods for generic convexregularizers (iterative soft-thresholding algorithms, bound-constrained methodsfor l1-norm regularized problems), second-order methods (proximal Newtonmethods, orthant-based methods). 5. Sparse hints to Artificial Intelligence/Machine Learning applications.

 Prerequisiti
 The course primarily targets graduate students. Students are requested to have a solidbackground in multivariate calculus and linear algebra and some programming experience.Further prerequisites are: foundations of numerical methods, statistics and probability.

 Modalità di valutazione
 The exam consists of a written test and an oral colloquium. The written test takes place inthe computer room. The written exam covers all the theoretical and practical argumentsconsidered during the lectures and lab sessions. The problems mainly focus on definition,application of important lemmas and theorems, and important examples. Light calculationsmay be needed. It is not allowed to use any form of course material. The questions will beanswered without books, notes, preparations, etc. The maximum grade of the written test is28/30. To get a grade >= 28, students are required to an oral exam. The assessment of theoral exam will be based on subject knowledge matters and quality of the oral delivery.

 Bibliografia
 G. Strang, Linear Algebra and Learning from Data , Editore: Wellesley-Cambridge Press, Anno edizione: 2019, ISBN: 978-0692196380 J. Nocedal, S. J. Wright, Numerical Optimization, Editore: Springer Nature, Anno edizione: 2006, ISBN: 978-0387303031 A. Griewank, A. Walther, Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation, Editore: SIAM, Anno edizione: 2008, ISBN: 978-0898716597 S. Raschka, V. Mirjalili, Python Machine Learning: Machine Learning and Deep Learning with Python, scikit-learn, and TensorFlow 2, 3rd Edition, Editore: Packt Publishing, Anno edizione: 2019, ISBN: 978-1789955750 I. Goodfellow, Y. Bengio, A. Courville, Deep Learning, Editore: Mit Press, Anno edizione: 2017, ISBN: 978-0262035613 http://www.deeplearningbook.org/

 Forme didattiche
Tipo Forma Didattica Ore di attività svolte in aula
(hh:mm)
Ore di studio autonome
(hh:mm)
Lezione
52:00
110:00
Esercitazione
28:00
60:00
Laboratorio Informatico
0:00
0:00
Laboratorio Sperimentale
0:00
0:00
Laboratorio Di Progetto
0:00
0:00
Totale 80:00 170:00

 Informazioni in lingua inglese a supporto dell'internazionalizzazione
 Insegnamento erogato in lingua Inglese Disponibilità di materiale didattico/slides in lingua inglese Disponibilità di libri di testo/bibliografia in lingua inglese Possibilità di sostenere l'esame in lingua inglese
 schedaincarico v. 1.6.5 / 1.6.5 Area Servizi ICT 27/09/2020