Ing Ind - Inf (Mag.)(ord. 270) - MI (471) BIOMEDICAL ENGINEERING - INGEGNERIA BIOMEDICA
054062 - MODEL IDENTIFICATION AND MACHINE LEARNING [I.C.]
054063 - MODEL IDENTIFICATION AND DATA ANALYSIS 2
054102 - MACHINE LEARNING
Analysis and design problems of engineering are typically solved by using mathematical models to describe the salient features of the systems under study, be they technological systems or industrial devices, biological or natural phenomena, economic or financial processes, etc. Due to the growing complexity of the problems that are nowadays faced, it is often the case that it is impossible to deduce the required mathematical models from simple laws of the various disciplines. It is instead necessary to derive the models inductively, that is, starting from experimental observations by means of automatic estimation procedures. The study of these procedures that allow the conversion of data into simple and suitable models is the objective of this course. The course is divided into two parts. A part of Model Identification, which mainly deals with problems related to linear dynamic systems, and a second part of Machine Learning, which deals mainly with problems related to static, possibly nonlinear systems and data-driven decision making.
The course is consistent with the overall curriculum profile and in pursuing the general objectives of the master program. In particular, the course contributes to the development of the following skills:
- to be able to understand the contexts in which it is appropriate to use model identification, machine learning and data analysis methods
- to understand trends, technologies and the main methodologies for model identification and machine learning
- to be able to use the methods provided in the course to solve real problems
Risultati di apprendimento attesi
As for the part of Model Identification, the objective of this course is to provide the basic knowledge for modeling dynamic systems starting from experimental data. Meanwhile the course will develops techniques for the prediction of variables and the estimation of parameters through virtual sensors. Upon completion of the course, the student:
- he/she will acquire the notion of stochastic systems and will be able to evaluate their main properties (DD1);
- he/she will be able to solve an estimation and optimal prediction problem and he/she will be able to evaluate the properties of the obtained solution (DD2);
- he/she will know the main algorithms for data processing in order to identify a model starting from experimental data(DD1,DD2);
- he/she will be able to evaluate the performance of the implemented algorithms (DD3);
- he/she will be able to make design choices in the field of identification in order to optimize the final result (DD3).
As for the part of Machine Learning, the course offers an introduction to machine learning and pattern recognition techniques and algorithms. It provides students with the insights and ideas that underlie the current methods of machine learning, along with a detailed discussion of the illustrated techniques. Upon completion of the course the students will have obtained:
- a good knowledge of the main components of machine learning: data, algorithms, selection and complexity of the models (DD1);
- the capability to appreciate data learning issues (DD1);
- the knowledge of a wide range of machine learning algorithms (DD1);
- the capability to apply algorithms to real problems (DD2);
- the capability to select models based on appropriate metrics (DD3);
- the knowledge of the mathematical structure of the main methods and the strengths and weaknesses of each of them (DD1);
- the knowledge of supervised and unsupervised learning paradigms (DD1).
FIRST PART: MODEL IDENTIFICATION
Models in engineering and science
Model accuracy and complexity. Estimation from experimental observations. Models for classification, prediction, control, simulation and management. Data processing techniques.
Stochastic dynamic models, spectral analysis and prediction
Stochastic processes. Input/output models for time series and cause / effect relationships (continuous and discrete time models, AR, MA, ARMA, ARX, ARMAX, Box-Jenkins models). Correlation analysis and spectral analysis. Kolmogorov-Wiener prediction theory.
Identification of input/output models
The problem of model identification starting from simple experimental tests. The Prediction Error Minimization (PEM) paradigm. The Least-Squares (LS) and Maximum Likelihood (ML) for the identification if AR, ARX, ARMA, ARMAX models. Asymptotic analysis of PEM identification. Choice of the complexity: FPE, AIC, MDL indicators. Spectrum estimate.
Kalman filtering and prediction
Stochastic state-space models. Filtering, prediction and regularization. Kalman filter. Steady-state Kalman filter. Extended Kalman filter. Use of the Kalman filter for model identification.
SECOND PART: MACHINE LEARNING
Introduction to Machine Learning
Motivations of machine learning. Machine learning, artificial intelligence and big data. Machine learning applications. Representation of input data. Machine learning process.
Exploratory data analysis
Data validation and cleansing, identification of outliers and missing values detection. Data transformation. Data reduction. Sampling. Features selection. Features extraction by filtering. Principal component analysis. Data discretization. Univariate analysis: graphical analysis, central tendency measurements, dispersion, relative positioning, heterogeneity, empirical density analysis. Bivariate analysis: graphical analysis, correlation, contingency tables. Multivariate analysis: graphical analysis, correlation indices.
Supervised learning: classification and regression
Taxonomy of supervised methods. Evaluation of classification models: holdout, cross-validation, confusion matrices, ROC curves, cumulative gain and lift. Treatment of categorical attributes. Nearest neighbor. Classification and regression trees: separation, arrest and pruning. Bayesian methods: naive Bayesians, Bayesian networks. Logistic regression. Neural networks: Rosenblatt perceptrons, multi-level feed-forward networks. Support vector machines: structural risk minimization; hyperplanes of maximum margin for linear separation; nonlinear separation. Simple and multiple linear regression. Assumptions relating to residues. Least square regression: residual normality and independence, significance of coefficients, analysis of variance, coefficients of determination and linear correlation, multicollinearity, confidence limits and prediction. Selection of predictive variables. Regression regression. Generalized linear regression.
Motivation and evaluation of association rules. Single size association rules. A priori algorithm: generation of frequent item sets; generation of strong rules. Other association rules.
Taxonomy of clustering models. Affinity measurements. Partition methods: K-means, K-medoids. Hierarchical methods: agglomeration and subdivision methods. Evaluation of clustering models.
Applications and use cases
Relational marketing applications. Web mining. Social market analysis. Speech recognition. Text mining. Fraud and anomaly detection. Bioinformatics.
As for the part of Model Identification, the student is required to possess a good background of systems theory and control. Knowledge of the basic concepts of probability and statistics is also required.
Machine Learning is a discipline on the boundary of mathematics and computer science. Therefore, a good background in probability, linear algebra and analysis, as well as experience with programming languages is required.
Modalità di valutazione
The exam consists of two written tests, one concerning the Model Identification part and one for the Machine Learning part. The student is required to give the two written tests simultaneously, on the same day. The exam cannot be split. Only students officially registered to the exam session can take the exam. Late registrations will not be allowed.
The Model Identification test consists of 2/3 numerical exercises and 3/2 open questions on the course topics. In some exercises, questions will be proposed to highlight the students' capability to develop links between the various topics of the course. Specifically, the student will be required:
- to be able to analyze the main properties of a stochastic dynamic system and to calculate its mean, the covariance function and the spectrum of the output stochastic process;
- to demonstrate knowledge of the main definitions and concepts inherent in stochastic dynamical systems, the problem of output and state prediction and the problem of model identification from experimental data;
- to know how to use the main analytical results developed in the course in order to calculate the predictor of I/O or Kalman systems; be able to evaluate its properties and change the parameter variation front; to be able to compare different results and choose the most appropriate one in relation to the situation of interest;
- to know how to numerically implement the main algorithms for data processing for the purpose of model identification;
- to demonstrate the capability to analyze the properties of the identification algorithms and to choose the most significant design parameters (for example the class of models and its complexity) in order to optimize the final result.
The test of Machine Learning includes questions in both open and closed form. Some more theoretical questions aim to verify the knowledge related to methods and algorithms. Other questions areof a more applicative nature and aim at verifying the capability to apply the methods and algorithms to real cases, to understand the outputs and to obtain the implications in the application context.
S. Bittanti, Model Identification and Data Analysis, Editore: Wiley
T. Söderström, Discrete-time stochastic systems, Editore: Springer
T. Söderström, P. Stoica, System Identification, Editore: Prentice Hall
S. Bittanti, Teoria della predizione e del filtraggio, Editore: Pitagora
S. Bittanti, Identificazione dei modelli e sistemi adattativi, Editore: Pitagora
C. Vercellis, Business intelligence: data mining and optimization for decision making, Editore: Wiley
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, Editore: Springer
E. Alpaydin, Introduction to Machine Learning, Editore: MIT press
A. Geron, Hands-On Machine Learning With Scikit-Learn and Tensorflow: Concepts, Tools, and Techniques to Build Intelligent Systems, Editore: O'Reilly
Tipo Forma Didattica
Ore di attività svolte in aula
Ore di studio autonome
Laboratorio Di Progetto
Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
Disponibilità di supporto didattico in lingua inglese