L'insegnamento prevede 1.0 CFU erogati con Didattica Innovativa come segue:

Blended Learning & Flipped Classroom

Soft Skills

Corso di Studi

Codice Piano di Studio preventivamente approvato

Da (compreso)

A (escluso)

Insegnamento

Ing Ind - Inf (Mag.)(ord. 270) - MI (487) MATHEMATICAL ENGINEERING - INGEGNERIA MATEMATICA

*

A

ZZZZ

095966 - STOCHASTIC DYNAMICAL MODELS

054074 - STOCHASTIC DYNAMICAL MODELS

Obiettivi dell'insegnamento

The aim of this course is to provide a simple and accessible introduction to discrete and time-continuous Markov chains, the simplest mathematical model for random phenomena evolving in time, with applications to queueing models, biological models, Markov chain Monte Carlo, reliability. Moreover, a wide variety of models will be illustrated to provide the student with useful tools to apply the methods in new situations. A brief introduction to martingale theory and methods with applications to Markov chains will also be given.

Lectures and exercise sessions will allow students to:

• Analyze and understand the goals, assumptions and requirements associated with Markovian models in discrete and continuous time • Identify situations in which the Markov property may hold or not • Learn the most relevant models (random walks, birth and death processes, random queues) • Learn methods for computing the relevant probabilities and mean times • Study the behaviour of the random evolution

Risultati di apprendimento attesi

Learn the fundamental mathematical models with discrete and continuous time Markov chains and martingale methods. Learn the constructive mathematical proofs of some key result of the theory. Understand the key structures underlying Markov chain models.

Analyze random phenomena discriminating those that can be described by Markov chains and design the corresponding models with a correct scientific approach.Develop new ideas and solutions to find quantitative rigorous solutions in new situations creating new models.

Acquire the ability to present in a clear and precise way the models and results of the theory.

Argomenti trattati

1. DISCRETE TIME MARKOV CHAINS. Transition kernels. Markov chains, classes of states and their structure, irreducibility, periodicity, transience and recurrence. Random walks, recurrence and transience of random walks, binary communication channels. Invariant distributions (discrete and continuous state space). Recurrence and transience. Harris recurrence. Stopping times and strong Markov property. Hitting times and absorption probabilities. Mean absorption times. Application to ruin problems (ruin probability and mean ruin time). Empirical means and ergodic theorem. Reversibility. Applications to queueing models and population models. Lyapunov functions and Foster criteria. Exponential convergence to invariant distributions and Doeblin condition. Monte Carlo methods, Metropolis algorithm, binomial model.

2. CONTINUOUS TIME MARKOV CHAINS. Consistent families of probability distributions, Kolmogorov’s theorem, canonical processes. Trajectories and modifications. Transition rates, Chapman-Kolmogorov equation and transition semigroup, forward and backward Kolmogorov equations. Transition rate matrices and their exponentials. Markov property and exponential sojourn times, jump chain and holding times of a continuous time Markov chain. Invariant distributions, ergodic theorem and convergence to invariant distributions. Poisson process, independence of increments. Birth and death processes. Non-minimal chains and explosion in finite time. M/M/1 and M/M/k queues and performance indices. Epidemic models.

Renewal processes: law of large numbers and central limit theorem. Failure rate and reliability.

3. MARTINGALES. Martingales, supermartingales and submartingales. Modelling a player’s fortune. Filtrations and information. Predictable processes and predictable strategies. Discrete time stochastic integrals and return of a strategy. Stopping theorem. Maximal inequality and Doob inequality. Martingales of a Markov chain, Lyapunov functions and submartingales.

Prerequisiti

The only prerequisite is a first course in probability. Knowledge of basic measure theory would be an advantage, but it is not a prerequisite in the strict sense.

Modalità di valutazione

The final exam is made of a preliminary written test, followed by an oral test. In both stages of the exam the student will have to demonstrate: knowledge of definitions and concept and some of the main constructive mathematical proofs, skills in the identification and construction of Markov chain models in real-world situations, critical reasoning skills in the application of the learnt methods.

The written test usually consists of 2 exercises, one on discrete time and the other on continuous time Markov chains, each one containing 4 – 5 questions. The candidate will be usually asked to construct the mathematical model that he will analyze to obtain quantitative information (probabilities, mean times …).

The oral exam consists of questioning on topics of the course. The candidate will be asked to illustrate some results with proofs from a part of the programme chosen by the candidate himself among: 1 discrete time Markov chains till the strong Markov property included, 2 discrete time Markov processes from hitting times and absorption probabilities, 3 continuous time Markov chains, 4 conditional expectation and martingales. Moreover, the candidate is supposed to be able to present rigorously, without proofs, all the results and models.