logo-polimi
Loading...
Risorse bibliografiche
Risorsa bibliografica obbligatoria
Risorsa bibliografica facoltativa
Scheda Riassuntiva
Anno Accademico 2015/2016
Scuola Scuola di Ingegneria Industriale e dell'Informazione
Insegnamento 095966 - STOCHASTIC DYNAMICAL MODELS
Docente Fagnola Franco
Cfu 8.00 Tipo insegnamento Monodisciplinare

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (487) MATHEMATICAL ENGINEERING - INGEGNERIA MATEMATICA*AZZZZ095966 - STOCHASTIC DYNAMICAL MODELS

Programma dettagliato e risultati di apprendimento attesi

Obiettivi e contenuti del corso

The aim of this course is to provide a straightforward and accessible introduction to Markov chains, the simplest mathematical model for random phenomena evolving in time, with applications to queueing models, biological models, Markov chain Monte Carlo, reliability. A brief introduction to martingale theory and methods with applications to Markov chains will also be given.

The only prerequisite is a first course in probability. Knowledge of basic measure theory would be an advantage, but it is not a strict prerequisite.

Descrizione degli argomenti trattati

1. DISCRETE TIME MARKOV CHAINS. Markov chains, classes of states and their structure, periodicity. Stopping times and strong Markov property. Invariant distributions. Recurrence and transience. Hitting times and absorption probabilities. Mean absorption times. Application to ruin problems (ruin probability and mean ruin time). Random walks, recurrence and transience of random walks, binary communication channels. Empirical means and ergodic theorem. Reversibility. Applications to queueing models and population models. Lyapunov functions and Foster criteria. Exponential convergence to invariant distributions and Doeblin condition. Monte Carlo methods, Metropolis algorithm, binomial model.

2. CONTINUOUS TIME MARKOV CHAINS. Consistent families of probability distributions, Kolmogorov’s theorem, canonical processes. Trajectories and modifications. Transition rates, Chapman-Kolmogorov equation and transition semigroup, forward and backward Kolmogorov equations. Transition rate matrices and their exponentials. Markov property and exponential sojourn times, jump chain and holding times of a continuous time Markov chain. Invariant distributions, ergodic theorem and convergence to invariant distributions. Poisson process, independence of increments. Birth and death processes. Non-minimal chains and explosion in finite time. M/M/1 and M/M/k queues and performance indices.

Renewal processes: law of large numbers and central limit theorem. Failure rate and reliability.

3. MARTINGALES. Martingales, supermartingales and submartingales. Modelling a player’s fortune. Filtrations and information. Predictable processes and predictable strategies. Discrete time stochastic integrals and return of a strategy. Stopping theorem. Maximal inequality and Doob inequality. Martingales of a Markov chain, Lyapunov functions and submartingales.

4. RANDOM FIELDS. Markov fields, gaussian fields, applications.  

Results will be presented in a rigorous way. However, only proofs deemed useful for understanding structures or clarifying models and methods will be discussed in detail.

Organizzazione del corso e modalità di verifica

The final exam is made of a preliminary written test, followed by an oral test.

 


Note Sulla Modalità di valutazione

The final exam is made of a preliminary written test, followed by an oral test.


Bibliografia

Mix Forme Didattiche
Tipo Forma Didattica Ore didattiche
lezione
50.0
esercitazione
30.0
laboratorio informatico
0.0
laboratorio sperimentale
0.0
progetto
0.0
laboratorio di progetto
0.0

Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua Inglese
schedaincarico v. 1.6.5 / 1.6.5
Area Servizi ICT
27/09/2020