Obiettivi e contenuti del corso
The aim of this course is to provide a straightforward and accessible introduction to Markov chains, the simplest mathematical model for random phenomena evolving in time, with applications to queueing models, biological models, Markov chain Monte Carlo, reliability. A brief introduction to martingale theory and methods with applications to Markov chains will also be given.
The only prerequisite is a first course in probability. Knowledge of basic measure theory would be an advantage, but it is not a strict prerequisite.
Descrizione degli argomenti trattati
1. DISCRETE TIME MARKOV CHAINS. Markov chains, classes of states and their structure, periodicity. Stopping times and strong Markov property. Invariant distributions. Recurrence and transience. Hitting times and absorption probabilities. Mean absorption times. Application to ruin problems (ruin probability and mean ruin time). Random walks, recurrence and transience of random walks, binary communication channels. Empirical means and ergodic theorem. Reversibility. Applications to queueing models and population models. Lyapunov functions and Foster criteria. Exponential convergence to invariant distributions and Doeblin condition. Monte Carlo methods, Metropolis algorithm, binomial model.
2. CONTINUOUS TIME MARKOV CHAINS. Consistent families of probability distributions, Kolmogorov’s theorem, canonical processes. Trajectories and modifications. Transition rates, Chapman-Kolmogorov equation and transition semigroup, forward and backward Kolmogorov equations. Transition rate matrices and their exponentials. Markov property and exponential sojourn times, jump chain and holding times of a continuous time Markov chain. Invariant distributions, ergodic theorem and convergence to invariant distributions. Poisson process, independence of increments. Birth and death processes. Non-minimal chains and explosion in finite time. M/M/1 and M/M/k queues and performance indices.
Renewal processes: law of large numbers and central limit theorem. Failure rate and reliability.
3. MARTINGALES. Martingales, supermartingales and submartingales. Modelling a player’s fortune. Filtrations and information. Predictable processes and predictable strategies. Discrete time stochastic integrals and return of a strategy. Stopping theorem. Maximal inequality and Doob inequality. Martingales of a Markov chain, Lyapunov functions and submartingales.
4. RANDOM FIELDS. Markov fields, gaussian fields, applications.
Results will be presented in a rigorous way. However, only proofs deemed useful for understanding structures or clarifying models and methods will be discussed in detail.
Organizzazione del corso e modalità di verifica
The final exam is made of a preliminary written test, followed by an oral test.