Risorse bibliografiche
 Risorsa bibliografica obbligatoria Risorsa bibliografica facoltativa
 Scheda Riassuntiva
 Anno Accademico 2017/2018 Scuola Scuola di Ingegneria Industriale e dell'Informazione Insegnamento 093294 - INFORMATION THEORY Docente Bellini Sandro Cfu 5.00 Tipo insegnamento Monodisciplinare

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (474) TELECOMMUNICATION ENGINEERING - INGEGNERIA DELLE TELECOMUNICAZIONI*AZZZZ093294 - INFORMATION THEORY
Ing Ind - Inf (Mag.)(ord. 270) - MI (476) ELECTRONICS ENGINEERING - INGEGNERIA ELETTRONICA*AZZZZ093294 - INFORMATION THEORY

 Programma dettagliato e risultati di apprendimento attesi
 Aims and learning outcomes This course deals with the theoretical foundations of source coding and of channel coding. Information measures are first introduced, and then applied to the analysis of the theoretical performance achievable in data compression and communication over noisy channels. Syllabus 1 - Entropy and source coding Introduction to information theory. Entropy of a memoryless source. Coding of memoryless sources. Prefix codes. Kraft inequality. Huffman codes and Shannon codes. Source coding theorems (for memoryless sources). Joint entropy and conditional entropy. Chain rules. Entropy of sources with memory. Source coding theorem. Practical methods for source coding. Universal codes. Arithmetic coding. Lempel-Ziv coding. 2 - Channel capacity Channel models. Discrete channels. Mutual information. Data processing inequality. Channel capacity. Coding of information for transmission on unreliable channels. Entropy, mutual information, and capacity for continuous channels. Gaussian AWGN channel. Channel coding theorem. Error exponent. Fano’s inequality. Converse of the channel coding theorem. Hints for practical channel codes. 3 - Rate distortion theory Rate-distortion function. Coding of discrete and continuous sources with a fidelity criterion. Vector quantization. Channel coding with a fidelity criterion. 4 - Network information theory Another look at source coding. Slepian-Wolf source coding. Multiple-access channels. Capacity regions. Gaussian multiple-access channel. Gaussian broadcast channel. Capacity regions. Prerequisites This course requires a solid understanding of probability theory and random variables. Further information Course web page: http://home.deib.polimi.it/bellini/inf_th/inf_th.html Lecture notes are available on this course web page.

 Note Sulla Modalità di valutazione
 Oral examination.

 Bibliografia
 Lecture notes by Sandro Bellini http://http://home.deib.polimi.it/bellini/inf_th/inf_theory.pdf T. M. Cover, J. A. Thomas, Elements of information theory (1st or 2nd edition), Editore: John Wiley & Sons, 1991 (1st ed.), 2006 (2nd ed.). First edition available online for PoliMi students (http://onlinelibrary.wiley.com/book/10.1002/0471200611) D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Editore: D. J. Cambridge University Press, 2003 R. G. Gallager, Information Theory and Reliable Communication, Editore: John Wiley & Sons, 1968

 Mix Forme Didattiche
Tipo Forma Didattica Ore didattiche
lezione
30.0
esercitazione
20.0
laboratorio informatico
0.0
laboratorio sperimentale
0.0
progetto
0.0
laboratorio di progetto
0.0

 Informazioni in lingua inglese a supporto dell'internazionalizzazione
 Insegnamento erogato in lingua Inglese Disponibilità di materiale didattico/slides in lingua inglese Disponibilità di libri di testo/bibliografia in lingua inglese Possibilità di sostenere l'esame in lingua inglese
 schedaincarico v. 1.6.5 / 1.6.5 Area Servizi ICT 17/05/2021