logo-polimi
Loading...
Risorse bibliografiche
Risorsa bibliografica obbligatoria
Risorsa bibliografica facoltativa
Scheda Riassuntiva
Anno Accademico 2017/2018
Scuola Scuola di Ingegneria Industriale e dell'Informazione
Insegnamento 093294 - INFORMATION THEORY
Docente Bellini Sandro
Cfu 5.00 Tipo insegnamento Monodisciplinare

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (474) TELECOMMUNICATION ENGINEERING - INGEGNERIA DELLE TELECOMUNICAZIONI*AZZZZ093294 - INFORMATION THEORY
Ing Ind - Inf (Mag.)(ord. 270) - MI (476) ELECTRONICS ENGINEERING - INGEGNERIA ELETTRONICA*AZZZZ093294 - INFORMATION THEORY

Programma dettagliato e risultati di apprendimento attesi

Aims and learning outcomes

This course deals with the theoretical foundations of source coding and of channel coding. Information measures are first introduced, and then applied to the analysis of the theoretical performance achievable in data compression and communication over noisy channels.

Syllabus

1 - Entropy and source coding

Introduction to information theory. Entropy of a memoryless source. Coding of memoryless sources. Prefix codes. Kraft inequality. Huffman codes and Shannon codes. Source coding theorems (for memoryless sources). Joint entropy and conditional entropy. Chain rules. Entropy of sources with memory. Source coding theorem. Practical methods for source coding. Universal codes. Arithmetic coding. Lempel-Ziv coding.

2 - Channel capacity

Channel models. Discrete channels. Mutual information. Data processing inequality. Channel capacity. Coding of information for transmission on unreliable channels. Entropy, mutual information, and capacity for continuous channels. Gaussian AWGN channel. Channel coding theorem. Error exponent. Fano’s inequality. Converse of the channel coding theorem. Hints for practical channel codes.

3 - Rate distortion theory

Rate-distortion function. Coding of discrete and continuous sources with a fidelity criterion. Vector quantization. Channel coding with a fidelity criterion.

4 - Network information theory

Another look at source coding. Slepian-Wolf source coding. Multiple-access channels. Capacity regions. Gaussian multiple-access channel. Gaussian broadcast channel. Capacity regions.

Prerequisites

This course requires a solid understanding of probability theory and random variables.

Further information

Course web page: http://home.deib.polimi.it/bellini/inf_th/inf_th.html

Lecture notes are available on this course web page.


Note Sulla Modalità di valutazione

Oral examination.


Bibliografia
Risorsa bibliografica obbligatoriaLecture notes by Sandro Bellini http://http://home.deib.polimi.it/bellini/inf_th/inf_theory.pdf
Risorsa bibliografica facoltativaT. M. Cover, J. A. Thomas, Elements of information theory (1st or 2nd edition), Editore: John Wiley & Sons, 1991 (1st ed.), 2006 (2nd ed.). First edition available online for PoliMi students (http://onlinelibrary.wiley.com/book/10.1002/0471200611)
Risorsa bibliografica facoltativaD. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Editore: D. J. Cambridge University Press, 2003
Risorsa bibliografica facoltativaR. G. Gallager, Information Theory and Reliable Communication, Editore: John Wiley & Sons, 1968

Mix Forme Didattiche
Tipo Forma Didattica Ore didattiche
lezione
30.0
esercitazione
20.0
laboratorio informatico
0.0
laboratorio sperimentale
0.0
progetto
0.0
laboratorio di progetto
0.0

Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua Inglese
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
schedaincarico v. 1.6.5 / 1.6.5
Area Servizi ICT
17/05/2021