logo-polimi
Loading...
Risorse bibliografiche
Risorsa bibliografica obbligatoria
Risorsa bibliografica facoltativa
Scheda Riassuntiva
Anno Accademico 2018/2019
Scuola Scuola di Ingegneria Industriale e dell'Informazione
Insegnamento 093294 - INFORMATION THEORY
Docente Magarini Maurizio
Cfu 5.00 Tipo insegnamento Monodisciplinare

Corso di Studi Codice Piano di Studio preventivamente approvato Da (compreso) A (escluso) Insegnamento
Ing Ind - Inf (Mag.)(ord. 270) - MI (474) TELECOMMUNICATION ENGINEERING - INGEGNERIA DELLE TELECOMUNICAZIONI*AZZZZ093294 - INFORMATION THEORY
Ing Ind - Inf (Mag.)(ord. 270) - MI (476) ELECTRONICS ENGINEERING - INGEGNERIA ELETTRONICA*AZZZZ093294 - INFORMATION THEORY

Obiettivi dell'insegnamento

This course deals with the theoretical foundations of source coding and of channel coding. Information measures are first introduced, and then applied to the analysis of the theoretical performance achievable in data compression and communication over noisy channels.
The goal of the course is to teach students how the concepts related to the source and channel coding are used to model, analyze and design modern communication systems in order to enable efficient information processing. Since the application of these concepts is a fundamental aspect in every modern communication system, another goal is to establish concrete links of these concepts with advanced technologies used to process the information in different telecommunication systems (wireless and optical).
Some of the topics will be presented using a more practical approach by means of examples built using commercial software tools.


Risultati di apprendimento attesi

1 - Knowledge and understanding
After the end of the course, students should be able to:
- define quantitative measure of information
- know theoretical and methodological skills related to the concept of information
- understand properties of entropy, mutual information as it applied to information
- comprehend what is the rate-distortion theory and its related consequences
2 Applying knowledge and understaning
After the end of the course, students should be able to:
- understand and solve complex problems in different interdisciplinary areas
- integrate and individually find and re-adapt known solutions to growing problems
- analyse uncertainty in terms of information entropy and other information measures
3 - Making judgements
After the end of the course, students should be able to:
- define mathematical models by considering the informational aspects of a system and the related measures
- evaluate concrete schemes for compression and communication


Argomenti trattati

1 - Entropy and source coding

Introduction to information theory. Entropy of a memoryless source. Coding of memoryless sources. Prefix codes. Kraft inequality. Huffman codes and Shannon codes. Source coding theorems (for memoryless sources). Joint entropy and conditional entropy. Chain rules. Entropy of sources with memory. Source coding theorem. Practical methods for source coding. Universal codes. Arithmetic coding. Lempel-Ziv coding.

2 - Channel capacity

Channel models. Discrete channels. Mutual information. Data processing inequality. Channel capacity. Coding of information for transmission on unreliable channels. Entropy, mutual information, and capacity for continuous channels. Gaussian AWGN channel. Channel coding theorem. Error exponent. Fano’s inequality. Converse of the channel coding theorem. Hints for practical channel codes.

3 - Rate distortion theory

Rate-distortion function. Coding of discrete and continuous sources with a fidelity criterion. Vector quantization. Channel coding with a fidelity criterion.

4 - Network information theory

Another look at source coding. Slepian-Wolf source coding. Multiple-access channels. Capacity regions. Gaussian multiple-access channel. Gaussian broadcast channel. Capacity regions.


Prerequisiti

This course requires a basic knowledge in computer science and solid understanding of probability theory and random variables.


Modalità di valutazione

The assessment will consist in a written exam followed by oral examination.
The written part consists of numerical exercises (2/3) and open questions (2/1) on all the topics covered in the course.
The written exam can assign up to 30 points, which constitute the starting point for the final mark after the oral.
The mark 30 cum laude will be assigned when the total score after the oral is greater or equal 31.
Optional projects, proposed during the course, can be selected to get assigned additional 3 points.


Bibliografia
Risorsa bibliografica obbligatoriaLecture notes https://beep.metid.polimi.it/
Risorsa bibliografica facoltativaT. M. Cover, J. A. Thomas, Elements of information theory (1st or 2nd edition), Editore: John Wiley & Sons, 1991 (1st ed.), 2006 (2nd ed.). First edition available online for PoliMi students (http://onlinelibrary.wiley.com/book/10.1002/0471200611)
Risorsa bibliografica facoltativaD. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Editore: D. J. Cambridge University Press, 2003
Risorsa bibliografica facoltativaR. G. Gallager, Information Theory and Reliable Communication, Editore: John Wiley & Sons, 1968

Forme didattiche
Tipo Forma Didattica Ore di attività svolte in aula
(hh:mm)
Ore di studio autonome
(hh:mm)
Lezione
30:00
45:00
Esercitazione
20:00
30:00
Laboratorio Informatico
0:00
0:00
Laboratorio Sperimentale
0:00
0:00
Laboratorio Di Progetto
0:00
0:00
Totale 50:00 75:00

Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua Inglese
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
Disponibilità di supporto didattico in lingua inglese
schedaincarico v. 1.6.1 / 1.6.1
Area Servizi ICT
27/01/2020