L'insegnamento prevede 1.0 CFU erogati con Didattica Innovativa come segue:
Cotutela con mondo esterno
Corso di Studi
Codice Piano di Studio preventivamente approvato
Ing Ind - Inf (Mag.)(ord. 270) - MI (474) TELECOMMUNICATION ENGINEERING - INGEGNERIA DELLE TELECOMUNICAZIONI
054322 - INFORMATION THEORY
This course is about how to measure, represent, and communicate information effectively. Why bits have become the universal currency for information exchange. How information theory bears on the design and operation of modern-day systems such as smartphones and the Internet. Information measures are first introduced, and then applied to the analysis of the theoretical performance achievable in data compression and communication over noisy channels. The goal of the course is to teach students how the concepts related to the source coding and channel capacity are used to model, analyze and design modern communication systems in order to enable efficient information processing. Since the application of these concepts is a fundamental aspect in every modern communication system, another goal is to establish concrete links of these concepts with advanced technologies used to process the information in different systems. To this aim, a project will be assigned to apply the concepts and principles learned during the lessons.
Risultati di apprendimento attesi
1 - Knowledge and understanding After the end of the course, students should be able to: - define quantitative measure of information - know theoretical and methodological skills related to the concept of information - understand properties of entropy, mutual information as it applied to information processing - define the multi-terminal source and channel coding problems and describe the coding theorems concerning distributed compression, multiple access, and broadcast.
2 - Applying knowledge and understaning After the end of the course, students should be able to: - understand and solve complex problems in different interdisciplinary areas - integrate and individually find and re-adapt known solutions to growing problems - analyse uncertainty in terms of information entropy and other information measures - summarize and present the results achieved during the analysis and specification activities
3 - Making judgements After the end of the course, students should be able to: - define mathematical models by considering the informational aspects of a system and the related measures - evaluate concrete schemes for compression and communication
1 – Introduction to information theory
Historical background. Introduction to information theory and its applications.
2 - Entropy and source coding
Entropy of a memoryless source. Coding of memoryless sources. Prefix codes. Kraft inequality. Huffman codes and Shannon codes. Source coding theorems (for memoryless sources). Joint entropy and conditional entropy. Chain rules. Entropy of sources with memory. Source coding theorem. Practical methods for source coding. Universal codes. Arithmetic coding.
3 - Channel capacity
Channel models. Discrete channels. Mutual information. Data processing inequality. Channel capacity. Coding of information for transmission on unreliable channels. Entropy, mutual information, and capacity for continuous channels. Gaussian AWGN channel. Channel coding theorem. Error exponent. Fano’s inequality. Converse of the channel coding theorem.
4 - Network information theory
Another look at source coding. Slepian-Wolf source coding. Multiple-access channels. Capacity regions. Gaussian multiple-access channel. Gaussian broadcast channel. Capacity regions.
5 - Project on selected topics in information theory
A project laboratory, which is integral part of the course, will be assigned during the semester. This activity, which constitutes the innovative teaching part of the course, will be done in cooperation with invited companies that will suggest projects related to the application of information theory in their specific field. The objective of projects is to help students in applying the concepts and principles learned during the lessons. The project, to be done in groups of two/three students, will be assigned in the first weeks of the semester. Project artifacts are expected to be released at the end of the course. The evaluation of projects will be based on the produced artifacts (documentation, code, …) and on a final presentation. The project laboratory will include class discussions supervised by the course instructors and researchers from invited companies.
This course requires a basic knowledge in computer science and solid understanding of probability theory and random variables.
Modalità di valutazione
The assessment will consist in a written exam and a presentation of the project laboratory. The written part consists of numerical exercises (2/3) and open questions (2/1) on all the topics covered in the course. The written exam can assign up to 23 points, which constitute the starting point for the final mark after the presentation of the project laboratory. The project can assign up to 10 points. The mark 30 cum laude will be assigned when the total score after the presentation of the project laboratory is equal to 33.
Lecture noteshttps://beep.metid.polimi.it/T. M. Cover, J. A. Thomas, Elements of information theory (1st or 2nd edition), Editore: John Wiley & Sons, 1991 (1st ed.), 2006 (2nd ed.). First edition available online for PoliMi students (http://onlinelibrary.wiley.com/book/10.1002/0471200611)
D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, Editore: D. J. Cambridge University Press, 2003
R. G. Gallager, Information Theory and Reliable Communication, Editore: John Wiley & Sons, 1968
Nessun software richiesto
Tipo Forma Didattica
Ore di attività svolte in aula
Ore di studio autonome
Laboratorio Di Progetto
Informazioni in lingua inglese a supporto dell'internazionalizzazione
Insegnamento erogato in lingua
Disponibilità di materiale didattico/slides in lingua inglese
Disponibilità di libri di testo/bibliografia in lingua inglese
Possibilità di sostenere l'esame in lingua inglese
Disponibilità di supporto didattico in lingua inglese