Information Theory

Course Title: Information Theory

Code: ФЕИТ10030

ECTS points: 6 ECTS

Number of classes per week:3+0+0+3

Lecturer: Prof. Dr.Venceslav Kafedjiski

Course Goals (acquired competencies): Upon finishing the course, it is expected that the student will get familiar with the notions and techniques of information theory and their application to solving different problems in information theory, know how to determine limits of compression and transmission rates, get familiar with  the connection of information theory with statistics, and be prepared to perform scientific work in the area of information theory

Subject of the course content: Entropy, relative entropy, and mutual information. Asymptotic equipartition property. Entropy rate of random processes. Lossless data compression. Huffman coding. Shannon-Fano-Elias coding. Universal coding of information sources. Arithmetic coding. Lempel-Ziv coding. Capacity of discrete memoryless channels. Capacity of Gaussian memoryless channels. Capacity of vector Gaussian channels. Lossy data compression. Rate distortion theory. Quantization. Rate distortion of Gaussian information sources. Network information theory. Fundamental problems of network information theory and approaches to their solution. Capacity of the multiple access channel and capacity of the broadcast channel. Capacity of the relay channel. Distributed coding of multiple information sources. Information theory and statistics.

Literature: