Course Title: Information Theory
ECTS points: 6 ECTS
Number of classes per week:3+0+0+3
Lecturer: Prof. d-r Venceslav Kafedziski
Course Goals (acquired competencies): Upon finishing the course, it is expected that the student will get familiar with the notions and techniques of information theory and their application to solving different problems in information theory, know how to determine limits of compression and transmission rates, get familiar with the connection of information theory with statistics, and be prepared to perform scientific work in the area of information theory
Subject of the course content: Entropy, relative entropy, and mutual information. Asymptotic equipartition property. Entropy rate of random processes. Lossless data compression. Huffman coding. Shannon-Fano-Elias coding. Universal coding of information sources. Arithmetic coding. Lempel-Ziv coding. Capacity of discrete memoryless channels. Capacity of Gaussian memoryless channels. Capacity of vector Gaussian channels. Lossy data compression. Rate distortion theory. Quantization. Rate distortion of Gaussian information sources. Network information theory. Fundamental problems of network information theory and approaches to their solution. Capacity of the multiple access channel and capacity of the broadcast channel. Capacity of the relay channel. Distributed coding of multiple information sources. Information theory and statistics.
|1.||T. M. Cover, J. A. Thomas||Elements of Information Theory||Wiley||2006|