Information Theory

Објавено: June 28, 2022
1. Course Title Information Theory
2. Code 4ФЕИТ10З029
3. Study program ТКИИ,КХИЕ
4. Organizer of the study program (unit, institute, department) Faculty of Electrical Engineering and Information Technologies
5. Degree (first, second, third cycle) First cycle
6. Academic year/semester III/5 7. Number of ECTS credits 6
8. Lecturer D-r Aleksandar Risteski
9. Course Prerequisites Passed: Mathematics 2
10. Course Goals (acquired competencies): By passing the course, students will be familiar with the properties of random signals, their autocorrelation functions and spectra and will be able to calculate them. They will be able to calculate how the properties of random signals change during their transmission through telecommunication systems. Students will learn the basic statistical model of a telecommunications system in terms of processing and transmission of information, with the necessary theoretical foundations. They will know how to calculate entropy and flux of an information source, apply entropy coding algorithms and calculate its performance, calculate transmission channel capacity, apply basic protection codes and calculate parameters for optimal decision making in the receiver for minimizing bit error probability.
11. Course Syllabus: Introduction. Overview of topics on probability, random variables, functional transformation, statistical mean values and their interpretation, basic types of distributions, distribution of sum and product of random variables and their application to random signals for information transmission. Statistical ensemble of random signals. Correlation functions and spectra. Wiener-Khinchine’s theorem. Principles for measurement of correlations and spectra. Examples. White Gaussian noise. Transmission of random signals through a linear transmission system. General statistical model of communication system. Definition of information. Information sources. Entropy. Information flux. Source coding. The basic theorem of source coding. First Shannon theorem. Procedures for optimal coding (Fano, Huffman). Efficiency. Statistical model of the transmission communication channel. Mutual information. Channel capacity. Properties of symmetrical channels. Confidentiality of the channel transmission. Probability of error. Channel coding. Principles and basic characteristics. Second Shannon theorem. Basic examples. Statistical theory of decision making. Optimum decision rule. Decision criteria (Bayes, minmax, Neyman-Pearson).
12. Learning methods: Lectures, tutorial and laboratory classes, individual student projects and seminar works.
13. Total number of course hours 3 + 1 + 1 + 0
14. Distribution of course hours 180
15. Forms of teaching 15.1. Lectures-theoretical teaching 45
15.2. Exercises (laboratory, practice classes), seminars, teamwork 30
16. Other course activities 16.1. Projects, seminar papers 10
16.2. Individual tasks 10
16.3. Homework and self-learning 85
17. Grading 17.1. Exams 10
17.2. Seminar work/project (presentation: written and oral) 0
17.3. Activity and participation 0
17.4. Final exam 90
18. Grading criteria (points) up to 50 points 5 (five) (F)
from 51to 60 points 6 (six) (E)
from 61to 70 points 7 (seven) (D)
from 71to 80 points 8 (eight) (C)
from 81to 90 points 9 (nine) (B)
from 91to 100 points 10 (ten) (A)
19. Conditions for acquiring teacher’s signature and for taking final exam Regular participation to lectures and tutorial classes and completion of all laboratory exercises.
20. Forms of assessment During the semester, two partial written exams are taken (at the middle and at the end of the semester, with a duration of up to 90 minutes) and tests , which are conducted during the classes. The final grade includes the points from the partial exams and tests.
During the exam sessions a written exam is taken (with a duration of up to 120 minutes). The final grade includes points form the exam tests.
A special instruction published before each exam regulates the manner of taking the exam and the use of teaching aids and electronic devices during the exam.
21. Language Macedonian and English
22. Method of monitoring of teaching quality Internal evaluation and polls.
23. Literature
23.1. Required Literature
No. Author Title Publisher Year
1 Tatjana Ulcar-Stavrova Teorija na informacii ETF Skopje 1995
2 Aleksandar Risteski Information Theory Lecture Notes Internal Lecture Notes 2021
3 Збирка решени задачи по Теорија на информации Интерна скрипта NULL