Information Theory

Објавено: October 12, 2018
  1.    Course Title Information Theory
  2.    Code 3ФЕИТ10З040
  3.    Study program KHIE, TKII
  4.    Organizer of the study program (unit, institute, department) Faculty of Electrical Engineering and Information Technologies
  5.    Degree (first, second, third cycle) First cycle
  6.    Academic year/semester II/3, III/5   7.    Number of ECTS credits 6.00
  8.    Lecturer Dr Aleksandar Risteski
  9.    Course Prerequisites Taken course: Mathematics 2

10.    Course Goals (acquired competencies):  Knowledge of properties of random signals, their auto-correlation functions and spectra and their transmission through telecommunication systems. Setting a statistical model for the basic components for information transmission and processing through a telecommunications system.

11.    Course Syllabus: Introduction. Probability. Random variables. Description of random variables. Functional transformation of random variables. Statistical ensemble of random signals. Statistical mean values and their physical interpretation. Basic types of distribution of random variables. Distribution of sum and product of random variables. Correlation functions and spectra of random signals. Definitions and properties. Wiener-Khinchine’s theorem. Procedures for the experimental determination of correlations and spectra of random signals. Correlations and spectra of selected random signals. White Gaussian noise. Transmission of random signals through a linear transmission system. General statistical model of communication system. Definition of information. Information sources. Entropy. Types of information sources and their entropy. Information flux. Source coding. Principles and basic characteristics. The basic theorem of source coding. First Shannon theorem. Procedures for optimal coding (Fano, Huffman). Efficiency of entropic coding. Statistical model of the transmission communication channel. Mutual information. Channel capacity. Properties of symmetrical channels. Confidentiality of the channel transmission. Probability of error. Statistical theory of decision making. Optimum decision rule. Decision criteria (Bayes, minmax, Neyman-Pearson). Channel coding. Principles and basic characteristics. Second Shannon theorem. Basic examples.

12.    Learning methods:  Lectures, tutorial and laboratory classes, individual student projects and seminar works.
13.    Total number of course hours 3 + 1 + 1 + 0
14.    Distribution of course hours 180
15.    Forms of teaching 15.1. Lectures-theoretical teaching 45
15.2. Exercises (laboratory, practice classes), seminars, teamwork 30
16.    Other course activities 16.1. Projects, seminar papers 10
16.2. Individual tasks 10
16.3. Homework and self-learning 85
17.    Grading 17.1. Exams 10
17.2. Seminar work/project (presentation: written and oral)
17.3. Activity and participation
17.4. Final exam 90
18.    Grading criteria (points) up to 50 points     5 (five) (F)
from 51 to 60 points     6 (six) (E)
from 61 to 70 points     7 (seven) (D)
from 71 to 80 points     8 (eight) (C)
from 81 to 90 points     9 (nine) (B)
from 91 to 100 points   10 (ten) (A)
19.    Conditions for acquiring teacher’s signature and for taking final exam Regular participation to lectures and tutorial classes and completion of all laboratory exercises.
20.  Forms of assessment During the semester, two partial written exams are taken (at the middle and at the end of the semester, with a duration of up to 90 minutes) and tests , which are conducted during the classes. The final grade includes the points from the partial exams and tests.
During the exam sessions a written exam is taken (with a duration of up to 120 minutes). The final grade includes points form the exam tests.
A special instruction published before each exam regulates the manner of taking the exam and the use of teaching aids and electronic devices during the exam
21.  Language Macedonian and English
22.  Method of monitoring of teaching quality Internal evaluation and polls.