Parallel and Distributed Machine Learning

Објавено: June 28, 2023
1. Course Title Parallel and Distributed Machine Learning
2. Code 4ФЕИТ07011
3. Study program 8-KM-INN, 19-MV, 21-PNMI 
4. Organizer of the study program (unit, institute, department) Faculty of Electrical Engineering and Information Technologies
5. Degree (first, second, third cycle) Second cycle
6. Academic year/semester I/1   7.    Number of ECTS credits 6.00
8. Lecturer Dr Daniel Denkovski
9. Course Prerequisites
10. Course Goals (acquired competencies):

Knowledge and understanding of the basic concepts in parallel and distributed machine learning. With this course, the student will acquire knowledge about the practical use of advanced tools, techniques and software, as well as the ability to solve practical machine learning problems in parallel and distributed environments.
11. Course Syllabus:

Parallelism in machine learning problems, with a particular focus on deep learning. Graphics processing units (GPU), tensor processing units (TPU) and their practical usage in machine learning. Data parallelism. Data partitioning and distribution, model synchronization and model update. Parallel reduction. The problem of distributed gradient computation. Model parallelism: horizontal (intra-layer model splitting, tensor parallelism) and vertical parallelism (inter-layer model splitting, pipeline parallelism). Advanced concepts of parallel and distributed learning. Federated learning. Preserving security and privacy in federated learning. Model aggregation algorithms. Horizontal, vertical and split federated learning. Personalization of models. Evaluation of performances and effectiveness of parallel and distributed learning.
12. Learning methods:

Lectures, independent work on project tasks and preparation of seminar work
13. Total number of course hours 180
14. Distribution of course hours 3 + 3
15. Forms of teaching 15.1 Lectures-theoretical teaching 45 hours
15.2 Exercises (laboratory, practice classes), seminars, teamwork 45 hours
16. Other course activities 16.1 Projects, seminar papers 30 hours
16.2 Individual tasks 30 hours
16.3 Homework and self-learning 30 hours
17. Grading
17.1 Exams 0 points
17.2 Seminar work/project (presentation: written and oral) 50 points
17.3. Activity and participation 0 points
17.4. Final exam 50 points
18. Grading criteria (points) up to 50 points 5 (five) (F)
from 51 to 60 points 6 (six) (E)
from 61 to 70 points 7 (seven) (D)
from 71 to 80 points 8 (eight) (C)
from 81 to 90 points 9 (nine) (B)
from 91 to 100 points 10 (ten) (A)
19. Conditions for acquiring teacher’s signature and for taking final exam Regular attendance on lectures
20. Forms of assessment

Each student must complete a mandatory project assignment. Final exam with a duration of 90 minutes. The final grade is determined based on the total points from the project assignment and the final exam.
21. Language Macedonian and English
22. Method of monitoring of teaching quality Self-evaluation
23. Literature
23.1.       Required Literature
No. Author Title Publisher Year
1. Guanhua Wang Distributed Machine Learning with Python: Accelerating model training and serving with distributed systems Packt Publishing 2022
2. Heiko Ludwig and Nathalie Baracaldo Federated Learning: A Comprehensive Overview of Methods and Applications Springer 2022
3.
23.2.       Additional Literature
No. Author Title Publisher Year
1.  Arun Kumar Sangaiah  Deep Learning and Parallel Computing Environment for Bioengineering Systems  Elsevier Science  2019