National Computing Education Accreditation Council
NCEAC
NCEAC.DOC.001-E
Course Log Template
COMPUTER SCIENCE, THE UNIVERSITY OF MIRPURKHAS
INSTITUTION
PROGRAM (S) TO BACHELOR OF SCIENCE (COMPUTER SCIENCE)
BE
EVALUATED
Course Name Parallel and Distributed Computing Cr. Hrs. 3
Catalog Number CC-510
Instructor Name Areeba Bashir Chachar | Teaching Assistant Computer Science
S.N Date Duration Topics Covered Evaluation Signatur
o Instruments e
used
1 Week 1 Introduction to Parallel and Homework
Distributed Computing
Overview of parallel vs. distributed
computing, basic principles,
architecture types
2 Week 2 Asynchronous and Synchronous Homework
Computation/Communication
Synchronous vs. Asynchronous
communication models, examples,
trade-offs, message passing, barriers
3 Week 3 Concurrency Control Reading
Assignment
Race conditions, mutual exclusion,
locks, semaphores, deadlock
prevention, synchronization
primitives
4 Week 4 Fault Tolerance in Parallel Systems Assignment
Techniques for fault detection and
recovery, redundancy, replication,
checkpointing
Page 1 of 4 NCEAC.DOC.001-E
National Computing Education Accreditation Council
NCEAC
NCEAC.DOC.001-E
5 Week 5 GPU Architecture and Homework
Programming
GPU architecture basics, CUDA
programming, parallelism in GPUs,
GPU memory model, performance
considerations
6 Week 6 Heterogeneity in Parallel Systems Reading
Assignments
Heterogeneous architectures
(CPU+GPU, clusters), handling
diverse resources, resource
management
7 Week 7 Interconnection Topologies Homework
Mesh, torus, tree, hypercube, and
other interconnection topologies, their
implications on communication
8 Week 8 Load Balancing in Parallel Systems Subject-
specific
research
Load balancing strategies, static vs.
dynamic balancing, task partitioning,
workload distribution
9 Week 9 Memory Consistency Model and Assignments
Hierarchies
Cache coherence, consistency models
(Sequential consistency, Relaxed
consistency), memory hierarchies
10 Week 10 Message Passing Interface (MPI) Class Test
MPI basics, point-to-point
communication, collective
communication, MPI libraries,
practical use-cases
11 Week 11 MIMD/SIMD Architectures Assignment
MIMD vs. SIMD, architectures and
programming paradigms, vector
processing, pipelining
Page 2 of 4 NCEAC.DOC.001-E
National Computing Education Accreditation Council
NCEAC
NCEAC.DOC.001-E
12 Week 12 Multithreaded Programming Assignments
Thread creation, synchronization,
thread-safe programming, shared
memory programming (OpenMP,
pthreads)
13 Week 13 Parallel Algorithms and Report
Architectures Writing
Designing parallel algorithms,
Amdahl's Law, work-efficiency trade-
offs, parallel vs. sequential algorithms
14 Week 14 Parallel I/O and Storage Systems Presentation
Parallel file systems, I/O bottlenecks,
storage systems (HDFS, Ceph),
parallel I/O libraries
15 Week 15 Performance Analysis, Tuning, and Assignments
Power Efficiency
Profiling tools, performance
bottlenecks, optimization techniques,
energy and power considerations
16 Week 16 Programming Models and Report
Scalability Studies Writing
Data parallelism, task parallelism,
process-centric, shared/distributed
memory, scalability, and performance
studies
Instructor Signature ____________________________
Date ____________________________
Page 3 of 4 NCEAC.DOC.001-E
National Computing Education Accreditation Council
NCEAC
NCEAC.DOC.001-E
CC-510 Parallel and distributed Computing Credit Hours: 3
Prerequisite. Operating Systems
Course Learning Outcomes (CLOs).
At the end of the course the students will be able to:
1. Learn about parallel and distributed computers.
2. Write portable programs for parallel or distributed architectures using Message-Passing
Interface (MPI) library
3. Analytical modelling and performance of parallel programs.
4. Analyze complex problems with shared memory programming with openMP.
Course Contents.
Asynchronous/synchronous computation/communication, concurrency control, fault tolerance,
GPU architecture and programming, heterogeneity, interconnection topologies, load balancing,
memory consistency model, memory hierarchies, Message passing interface (MPI),
MIMD/SIMD, multithreaded programming, parallel algorithms & architectures, parallel I/O,
performance analysis and tuning, power, programming models (data parallel, task parallel,
process-centric, shared/distributed memory), scalability and performance studies, scheduling,
storage systems, synchronization, and tools (Cuda, Swift, Globus, Condor, Amazon AWS,
OpenStack, Cilk, gdb, threads, MPICH, OpenMP, Hadoop, FUSE)
Teaching Methodology.
Lectures, Assignments, Practical labs, Semester Project, Presentations
Reference Materials.
1. Distributed Systems: Principles and Paradigms, A. S. Tanenbaum and M. V. Steen,
Prentice Hall, 2nd Edition.
2. Distributed and Cloud Computing: Clusters, Grids, Clouds, and the Future Internet, K
Hwang, J Dongarra and GC. C. Fox, Elsevier, 1st Ed.
Page 4 of 4 NCEAC.DOC.001-E
