Defence University, College of Engineering
Department of Computer Science and Engineering
Course Syllabus
1. Instructor Information
Name
Office Location
Phone Number
E-mail
Consultation / Office Hours
2. Technical Assistant Contact Information
Name / Phone Number
E-mail
3. Course Information
Course Name Parallel Computer Architectures &
Programming
Course Code
Credit hours C Le T La
3 2 0 3
Pre-requisite ECCS 3212, ECCS 4221
Course Status Major Course
4. Course Description
Aim: To increase available computation power for faster application is processing and
problem solving
Description
Introduction: Overview of parallel architectures, Levels of parallelism (instruction,
transaction, task, thread, machine); Parallel computer architectures: Shared memory
systems and cache coherency, Scalable distributed memory multiprocessors,
Heterogeneous systems; Parallel programing models: message passing scalable systems
(introduction to programming with MPI), shared memory systems (introduction to parallel
programming with POSIX threads and OpenMP), Heterogeneous data-parallel systems
(Introduction to CUDA); Parallel algorithm design.
5. Method of Instruction
Lectures This course has 16 weeks of lectures.
Each week has 2 hours lecturing.
Class delivery is supported by teaching aids like
the beaming of PPT slides, demonstration
important programs/tools/experiments.
Every student need to attentively attend all the
class and collect all supporting material for each
class session in advance.
Lecturer should prepare the chapter slides in
advance and should be distributed to the students
beforehand.
Every student should maintain a notebook for the
course and should be checked periodically.
For practical sessions students need to maintain a
1
separate book and should be evaluated
periodically. Well maintained book with proper
algorithm for the problem given, the complete
program with comments and results achieved for
various classes of inputs be given appropriate
credit by allocating a minor percentage of mark
optionally.
In-class Tutorial Every assignment problem should be given in
time and student should prepare and submit it on
time
Every tutorial portion should be prepared by
identifying the topics to be covered after
discussion with the students.
During tutorial sessions, the lecturers should
solve more example problems to train each of the
students in the class in respective areas.
Study of lecture notes Detailed notes should be prepared on all the
topics of the syllabus and should be supplied as
supporting material.
Lecture notes supplied should be properly read
by the students in time.
Demonstrations All programs and tools should be demonstrated
by the instructor and should assignments to
further strengthen the knowledge of students in
the respective areas.
Lab Assignments / Practice Lab manual has to be prepared and should be
given to the students well in advance
Students should come prepare to lab for the
assigned programming and related works
Every completed assignment should be
recorded in the report and should be submitted
to the instructor and has to be approved by the
instructor before the conduction of the
subsequent lab works.
Instructor should prepare a template for
recording program outputs and procedures.
Every student should follow the instructions of
the lectures in the conduction of the
experiments.
Team Project / Mini-Project / A group assignment has to be conducted by the
Group Assignment students.
A group should not contain more than 4
students.
All group assignment problems should be
approved by the sections.
Group assignment problems should be
submitted to the student in the first four weeks
of the semester.
2
Evaluation of the assignments should be
conducted during the last week of the semester.
Every assignment should be supported by a well
written report.
Role of each group member should be clearly
defined. Hence every individual in the group
should be evaluated systematically.
All assignments should present and
demonstrated to the concerned instructor or to a
committee formed by the department.
Quizzes/ Individual Assignments At least two tests should be conducted in the
semester. One test can be before the mid
examination and the other has to be conducted
after the mid exam.
Practical intensive courses can have tests based
on the practical assignments.
6. Learning Outcomes
After the completion of the course the student will have the following attributes:
6.1 Knowledge
6.1.1 Knowledge:
To gain a comprehensive understanding of the fundamental concepts on how
to design and program computer systems that can execute multiple tasks or
processes simultaneously, covering topics such as parallel computer
architectures, programming models, algorithms, languages, performance
analysis, and tools.
Comprehension:
To gain a deep understanding on parallel architectures and to develop
programs and design computer systems that can execute multiple tasks or
processes simultaneously using parallel computing techniques.
Application:
To gain hands-on experience in designing, implementing, and evaluating
architectures to solve complex problems in various domains such as
scientific computing, financial modeling, machine learning, and computer
graphics where high-performance computing is essential.
Analysis:
To measure and optimize the performance of parallel architectures and
programs and computing systems, using techniques such as profiling,
benchmarking, and performance modeling.
Synthesis:
To develop the ability in designing and implementing effective parallel
programs by integrating knowledge of parallel computer architectures,
programming models, algorithms, languages, performance analysis, and
tools.
Evaluation: Help elaborate the capability of the students' understanding and
proficiency in designing, implementing, and optimizing parallel
architectures and programs, using criteria such as code correctness,
performance, scalability, and usability.
3
6.2 Practical skills
6.2.1 Perception:
To perceive components and steps in various parallel computer architectures,
programming models, algorithms and to have them customized for
application domains.
Set:
To setup experimentation platforms to develop and test parallel computer
architectures, programming models, algorithms with appropriate
benchmarking tools and platforms.
Guided Response:
Follow the steps to design and develop parallel programming models,
algorithms models and to test them.
6.3 Attitude and behavior
6.3.1 Receiving:
Listen the concepts of parallel architectures and to develop programs and
design computer systems that can execute multiple tasks or processes
simultaneously using parallel computing techniques.
Responding:
Responding with questions, new ideas, concepts, models, etc. in order to
fully understand the concepts.
Valuing:
Appreciate the role of parallel architectures and algorithms in various
domains such as scientific computing, machine learning, and computer
graphics where high-performance computing is essential.
Organizing:
Customize algorithms and place at right components of parallel architectures.
Internalizing:
Internalize the concepts in sufficient detail to develop efficient designs.
7. Course Outline
Chapter Week Topics to be covered Learning Assignments/activity
Outcomes (Tutorial hours)
(Lecture hours)
Overview of parallel architectures
Parallel Architecture Models
Levels of parallelism
1. Introduction to o instruction,
Parallel Computer 1 o transaction,
Architectures (2 Hrs.)
o task,
o thread,
o machine
2. Parallel Computer Shared memory systems
Architectures (4 Hrs.) 2 cache coherency
3 Scalable distributed memory
multiprocessors,
Heterogeneous systems
4
4
Approval of Group
5 Assignments
Generalized structure of a parallel
computer
Classification of parallel
computers
o Flynn’s classification
o Coupling between processing
elements
5 o Classification based on mode
of accessing memory
o Classification based on grain
size
Vector computers
Array processors
Systolic array processors
Shared memory parallel
computers
o Synchronization of
processes in shared memory
computers
o Shared bus architecture
o Cache coherence in shared
6 bus multiprocessor
4. Structure of o MESI cache coherence
Parallel Computers (9 protocol
Hrs.) o MOESI protocol
o Memory consistency models
o Shared memory parallel
computer using an
interconnection network
Interconnection networks
o Networks to interconnect
processors to memory or
computers to Computers
o Direct interconnection of
computers
o Routing techniques for
directly connected
multicomputer systems
7 Distributed shared memory
parallel computers
Cache coherence in DSM
Message passing parallel
computers
Computer cluster
o Computer cluster using
system area networks
o Computer cluster applications
Warehouse scale computing
5
A generalized structure of Chip
Multiprocessors
MultiCore Processors or Chip
MultiProcessors (CMPs)
Cache Coherence in Chip
Multiprocessor
ARM Cortex A9 and Intel i7
Multicore Processors
Chip Multiprocessors using
6. Core Level Parallel Interconnection Networks
Processing (3 Hrs.) 8 o Ring Interconnection of
Processors
o Ring Bus Connected Chip
Multiprocessors
o Intel Xeon Phi Coprocessor
Architecture
o Mesh Connected Many Core
Processors
o Intel Teraflop Chip
General Purpose Graphics
Processing Unit (GPGPU)
9 Mid Examination Period
Grid Computing
o Enterprise Grid
Cloud computing
o Virtualization
o Cloud Types
o Cloud Services
6. 6. Grid and Cloud
10 o Advantages of Cloud
Computing (3 Hrs.)
Computing
o What has Led to the
Acceptance of Cloud
Computing
o Applications Appropriate
for Cloud Computing
6
Parallel Algorithms
Models of Computation
o The Random-
Access Machine
(RAM)
o The Parallel
Random-Access
Machine
(PRAM)
o Interconnection
Networks
11 o Combinational
Circuits
Analysis of Parallel Algorithms
o Running Time
o Number of Processors
o Cost
Prefix Computation
o Prefix Computation on the
7. Parallel Algorithms
PRAM
(6 Hrs.)
o Prefix Computation on a
Linked List
Sorting
o Combinational Circuits for
Sorting
o Sorting on PRAM Models
o Sorting on Interconnection
Networks
Searching
o Searching on PRAM Models
12 Analysis
o Searching on Interconnection
Networks
Matrix operations
Practical Models Of Parallel
Computation
o Bulk Synchronous Parallel
(BSP) Model
o LogP Model
8. Parallel 13 Message Passing Programming
Programming (6 Hrs.) Message Passing Programming
With MPI
o Message Passing Interface
(MPI)
o MPI Extensions
Shared Memory Programming
Shared Memory Programming
with OpenMP
7
Heterogeneous Programming
with CUDA and OpenCL
CUDA (Compute Unified
Device Architecture)
14 OpenCL (Open Computing
Language)
Programming iIn Big Data Era
o MapReduce
o Hadoop
Target Architectures
Pipelines
Multiple Functional Units
Vector Architectures
Multiprocessor And
9. Compiler Multicore Architectures
Transformations for Dependence Analysis
15
Parallel Computers (3 Transformations
Hrs.) Fine-Grained Parallelism
Instruction Scheduling
Trace Scheduling
Software Pipelining
Transformation Framework
Parallelizing Compilers
Resource Management
Task Scheduling in Message
Passing Parallel Computers
Dynamic Scheduling
Task Scheduling in Shared
Memory Parallel Computers
Task Scheduling for
10. Operating Multicore Processor
Systems for Systems
Parallel 16 Process Management
Computers Threads
(3 Hrs.) Process Synchronization
Transactional Memory
Inter-Process Communication
Memory Management
Input/Output (Disk Arrays)
Data Striping
Redundancy Mechanisms
RAID Organizations
8
Basics of Performance Group Assignments
Evaluation Presentation and
Performance Metrics Valuation
Performance Measures and
Benchmarks
Sources of Parallel Overhead
Inter-processor
Communication
11. Performance Load Imbalance
Evaluation of Parallel Inter-task Synchronization
17
Extra Computation
Computers (3 Hrs.)
Other Overheads
Parallel Balance Point
Speedup Performance Laws
Amdahl’s Law
Gustafson’s Law
Sun and Ni’s Law
Scalability Metric
Isoefficiency Function
Performance Analysis
18&19 Final Examination Period
8. Laboratory Activities
No Experiment Titles
1
2
3
4
5
9. Required Resource
Text Book(s) 1. V.Rajaraman, C.Siva Ram Murthi, “Parallel Computers
Architecture and Programming”, Second Edition, PHI
Learning Private Limited, Delhi, India,2016.
2. Rauber, T., & Rünger, G. “Parallel Programming: for
Multicore and Cluster Systems”, 2nd ed., Springer, 2015.
3. Grama, A., Kumar, V., Karypis, G., & Gupta, A.
“Introduction to Parallel Computing”, 2nd ed., Pearson,
2003.
Reference books 1. Hwu, W. K. D. B. W. “Programming Massively Parallel
Processors”, 2nd ed., elsevier india, 2021.
2. Hennessy, J. L., & Patterson, D. A. “Computer Architecture:
A Quantitative Approach (The Morgan Kaufmann Series in
Computer Architecture and Design)”, 6th ed., Morgan
Kaufmann, 2017.
Software
Required
Course website
10. Assessment
Type Weigh Due date Behavior and Criteria
t
Mid Exam 25% 9th week of the Examination will be set to
semester address learning outcomes
9
### and ### and the
criteria is to get all
questions answered
correctly
Final Exam 35% 18th & 19th weeks Examination will be set to
of the semester address learning outcomes
### and ### and the
criteria is to get all
questions answered
correctly
Quizzes/ Individual 20% TBD Individual
Assignments/Participation Assignments/Quizzes
unique questions will be
given to check the level of
confidence of each student
and the criteria is to get
correct answer.
Lab Exam/Report 10% TBD Perform all the stated
experiments completely
and report accordingly
Team Project/Mini-project/ 10% TBD Problems related to the subject
Group Assignment matter will be given and the
criteria are to produce relevant
document, scientific approach to
solve the problem in question,
presentation skill and to develop
team spirit.
11. Attendance Requirement
Minimum of 90% attendance during lecture hours; and 100% of attendance during
tutorial/practical laboratory sessions, except some unprecedented mishaps. Students are expected
to attend classes regularly and come to class on time with appropriate military/civil uniform.
Coming to class after stipulated time is completely prohibited and will be considered absent. If
students are absent, they are responsible for learning materials covered in class.
12. Academic Honesty
Copying from any outside sources (e.g. Fellow students, and Internet, etc.) on any material to be
graded is not permitted, and will be considered cheating. Cheating will result in failure of the
assignment, failure of the class and/or face possible disciplinary action. Each student is responsible
for securing his or her work from copying. Each student is expected to abide by college policies on
academic conduct.
13. Due Date
All assignments must be turned in the class on the due date for full credit. No assignment will be
accepted after class on the due date. Since Team Project/Mini-project/ Group Assignment is
due in week --, papers for this presentation should be submitted before one week of the -- week.
Failure of submission and presentation of the group assignment in week -- will be awarded as zero
out of -- points.
14. Classroom Behavior
Instructor should make every reasonable attempt to create an atmosphere in which each student
feels comfortable voicing their argument without fear of being personally attacked, mocked,
demeaned, or devalued. Anything that disturbs your instructor or your colleagues during the class
period is considered a troublesome behavior. Examples include: Inappropriate use of
mobiles/PDA, making offensive remarks, sleeping, working on assignments related to other
10
courses, leaving the class prior to the Instructor etc. troublesome behaviors are completely
prohibited. Any behavior (including harassment, sexual harassment, and racially and/or culturally
derogatory language) that threatens the class atmosphere will not be tolerated. Students should
alert the Instructor immediately if they feel threatened, dismissed, or silenced at any point during
class lecture and/or if student’s engagement in discussion has been in some way hindered by the
learning environment.
15. Email communication with Instructor
In addition to Consultation / Office Hours scheduled above, students may use respective email to
reach out and communicate with the Instructor. However, students SHOULD NOT email for
questions that are easily found in the syllabus (i.e. When is this assignment due? How much is it
worth? etc.); instead reach out about personal, academic, and intellectual concerns/questions. You
are responsible for checking your email with regular frequency.
16. Approval (Affidavit)
Name Signature Date
Instructor:
Section Head:
Department Head:
11
