1
Lahore Garrison University
Parallel and Distributed
Computing
Session Spring 2023
Lecture – 15
INSTRUCTOR: MUHAMMAD ARSALAN RAZA
2
Lahore Garrison University
NAME: EMAIL: PHONE NUMBER: VISITING HOURS:
MUHAMMAD ARSALAN RAZA MARSALANRAZA@LGU.EDU.P 0333-7843180 (PRIMARY) MON-FRIDAY:(10:00AM - 1:00PM)
K 0321-4069289 Thursday: (11:00Am - 12:00pm)
3
Preamble
Memory Models
Message Passing Interface (MPI)
Making Sockets Easier to Work With
MPI Primitives and Meanings
Lahore Garrison University
4
Lesson Plan
OpenMP
OpenMP Essentials
What is OpenMP and What’s Not?
Goals of OpenMP
OpenMP Programming Model
Programming
Lahore Garrison University
5
OpenMP
OpenMP is an Application Program Interface (API), jointly defined by a group of major computer hardware
and software vendors.
OpenMP provides a portable, scalable model for developers of shared memory parallel applications.
OpenMP Essentials
The API supports C/C++ and Fortran on a wide variety of architectures.
This lecture covers most of the major features of OpenMP 3.1, including its various constructs and
directives for specifying parallel regions, work sharing, synchronization, and data environment.
Runtime library functions and environment variables are also covered.
Lahore Garrison University
6
What is OpenMP?
An Application Program Interface (API) that may be used to explicitly direct multi-threaded, shared
memory parallelism.
Comprised of three primary API components:
Compiler Directives*
Runtime Library Routines**
Environment Variables
An abbreviation for: Open Multi-Processing
*See References on the last slide.
Lahore Garrison University
7
OpenMP is Not:
Necessarily implemented identically by all vendors
Guaranteed to make the most efficient use of shared memory
Required to check for data dependencies, data conflicts, race conditions, deadlocks, or code sequences
that cause a program to be classified as non-conforming
Designed to handle parallel I/O. The programmer is responsible for synchronizing input and output.
Lahore Garrison University
Goals of OpenMP 8
Standardization:
Provide a standard among a variety of shared memory architectures/platforms
Jointly defined and endorsed by a group of major computer hardware and software vendors
Lean and Mean:
Establish a simple and limited set of directives for programming shared memory machines.
Significant parallelism can be implemented by using just 3 or 4 directives.
This goal is becoming less meaningful with each new release, apparently.
Ease of Use:
Provide capability to incrementally parallelize a serial program, unlike message-passing libraries which typically require an all or nothing
approach
Provide the capability to implement both coarse-grain and fine-grain parallelism
Portability:
The API is specified for C/C++ and Fortran
Public forum for API and membership
Most major platforms have been implemented including Unix/Linux platforms and Windows
Lahore Garrison University
9
OpenMP Programming Model
Lahore Garrison University
10
Hybrid Programming
Hierarchical parallel architectures are gaining more importance.
Uniform programming interface is possible but might not be most efficient.
MPI requires optimized library for intranode communication.
OpenMP requires global address space implemented in software.
Hybrid programming allows to optimize both levels of the hierarchy.
MPI across nodes
OpenMP within a node
In addition to increased programming complexity, it requires additional performance-relevant design decisions
prior to coding.
Lahore Garrison University
11
Lahore Garrison University
Programming Interfaces
OpenMP
www.openmp.org
12
OpenMP: Description-based SM
Parallelization
OpenMP is a standard shared memory programming interface (1997)
Directives for Fortran 77 and C/C++
Fork-join model resulting in a global program
It includes:
Parallel loops
Parallel sections
Parallel regions
Shared and private data
Synchronization primitives
barrier
critical region
Lahore Garrison University
13
OpenMP
Example
Lahore Garrison University
14
C Language Example
// OpenMP program to print Hello World
// using C language
// OpenMP header For CMD
#include <omp.h>
#include <stdio.h> gcc -o hello -fopenmp hello.c
#include <stdlib.h>
int main(int argc, char* argv[]) ./hello
{
// Beginning of parallel region
#pragma omp parallel
{
printf("Hello World... from thread = %d\n",
omp_get_thread_num());
}
// Ending of parallel region
}
Lahore Garrison University Tutorial Link: https://www.geeksforgeeks.org/openmp-hello-world-program/
15
Notes & Restrictions
The dynamic threads mechanism modifies the effect of this routine.
Enabled: specifies the maximum number of threads that can be used for any parallel region by the dynamic
thread's mechanism.
Disabled: specifies exact number of threads to use until next call to this routine.
Lahore Garrison University
16
Lesson Review
Tools and Development
OpenMP
OpenMP Essentials
What is OpenMP and What’s Not?
Goals of OpenMP
OpenMP Programming Model
Notes & Restrictions
Lahore Garrison University
17
Next Lesson Preview
Tools and Development
MPI using C#
Lahore Garrison University
18
References
To cover this topic, different reference material has been used for consultation.
Textbook:
Distributed Systems: Principles and Paradigms, A. S. Tanenbaum and M. V. Steen, Prentice Hall, 2 nd Edition, 2007.
Distributed and Cloud Computing: Clusters, Grids, Clouds, and the Future Internet, K. Hwang, J. Dongarra and GC. C.
Fox, Elsevier, 1st Ed.
Google Search Engine
*https://docs.microsoft.com/en-us/cpp/parallel/openmp/reference/openmp-directives?view=msvc-170
**https://www.cita.utoronto.ca/~merz/intel_f10b/main_for/mergedProjects/optaps_for/common/optaps_par_libr.htm
Lahore Garrison University
