Lecture 3: Computer Architectures
Basic Computer Architecture
Memory
instruction data
Von Neumann Architecture
Input unit
ALU Processor CU Reg.
Output unit
Levels of Parallelism
Bit level parallelism
Within arithmetic logic circuits Multiple instructions execute per clock cycle Overlap of memory operations with computation More than one processor Multiple jobs run in parallel on SMP
Instruction level parallelism
Memory system parallelism
Operating system parallelism
Loop level Procedure level
Levels of Parallelism
Bit Level Parallelism
Within arithmetic logic circuits
Levels of Parallelism
Instruction Level Parallelism (ILP)
Multiple instructions execute per clock cycle
Pipelining (instruction - data) Multiple Issue (VLIW)
Levels of Parallelism
Memory System Parallelism
Overlap of memory operations with computation
Levels of Parallelism
Operating System Parallelism
There are more than one processor Multiple jobs run in parallel on SMP
Loop level Procedure level
Flynns Taxonomy
Single Instruction stream - Single Data stream (SISD) Single Instruction stream - Multiple Data stream (SIMD) Multiple Instruction stream - Single Data stream (MISD) Multiple Instruction stream - Multiple Data stream (MIMD)
Single Instruction stream Single Data stream (SISD)
Memory
instruction data
Von Neumann Architecture
CU
ALU
Processor
Flynns Taxonomy
Single Instruction stream - Single Data stream (SISD) Single Instruction stream - Multiple Data stream (SIMD) Multiple Instruction stream - Single Data stream (MISD) Multiple Instruction stream - Multiple Data stream (MIMD)
Single Instruction stream Multiple Data stream (SIMD)
PE
data
Instructions of the program are broadcast to more than one processor Each processor executes the same instruction synchronously, but using different data
Memory
instruction CU
PE
data
PE
data
PE
data
instruction
Used for applications that operate upon arrays of data
Flynns Taxonomy
Single Instruction stream - Single Data stream (SISD) Single Instruction stream - Multiple Data stream (SIMD) Multiple Instruction stream - Single Data stream (MISD) Multiple Instruction stream - Multiple Data stream (MIMD)
Multiple Instruction stream Multiple Data stream (MIMD)
Each processor has a separate program
An instruction stream is generated for each program on each processor Each instruction operates upon different data
Multiple Instruction stream Multiple Data stream (MIMD)
Shared memory
Distributed memory
Shared vs Distributed Memory
M
P
M
P
M
P
M
P
Distributed memory
Network
Each processor has its own local memory Message-passing is used to exchange data between processors
Shared memory
P Bus
Memory
Single address space All processes have access to the pool of shared memory
Distributed Memory
M P NI M P NI M P NI M P NI
Processors cannot directly access another processors memory Each node has a network interface (NI) for communication and synchronization
Network
Distributed Memory
M
instr data
CU
PE
data
data
instr data
CU
PE Network
instr data
data
CU
PE
Each processor executes different instructions asynchronously, using different data
instr data
data
CU
PE
Shared Memory
CU PE
data
data
data
CU
PE
data
CU
PE
instruction
Memory
CU
PE
Each processor executes different instructions asynchronously, using different data
Shared Memory
P P
Bus Memory
Uniform memory access (UMA)
Each processor has uniform access to memory (symmetric multiprocessor - SMP)
P Bus
P Bus
Non-uniform memory access (NUMA)
Memory Network
Memory
Time for memory access depends on the location of data Local access is faster than nonlocal access Easier to scale than SMPs
Distributed Shared Memory
Making the main memory of a cluster of computers look as if it is a single memory with a single address space Shared memory programming techniques can be used
Distributed Systems
Clusters
Individual computers, that are tightly coupled by
software, in a local environment, to work together on single problems or on related problems
Grid
Many individual systems, that are geographically
distributed, are tightly coupled by software, to work together on single problems or on related problems
