CSC334 Parallel & Distributed Computing
Lecture # 03
Distributed Systems
What is a distributed
system
•
A distributed system is one in which components
located at networked computers communicate and
coordinate their actions only by passing messages
• A distributed system is a collection of
autonomous computers linked by a computer
network that appear to the users of the system as a
single computer.
4
What is a distributed
system (cont’d)
- Personal workstations +
processors not assigned to
specific users.
- For a certain command the
system can look for the
best place (workstation) to
execute it.
Examples of Distributed Systems
5
The internet and intranets
•
The Internet is a very large distributed system
consisting of many interconnected computer
networks.
•
The implementation of the internet and the
services that it supports, has entailed the
development of practical solutions to many
distributed system issues.
•
An intranet is a “mini-internet” which uses the
same technologies as the internet, but is controlled
by an organization. 6
Mobile computing
•
Today's computing power makes can benefit from
network information
•
Examples:
–
Laptops
–
PDA
–
Mobile phone
•
These devices can either surf the internet, or it is
possible to exchange business information at a
meeting
7
Mobile Computing
• Mobile computing refers to the ability to
use computing devices wirelessly while
being mobile. It enables users to access
data, applications, and communication
services from anywhere without being
restricted to a fixed location.
• Portability – Devices can be carried easily (e.g.,
smartphones, tablets, laptops).
• Wireless Connectivity – Uses Wi-Fi, cellular
networks (4G, 5G), and Bluetooth for
communication.
• Ubiquitous Access – Users can access data and
applications anytime, anywhere.
• Cloud Integration – Many services rely on cloud
storage and processing.
• Energy Efficiency – Battery-powered devices
ensure mobility.
Ubiquitous computing
•
Ubiquitous computing, also known as pervasive
computing, is a computing concept where
technology is seamlessly integrated into everyday
life, making computing services available anytime
and anywhere without requiring explicit user
interaction.
8
Key Features
• Embedded Computing – Devices and
sensors are embedded in everyday objects
(e.g., smart appliances, wearable devices).
• Seamless Connectivity – Uses IoT, Wi-Fi,
Bluetooth, and cloud computing to ensure
continuous communication.
• Context Awareness – Systems can adapt
based on the user's environment (e.g., smart
thermostats adjusting temperature).
• Automation & Intelligence – Uses AI and
machine learning to automate tasks and
enhance decision-making.
• User-Friendly & Invisible Computing –
Works in the background without requiring
active user input.
Resources and
Components
•
The terms “resources” and “components” are used
to describe how a distributed system is built up.
•
These terms are very abstract, but the range of
things that can be connected and shared is very
large.
11
Important terms
•
Service
– A distinct part of a computer system that manages a collection of
related resources and presents their functionality to users and
applications
– Internet services: http, telnet, pop3...
•
Server
– A running program (a process) on a networked computer that
accepts requests from programs running on other computers to
perform a service, and responds appropriately
– Internet servers: Apache, Sendmail, WU-ftpd, OpenSSH
•
Client
– The requesting process
12
• Scalability: The ability of a system to
handle increasing workloads by adding
more resources (e.g., nodes, storage).
• Concurrency – Multiple processes
executing simultaneously in a distributed
environment.
• Fault Tolerance – The system's ability to
continue functioning even when some
components fail.
• Replication – Storing copies of data across
multiple nodes to improve reliability and
availability.
• Latency – The time delay in
communication between distributed
components.
• Load Balancing – Distributing workloads
across multiple servers or resources to
prevent bottlenecks.
• Challenges in Distributed Systems
Heterogeneity
•
Different networks, hardware, operating systems,
programming languages, developers
•
We set up protocols to solve these heterogeneities
•
Middleware: a software layer that provides a
programming abstraction as well as masking the
heterogeneity
•
Mobile code: code that can be sent from one
computer to another and run at the destination
17
Security
•
Security for information resources has three
components:
–
Confidentiality: protection against disclosure to
unauthorized individuals
–
Integrity: protection against alteration or corruption
–
Availability: protection against interference with the
means to access the resources
19
Scalability
•
A system is described as scalable if it remains
effective when there is a significant increase in the
number of resources and the number of users
•
Challenges:
–
Controlling the cost of resources
–
Controlling loss of performance
–
Preventing software resources from running out
–
Avoiding performance bottlenecks
20
Fault Tolerance & Reliability
• Systems must continue operating even
when components fail.
• Implementing redundancy and replication
mechanisms.
• Handling network failures, hardware
crashes, and process failures.
Concurrency & Synchronization
• Managing multiple processes running
simultaneously.
• Ensuring consistent access to shared
resources (e.g., databases).
• Avoiding deadlocks and race conditions.
Consistency & Data
Synchronization
• Ensuring data is updated correctly across all
nodes.
• Managing trade-offs between strong
consistency (strict updates) and eventual
consistency (delayed updates).
Network Latency &
Communication Overhead
• Delays in data transmission due to network
congestion.
• High overhead in synchronizing and
communicating between distributed nodes.
• Handling partial failures where only some
parts of the system fail.
Cost Management
• Running distributed systems across cloud
providers incurs significant costs.
• Optimizing storage, computing, and
network costs while maintaining
performance.
Summary
•
Distributed systems are everywhere
•
Resource sharing is the main motivating factor for
constructing distribute systems
•
The construction of a distributed system produces
many challenges
24
