CSE 5306
Distributed Systems
Distributed File Systems
1
Outline
• Architecture
• Communication
• Naming
• Synchronization
• Consistency and replication
• Fault tolerance
• Security
2
Architecture
• Client-Server architecture
• Cluster-based architecture
• Symmetric architecture
– Ivy distributed file system
3
Client-Server Architecture (1)
(a) The remote access model. (b) The upload/download
model.
4
Client-Server Architecture (2)
The basic NFS architecture for UNIX systems.
5
File System Operations (1)
An incomplete list of file system operations supported
by NFS.
6
File System Operations (2)
An incomplete list of file system operations supported
by NFS.
7
Cluster-Based Distributed FS (1)
The difference between (a) distributing whole files across
several servers and (b) striping files for parallel access.
8
Cluster-Based Distributed FS (2)
The organization of a Google cluster of servers.
9
Ivy Distributed File Systems
Computing look-up keys: content-based or public key based
10
Communication
• RPCs in NFS
• Coda RPC2 subsystem
– An enhancement of traditional RPC
• File-oriented communication in Plan 9
11
RPC calls in NFS
(a) Reading data from a file in NFS version 3.
(b) Reading data using a compound procedure in version 4.
12
Coda RPC2 Subsystem (1)
e.g. video stream
Side effects in Coda’s RPC2 system allows application
specific protocols during communication
13
Coda RPC2 Subsystem (2)
A file is modified, and all outdated copies need to be invalidated
(a) Sending an invalidation message one at a time.
(b) multicasting: sending invalidation messages in parallel.
14
File-Oriented Comm. in Plan 9
Files associated with a single TCP connection in Plan 9.
15
Naming in NFS (1)
Mounting (part of) a remote file system in NFS.
16
Naming in NFS (2)
Mounting nested directories from
multiple servers in NFS.
17
Challenges
• File sharing between many clients
– Clients may name the same file differently
• Partially standardize the naming
• e.g., /home/alice for accessing alice’s files
– Automounting
• On-demand mounting
18
Automounting
A simple automounter for NFS.
19
Synchronization
• Semantics of file sharing
• File locking
• Sharing files in Coda
20
Semantics of File Sharing (1)
On a single processor,
when a read follows a
write, the value returned
by the read is the value
just written.
21
Semantics of File Sharing (2)
(b) In a distributed system with
caching, obsolete values
may be returned.
22
Semantics of File Sharing (3)
group of files
Four ways of dealing with the shared files
in a distributed system.
23
File Locking
NFSv4 operations related to file locking.
24
File Sharing in Coda
The transactional behavior in sharing files in Coda.
25
Consistency and Replication
• Client-side caching
– Caching in NFS
– Caching in Coda
• Server-side replication
– Server replication in Coda
26
NFS Client-Side Caching (1)
Client-side caching in NFS.
27
NFS Client-Side Caching (2)
Using the NFSv4 callback mechanism
to recall file delegation.
28
Client-Side Caching in Coda
The use of local copies when opening a session in Coda.
29
Server Replication in Coda
Coda uses a variant of replicated-write protocol, ROWA(Read-one, write all)
Two clients with a different AVSG for the same replicated file.
30
Handling Byzantine Failures
3k+1 replicas for k faulty tolerance
The different phases in Byzantine fault tolerance.
31
Security in NFS
The NFS security architecture.
32
Secure RPCs
Secure RPC in NFSv4.
33
Access Control
The various kinds of users and processes distinguished
by NFS with respect to access control.
34
Secure Collaborative Storage
Storage claims in the Samsara peer-to-peer system.
Produced by Q’s
private key, and
cannot be compressed
Size=what is asked by Q
Question: How to verify that P did allocate space of the requested size?
35
