OPERATING SYSTEMS

(BIT-202)
1
Dr. Rahul Sachdeva
2
FILE-SYSTEM INTERFACE
FILE CONCEPT
• A file is a collection of related data stored on a computer’s memory or storage
device.
• It can contain various types of data: text, images, audio, video, executables, etc.
• Stored as bits and bytes, managed by the operating system’s file system.

3
FILE CONCEPT
• File Name
• Used to identify and access the file.
• Typically includes a name and extension (e.g., report.txt, image.png).

• File Format or Type

• Determines how data is stored and which application can open it.
• Examples:
• .txt – plain text
• .jpg – image
• .mp4 – video

• File Organization
• Files are stored in a directory structure (folders and subfolders).
• Allows for easy data management and navigation.

4
FILE CONCEPT
Functions of a File System
• Allocates space for new files and directories.
• Updates existing files when changes are made.
• Deletes files and reclaims storage space.
• Tracks file metadata like size, timestamps, ownership, and permissions.

Importance of the File Concept

• Fundamental to all operating systems and software applications.
• Enables systematic storage, retrieval, and management of data.
• Simplifies user interaction with large volumes of information.
5
FILE ATTRIBUTES
▪ Name – only information kept in human-readable form
▪ Identifier – unique tag (number) identifies file within file system
▪ Type – needed for systems that support different types
▪ Location – pointer to file location on device
▪ Size – current file size
▪ Protection – controls who can do reading, writing, executing
▪ Time, date, and user identification – data for protection, security, and usage
monitoring
▪ Information about files are kept in the directory structure, which is maintained on the
disk
▪ Many variations, including extended file attributes such as file checksum
▪ Information kept in the directory structure

6
FILE INFO WINDOW

7
FILE OPERATIONS
▪ File is an abstract data type
▪ Create
▪ Write – at write pointer location
▪ Read – at read pointer location
▪ Reposition within file - seek
▪ Delete
▪ Truncate
▪ Open(Fi) – search the directory structure on disk for entry Fi, and move the content
of entry to memory
▪ Close (Fi) – move the content of entry Fi in memory to directory structure on disk

8
OPEN FILES
▪ Several pieces of data are needed to manage open files:
▪ Open-file table: tracks open files
▪ File pointer: pointer to last read/write location, per process that has the file open
▪ File-open count: counter of number of times a file is open – to allow removal of data from
open-file table when last processes closes it
▪ Disk location of the file: cache of data access information
▪ Access rights: per-process access mode information

9
OPEN FILE LOCKING
▪ Provided by some operating systems and file systems
▪ Similar to reader-writer locks
▪ Shared lock similar to reader lock – several processes can acquire concurrently
▪ Exclusive lock similar to writer lock

▪ Mediates access to a file

▪ Mandatory or advisory:
▪ Mandatory – access is denied depending on locks held and requested
▪ Advisory – processes can find status of locks and decide what to do

10
FILE TYPES – NAME, EXTENSION

11
FILE STRUCTURE
▪ None - sequence of words, bytes
▪ Simple record structure
▪ Lines
▪ Fixed length
▪ Variable length

▪ Complex Structures
▪ Formatted document
▪ Relocatable load file

▪ Can simulate last two with first method by inserting appropriate control characters
▪ Who decides:
▪ Operating system
▪ Program

12
SEQUENTIAL-ACCESS FILE

13
ACCESS METHODS
▪ Sequential Access
read next
write next
reset
no read after last write
(rewrite)
▪ Direct Access – file is fixed length logical records
read n
write n
position to n
read next
write next
rewrite n
n = relative block number

▪ Relative block numbers allow OS to decide where file should be placed

▪ See allocation problem in Ch 12

14
SIMULATION OF SEQUENTIAL ACCESS ON
DIRECT-ACCESS FILE

15
EXAMPLE OF INDEX AND RELATIVE FILES

16
DIRECTORY STRUCTURE
▪ A collection of nodes containing information about all files

Directory

Files
F1 F2 F4
F3
Fn

Both the directory structure and the files reside on disk

17
DISK STRUCTURE
▪ Disk can be subdivided into partitions
▪ Disks or partitions can be RAID protected against failure
▪ Disk or partition can be used raw – without a file system, or formatted with a file
system
▪ Partitions also known as minidisks, slices
▪ Entity containing file system known as a volume
▪ Each volume containing file system also tracks that file system’s info in device
directory or volume table of contents
▪ As well as general-purpose file systems there are many special-purpose file
systems, frequently all within the same operating system or computer

18
A TYPICAL FILE-SYSTEM ORGANIZATION

19
TYPES OF FILE SYSTEMS
▪ We mostly talk of general-purpose file systems
▪ But systems frequently have may file systems, some general- and some special-
purpose
▪ Consider Solaris has
▪ tmpfs – memory-based volatile FS for fast, temporary I/O
▪ objfs – interface into kernel memory to get kernel symbols for debugging
▪ ctfs – contract file system for managing daemons
▪ lofs – loopback file system allows one FS to be accessed in place of another
▪ procfs – kernel interface to process structures
▪ ufs, zfs – general purpose file systems

20
OPERATIONS PERFORMED ON DIRECTORY
▪ Search for a file

▪ Create a file

▪ Delete a file

▪ List a directory

▪ Rename a file

▪ Traverse the file system

21
DIRECTORY ORGANIZATION
▪ The directory is organized logically to obtain

▪ Efficiency – locating a file quickly

▪ Naming – convenient to users
▪ Two users can have same name for different files
▪ The same file can have several different names

▪ Grouping – logical grouping of files by properties, (e.g., all Java programs, all
games, …)

22
SINGLE-LEVEL DIRECTORY
▪ A single directory for all users

▪ Naming problem
▪ Grouping problem

23
TWO-LEVEL DIRECTORY
▪ Separate directory for each user

Path name
Can have the same file name for different user
Efficient searching
No grouping capability
24
TREE-STRUCTURED DIRECTORIES

25
TREE-STRUCTURED DIRECTORIES (CONT.)
▪ Efficient searching

▪ Grouping Capability

▪ Current directory (working directory)

▪ cd /spell/mail/prog
▪ type list

26
ACYCLIC-GRAPH DIRECTORIES
▪ Have shared subdirectories and files

27
GENERAL GRAPH DIRECTORY

28
FILE SHARING
▪ Sharing of files on multi-user systems is desirable
▪ Sharing may be done through a protection scheme
▪ On distributed systems, files may be shared across a network
▪ Network File System (NFS) is a common distributed file-sharing method
▪ If multi-user system
▪ User IDs identify users, allowing permissions and protections to be per-user
Group IDs allow users to be in groups, permitting group access rights
▪ Owner of a file / directory
▪ Group of a file / directory

29
FILE SHARING – REMOTE FILE SYSTEMS
▪ Uses networking to allow file system access between systems
▪ Manually via programs like FTP
▪ Automatically, seamlessly using distributed file systems
▪ Semi automatically via the world wide web

▪ Client-server model allows clients to mount remote file systems from servers
▪ Server can serve multiple clients
▪ Client and user-on-client identification is insecure or complicated
▪ NFS is standard UNIX client-server file sharing protocol
▪ CIFS is standard Windows protocol
▪ Standard operating system file calls are translated into remote calls

▪ Distributed Information Systems (distributed naming services) such as LDAP,

DNS, NIS, Active Directory implement unified access to information needed for
remote computing

30
FILE SHARING – FAILURE MODES
▪ All file systems have failure modes
▪ For example corruption of directory structures or other non-user data, called metadata

▪ Remote file systems add new failure modes, due to network failure, server failure
▪ Recovery from failure can involve state information about status of each remote
request
▪ Stateless protocols such as NFS v3 include all information in each request, allowing
easy recovery but less security

31
FILE SHARING – CONSISTENCY
SEMANTICS
▪ Specify how multiple users are to access a shared file simultaneously
▪ Similar to Ch 5 process synchronization algorithms
▪ Tend to be less complex due to disk I/O and network latency (for remote file systems
▪ Andrew File System (AFS) implemented complex remote file sharing semantics
▪ Unix file system (UFS) implements:
▪ Writes to an open file visible immediately to other users of the same open file
▪ Sharing file pointer to allow multiple users to read and write concurrently
▪ AFS has session semantics
▪ Writes only visible to sessions starting after the file is closed

32
PROTECTION
▪ File owner/creator should be able to control:
▪ what can be done
▪ by whom

▪ Types of access
▪ Read
▪ Write
▪ Execute
▪ Append
▪ Delete
▪ List

33
ACCESS LISTS AND GROUPS
▪ Mode of access: read, write, execute
▪ Three classes of users on Unix / Linux
RWX
a) owner access 7  111
RWX
b) group access 6  110
RWX
c) public access 1  001

▪ Ask manager to create a group (unique name), say G, and add some users to the
group.
▪ For a particular file (say game) or subdirectory, define an appropriate access.

Attach a group to a file

chgrp G game 34
WINDOWS 7 ACCESS-CONTROL LIST
MANAGEMENT

35
A SAMPLE UNIX DIRECTORY LISTING

36
37
FILE-SYSTEM
IMPLEMENTATION
FILE-SYSTEM STRUCTURE
▪ File structure
▪ Logical storage unit
▪ Collection of related information

▪ File system resides on secondary storage (disks)

▪ Provided user interface to storage, mapping logical to physical
▪ Provides efficient and convenient access to disk by allowing data to be stored, located
retrieved easily
▪ Disk provides in-place rewrite and random access
▪ I/O transfers performed in blocks of sectors (usually 512 bytes)

▪ File control block – storage structure consisting of information about a file

▪ Device driver controls the physical device
▪ File system organized into layers

38
LAYERED FILE SYSTEM

39
FILE SYSTEM LAYERS
▪ Device drivers manage I/O devices at the I/O control layer
▪ Given commands like “read drive1, cylinder 72, track 2, sector 10, into memory location
1060” outputs low-level hardware specific commands to hardware controller
▪ Basic file system given command like “retrieve block 123” translates to device
driver
▪ Also manages memory buffers and caches (allocation, freeing, replacement)
▪ Buffers hold data in transit
▪ Caches hold frequently used data

▪ File organization module understands files, logical address, and physical blocks
Translates logical block # to physical block #
Manages free space, disk allocation

40
FILE SYSTEM LAYERS (CONT.)
▪ Logical file system manages metadata information
▪ Translates file name into file number, file handle, location by maintaining file control
blocks (inodes in UNIX)
▪ Directory management
▪ Protection

▪ Layering useful for reducing complexity and redundancy, but adds overhead and
can decrease performanceTranslates file name into file number, file handle,
location by maintaining file control blocks (inodes in UNIX)
▪ Logical layers can be implemented by any coding method according to OS designer

41
FILE SYSTEM LAYERS (CONT.)
▪ Many file systems, sometimes many within an operating system
▪ Each with its own format (CD-ROM is ISO 9660; Unix has UFS, FFS; Windows has FAT,
FAT32, NTFS as well as floppy, CD, DVD Blu-ray, Linux has more than 40 types, with
extended file system ext2 and ext3 leading; plus distributed file systems, etc.)
▪ New ones still arriving – ZFS, GoogleFS, Oracle ASM, FUSE

42
FILE-SYSTEM IMPLEMENTATION
▪ We have system calls at the API level, but how do we implement their functions?
▪ On-disk and in-memory structures

▪ Boot control block contains info needed by system to boot OS from that volume
▪ Needed if volume contains OS, usually first block of volume

▪ Volume control block (superblock, master file table) contains volume details
▪ Total # of blocks, # of free blocks, block size, free block pointers or array

▪ Directory structure organizes the files

▪ Names and inode numbers, master file table

43
FILE-SYSTEM IMPLEMENTATION (CONT.)
▪ Per-file File Control Block (FCB) contains many details about the file
▪ inode number, permissions, size, dates
▪ NFTS stores into in master file table using relational DB structures

44
IN-MEMORY FILE SYSTEM STRUCTURES
▪ Mount table storing file system mounts, mount points, file system types
▪ The following figure illustrates the necessary file system structures provided by the
operating systems
▪ Figure 12-3(a) refers to opening a file
▪ Figure 12-3(b) refers to reading a file
▪ Plus buffers hold data blocks from secondary storage
▪ Open returns a file handle for subsequent use
▪ Data from read eventually copied to specified user process memory address

45
IN-MEMORY FILE SYSTEM STRUCTURES

46
PARTITIONS AND MOUNTING
▪ Partition can be a volume containing a file system (“cooked”) or raw – just a
sequence of blocks with no file system
▪ Boot block can point to boot volume or boot loader set of blocks that contain
enough code to know how to load the kernel from the file system
▪ Or a boot management program for multi-os booting

▪ Root partition contains the OS, other partitions can hold other Oses, other file
systems, or be raw
▪ Mounted at boot time
▪ Other partitions can mount automatically or manually

▪ At mount time, file system consistency checked

▪ Is all metadata correct?
▪ If not, fix it, try again
▪ If yes, add to mount table, allow access

47
DIRECTORY IMPLEMENTATION
▪ Linear list of file names with pointer to the data blocks
▪ Simple to program
▪ Time-consuming to execute
▪ Linear search time
▪ Could keep ordered alphabetically via linked list or use B+ tree

▪ Hash Table – linear list with hash data structure

▪ Decreases directory search time
▪ Collisions – situations where two file names hash to the same location
▪ Only good if entries are fixed size, or use chained-overflow method

48
ALLOCATION METHODS - CONTIGUOUS
▪ An allocation method refers to how disk blocks are allocated for files:
▪ Contiguous allocation – each file occupies set of contiguous blocks
▪ Best performance in most cases
▪ Simple – only starting location (block #) and length (number of blocks) are required
▪ Problems include finding space for file, knowing file size, external fragmentation, need
for compaction off-line (downtime) or on-line

49
CONTIGUOUS ALLOCATION
▪ Mapping from logical to physical Q

LA/512

R
Block to be accessed = Q +
starting address
Displacement into block = R

50
EXTENT-BASED SYSTEMS
▪ Many newer file systems (i.e., Veritas File System) use a modified contiguous
allocation scheme

▪ Extent-based file systems allocate disk blocks in extents

▪ An extent is a contiguous block of disks

▪ Extents are allocated for file allocation
▪ A file consists of one or more extents

51
ALLOCATION METHODS - LINKED
▪ Linked allocation – each file a linked list of blocks
▪ File ends at nil pointer
▪ No external fragmentation
▪ Each block contains pointer to next block
▪ No compaction, external fragmentation
▪ Free space management system called when new block needed
▪ Improve efficiency by clustering blocks into groups but increases internal fragmentation
▪ Reliability can be a problem
▪ Locating a block can take many I/Os and disk seeks

52
ALLOCATION METHODS – LINKED (CONT.)
▪ FAT (File Allocation Table) variation
▪ Beginning of volume has table, indexed by block number
▪ Much like a linked list, but faster on disk and cacheable
▪ New block allocation simple

53
LINKED ALLOCATION
▪ Each file is a linked list of disk blocks: blocks may be scattered anywhere on the
disk
block = pointer

Mapping
Q
LA/511
R

Block to be accessed is the Qth block in the linked chain of blocks

representing the file.

Displacement into block = R + 1

54
LINKED ALLOCATION

55
FILE-ALLOCATION TABLE

56
ALLOCATION METHODS - INDEXED
▪ Indexed allocation
▪ Each file has its own index block(s) of pointers to its data blocks

▪ Logical view

index table
57
EXAMPLE OF INDEXED ALLOCATION

58
INDEXED ALLOCATION (CONT.)
▪ Need index table

▪ Random access

▪ Dynamic access without external fragmentation, but have overhead of index block
Q
LA/512
R
▪ Mapping from logical to
▪ physical in a file of maximum size of 256K bytes and block size of 512 bytes. We
need only 1 block for index table
Q = displacement into index table
R = displacement into block
59
PERFORMANCE
▪ Best method depends on file access type
▪ Contiguous great for sequential and random

▪ Linked good for sequential, not random

▪ Declare access type at creation -> select either contiguous or linked
▪ Indexed more complex
▪ Single block access could require 2 index block reads then data block read
▪ Clustering can help improve throughput, reduce CPU overhead

60
PERFORMANCE (CONT.)
▪ Adding instructions to the execution path to save one disk I/O is reasonable
▪ Intel Core i7 Extreme Edition 990x (2011) at 3.46Ghz = 159,000 MIPS
▪ http://en.wikipedia.org/wiki/Instructions_per_second
▪ Typical disk drive at 250 I/Os per second
▪ 159,000 MIPS / 250 = 630 million instructions during one disk I/O
▪ Fast SSD drives provide 60,000 IOPS
▪ 159,000 MIPS / 60,000 = 2.65 millions instructions during one disk I/O

61
FREE-SPACE MANAGEMENT
▪ File system maintains free-space list to track available blocks/clusters
▪ (Using term “block” for simplicity)

▪ Bit vector or bit map (n blocks)

0 1 2 n-1
…


bit[i] =
1  block[i] free
0  block[i] occupied
Block number calculation
(number of bits per word) *
(number of 0-value words) +
offset of first 1 bit
62
CPUs have instructions to return offset within word of first “1” bit
FREE-SPACE MANAGEMENT (CONT.)
▪ Bit map requires extra space
▪ Example:

block size = 4KB = 212 bytes

disk size = 240 bytes (1 terabyte)
n = 240/212 = 228 bits (or 32MB)
if clusters of 4 blocks -> 8MB of memory

▪ Easy to get contiguous files

63
LINKED FREE SPACE LIST ON DISK
Linked list (free list)
Cannot get contiguous
space easily
No waste of space
No need to traverse the
entire list (if # free blocks
recorded)

64
FREE-SPACE MANAGEMENT (CONT.)
▪ Grouping
▪ Modify linked list to store address of next n-1 free blocks in first free block, plus a pointer
to next block that contains free-block-pointers (like this one)

▪ Counting
▪ Because space is frequently contiguously used and freed, with contiguous-allocation
allocation, extents, or clustering
▪ Keep address of first free block and count of following free blocks
▪ Free space list then has entries containing addresses and counts

65
FREE-SPACE MANAGEMENT (CONT.)
▪ Space Maps
▪ Used in ZFS
▪ Consider meta-data I/O on very large file systems
▪ Full data structures like bit maps couldn’t fit in memory -> thousands of I/Os
▪ Divides device space into metaslab units and manages metaslabs
▪ Given volume can contain hundreds of metaslabs
▪ Each metaslab has associated space map
▪ Uses counting algorithm
▪ But records to log file rather than file system
▪ Log of all block activity, in time order, in counting format
▪ Metaslab activity -> load space map into memory in balanced-tree structure, indexed by
offset
▪ Replay log into that structure
▪ Combine contiguous free blocks into single entry

66
EFFICIENCY AND PERFORMANCE
▪ Efficiency dependent on:
▪ Disk allocation and directory algorithms
▪ Types of data kept in file’s directory entry
▪ Pre-allocation or as-needed allocation of metadata structures
▪ Fixed-size or varying-size data structures

67
EFFICIENCY AND PERFORMANCE (CONT.)
▪ Performance
▪ Keeping data and metadata close together
▪ Buffer cache – separate section of main memory for frequently used blocks
▪ Synchronous writes sometimes requested by apps or needed by OS
▪ No buffering / caching – writes must hit disk before acknowledgement
▪ Asynchronous writes more common, buffer-able, faster
▪ Free-behind and read-ahead – techniques to optimize sequential access
▪ Reads frequently slower than writes

68
PAGE CACHE
▪ A page cache caches pages rather than disk blocks using virtual memory
techniques and addresses

▪ Memory-mapped I/O uses a page cache

▪ Routine I/O through the file system uses the buffer (disk) cache

▪ This leads to the following figure

69
I/O WITHOUT A UNIFIED BUFFER CACHE

70
UNIFIED BUFFER CACHE
▪ A unified buffer cache uses the same page cache to cache both memory-mapped
pages and ordinary file system I/O to avoid double caching

But which caches get priority, and what replacement algorithms to use?

71
I/O USING A UNIFIED BUFFER CACHE

72
RECOVERY
▪ Consistency checking – compares data in directory structure with data blocks on
disk, and tries to fix inconsistencies
▪ Can be slow and sometimes fails

▪ Use system programs to back up data from disk to another storage device
(magnetic tape, other magnetic disk, optical)

▪ Recover lost file or disk by restoring data from backup

73
LOG STRUCTURED FILE SYSTEMS
▪ Log structured (or journaling) file systems record each metadata update to the
file system as a transaction
▪ All transactions are written to a log
▪ A transaction is considered committed once it is written to the log (sequentially)
▪ Sometimes to a separate device or section of disk
▪ However, the file system may not yet be updated

▪ The transactions in the log are asynchronously written to the file system structures
▪ When the file system structures are modified, the transaction is removed from the log

▪ If the file system crashes, all remaining transactions in the log must still be
performed
▪ Faster recovery from crash, removes chance of inconsistency of metadata

74
75
THANK YOU

Any QUESTIONS?