UNIX INTERNALS –
INTRODUCTION TO KERNEL
Student Copy – JUNE 2015
After this Lecture ;
1. The student shall be able to “draw & brief” UNIX Kernel Architecture
2. The student shall be able to list the CHARACTERS of File & Process Sub-
system
3. The student shall be able to brief the File Sub System - Buffering
mechanism
4. The student shall be able to narrate on Programs, Libraries & System calls
& its interaction
Student Copy – JUNE 2015
After this Lecture 2;
1. The student shall be able to explain the working of File Sub-system &
inodes
2. The student shall be able to list the CHARACTERS of File & Process Sub-
system
3. The student shall be able to “draw & brief” File Sub-system Layout & its
entities
4. The student shall be able to “brief” Process Sub-system.
5. The student shall be able to “draw & brief” Kernel & User Stacks..
6. The student shall be able to list & explain Process states & its transitions
Student Copy – JUNE 2015
INTRODUCTION TO KERNEL
1. Architecture Of The Unix Operating System
2. Introduction To System Concepts & Kernel Data Structures
Student Copy – JUNE 2015
Introduction To Kernel
UNIX system has TWO major Entities
Files and Processes.
UNIX kernel has TWO major Components
Process Control Subsystem & File Subsystem
UNIX system has THREE System levels
user, kernel, and hardware.
Student Copy – JUNE 2015
Student Copy – JUNE 2015
UNIX Kernel Architecture – PROGRAMS & LIBRARIES
System calls (used in Programs) looks like ordinary function
calls in C programs, libraries map these function calls to
primitives of OS.
Yet, Assembly language programs may invoke system calls
directly without a system call library
Student Copy – JUNE 2015
UNIX Kernel Architecture – PROGRAMS & LIBRARIES
During Compilation,
what happens to System calls & Libraries ?
HINT : Last lecture
Student Copy – JUNE 2015
UNIX Kernel Architecture – SYSTEM CALLS
The set of SYSTEM CALLS is partitioned into those that
interact with the file subsystem and those that interact with
the process control subsystem.
Student Copy – JUNE 2015
UNIX Kernel Architecture - FILE SUBSYSTEM
THE FILE SUBSYSTEM
1. Manages files
2. Allocating file space
3. Administering free space
4. Controlling access to files
5. Retrieving data for users
Student Copy – JUNE 2015
UNIX Kernel Architecture - FILE SUBSYSTEM
THE FILE SUBSYSTEM
The file subsystem accesses file
data using a buffering mechanism
Buffering Mechanism
Regulates data flow between the kernel and secondary storage devices.
Interacts with block I/O device drivers to initiate data transfer to and
from the kernel.
where, Block I/O devices are random access storage devices.
Student Copy – JUNE 2015
UNIX Kernel Architecture - FILE SUBSYSTEM
THE FILE SUBSYSTEM
It interacts directly with "raw" I/0 device drivers without the
intervention of a buffering mechanism
Raw devices character devices
More info on Block & Char Dev -Refer :
http://unix.stackexchange.com/questions/60034/what-are-character-special-and-
block-special-files-in-a-unix-system
Student Copy – JUNE 2015
UNIX Kernel Architecture - FILE SUBSYSTEM
THE FILE SUBSYSTEM
More info on Block & Char Dev -Refer :
http://unix.stackexchange.com/questions/60034/what-are-
character-special-and-block-special-files-in-a-unix-system
GOOGLE : unix stackexchange 60034
Student Copy – JUNE 2015
UNIX Kernel Architecture - THE PROCESS CONTROL SUBSYSTEM
THE PROCESS CONTROL SUBSYSTEM
Its responsible for;
1. Process Synchronization
2. Inter Process Communication
3. Memory Management
4. Process Scheduling.
Student Copy – JUNE 2015
Student Copy – JUNE 2015
UNIX Kernel Architecture - THE PROCESS CONTROL SUBSYSTEM
THE PROCESS CONTROL SUBSYSTEM
The process subsystem read executable files into memory before
executing them.
The system calls for controlling processes are:
fork (create a new process)
exec (overlay the image of a program onto the running process)
exit (finish executing a process)
wait (synchronize process execiition)
brk (control the size of memory allocated to a process)
signal
Student Copy – JUNE 2015
UNIX Kernel Architecture – INTERACTION
FILE SUBSYSTEM & PROCESS CONTROL SUBSYSTEM
FILE SUBSYSTEM & PROCESS CONTROL SUBSYSTEM
Processes interact with the file subsystem via a specific set of system
calls;
open
close
Read
write
stat
chown
chmod
Student Copy – JUNE 2015
UNIX Kernel Architecture –
THE PROCESS CONTROL SUBSYSTEM
MEMORY MANAGEMENT MODULE
It Controls the allocation of memory of a Process.
If system does not have enough physical memory for all processes, the
kernel moves them between main memory and secondary memory
called as swapping
The swapper process is sometimes called the scheduler, because it
"schedules" the allocation of memory for processes and influences the
operation of the CPU scheduler.
Remember !!!
swap device -- swapping system -- paging system
Student Copy – JUNE 2015
UNIX Kernel Architecture –
THE PROCESS CONTROL SUBSYSTEM
SCHEDULER
The scheduler module allocates the CPU to processes
It schedules them to run in-turn until;
1. Voluntarily exit or
2. Kernel preempts them when their recent run time exceeds a time
quantum
INTER-PROCESS COMMUNICATION
Inter-process communication, ranging from asynchronous signaling of
events to synchronous transmission of messages between processes
Student Copy – JUNE 2015
Student Copy – JUNE 2015
Thus we have discussed,
Introduction To Kernel
Programs, Libraries & System calls & its interaction
Programs,
The File Sub System - Responsibilities & Working
The File Sub System - Buffering mechanism
The
The Process Control Sub System - Responsibilities & Working
Interaction <--> File Sub System & Process Control Sub System
The Process Control Sub System - IPC, Scheduler & Memory Mgmt
Student Copy – JUNE 2015
After this Lecture 2;
1. The student shall be able to explain the working of File Sub-system &
inodes
2. The student shall be able to list the CHARACTERS of File & Process Sub-
system
3. The student shall be able to “draw & brief” File Sub-system Layout & its
entities
4. The student shall be able to “brief” Process Sub-system.
5. The student shall be able to “draw & brief” Kernel & User Stacks..
6. The student shall be able to list & explain Process states & its transitions
Student Copy – JUNE 2015
INTRODUCTION TO KERNEL
1. Architecture Of The Unix Operating System
2. Introduction To System Concepts & Kernel Data Structures
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures
1. Overview of File Sub-system
2. PROCESSES
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
About inode :
inode it gives INTERNAL REPRESENTATION of a file.
INTERNAL REPRESENTATION ?
• Description of Disk Layout of a FILE.
• FILE Owner
• Access Permissions
• Access Times
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
About inode :
Inodes are stored in the file system
Every file has one inode.
But it(file) may have several names, all of which map into
the inode, Each name is called a link .
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
When a PROCESS refers to a FILE by “name”, the kernel does
the following:
1. Parses the FILE NAME.
2. Checks if the PROCESS has permission to search the
directories in the path.
3. Retrieves the inode for the file.
open ("/fs2/mjb/study/sourcefile", 1);
(The first parameter path in specifies the file name to be opened)
(The second parameter (flags) specifies how the file may be used)
REFER - http://www.cs.uofs.edu/~bi/2008f-html/cs352/syscalls-file.htm#open
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
When a PROCESS “creates” a new file, the kernel assigns it
an unused inode.
The kernel contains two other data structures,
1.File Table
2.User File Descriptor Table
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
When a process opens or creats a file, the kernel allocates an
entry to each table (file descriptor table, file table, and inode
table)
all 3 tables maintain the “State Of The File” and “User's
Access To It”
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
FILE TABLE
It keeps track of the Byte Offset in the file
(where the user's next read or write will start)
Access Rights allowed to the “opening process”.
USER FILE DESCRIPTOR TABLE
It identifies all “open files” for a process
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
The kernel returns a file descriptor for the open and creat
system calls, which is an index into the user file
descriptor table.
When executing read and write system calls, the kernel
uses the file descriptor to access the user file descriptor
table.
(follows pointers to the file table and inode table entries, and, from the
inode, finds the data in the file)
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
About File Sub-System & Disks
In Unix a disk is Partitioned into several File Systems.
(makes it easier for administrators to manage the data stored)
The kernel deals on a LOGICAL LEVEL with File Systems
rather than with disks.
Treating each one as a LOGICAL DEVICE identified by a
LOGICAL DEVICE NUMBER.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
About File Sub-System & Disks
The logical device (file system) addresses and physical
device (disk) addresses conversion is done by the DISK
DRIVER.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
About File Sub-System & Disks
A File System consists of a sequence of logical blocks -
containing 512, 1024, 2048 Bytes.
Large logical blocks increases the effective data transfer rate
between disk and memory.
Coz, kernel can transfer more data per disk operation.
However, If Logical block is too large, effective storage capacity may
drop
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
OVERVIEW OF FILE SUB-SYSTEM
boot block typically the first sector, and may contain the
bootstrap code
super block describes the “state of a file system” –
• how large it is,
• how many files it can store,
• About free space on the file system, and other information.
inode list is a list of inodes, One inode is the root inode of the
file system
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures
1. Overview of File Sub-system
2. PROCESSES
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES
We will see process subsystem more closely
1. Structure of a process
2. Process data structures used for memory management.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
Every process (is an Entity) is created by the fork system
call. (execpt process 0 – created on boot)
Invoked process is Parent
Created process is Child
Every process has one parent process, but a process can have
many child processes.
Every process has a PID
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
A user compiles the source code of a program to create an
executable file, which consists of several parts:
= A set of "HEADERS" (that describe the attributes of the file)
= The program TEXT
(pattern of bytes during execution of a program that the CPU interprets as
machine instructions)
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
A user compiles the source code of a program to create an
executable file, which consists of several parts:
== The DATA(machine language representation) that has
initial values when the program starts execution,
and How much space the kernel should allocate for
uninitialized data called BSS
== Symbol table information.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The kernel loads an executable file into memory
(during an exec system call)
The loaded process consists of at least three parts, called
REGIONS.
1. Text Regions
2. Data Regions
3. Stack Regions
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The loaded process consists of at least three parts, called
REGIONS.
The text and data regions correspond to the text and data-bss
sections of the executable file.
The stack region is automatically created and its size is
dynamically adjusted by the kernel at run time
Eg:
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The stack consists of FRAMES (logical stack).
Frames that are PUSHED when calling a function
Frames that are POPPED while returning a function
STACK POINTER (special register) indicates the current stack
depth.
So, What does Stack frame contains?
Student Copy – JUNE 2015
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The Stack frame contains;
• Function parameters/arguments
• Local variables
• Value of the program counter and stack pointer
• Data necessary to recover the previous stack frame
Student Copy – JUNE 2015
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
Every Process has an entry in the kernel process table.
Each Process is allocated a U AREA
(contains private data manipulated only by the kernel)
The Process table contains/maps to a per process region
table, whose entries point to entries in a region table.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
What's a REGION?
Region is Process Address Space, such as text, data, and stack.
Region table entries the attributes of the region.
• whether it contains text or data
• whether it is shared or private
• Memory location of "data”
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The mapping between “per process region table” to the
“region table” allows independent processes to share
regions.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
We know, When a process invokes the exec system Call (Loads
a executable to memory)
The kernel allocates regions for its text, data, and stack.
When a process invokes fork, it duplicates the address space
of the old process.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The process table entry and the u area contain control and
status information about the process.
Fields in the process table are;
1. a state field,
2. identifiers-UID indicating the user who owns the
process
3. an event descriptor set when a process is suspended
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
The u area contains information describing the process that
needs to be accessible only when the process is executing.
1. a pointer to the process table slot
2. file descriptors for all open files
3. current directory and current root
4. process and file size limits.
5. parameters of the current system call, return values and
error codes
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
1. Context of a process
2. Process states
3. State transitions
4. Sleep and wakeup
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
CONTEXT OF A PROCESS
The context of a process is its state, as defined by:
• Text
• Values of its global user variables and data structures
• Contents of its user and kernel stacks.
• Values stored in its process table slot and u area
• Values of machine registers it uses
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
CONTEXT OF A PROCESS
When executing a process, the system is said to be
executing in the context of the process
So When the kernel decides that it should execute another
process, it does a CONTEXT SWITCH
Kernel allows a context switch only under specific
conditions
When doing a context switch, the kernel saves enough information so
that it can later switch back to the first process and resume its
execution Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
1. Context of a process
2. Process states
3. State transitions
4. Sleep and wakeup
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
PROCESS STATES
The lifetime of a process can be divided into a set of states.
• The process is currently executing in user mode.
• The process is currently executing in kernel mode.
• The process is not executing, but it is ready to run as soon
as the scheduler chooses it.
• The process is sleeping. A process puts itself to sleep when
it can no longer continue executing
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
1. Context of a process
2. Process states
3. State transitions
4. Sleep and wakeup
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
STATE TRANSITIONS
Nodes represent the states a process can enter.
Edges represent the events that cause a process to move
from one state to another.
State transitions are legal between two states if there
exists an edge
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
STATE TRANSITIONS
The kernel allows a context switch only when a process
moves from the state "kernel running" to the state "asleep
in memory."
Processes running in kernel mode cannot be preempted by
other processes.
Therefore the kernel is sometimes said to be non-
preemptive,
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
1. Context of a process
2. Process states
3. State transitions
4. Sleep and wakeup
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
SLEEP AND WAKEUP
Processes go to sleep because they are awaiting the
occurrence of some event.
When an event occurs, at which time they wake up and
enter the state "ready to run”
Many processes can simultaneously sleep on an event;
when an event occurs, all processes sleeping on the event
wake up & where it is eligible for later scheduling; it does
not execute immediately.
Student Copy – JUNE 2015
Intro To System Concepts & Kernel Data Structures –
PROCESSES on UNIX System
Student Copy – JUNE 2015
INTRODUCTION TO KERNEL &
KERNEL DATA STRUCTURES
Architecture Of The Unix Operating System
Introduction To System Concepts & Kernel Data Structures
That’s it.
Student Copy – JUNE 2015
After this Lecture ;
1. The student shall be able to “draw & brief” UNIX Kernel Architecture
2. The student shall be able to list the CHARACTERS of File & Process Sub-
system
3. The student shall be able to brief the File Sub System - Buffering
mechanism
4. The student shall be able to narrate on Programs, Libraries & System calls
& its interaction
Student Copy – JUNE 2015
After this Lecture 2;
1. The student shall be able to explain the working of File Sub-system &
inodes
2. The student shall be able to list the CHARACTERS of File & Process Sub-
system
3. The student shall be able to “draw & brief” File Sub-system Layout & its
entities
4. The student shall be able to “brief” Process Sub-system.
5. The student shall be able to “draw & brief” Kernel & User Stacks..
6. The student shall be able to list & explain Process states & its transitions
Student Copy – JUNE 2015
