Advanced Operating

System
Professor Mangal Sain
 Lecture 10

Security and Protection

Lecture 10 – Part 1

Protection
OBJECTIVES

 Discuss the goals and principles of protection

in a modern computer system
 Explain how protection domains combined
with an access matrix are used to specify the
resources a process may access
 Examine capability and language-based
protection systems
GOALS OF PROTECTION

 In one protection model, computer consists of

a collection of objects, hardware or software
 Each object has a unique name and can be
accessed through a well-defined set of
operations
 Protection problem - ensure that each object
is accessed correctly and only by those
processes that are allowed to do so
PRINCIPLES OF PROTECTION

 Guiding principle – principle of least privilege

 Programs, users and systems should be given just enough
privileges to perform their tasks
 Properly set permissions can limit damage if entity has a
bug, gets abused
 Can be static (during life of system, during life of process)
 Or dynamic (changed by process as needed) – domain
switching, privilege escalation
 Compartmentalization a derivative concept regarding
access to data
 Process of protecting each individual system component through the
use of specific permissions and access restrictions
PRINCIPLES OF PROTECTION (CONT.)

 Must consider “grain” aspect

 Rough-grained privilege management easier, simpler, but
least privilege now done in large chunks
 For example, traditional Unix processes either have abilities of
the associated user, or of root
 Fine-grained management more complex, more overhead,
but more protective
 File ACL lists, RBAC
 Domain can be user, process, procedure
 Audit trail – recording all protection-orientated
activities, important to understanding what
happened, why, and catching things that shouldn’t
 No single principle is a panacea for security
vulnerabilities – need defense in depth
PROTECTION RINGS

 Components ordered by amount of privilege and

protected from each other
 For example, the kernel is in one ring and user
applications in another
 This privilege separation requires hardware support
 Gates used to transfer between levels, for example the
syscall Intel instruction
 Also traps and interrupts
 Hypervisors introduced the need for yet another ring
 ARMv7 processors added TrustZone(TZ) ring to protect
crypto functions with access via new Secure Monitor
Call (SMC) instruction
 Protecting NFC secure element and crypto keys from even
the kernel
PROTECTION RINGS (MULTICS)
 Let Di and Dj be any two domain rings
 If j < I  Di  Dj
ANDROID USE OF TRUSTZONE
ARM CPU ARCHITECTURE
 DOMAIN OF PROTECTION
 Rings of protection separate functions into domains and order them
hierarchically
 Computer can be treated as processes and objects
 Hardware objects (such as devices) and software objects (such as files,
programs, semaphores
 Process for example should only have access to objects it currently
requires to complete its task – the need-to-know principle
 Implementation can be via process operating in a protection domain
 Specifies resources process may access
 Each domain specifies set of objects and types of operations on them
 Ability to execute an operation on an object is an access right
 <object-name, rights-set>
 Domains may share access rights
 Associations can be static or dynamic
 If dynamic, processes can domain switch
DOMAIN STRUCTURE

 Access-right = <object-name, rights-set>

where rights-set is a subset of all valid
operations that can be performed on the object
 Domain = set of access-rights
DOMAIN IMPLEMENTATION (UNIX)

 Domain = user-id
 Domain switch accomplished via file system
 Each file has associated with it a domain bit (setuid bit)
 When file is executed and setuid = on, then user-id is set
to owner of the file being executed
 When execution completes user-id is reset
 Domain switch accomplished via passwords
 su command temporarily switches to another user’s
domain when other domain’s password provided
 Domain switching via commands
 sudo command prefix executes specified command in
another domain (if original domain has privilege or
password given)
DOMAIN IMPLEMENTATION (ANDROID APP IDS)

In Android, distinct user IDs are provided on a per-application

basis
When an application is installed, the installd daemon assigns it a
distinct user ID (UID) and group ID (GID), along with a private
data directory (/data/data/<appname>) whose ownership is
granted to this UID/GID combination alone.
Applications on the device enjoy the same level of protection
provided by UNIX systems to separate users
A quick and simple way to provide isolation, security, and privacy.
The mechanism is extended by modifying the kernel to allow
certain operations (such as networking sockets) only to members
of a particular GID (for example, AID INET, 3003)
A further enhancement by Android is to define certain UIDs as
“isolated,” prevents them from initiating RPC requests to any but
a bare minimum of services
Lecture 10 – Part 2

Protection
 ACCESS MATRIX
 View protection as a matrix (access matrix)
 Rows represent domains
 Columns represent objects
 Access(i, j) is the set of operations that a process
executing in Domaini can invoke on Objectj
USE OF ACCESS MATRIX
 If a process in Domain Di tries to do “op” on object Oj,
then “op” must be in the access matrix
 User who creates object can define access column for
that object
 Can be expanded to dynamic protection
 Operations to add, delete access rights
 Special access rights:
 owner of Oi
 copy op from Oi to Oj (denoted by “*”)

 control – Di can modify Dj access rights

 transfer – switch from domain Di to Dj

 Copy and Owner applicable to an object

 Control applicable to domain object
USE OF ACCESS MATRIX (CONT.)

 Access matrix design separates mechanism

from policy
 Mechanism
 Operating system provides access-matrix + rules
 If ensures that the matrix is only manipulated by

authorized agents and that rules are strictly enforced

 Policy
 User dictates policy
 Who can access what object and in what mode

 But doesn’t solve the general confinement

problem
ACCESS MATRIX OF FIGURE A WITH DOMAINS AS OBJECTS
ACCESS MATRIX WITH COPY RIGHTS
ACCESS MATRIX WITH OWNER RIGHTS
MODIFIED ACCESS MATRIX OF FIGURE B
IMPLEMENTATION OF ACCESS MATRIX

 Generally, a sparse matrix

 Option 1 – Global table
 Store ordered triples <domain, object, rights-
set> in table
 A requested operation M on object Oj within domain
Di -> search table for < Di, Oj, Rk >
 with M ∈ Rk
 But table could be large -> won’t fit in main memory
 Difficult to group objects (consider an object that all
domains can read)
IMPLEMENTATION OF ACCESS MATRIX (CONT.)

 Option 2 – Access lists for objects

 Each column implemented as an access list for one
object
 Resulting per-object list consists of ordered pairs
<domain, rights-set> defining all domains with
non-empty set of access rights for the object
 Easily extended to contain default set -> If M ∈ default
set, also allow access
IMPLEMENTATION OF ACCESS MATRIX (CONT.)

 Each column = Access-control list for one object

Defines who can perform what operation
Domain 1 = Read, Write
Domain 2 = Read
Domain 3 = Read

 Each Row = Capability List (like a key)

For each domain, what operations allowed on what
objects
Object F1 – Read
Object F4 – Read, Write, Execute
Object F5 – Read, Write, Delete, Copy
 IMPLEMENTATION OF ACCESS MATRIX (CONT.)
 Option 3 – Capability list for domains
 Instead of object-based, list is domain based
 Capability list for domain is list of objects together with
operations allows on them
 Object represented by its name or address, called a capability
 Execute operation M on object Oj, process requests operation and
specifies capability as parameter
 Possession of capability means access is allowed

 Capability list associated with domain but never directly

accessible by domain
 Rather, protected object, maintained by OS and accessed

indirectly
 Like a “secure pointer”

 Idea can be extended up to applications

IMPLEMENTATION OF ACCESS MATRIX (CONT.)
 Option 4 – Lock-key
 Compromise between access lists and
capability lists
 Each object has list of unique bit patterns,
called locks
 Each domain as list of unique bit patterns
called keys
 Process in a domain can only access object if
domain has key that matches one of the locks
COMPARISON OF IMPLEMENTATIONS

 Many trade-offs to consider

 Global table is simple, but can be large
 Access lists correspond to needs of users
 Determining set of access rights for domain non-
localized so difficult
 Every access to an object must be checked

 Many objects and access rights -> slow

 Capability lists useful for localizing information

for a given process
 But revocation capabilities can be inefficient
 Lock-key effective and flexible, keys can be
passed freely from domain to domain, easy
revocation
 COMPARISON OF IMPLEMENTATIONS (CONT.)

 Most systems use combination of access

lists and capabilities
 First access to an object -> access list searched
 If allowed, capability created and attached to
process
 Additional accesses need not be checked

 After last access, capability destroyed

 Consider file system with ACLs per file

REVOCATION OF ACCESS RIGHTS

 Various options to remove the access right of

a domain to an object
 Immediate vs. delayed
 Selective vs. general
 Partial vs. total
 Temporary vs. permanent

 Access List – Delete access rights from

access list
 Simple – search access list and remove entry
 Immediate, general or selective, total or
partial, permanent or temporary
REVOCATION OF ACCESS RIGHTS (CONT.)

 Capability List – Scheme required to locate

capability in the system before capability can be
revoked
 Reacquisition – periodic delete, with require and denial
if revoked
 Back-pointers – set of pointers from each object to all
capabilities of that object (Multics)
 Indirection – capability points to global table entry
which points to object – delete entry from global table, not
selective (CAL)
 Keys – unique bits associated with capability, generated
when capability created
 Master key associated with object, key matches master key for
access
 Revocation – create new master key
 Policy decision of who can create and modify keys – object
owner or others?
Lecture 10 – Part 3

Protection
ROLE-BASED ACCESS CONTROL

 Protection can be applied to

non-file resources
 Oracle Solaris 10 provides
role-based access control
(RBAC) to implement least
privilege
 Privilege is right to execute
system call or use an option
within a system call
 Can be assigned to processes
 Users assigned roles granting
access to privileges and
programs
 Enable role via password to
gain its privileges
 Similar to access matrix
 MANDATORY ACCESS CONTROL (MAC)
 Operating systems traditionally had discretionary access
control (DAC) to limit access to files and other objects (for
example UNIX file permissions and Windows access control
lists (ACLs))
 Discretionary is a weakness – users / admins need to do something to
increase protection
 Stronger form is mandatory access control, which even root
user can’t circumvent
 Makes resources inaccessible except to their intended owners
 Modern systems implement both MAC and DAC, with MAC usually a
more secure, optional configuration (Trusted Solaris, TrustedBSD
(used in macOS), SELinux), Windows Vista MAC)
 At its heart, labels assigned to objects and subjects (including
processes)
 When a subject requests access to an object, policy checked to
determine whether or not a given label-holding subject is allowed to
perform the action on the object
 CAPABILITY-BASED SYSTEMS
 Hydra and CAP were first capability-based systems
 Now included in Linux, Android and others, based on POSIX.1e (that
never became a standard)
 Essentially slices up root powers into distinct areas, each
represented by a bitmap bit
 Fine grain control over privileged operations can be achieved by
setting or masking the bitmap
 Three sets of bitmaps – permitted, effective, and inheritable
 Can apply per process or per thread

 Once revoked, cannot be reacquired

 Process or thread starts with all privs, voluntarily decreases set

during execution
 Essentially a direct implementation of the principle of least

privilege
 An improvement over root having all privileges but inflexible (adding
new privilege difficult, etc)
CAPABILITIES IN POSIX.1E
 OTHER PROTECTION IMPROVEMENT METHODS
 System integrity protection (SIP)
 Introduced by Apple in macOS 10.11
 Restricts access to system files and resources, even by root
 Uses extended file attribs to mark a binary to restrict
changes, disable debugging and scrutinizing
 Also, only code-signed kernel extensions allowed and
configurably only code-signed apps
 System-call filtering
 Like a firewall, for system calls
 Can also be deeper –inspecting all system call arguments
 Linux implements via SECCOMP-BPF (Berkeley packet
filtering)
OTHER PROTECTION IMPROVEMENT METHODS (CONT.)

 Sandboxing
 Running process in limited environment
 Impose set of irremovable restrictions early in
startup of process (before main())
 Process then unable to access any resources beyond
its allowed set
 Java and .net implement at a virtual machine level
 Other systems use MAC to implement
 Apple was an early adopter, from macOS 10.5’s
“seatbelt” feature
 Dynamic profiles written in the Scheme language,
managing system calls even at the argument level
 Apple now does SIP, a system-wide platform profile
OTHER PROTECTION IMPROVEMENT METHODS (CONT.)

 Code signing allows a system to trust a program or

script by using crypto hash to have the developer
sign the executable
 So code as it was compiled by the author
 If the code is changed, signature invalid and (some)
systems disable execution
 Can also be used to disable old programs by the operating
system vendor (such as Apple) cosigning apps, and then
invaliding those signatures so the code will no longer run
LANGUAGE-BASED PROTECTION

 Specification of protection in a programming

language allows the high-level description of policies
for the allocation and use of resources
 Language implementation can provide software for
protection enforcement when automatic hardware-
supported checking is unavailable
 Interpret protection specifications to generate calls
on whatever protection system is provided by the
hardware and the operating system
 PROTECTION IN JAVA 2

 Protection is handled by the Java Virtual Machine

(JVM)
 A class is assigned a protection domain when it is
loaded by the JVM
 The protection domain indicates what operations the
class can (and cannot) perform
 If a library method is invoked that performs a
privileged operation, the stack is inspected to
ensure the operation can be performed by the library
 Generally, Java’s load-time and run-time checks
enforce type safety
 Classes effectively encapsulate and protect data
and methods from other classes
STACK INSPECTION