01intro

CS 167: Operating Systems

CS 167 I–1 Copyright © 2006 Thomas W. Doeppner. All rights reserved.

I–1

Workload: 167

• Write five easy-to-moderate-to-difficult
programs (40%)
– shell
– multithreaded programming/high-performance
I/O
– user-level threads package
– system calls and high-level file system
– low-level file system
• Hand in five short homeworks (20%)
• One in-class midterm exam (10%)
• Final exam (30%)
• See http://www.cs.brown.edu/courses/cs167/docs/syllabus.167.htm

I–2

Workload: 169

• One large program with a number of
milestones, resulting in a working operating
system
– Kernel 1
– Kernel 2
– Virtual file system
– S5 file system
– Virtual memory
• See http://www.cs.brown.edu/courses/cs167/docs/syllabus.169.htm


I–3

Skills Needed

• Ability to write and debug largish programs in
C with POSIX threads
– CS 36
– or CS32 with
- C/C++ minicourse (this week)
- POSIX threads lectures/labs (next week)
• Basic computer architecture
– CS 31


I–4

What are Operating Systems?

• Possible definitions:
– the code that {Microsoft, Sun, Apple, Linus}
provides
– the code that you didn’t write
– the code that runs in privileged mode
– the code that makes things work
– the code that makes things crash
– etc.


I–5

Operating Systems

• Abstraction
– providing an “appropriate” interface for
applications
• Concerns
– performance
- time and space
– sharing and resource management
– failure tolerance
– security
– marketability


The purpose of an operating system is to make the underlying hardware useful. Rather than
forcing programmers to deal with the complicated details of such hardware, more convenient
idealized abstractions of the hardware are devised, implemented, and made available, replacing
the direct interface to the hardware. Thus, as discussed in upcoming slides, we deal with files
rather than disks, virtual memory rather than real memory, threads rather than processors, etc.
However, in operating systems, even more so than in “ordinary” applications we must cope with
the concerns listed in the slide. Performance is a huge issue. Some portions of the OS might be
invoked often, say thousands of times per second. Reducing the number of instructions used is
extremely important; we must also be concerned with how effectively the code uses the
processor’s caches. Use of memory, both primary and secondary, is also important. If too much
memory is used, the operating system might not “fit” in the available resources. Furthermore, too
much memory use may require shuffling of code and data between primary and secondary
storage, thus wasting time.
We must also provide various forms of sharing: not only must different applications coexist on a
computer at once, but different users might be using the same computer at the same time. Our
operating system must be tolerant of failures: bugs in one application must not bring down
another. Problems with hardware should not cause catastrophic failure.
Security is increasingly a problem. We must protect the operating system, the applications
running on it, and data maintained by it, from attack. (This is clearly not a solved problem!)
Finally, for an OS to be successful, people must want to use it. It’s one thing to build an
operating system that does an excellent job dealing with everything mentioned above, but if none
of our existing applications can be made to run on it and we have to write everything from
scratch, no one is going to want to use it.

I–6

Hardware

r
so
es
oc
Pr
Network

Memory
or
ss
ro ce
P

Disk Disk


As we’ve just mentioned, one of the major purposes of an operating system is to give its users
an easy-to-use abstraction of a fairly complicated reality. A typical computer system consists of a
number of disk drives (which the operating system accesses through specialized hardware called
controllers), a network interface (also accessed via a controller), some sort of
display/keyboard/mouse combination, and some amount of primary storage.

I–7

Abstractions

• Hardware • Operating system
– disks – files
– memory – programs
– processors – threads of control
– network – communication
– monitor – windows,
– keyboard graphics
– mouse – input
– locator


The typical user has no desire to deal with the complexities of the underlying hardware,
preferring instead to deal with abstractions and depend upon the operating system to map these
abstractions into reality. For example, the user may have information that needs to be accessed
and perhaps updated over long periods of time. While such information may well be stored on
disk, the user probably doesn’t care how.
The user’s computation will certainly use temporary storage that need not exist longer than the
computation, but should be quickly accessible during the lifetime of the computation. The user’s
computation will need a “compute engine” to carry out the computation; in some cases it will need
multiple compute engines. Finally, the user may need to communicate with other users or
computations on other computers, and may need to access information stored elsewhere.
Designing the most suitable abstractions for an operating system to present to its users is a
subject of continuing research. Soon we examine the abstractions provided by a particular
operating system, Unix, that is in widespread use on computer workstations and larger machines.
Later in the course we discuss alternate abstractions.
Unix provides a relatively simple collection of abstractions (though many argue that Unix has
become much too complicated). In particular, permanent storage is represented as files. We think
of temporary storage as being built from the abstraction of primary memory, known as virtual
memory. Each computation has its own private virtual memory—computations are known in Unix
as processes. Traditionally, each process is embodied as a single abstract compute engine, but
more recently the concept of multithreaded processes has been introduced. In this model, each
process may contain multiple compute engines, or threads, each capable of independent
computation. Communication is not a very well developed concept in Unix systems. The general
idea is that independent processes can transfer data back and forth among one another. We
explore this idea and newer approaches to handling communication in the last section of the
course.

I–8

Files

Memory

Disk Disk


File systems provide a simple abstraction of permanent storage, i.e., storage that doesn’t go
away when your program terminates or you shut down the computer. A user accesses
information, represented as files, and the operating system is responsible for retrieving and
storing this information. We expect files to last a long time and we associate names (as opposed to
addresses) with files.

I–9

Issues

• Naming
• Allocating space on disk (permanent storage)
– organized for fast access
– minimize waste
• Shuffling data between disk and memory
(high-speed temporary storage)
• Coping with crashes


Among the issues in implementing the notion of files are those mentioned above.

I–10

Programs

Memory

Data

Code Disk Disk

r r
so so
es es
P roc P roc


A program is an abstraction that we view as consisting of data and code. Somehow we’ve got to
build programs from the pieces of hardware available to us, in particular memory, disks, and
processors.

I–11

Memory
Sharing (1)

Program 1

Program 2

Program 3

Operating
System

Memory


So that multiple programs, each with their own private libraries and each accessing shared
libraries, can coexist without destroying each other, we must have some means of protecting one
program’s memory from another’s. Furthermore, we must protect the operating system from being
inadvertently or maliciously attacked by programs.
Using various hardware features, each program is prevented from accessing private portions of
other programs. Shared portions can be accessed, but only in a read-only fashion. When the
operating system is executing, it can access the user programs. But when user programs are
executing, they can access the operating system only to do prescribed requests.

I–12

Memory
Sharing (2)
Program 1

Program 2

Program 3 Memory


Another memory-sharing issue arises when the programs that are to share memory are too big
to fit in all at once (or perhaps even individually).

I–13

Virtual Memory
Program 1

Memory
Program 2

Disk Disk

Program 3


One popular approach for dealing with memory-protection and -fitting issues is to employ
virtual memory. A program executes in a “virtual address space,” which is implemented partially
on real memory and partially on disks. A facility supported as a joint effort of hardware and the
operating-system kernel “maps” each program’s references of virtual memory into real memory
and disk space. Since the operating system controls this mapping, it determines who can access
what.

I–14

History of Virtual Memory

• 1961: Atlas computer, University of
Manchester, UK
• 1962: Burroughs B5000
• 1972: IBM OS/370
• 1979: 3 BSD Unix, UC Berkeley
• 1993: Microsoft Windows NT 3.1
• 2000: Apple Macintosh OS X


The slide lists milestones in the history of virtual memory, from its first instance on
Manchester’s Atlas computer in 1961 (when a working prototype was completed), to Apple’s
recent announcement of VM support for the Macintosh.

I–15

Apple Invents VM …

Welcome to the Brave New World of Crash-Resistant
Computing
Let’s start with the notion of protected memory, and why
it’s so important. … One of the ways an operating
system ensures reliability is by protecting applications
through a mechanism called protected memory
(essentially walling off applications from each other). …
Along with the protected memory mechanism, Darwin
provides a super-efficient virtual memory manager to
handle that protected memory space. So you no longer
have to worry about how much memory an application
like Photoshop needs to open large files. …
— Apple website, September 2000

Apple is the latest of a long line of companies to “discover” virtual memory. The passage quoted
in the slide is from Apple’s web page and is copyright (by Apple) 2000. “Darwin” was the code
name of the kernel of Apple’s OS X operating system.

I–16

Concurrency

r r r
so so so
es es es
P roc P roc P roc
al al al
Vi rtu Vi rtu Vi rtu

or
ss
o ce
Pr


Computers are typically multiplexed among a number of users. Multiplexing is a concept that is
well over thirty years old—many users share a computer by dividing its processing time among
one another. It’s often known as multitasking.
Another equally old, if not older, concept is the notion of concurrency within a computation.
This is the ability of one computation to utilize multiple logical “compute engines,” or threads.
Concurrency means that multiple threads are in progress at one time; on a computer that
employs only a single real processor, the execution of these threads is multiplexed over time.
Why do this?
• Optimize the use of computers
— programs typically compute, perform I/O, compute, perform I/O …
— when one program is performing I/O, another should be computing
• Optimize the use of people
— want several applications active at once
– interactive applications
· editing
· drawing
– background applications
· mail monitor
· printing
· file transfer

I–17

Parallelism

r r r
so so so
es es es
P roc P roc P roc
al al al
Vi rtu Vi rtu Vi rtu

or or or
ss ss ss
o ce o ce o ce
Pr Pr Pr


Parallelism means that multiple threads are executing simultaneously: parallelism requires
multiple processors.
In this course we make this distinction between parallelism and concurrency; note, however,
that not everyone distinguishes between these terms in this way.

I–18

History of Concurrency

• Multiprogramming
– 1961, 1962: Atlas, B5000
– 1965: OS/360 MFT, MVT
• Timesharing
– 1961: CTSS (developed by MIT for IBM 7094);
BBN time-sharing system for DEC PDP-1
– mid 60s
- Dartmouth Timesharing System (DTSS)
- TOPS-10 (DEC)
– late 60s
- Multics (MIT, GE, Bell Labs)
- Unix (Bell Labs)

Multiprogramming refers to the notion of having multiple programs active at the same time so
that when the current program can no longer run (for example because it’s blocked waiting for
I/O), another is available to run. Timesharing is an extension of multiprogramming in which the
execution of the active programs is time-sliced: each program runs for a short period of time, then
another is run. A good web site with useful information on Multics is http://www.multicians.org/.
A brief description of CTSS can be found at http://www.multicians.org/thvv/7094.html.

I–19

Another Apple Innovation …

With Preemptive Multitasking, Everything Happens
at Once
In today’s fast-paced world, you rarely get to do one
thing at a time. Even in the middle of transforming, say,
a Photoshop file, you may need to find a crucial piece of
information on the web while you compose an urgent
reply to a customer. What you need is a computer that
can handle several different tasks at once, giving
priority to your primary application, but still crunching
away at other jobs in the background. …
Darwin makes this possible by incorporating a powerful
concept called preemptive multitasking. …
— ibid.

So what’s preemptive multitasking? It’s multiprogramming in which one program can preempt
the execution of another. In other words, it’s a necessary ingredient of timesharing, the notion
that was first implemented in the 1960s.
(Note that Microsoft “innovated” this idea in 1993 with their introduction of Windows NT.)

I–20

Brief History of
Operating Systems
• Hardware technology
– first generation: vacuum tubes
– second generation: transistors
– third generation: integrated circuits
– fourth generation: graphics, personal
computers, networking
• OS architecture
– monolithic operating systems
– microkernels
– personal-computer operating systems
– network operating systems


In the next several slides we look at the history of operating systems, showing that the design
for the CS169 final project is the culmination of years of operating-system research and
development. We look first at the history of hardware developments and how they affected OS
design, and then at various architectures of operating systems.

I–21

First-Generation Hardware

• “What you write is all you’ve got”
– no operating system
– no libraries
– no compilers
– no assemblers
– no nothing


The earliest electronic computers were, by our standards, amazingly primitive. The hardware
was characterized by the use of vacuum tubes. The software was characterized by its paucity.
Programmers supplied all software that they were to use. Furthermore, this software was written
directly in “machine language” and input to the computer via awkward means, such as paper
tape.

I–22

IBM 701


The IBM 701 is, reportedly, the first computer to have an operating system (designed by General
Motors (not IBM) and called the Input/Output System). The man on the right is computer pioneer
Herb Grosch, in whose hotel room the notion of an operating system was hashed out in 1953. He
later served as president of ACM. The man on the left is an actor who later served as president of
a large country. The picture is from http://www.fwtunesco.org/musi/museu8.html (the link no
longer works), the web site of FWT UNESCO Computer Museum, Padova, Italy, and was taken in
1956.

I–23

Second-Generation Hardware

• Batch systems
• Concurrent I/O and computation
• Compilers
• Assemblers
• Libraries


The second generation of computer hardware is characterized by the use of transistors.
Improvements in hardware came along with major improvements in software—compilers,
assemblers, and loaders were invented. Libraries were developed and linkers were made available
to ease their use. The early operating systems were known as “batch” systems—a collection, or
batch, of jobs was submitted and their output was written to magnetic tape. After the batch was
completed, the operator would take the tape to another computer that drove a printer and start
the next batch. Thus while one batch was running, the results of the previous batch were printed.
Also at this time the use of I/O devices such as disks, tapes, and drums became more
advanced. In early systems, all activity on a machine ceased while the computation was waiting
for an I/O operation to complete. In the second generation, the use of separate hardware to
manage I/O and the concept of interrupts made it possible for a program to continue to compute
while I/O was being performed. The difficulty of managing such concurrency helped to spur the
development of operating systems.

I–24

IBM 7094


The picture, from http://www.ecs.csun.edu/~dxs/history/histcomp.html, is of an IBM 7094, a midrange
second-generation scientific computer. Among its features were core memory, a disk drive, and a
subroutine-call instruction.

I–25

Early Third-Generation
Hardware
• Multiprogramming
• Time sharing

• Big operating systems
– OS/360
– Multics
• Little operating systems

– Unix


In the third generation, characterized by the use of (small-scale) integrated circuits, OS
technology begins to resemble what we have today. Large machines were multiprogrammed,
allowing several jobs to run concurrently. This made security very important, since jobs now had
to be protected from one another. Another important development was time sharing, in which a
programmer could actually interact with a running program. This was considered a rather
expensive frill at first, but has since become the dominant style of computing.
The most commercially important OS of the day was IBM’s OS/360. This system was based on
the radical concept that a whole range of hardware, from small, slow machines to huge, fast
machines, could all run the same operating system. Unfortunately, it didn’t work exactly as
planned, and OS/360 had to be split into a number of different versions, each targeted to a
particular class of machines. The largest of these OSs, known as OS/MVT, is the forerunner of
the OSs run on large data-processing machines today.
The most interesting OS of the day was Multics, which was initially a joint project of GE, MIT,
and Bell Laboratories. The latter two eventually dropped out and GE’s computer business was
bought by Honeywell, but under the Honeywell nameplate this OS survived into the ’80s. Many of
the concepts that have just now become commercially viable were pioneered in the Multics
system.
The dark-horse operating system was Unix, which was developed at Bell Labs after it dropped
out of the Multics project. It gradually became popular among researchers at Bell Labs. Bell Labs
made the radical decision to release of the source code of this OS to universities, and it soon
became very popular in a large number of computer science departments and a few enlightened
industrial organizations. Unix ran principally on DEC PDP-11s, but was ported to a few other
machines and gained the reputation of being the first portable operating system (another very
radical concept for software that had traditionally been written in assembler language and took
advantage of every arcane feature a machine had to offer). A group at the University of California
at Berkeley took Bell Lab’s port of Unix to DEC’s new architecture, the VAX, and embellished it
considerably. This version of Unix, known as BSD (Berkeley Software Distribution), or simply
Berkeley Unix, was adopted by a large number of academic CS departments around the world.

I–26

$

• If you had $10,000 to invest on January 1,
1962, would you have put it in IBM?


I–27

Later Third-Generation
Hardware
• Minicomputers
– proliferation of platforms and OSes
- Digital PDP-11
• RSX-11, RSTS-11, RT-11, etc.
- Data General Nova, Eclipse
• RDOS, SOS, AOS
- Hewlett Packard 2100, 3000
• BCS, DOS, MPE, MTS
– see http://www.cs.wvu.edu/~jdm/classes/cs258/OScat/minis.html


The ’70s saw an amazing number of minicomputers.

I–28

$$

• It’s now January 1, 1970; your stake in IBM is
worth $22,557


I–29

PDP-11


The PDP-11 shown in the left half of the picture (a relatively small configuration) is around 6-
feet tall, making one wonder about the appellation “mini”! On the right is a closeup of the console
of a model 45 (an early high-end model) and a closeup of the backplane of a model 50 (a later,
midrange model).

I–30

Late 70s, Early 80s

• Super minicomputers
– DEC VAX-11, etc.
• Workstations
– Apollo, Sun
• Personal computers
– MITS Altair, Apple II, TRS-80, IBM PC, etc.
• Xerox PARC
– Alto, Dorado, etc.


During the late ’70s and early ’80s, the minicomputer manufacturers (particularly DEC) thrived
with big sales of high-end minicomputers. The early 80s saw the beginning of workstation-class
computers, with first the establishment of Apollo, then Sun.
Meanwhile, computer hobbyists got into the act with the development of the MITS Altair (the
first affordable piece of hardware that you could reasonably call a computer; it came complete
with 256 8-bit words of memory). Steve Wozniak and Steve Jobs developed the successful Apple II
computer. Then IBM galvanized the entire industry with the introduction of the IBM PC in the
early ’80s.
Perhaps the most important work was being done at Xerox PARC, where the whole notion of
computer workstations and personal computers was invented. (This work actually started in the
early-to-mid ’70s.)

I–31

¢¢

• It’s now January 1, 1975; your stake in IBM
has gone down to $15,323


I–32

VAX-11

• Two operating systems
– VMS (sort of the predecessor of Windows XP)
– Berkeley Unix (definitely the predecessor of
Solaris, Linux, and FreeBSD)


The Digital VAX-11 (a model 780, the first, is shown in the picture) was the most successful of
the superminis. While Digital developed, marketed, and pushed their VMS operating system,
Unix, as modified and developed by a group at UC Berkeley, became extremely popular as well.

I–33

Apollo


The CS Department at Brown was one of Apollo’s first customers (receiving the 5th and 6th
computers delivered to customers).

I–34

Apple II

• See http://apple2history.org/


The Apple II was a very successful hobbyist computer that eventually made it into the business
world, as well as into the educational world.

I–35

PC Operating Systems

• Apple II
– DOS 3.x
– SOS, ProDOS
• Z80, 8080
– CPM
- see http://en.wikipedia.org/wiki/Gary_Kildall
• 8086
– MS-DOS
- See, for example,
http://en.wikipedia.org/wiki/MS-DOS


The early history of PC operating systems is controversial, interesting, entertaining, etc. Check
out the links given in the slide.

I–36

$$$

• It’s now January 1, 1980; your IBM stake is
worth $27,493


I–37

Further PC Development

• Apple introduces the Apple III
– thud …
• Microsoft, selling software, starts doing better
in the PC market than IBM, selling hardware
– $$$$$$$$$$$$$$$
• Apple introduces the Lisa …
– thud …
• Apple introduces the Macintosh

–BOINNNG!!!


I–38

$$$$

• It’s January 1, 1985; your IBM stake has
soared to $76,340


I–39

$$$$$

• It’s March 13, 1986. Microsoft is going public.
Do you put $10,000 into their IPO?


(you didn’t …)

(though my broker has a client who did …)

(and it wasn’t me)

($10,000 got you approximately 477 shares @ $21)

I–40

Toy Operating Systems
• 1987: Andrew Tanenbaum of Vrije
Universiteit, Amsterdam, publishes Operating
Systems: Design and Implementation
– included is source code for a complete, though
toy, operating system: Minix, sort of based on
Unix
• 1991: Linus Torvalds buys an Intel 386 PC
– MS-DOS doesn’t support all its features (e.g.,
memory protection, multi-tasking)
– “soups up” Minix to support all this
• January 1992: Torvalds releases Linux 0.12
• January 1992: Tanenbaum declares Linux
obsolete

I–41

$$$$$$

• It’s January 1, 1990. Your stake in IBM is
worth $64,775.
• The $10,000 you didn’t put into Microsoft
would now be worth $66,071.


(though note that, unlike Microsoft’s, IBM’s stock has always returned dividends. Microsoft
started returning dividends in 2004)

I–42

Late 80s/Early 90s
• 1988: Most major Unix vendors get together
and form OSF to produce a common Unix:
OSF/1, based on IBM’s AIX
• 1989: Microsoft begins work on NT
• 1990: OSF abandons AIX, restarts with Mach
• 1991: OSF releases OSF/1
• 1992: Sun releases Solaris 2
– many SunOS (Solaris 1) programs are broken
• 1993: All major players but DEC have
abandoned OSF/1
• 1993: Microsoft releases Windows NT 3.1
• 1994: Linux 1.0 released

I–43

$$$$$$$

• It’s January 1, 1995. Your IBM stake has
dropped to $57,476
• Your Microsoft stake would have been worth
$391,339


I–44

Late 90s

• IBM has three different versions of Unix, all
called “AIX”
• 1996: DEC renames its OSF/1 “Digital Unix”
• 1996: Microsoft releases Windows NT 4
• 1996: Linux 2.0 released
• 1998: DEC is purchased by Compaq; “Digital
Unix” is renamed “Tru64 Unix”
• 1999: Sun’s follow-on to Solaris 2.6 is called
Solaris 7


I–45

The ’00s
• 2000: Microsoft releases Windows 2000 and Windows
Me
• 2000: Linux 2.2 is released
• 2000: IBM “commits” to Linux (on servers)
• ~2000: Apple releases OS X, based on Unix (in
particular, OSF/1)
• 2001: Microsoft releases Windows XP
• 2002: Compaq is purchased by HP
• 2003: SCO claims their code is in Linux, sues IBM; IBM
countersues
– (trial currently scheduled for Feb. 2007)
• 2005: IBM sells PC business to Lenovo


I–46

$$$$$$$$$$$$$$$$

• It’s January 1, 2000
• Your IBM stake is worth $377,423
• Your Microsoft stake would have been worth
$5,927,143
• You should have sold instantly
– on September 5, 2006
- your IBM stake is worth $282,894
- your Microsoft stake would have been
worth $3,518,199
• If you’d traded all your MS stock for Google at
their IPO on 8/18/2004, you’d now have
$17,209,146

On September 5, 2006 you own 3499 shares of IBM stock. Check out
http://www.microsoft.com/msft/download/IPOsharecalc.xls for information on Microsoft stock.
On 8/18/2004, your Microsoft investment would have been worth $3,765,942. Google sold their
stock for $85/share at their IPO on 8/18/2004. On 9/5/2006, it sold for $384.36/share.

I–47

CS167/169 Operating Systems
An Abbreviated History—Part 1
• Prehistory
– IBM 360-like simulator written in PL/1
– simple OS written in IBM 360 assembler
• Dawn of civilization
– Intel 8086-like architecture
– simple OS written in pseudo code
- paper-only design
• Classical period
– Motorola 68000 simulator written in C
- first ran on 68000-based Apollos, then on 68000-
based Suns, then on SPARC-based Suns
– very simple OS design
– OSes first written in 68000 assembler, later in C

We finish this lecture with a brief history of the OS project of CS 167 and 169.

I–48


• The Enlightenment
– new simulator for “simple architecture” written in C
- no virtual memory
- much faster than previous simulators
– first OS: Unix-like, no virtual memory
– second OS: SLO (Solaris-Like OS)
- first to have a name
- microkernel architecture
• an architecture designed to ease OS writing
• few students completed the project ...


I–49


• The Industrial Revolution: Objects for the
Masses
– same simulator as used in the enlightenment
– OS design based on Sun’s Spring object-
oriented OS; OSes written in C++
- FallOS
- Doors 95
- Doors NT
- OpenDoors


I–50


• The Post-Modern Age
– new simulator, written in C
- supports dynamic address translation
– OS design based on Linux
- virtual memory
- “network capable”
- completely documented (well, almost …)
- marketed under the name Weenix™
– consumer version
Gone the way of
- Weenie™ Windows Me


I–51

01intro

More Related Content

What's hot

Similar to 01intro

Recently uploaded

01intro