Systems

Programming

Samia Riaz
samiariaz@gmail.com
 Welcome to Systems Programming

• UNIX Environment
• Shell Scripting
• The basic vi editor
• Unix Files System Calls
• Standard I/O, File & Directory Maintenance

12/08/21 Samia Riaz 2

 Welcome to Systems Programming

• Processes. Process Types, Process Creation,

Input/Output Redirection
• Inter Process Communication, Pipes, Named
Pipes Client/Server using FIFOs Sockets,
Connections, APIs
• Network Programming: Client/Server Program
• Threads: creation , synchronization, thread
attributes, canceling.
• Getting user info in unix.

12/08/21 Samia Riaz 3

 Text Book

• Beginning LINUX Programming

2nd Edition
By Richard Stones & Neil Matthew
• Unix™ Systems Programming: Communication,
Concurrency, and Threads
By Kay A. Robbins, Steven Robbins

12/08/21 Samia Riaz 4

 Introduction & Definition

• System programming (or systems

programming) is the activity of building and
maintaining, low-level,   system software for
  computers, including  operating systems,
 system utilities,  compilers etc.

12/08/21 Samia Riaz 5

 Software

• Application programs
– Machine independent
– Solves a specific problem using computer as tool.
– Examples
• Spreadsheets, accounting packages, graphic sw tools,
internet browsers etc…
• Programs to sort, search, etc.

12/08/21 Samia Riaz 6

 Software (cont…)
• Systems programs
– Machine dependent
– Support computers operation
• Compiler
– Translates application programs to intermediate code
• Assemblers
– Translates intermediate code to machine code
• Linkers
– Links the machine code modules into one
• Loaders
– Loads the machine code in memory
• OS
– Controls the operation of the computer

12/08/21 Samia Riaz 7

 Unix History
• 1969 – origin of Unix, inspired by Ken Thompson
• First version written by Thompson in one month with
assembly language.
• 1970 – Rewritten by Thompson and Dennis Ritchie in B (a
scaled down version of BCPL) for a PDP-11 (included line
editor ed, text rendering programs).
• The first user was Bell's Patent Department.
• First edition of Unix Programmer's Manual by Thompson
and Ritchie (dated Nov. 3, 1971) included over 60
commands like: b (compile b program), boot,cat, chdir,
chmod, chown, cp, ls, mv, wc.

12/08/21 Samia Riaz 8

 Unix History
• 1972-1973 – B rewritten into new language called
C,Thompson creates pipes as mechanism for connecting
output of one program to input of another (groundwork for
Unix philosophy), Unix rewritten in C
• University of Toronto on the first mailing list in 1975.
• Software Tools User Group formed in 1978
• Berkeley Software Distribution grew out of collecting and
distributing bug fixes. (Led to FreeBSD, NetBSD)
• Today: many different (proprietary) versions
• UNIX is trademark administrated by X/Open
– Superset of POSIX specifications actively developed by IEEE
• UNIX like system – Commercial (Solaris) and free
(freeBSD & Linux)

12/08/21 Samia Riaz 9

 Why Unix?

• Available on a number of platforms.

• Multi-user, multi-programmed.
• Shares computer resources sensibly.
• Permits manipulation of files, devices,processes,
and programs.
• Allows inter-process and inter-machine
communication.
• Permits access to its operating features.
• A core technology of the Internet.

12/08/21 Samia Riaz 10

 UNIX Philosophy
• The designers of UNIX used the following
maxim while writing the new operating
system.
– Make each program do one thing well. These
simple programs would be called "tools.“
– Expect the output of every program to be the input
to another program.
– Don't stop building new "tools" to do a job. The
library of tools should keep increasing.

12/08/21 Samia Riaz 11

 UNIX Philosophy

• Simplicity
• Focus
• Reusable Components
• Filters
• Open File Formats
• Flexibility

12/08/21 Samia Riaz 12

 BASIC UNIX IDEAS
• UNIX operating system: A set of programs that act as
a link b\w the computer and the user
• Kernel: The programs that allocate the system
resources and coordinate all the details of the
computer's internals
• Shell:
– Users communicate with the kernel through shell
– command line interpreter

12/08/21 Samia Riaz 13

 BASIC UNIX IDEAS
Spread Sheet Compilers
Calculators ³
³ ³
V V
ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿
³ The Shell ³ Mail and
³ ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿ ³<- Message
Inventory ³ ³ UNIX system kernel ³ ³ Facilities
Control --->³ ³ ÚÄÄÄÄÄÄÄÄÄÄÄÄÄÄ¿ ³ ³
Systems ³ ³ ³ ³ ³ ³<- Interpreters

³ ³ ³ Hardware ³ ³ ³
Formatters ->³ ³ ³ ³ ³ ³<- DBMS
³ ³ ³ ³ ³ ³
Calendar ³ ³ ÀÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ ³ ³ Word
Systems ---->³ ³ ³ ³<- Processors
³ ÀÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ ³
Editors ---->³ ³<- FTP
ÀÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÄÙ

12/08/21 Samia Riaz 14

 Linux
• Developed by Linus Torvalds at University of Helsinki
• A “Unix-like” system, sources not derived from Unix source but
interfaces intentionally like Unix.
• Linux & UNIX programs are similar
• Linux is actually just a kernel
• Open source: source code is freely available, can be freely
modified
• Linux Systems
– Many distributions Fedora, Suse,Debians, RedHat, Slackware etc
• Linux Community supports the concept of free software (GNU
General Public License)
• Free Software Foundation
– Richard Stallman, the author of GNU Emacs
• Copyleft instead of Copyright

12/08/21 Samia Riaz 15

 Linux or UNIX
• Two types of executables i,e programs and scripts
• No special type of formats like DOS, exe, com etc
• Path /bin: /usr/bin: /usr/local/bin: /sbin: . : /usr/sbin: /opt
• gcc, cc, g++, c89 (type info gcc)
• Header Files /usr/include and /usr/include/sys or
/usr/include/linux
• Library Files /lib /usr/lib /usr/lib/libc (.a and .so or .sa)

12/08/21 Samia Riaz 16

 Programming languages

• Here's a partial list of programming languages

available to the UNIX programmer:

12/08/21 Samia Riaz 17

 Linux Basics
• Kernel
• Shell (BASH, CSH,KSH)
– Shell is an command language interpreter that
executes commands read from the standard input
device
– $ echo $SHELL
• Redirectors, Pipes, Filters etc
Your Linux Kernel
Command Converted to binary
Linux Shell uderstand
or Shell language by shell

Script the Request

12/08/21 Samia Riaz 18

 Some Linux Commands

date who pwd

ls /dirs rm {filename} mv {f1} {f2}
Cat > {filename} mail grep {w} {file}
(need ^D) mail {username}
pr {filename} tail -|+ {linenumber} {filename}

shutdown chmod {u|g|o|a} {+|-} {filename}

ps kill {pid}, ps –ag , linux_command &

12/08/21 Samia Riaz 19

 Linux Commands (cont..)
• Redirection
– Linux-command > file
– Linux-command >> file
– Linux-command < file
• Pipes
Pipe is used to connect the output of one program to the
input of another program without any temporary file

Take input
Output of from Ist
First command for
Command this command

12/08/21 Samia Riaz 20

 Introduction to Shell Scripting

Shell Scripting

12/08/21 Samia Riaz 21

 Filename Conventions
• In UNIX, a file is the basic component for data storage.
• UNIX considers everything it interacts with as a file,
even attached devices such as monitors.
• A file system is the UNIX system’s way of organizing
files on mass storage devices such as hard and floppy
disks.
• Directory is merely a special type of file
• Some applications give the same name to all the
output files they generate

12/08/21 Samia Riaz 22

 Unix File System Structure

• Every item in the file system is a file (or a link).

• A directory is a special kind of file can contain
other files and directories (called
subdirectories).
• The subdirectory is considered as the child of
the parent directory.
• Each directory may have only one parent.

12/08/21 Samia Riaz 23

 File and Directory Names
• Valid names can be made up of:
– Uppercase letters (A to Z)
– Lower case letters (a to z)
– Numbers (0 to 9)
– period (.), underscore (_), commas (,).
• Should not contain spaces or the following:
– &*\|[]{}$<>()#?‘“/;^!~%
– You should also avoid naming files or directories with Unix
commands.
– Should not start with “-”
• Filenames and pathnames have limits on lengths
– 1024 characters typically
– these are pretty long (much better than MS-DOS days)

12/08/21 Samia Riaz 24

 Extension of the file concept

• Unix files come in other flavors as well, such as

– Directories
• a file containing pointers to other files
• equivalent of a “folder” on a Mac or Windows
– Links
• a pointer to another file
• used like the file it points to
• similar to “shortcuts” in Windows, but better
– Devices
• access a device (like a soundcard, or mouse, or ...) like it is a
file
• You can read and write the devices

12/08/21 Samia Riaz 25

12/08/21 Samia Riaz 26
 Some Standard Directories

• Root directory (/)

– at the top of the file system hierarchy. Contains some standard files
and directories.
• /bin
– binaries, or executables, which are the programs needed to start the
system and perform other essential system tasks.
• /dev
– Files in /dev references system devices. They access system devices
and resources such as hard disks, printers, …
• /etc
– configuration files that the system uses when the computer starts,
such as passwd (user info file), rc (scripts or directories of scripts)

12/08/21 Samia Riaz 27

 Lectures Out line

• Topic One Navigating the File System

• Topic Two
• Topic Three
• Topic Four
• Topic Five

12/08/21 Samia Riaz 28

 Topic One

• Listing files and directories

• Making Directories
• Changing to a different Directory
• The directories . and ..
• Pathnames

12/08/21 Samia Riaz 29

 Listing Files & Directories

• ls (list)
• ls -a

12/08/21 Samia Riaz 30

 Making Directories

• mkdir (make directory)

• mkdir dir_name

12/08/21 Samia Riaz 31

 Changing to a different directory 

• cd (change directory)
• cd dir_name

12/08/21 Samia Riaz 32

 The directories . and ..

• cd . denotes the current working directory

• cd .. denotes the parent of the current working
directory
• Typing cd with no argument always returns
you to your home directory
• * There is a space between cd and the dot

12/08/21 Samia Riaz 33

 Pathnames
• pwd (print working directory)
• where you are in relation to the whole file-system
• pwd
• ~ (your home directory)
• To describe an object in the file system you specify a path.
• Paths are either absolute, starting with the root level:
/users/faculty/sarwar/courses/ee446
• Or they are relative, staring with the current working directory or
a user’s home directory
~/courses/ee446
./courses/ee446
• Exercise
– ls ~ ?
– ls ~/.. ?
– What would be the out put?

12/08/21 Samia Riaz 34

 Navigating the File System

pwd print working directory name

cd <dir> change to <dir>
cd .. change to parent dir
ls list directory contents
ls -l long listing
ls -a list all files
ls -al long listing all files
ls –F mark directories with /, executable files with *
ls –R Recursively lists subdirectories encountered

12/08/21 Samia Riaz 35

 Summary

ls list files and directories

ls -a list all files and directories
mkdir make a directory

cd directory change to named directory

cd change to home-directory
cd ~ change to home-directory
cd .. change to parent directory
pwd display the path of the current directory

12/08/21 Samia Riaz 36

 Topic Two

• Copying Files
• Moving Files
• Removing Files and directories
• Displaying the contents of a file on the screen
• Searching the contents of a file

12/08/21 Samia Riaz 37

 Copying Files
• cp (copy)
• Syntax
– cp file1 file2
– cp [options] source target
• Common options:
-i : interactive; prompts before overwriting an existing file
-r : recursive; cp will copy the directory and all its files, including
any subdirectories and their files to target
• Example
– cp goodies goodies.old
– cp /vol/examples/tutorial/science.txt .
• Exercise
– Create a backup of your file1.txt file by copying it to a file called
file1.bak

12/08/21 Samia Riaz 38

 Moving files
• mv (move)
• Syntax
– mv file1 file2
• Examples
– mv [-fi] source target_file
• moves the file named by source to the destination specified by
target_file.
– mv [-fi] source... target_dir
• moves each file named by source to a destination file in the
existing directory named by target_dir.
-f move the file(s) without prompting even if it is writing over an
existing target.
-i prompt for confirmation whenever the move would overwrite an
existing target.

12/08/21 Samia Riaz 39

 Moving files

Examples:
• mv file1 file2
– Rename the file named file1 in the current working
directory to file2
• mv file2 .. ?

12/08/21 Samia Riaz 40

 Removing files and directories

• rm (remove)
• This is destructive!
• They are gone!
• They cannot be retrieved!!!
• rmdir (remove directory)
• Syntax
– rm file1
• Exercise
– Create a directory called tempstuff using mkdir ,
then remove it using the rmdir command
12/08/21 Samia Riaz 41
 Displaying the contents of a file on the
screen
• clear (clear screen)
• cat (concatenate)
• less

12/08/21 Samia Riaz 42

 Filters
• A filter is a command that takes data from
standard in and performs some simple
transformation on it, the result of which is sent on
to standard out.
• Examples:
– sort - sort standard in
– grep (and its derivatives) - search for keyword
information
– head - send on only the front end
– tail - send on only the tail end
– wc - count words, lines, and/or characters
– crypt - encrypt standard in (use with caution!)
12/08/21 Samia Riaz 43
 Sort

• sort (which does nothing more than sort a file)

– Example:
Sort
B
A
C
– finally enter an EOF (<Ctrl> d) at which point the
command will sort your input and output it to standard
output.

12/08/21 Samia Riaz 44

 Searching the contents of a file

• Simple searching using less

• grep (global regular expression - print)
– grep -n 'symbval' pass2.c
– gives the line numbers and prints the lines containing
the specific symbol "symbval".
• wc (word count)
– wc -lwc myfile counts lines, words, and characters in
the file.
– If any one of l, w, or c is omitted, that count is not
provided.

12/08/21 Samia Riaz 45

 Displaying the contents of a file on the
screen
• CONCEPT: Text files are intended for direct viewing, and other
files are intended for computer interpretation.
• The UNIX file command allows you to determine whether an
unknown file is in text format, suitable for direct viewing.
– EXERCISE: Type the command
• file /bin/sh
– to see what kind of file the shell is. (1)
• Head send on only the front end (2)
– head science.txt
– Then type
– $ head -5 science.txt
– What difference did the -5 do to the head command?
• tail send on only the tail end (3)
– How can you view the last 15 lines of the file?

12/08/21 Samia Riaz 46

 Summary

cp file1 file2 copy file1 and call it file2

mv file1 file2 move or rename file1 to file2
rm file remove a file
rmdir directory remove a directory
cat file display a file
more file display a file a page at a time
head file display the first few lines of a file
tail file display the last few lines of a file
grep 'keyword' file search a file for keywords
wc file count number of lines/words/characters in file

12/08/21 Samia Riaz 47

 Topic Three

• Redirection
• Redirecting the Output
• Redirecting the Input
• Pipes
• Two commands can be executed from a single
line; e.g.,
– date; cal

12/08/21 Samia Riaz 48

 Redirection (Introduction)
• standard output (terminal screen )
• standard input (keyboard)
• standard error (by default, to the terminal
screen)
• type
– cat
abc
– [Return]
– [Ctrl + d ]
• The out put would be
abc

12/08/21 Samia Riaz 49

 Redirection
• Redirection: is a means of instructing the shell to
use something other than the default for standard
input or standard output.
• The formats for standard in and standard out are:
– <command> < <file-name>
• for <command> to get its input from <file-name> rather than
standard in.
– <command> > <file-name>
• for <command> to send its output to <file-name> rather than
standard out.
– <command> >> <file-name>
• for <command> to append its output to <file-name>.

12/08/21 Samia Riaz 50

 Redirecting the Output
• Symbols used
– >
– >>
• Eg
– cat > topics
Copying Files
Moving Files
Removing Files and directories
– ^D (Control D to stop)
• Eg
– cat >> topics
Displaying the contents of a file on the screen
Searching the contents of a file
– ^D (Control D to stop)
• Exercise
– use the cat command to join (concatenate) file1 and file2 into a new
file (final.txt)

12/08/21 Samia Riaz 51

 Redirecting the Input
• Symbol used
– <
• Eg
– sort
C
A
B
– ^D (Control D to stop)
• Output will be
A
B
C
• Exercise
– Use sort to sort the contents of final.txt & redirect the output
to a new file sorted.txt

12/08/21 Samia Riaz 52

 Pipes
• Symbol used
– vertical bar |
– <command-1> | <command-2>
– denote that standard out from <command-1> is to be piped
<command-2> as standard in.
• Eg
– who (To see who is on the system with you )
– How to get a sorted list of names?
• Mehtod1
– who > names.txt
– sort < names.txt
• More efficient method is
– who | sort
• Exercise
– Using pipes, print all lines of file1 and file2 containing the
letter 'p', sort the result, and print to the printer.

12/08/21 Samia Riaz 53

 Summary
command > file redirect standard output to a file

command >> file append standard output to a file

command < file redirect standard input from a file

command1 | command2 pipe the output of command1 to the input of command2

cat file1 file2 > file0 concatenate file1 and file2 to file0

sort sort data

who list users currently logged in

a2ps -Pprinter textfile print text file to named printer

lpr -Pprinter psfile print postscript file to named printer

12/08/21 Samia Riaz 54

 Topic Four

• Wildcards
• Filename Conventions
• Getting Help
• Finding files
• Unix file times
• Generating calendar

12/08/21 Samia Riaz 55

 Wildcards (meta char )
• The characters * and ?
• *
– will match against none or more character(s) in a file (or directory)
name
– Eg
• ls list*
• This will list all files in the current directory starting with list....
• ?
– Will match exactly one character
– Eg
• ls ?ouse
• will match files like house and mouse, but not grouse
• [...] matches any one character between the braces; can use “-” to
indicate arange of characters
– Example: b[aei]t matches bat, bet, or bit, not baet
– Example: b[a-c]t matches bat, bbt, or bct, not bxt
– ls [abc]*.?

12/08/21 Samia Riaz 56

 Wildcards (meta char )
• Examples:
– ls *.c
– rm file[1-6].?
– cd ~/bin
– ls ~reid
– ls *.[^oa] - ^ in csh, ! in sh
• Explanation:
– * stands in for 0 or more characters
– ? stands in for exactly one character
– [1-6] stands in for one of 1, 2, 3, 4, 5, 6
– [^oa] stands in for any char except o or a
– ~/ stands in for your home directory
– ~reid stands in for reid’s home directory

12/08/21 Samia Riaz 57

 Getting Help

• man wc
• whatis wc
• apropos keyword

12/08/21 Samia Riaz 58

 Finding files
• find <path> <options>
• path is one or more directories in which find will
begin to search
• Find will recursively search all sub-directories.
• options
– name, perm, type, user, group, size, inum, atime,
mtime, ctime, newer
– Many more
• example: find all the *.c* and *.h* files in and
below the current directory
– find . –name ’*.[ch]*’

12/08/21 Samia Riaz 59

 UNIX file times
• Three Unix file times
– Change time (ctime)
• a change modifies the attributes of the file
– Modification time
• a change modifies the contents of the file
– Access time (atime)
• the last time the file was read or written
• Note:
– Reading a file updates its access time but not its change
time or modification time
– These time attributes can also be used in the ‘find’
command

12/08/21 Samia Riaz 60

 Generating calendar
• Show a calendar for this month
– cal
• Show a calendar for a specific month
– cal 12 2000
• Show a calendar for a specific year
– cal 2000

12/08/21 Samia Riaz 61

 Summary

* match any number of characters

? match one character

man command read the online manual page for a command

whatis command brief description of a command

apropos keyword match commands with keyword in their man pages

12/08/21 Samia Riaz 62

 Topic Five

• File system security (access rights)

• Changing access rights
• Processes and Jobs
• Listing suspended and background processes
• Killing a process

12/08/21 Samia Riaz 63

 File system security (access rights)
• ls -l (l for long listing!) [diff b/w ls –l & ls -lg]*
– E.g
– -rwxrw-r-- 1 ee51ab beng95 2450 Sept29 11:52 file1
• In the left-hand column is a 10 symbol string
consisting of the symbols d, r, w, x, -
– 1st d or –
– The 9 remaining symbols indicate the permissions, or access
rights, and are taken as three groups of 3.
• The left group of 3 gives the file permissions for the user that
owns the file (or directory) (ee51ab in the above example);
• The middle group gives the permissions for the group of people
to whom the file (or directory) belongs (beng95 in the above
example);
• The rightmost group gives the permissions for all others.

12/08/21 Samia Riaz 64

 File system security (access rights)
• Access rights on files
– r (or -), indicates read permission (or otherwise)
– w (or -), indicates write permission (or otherwise)
– x (or -), indicates execution permission
• Access rights on directories
– r allows users to list files in the directory;
– w means that users may delete files from the
directory or move files into it;
– x means the right to access files in the directory.
This implies that you may read files in the directory
provided you have read permission on the
individual files

12/08/21 Samia Riaz 65

 File system security (access rights)

• E.g

-rwxrwxrwx a file that everyone can read, write and execute (and delete).
a file that only the owner can read and write - no-one else
-rw------- can read or write and no-one has execution rights (e.g. your
mailbox file).

12/08/21 Samia Riaz 66

 Changing access rights
Symbol Meaning
• Only the owner of a file
can use chmod u user

• chmod (changing a file g group

mode) o other
• E.g
a all
– chmod go-rwx biglist
r read

w write (and delete)

execute (and access
x
directory)
+ add permission

- take away permission

12/08/21 Samia Riaz 67

 More on chmod . . .
• chmod ijk <name>
• sets permissions for
– owner (i)
– group (j)
– all others (k)
• where each of i, j, and k are octal digits (0-7). Viewed bitwise,
• 4 = read, 2 = write, 1 = execute.
• Since 7=4+2+1
• E.g
– chmod 700 <file-name>
– sets read, write, and execute permission for the file owner and for no
one else
• Suppose that you have a subdirectory named “mywork” in your
current working directory. Then
– chmod 700 mywork ????

12/08/21 Samia Riaz 68

 Processes and Jobs

• A job is a program whose execution has been

initiated by the user.
• At any moment, a job can be running or
suspended.
• Foreground job:
– a program which has control of the terminal
• Background job:
– runs concurrently with the parent shell and does not
take control of the keyboard.

12/08/21 Samia Riaz 69

 Processes and Jobs
• ps (process status)
• In general the shell does not return the UNIX prompt until the
current process has finished executing
• Running background processes
– Backgrounding a long process has the effect that the UNIX prompt
is returned immediately
– Symbol used is &
– E.g
– sleep 10 &
[1] 6259
• Backgrounding a current foreground process
– E.g
– sleep 100 (the current process)
– [control] key and typing [z] (^Z will suspend the process)
– bg

12/08/21 Samia Riaz 70

 Listing suspended and background
processes
• jobs
• E.g (output)
– [1] Suspended sleep 100
[2] Running netscape
[3] Running nedit
• foreground a suspended processes
– fg jobnumber
– E.g.
– fg 1
– * Typing fg with no job number??? *

12/08/21 Samia Riaz 71

 Killing a process
• kill (terminate or signal a process)
• kill a job running in the foreground
– ^C
• kill a suspended or background process
– kill jobnumber
• ps (process status) to find out the PID then use it to
kill a process
– kill PID_number
• * If a process refuses to be killed????????*

12/08/21 Samia Riaz 72

 Summary
ls -lag list access rights for all files
chmod [options] file change access rights for named file
command & run command in background
^C kill the job running in the foreground
^Z suspend the job running in the foreground
bg background the suspended job
jobs list current jobs
fg 1 foreground job number 1
kill 1 kill job number 1
ps list current processes
kill 26152 kill process number 26152

12/08/21 Samia Riaz 73

 Work to do …
1. Quota
– Built in kernel utility that keeps you from exceeding values for
disk space set by the Gurus who run your system
2. df
– how they are partitioned,
– how big they are,
– how much of that is used, and available
3. Du
– disk usage
– expresses itself in Kbytes
– du -s to get a summary of all the files you own, underneath that
point
4. compress
5. gzip
6. File
7. History
8. awk

12/08/21 Samia Riaz 74