Systems Programming: Samia Riaz

Systems Programming
Samia Riaz samiariaz@gmail.com
Text Book
Unix Systems Programming: Communication, Concurrency, and Threads by
Kay A. Robbins, Steven Robbins
3/9/2012
Samia Riaz
Unix History
1969 origin of Unix, inspired by Ken Thompson First version written by Thompson in one month with assembly language. 1970 Rewritten by Thompson and Dennis Ritchie in B (a scaled down version of BCPL) for a PDP-11 (included line editor ed, text rendering programs). The first user was Bell's Patent Department. First edition of Unix Programmer's Manual by Thompson and Ritchie (dated Nov. 3, 1971) included over 60 commands like: b (compile b program), boot,cat, chdir, chmod, chown, cp, ls, mv, wc.
3/9/2012
Samia Riaz
Unix History
1972-1973 B rewritten into new language called C,Thompson creates pipes as mechanism for connecting output of one program to input of another (groundwork for Unix philosophy), Unix rewritten in C University of Toronto on the first mailing list in 1975. Software Tools User Group formed in 1978 Berkeley Software Distribution grew out of collecting and distributing bug fixes. (Led to FreeBSD, NetBSD) Today: many different (proprietary) versions UNIX is trademark administrated by X/Open
Superset of POSIX specifications actively developed by IEEE
UNIX like system Commercial (Solaris) and free (freeBSD & Linux)
3/9/2012
Samia Riaz
Why Unix?
Available on a number of platforms. Multi-user, multi-programmed. Shares computer resources sensibly. Permits manipulation of files, devices,processes, and programs. Allows inter-process and inter-machine communication. Permits access to its operating features. A core technology of the Internet.
3/9/2012 Samia Riaz 5
UNIX Philosophy
The designers of UNIX used the following maxim while writing the new operating system.
Make each program do one thing well. These simple programs would be called "tools. Expect the output of every program to be the input to another program. Don't stop building new "tools" to do a job. The library of tools should keep increasing.
3/9/2012
Samia Riaz
UNIX Philosophy
Simplicity Focus Reusable Components Filters Open File Formats Flexibility
3/9/2012
Samia Riaz
BASIC UNIX IDEAS

UNIX operating system: A set of programs that act as a link b\w the computer and the user Kernel: The programs that allocate the system resources and coordinate all the details of the computer's internals Shell:
Users communicate with the kernel through shell command line interpreter
3/9/2012
Samia Riaz
BASIC UNIX IDEAS

Spread Sheet Compilers Calculators V V The Shell <Inventory UNIX system kernel Control ---> Systems < Hardware Formatters -> < Calendar Systems ----> < Editors ----> <
Mail and Message Facilities Interpreters DBMS Word Processors FTP
3/9/2012
Samia Riaz
Linux
Developed by Linus Torvalds at University of Helsinki A Unix-like system, sources not derived from Unix source but interfaces intentionally like Unix. Linux & UNIX programs are similar Linux is actually just a kernel Open source: source code is freely available, can be freely modified Linux Systems
Many distributions Fedora, Suse,Debians, RedHat, Slackware etc
Linux Community supports the concept of free software (GNU General Public License) Free Software Foundation
Richard Stallman, the author of GNU Emacs
Copyleft instead of Copyright
3/9/2012
Samia Riaz
10
Linux or UNIX
Two types of executables i,e programs and scripts No special type of formats like DOS, exe, com etc Path /bin: /usr/bin: /usr/local/bin: /sbin: . : /usr/sbin: /opt gcc, cc, g++, c89 (type info gcc) Header Files /usr/include and /usr/include/sys or
/usr/include/linux /usr/lib /usr/lib/libc (.a and .so or .sa)
Library Files /lib
3/9/2012
Samia Riaz
11
Programming languages
Here's a partial list of programming languages available to the UNIX programmer:
3/9/2012
Samia Riaz
12
Linux Basics
Kernel Shell (BASH, CSH,KSH)
Shell is an command language interpreter that executes commands read from the standard input device $ echo $SHELL
Redirectors, Pipes, Filters etc

Your Command or Shell Script Linux Kernel
Linux Shell
Converted to binary language by shell
uderstand the Request
3/9/2012
Samia Riaz
13
Some Linux Commands

date ls /dirs Cat > {filename}
(need ^D)
who rm {filename} mail mail {username}
pwd mv {f1} {f2}
grep {w} {file}
pr {filename}
shutdown ps
tail -|+ {linenumber} {filename}

chmod {u|g|o|a} {+|-} {filename}
kill {pid}, ps ag , linux_command &
3/9/2012
Samia Riaz
14
Linux Commands (cont..)

Redirection
Linux-command > file Linux-command >> file Linux-command < file
Pipes Pipe is used to connect the output of one program to the input of another program without any temporary file
Output of First Command

3/9/2012 Samia Riaz
Take input from Ist command for this command
15
Introduction to Shell Scripting
Shell Scripting
3/9/2012
Samia Riaz
16
Filename Conventions
In UNIX, a file is the basic component for data storage.
UNIX considers everything it interacts with as a file, even attached devices such as monitors.
A file system is the UNIX systems way of organizing files on mass storage devices such as hard and floppy disks. Directory is merely a special type of file Some applications give the same name to all the output files they generate
3/9/2012
Samia Riaz
17
Unix File System Structure

Every item in the file system is a file (or a link). A directory is a special kind of file can contain other files and directories (called subdirectories). The subdirectory is considered as the child of the parent directory. Each directory may have only one parent.
3/9/2012
Samia Riaz
18
File and Directory Names

Valid names can be made up of:
Uppercase letters (A to Z) Lower case letters (a to z) Numbers (0 to 9) period (.), underscore (_), commas (,).
Should not contain spaces or the following:

&*\|[]{}$<>()#?/;^!~% You should also avoid naming files or directories with Unix commands. Should not start with -
Filenames and pathnames have limits on lengths

1024 characters typically these are pretty long (much better than MS-DOS days)
3/9/2012
Samia Riaz
19
Extension of the file concept

Unix files come in other flavors as well, such as
Directories
a file containing pointers to other files equivalent of a folder on a Mac or Windows
Links
a pointer to another file used like the file it points to similar to shortcuts in Windows, but better
Devices
access a device (like a soundcard, or mouse, or ...) like it is a file You can read and write the devices
3/9/2012
Samia Riaz
21
Some Standard Directories

Root directory (/)
at the top of the file system hierarchy. Contains some standard files and directories.
/bin
binaries, or executables, which are the programs needed to start the system and perform other essential system tasks.
/dev
Files in /dev references system devices. They access system devices and resources such as hard disks, printers,
/etc
configuration files that the system uses when the computer starts, such as passwd (user info file), rc (scripts or directories of scripts)
3/9/2012
Samia Riaz
22
Lectures Out line

Topic One Navigating the File System Topic Two Topic Three Topic Four Topic Five
3/9/2012
Samia Riaz
23
Topic One
Listing files and directories Making Directories Changing to a different Directory The directories . and .. Pathnames
3/9/2012
Samia Riaz
24
Listing Files & Directories

ls (list) ls -a
3/9/2012
Samia Riaz
25
Making Directories
mkdir (make directory) mkdir dir_name
3/9/2012
Samia Riaz
26
Changing to a different directory

cd (change directory) cd dir_name
3/9/2012
Samia Riaz
27
The directories . and ..

cd . denotes the current working directory cd .. denotes the parent of the current working directory Typing cd with no argument always returns you to your home directory * There is a space between cd and the dot
3/9/2012
Samia Riaz
28
Pathnames
pwd (print working directory) where you are in relation to the whole file-system pwd ~ (your home directory) To describe an object in the file system you specify a path. Paths are either absolute, starting with the root level:
/users/faculty/sarwar/courses/ee446
Or they are relative, staring with the current working directory or a users home directory
~/courses/ee446 ./courses/ee446
Exercise
ls ~ ? ls ~/.. ? What would be the out put?
Navigating the File System

pwd print working directory name cd <dir> change to <dir> cd .. change to parent dir ls list directory contents ls -l long listing ls -a list all files ls -al long listing all files ls F mark directories with /, executable files with * ls R Recursively lists subdirectories encountered
3/9/2012
Samia Riaz
30
Summary
ls
list files and directories
ls -a
mkdir cd directory cd cd ~ cd .. pwd
list all files and directories

make a directory change to named directory change to home-directory change to home-directory change to parent directory display the path of the current directory
3/9/2012
Samia Riaz
31
Topic Two
Copying Files Moving Files Removing Files and directories Displaying the contents of a file on the screen Searching the contents of a file
3/9/2012
Samia Riaz
32
Copying Files
cp (copy) Syntax
cp file1 file2
cp [options] source target Common options: -i : interactive; prompts before overwriting an existing file -r : recursive; cp will copy the directory and all its files, including any subdirectories and their files to target Example cp goodies goodies.old
cp /vol/examples/tutorial/science.txt .
Exercise
Create a backup of your file1.txt file by copying it to a file called file1.bak
3/9/2012
Samia Riaz
33
Moving files
mv (move) Syntax
mv file1 file2
Examples
mv [-fi] source target_file
moves the file named by source to the destination specified by target_file.
mv [-fi] source... target_dir

moves each file named by source to a destination file in the existing directory named by target_dir. -f move the file(s) without prompting even if it is writing over an existing target. -i prompt for confirmation whenever the move would overwrite an existing target.
Moving files
Examples: mv file1 file2
Rename the file named file1 in the current working directory to file2
mv file2 .. ?
3/9/2012
Samia Riaz
35
Removing files and directories

rm (remove) This is destructive! They are gone! They cannot be retrieved!!! rmdir (remove directory) Syntax
rm file1
Exercise
Create a directory called tempstuff using mkdir , then remove it using the rmdir command
Displaying the contents of a file on the screen

clear (clear screen) cat (concatenate) less
3/9/2012
Samia Riaz
37
Filters
A filter is a command that takes data from standard in and performs some simple transformation on it, the result of which is sent on to standard out. Examples:
sort - sort standard in grep (and its derivatives) - search for keyword information head - send on only the front end tail - send on only the tail end wc - count words, lines, and/or characters crypt - encrypt standard in (use with caution!)
Sort
sort (which does nothing more than sort a file)
Example: Sort B A C finally enter an EOF (<Ctrl> d) at which point the command will sort your input and output it to standard output.
3/9/2012
Samia Riaz
39
Searching the contents of a file

Simple searching using less grep (global regular expression - print)
grep -n 'symbval' pass2.c gives the line numbers and prints the lines containing the specific symbol "symbval".
wc (word count)
wc -lwc myfile counts lines, words, and characters in the file. If any one of l, w, or c is omitted, that count is not provided.
3/9/2012
Samia Riaz
40
Displaying the contents of a file on the screen

CONCEPT: Text files are intended for direct viewing, and other files are intended for computer interpretation. The UNIX file command allows you to determine whether an unknown file is in text format, suitable for direct viewing.
EXERCISE: Type the command
file /bin/sh
to see what kind of file the shell is. (1)
Head

send on only the front end (2)
head science.txt Then type $ head -5 science.txt What difference did the -5 do to the head command?
tail send on only the tail end (3)

How can you view the last 15 lines of the file?
3/9/2012
Samia Riaz
41
Summary
cp file1 file2 mv file1 file2 rm file rmdir directory cat file more file
copy file1 and call it file2 move or rename file1 to file2 remove a file remove a directory display a file display a file a page at a time
head file
tail file grep 'keyword' file wc file
display the first few lines of a file

display the last few lines of a file search a file for keywords count number of lines/words/characters in file
3/9/2012
Samia Riaz
42
Topic Three
Redirection Redirecting the Output Redirecting the Input Pipes Two commands can be executed from a single line; e.g.,
date; cal
3/9/2012
Samia Riaz
43
Redirection (Introduction)
standard output (terminal screen ) standard input (keyboard) standard error (by default, to the terminal screen) type
cat
abc
[Return] [Ctrl + d ]
The out put would be

abc
Redirection
Redirection: is a means of instructing the shell to use something other than the default for standard input or standard output. The formats for standard in and standard out are:
<command> < <file-name>
for <command> to get its input from <file-name> rather than standard in.
<command> > <file-name>

for <command> to send its output to <file-name> rather than standard out.
<command> >> <file-name>

for <command> to append its output to <file-name>.
3/9/2012
Samia Riaz
45
Redirecting the Output

Symbols used
> >>
Eg
cat > topics
Copying Files Moving Files Removing Files and directories
^D (Control D to stop)
Eg
cat >> topics
Displaying the contents of a file on the screen Searching the contents of a file
Exercise
use the cat command to join (concatenate) file1 and file2 into a new file (final.txt)
3/9/2012
Samia Riaz
46
Redirecting the Input

Symbol used
<
Eg
sort
C A B
Output will be
A B C
Exercise
Use sort to sort the contents of final.txt & redirect the output to a new file sorted.txt
3/9/2012
Samia Riaz
47
Pipes
Symbol used
vertical bar | <command-1> | <command-2> denote that standard out from <command-1> is to be piped <command-2> as standard in.
Eg
who (To see who is on the system with you ) How to get a sorted list of names?
Mehtod1
who > names.txt sort < names.txt
More efficient method is

who | sort
Exercise
Using pipes, print all lines of file1 and file2 containing the letter 'p', sort the result, and print to the printer.
Summary
command > file command >> file command < file command1 | command2
cat file1 file2 > file0 sort who a2ps -Pprinter textfile lpr -Pprinter psfile
redirect standard output to a file append standard output to a file redirect standard input from a file pipe the output of command1 to the input of command2 concatenate file1 and file2 to file0 sort data list users currently logged in print text file to named printer print postscript file to named printer
Samia Riaz 49
3/9/2012
Topic Four
Wildcards Filename Conventions Getting Help Finding files Unix file times Generating calendar
3/9/2012
Samia Riaz
50
Wildcards (meta char )

The characters * and ? *
will match against none or more character(s) in a file (or directory) name Eg
ls list* This will list all files in the current directory starting with list....
?
Will match exactly one character Eg
ls ?ouse will match files like house and mouse, but not grouse
[...] matches any one character between the braces; can use - to indicate arange of characters
Example: b[aei]t matches bat, bet, or bit, not baet Example: b[a-c]t matches bat, bbt, or bct, not bxt ls [abc]*.?
3/9/2012
Samia Riaz
51
Wildcards (meta char )

Examples:
ls *.c rm file[1-6].? cd ~/bin ls ~reid ls *.[^oa] - ^ in csh, ! in sh * stands in for 0 or more characters ? stands in for exactly one character [1-6] stands in for one of 1, 2, 3, 4, 5, 6 [^oa] stands in for any char except o or a ~/ stands in for your home directory ~reid stands in for reids home directory
Explanation:
3/9/2012
Samia Riaz
52
Getting Help
man wc whatis wc apropos keyword
3/9/2012
Samia Riaz
53
Finding files
find <path> <options> path is one or more directories in which find will begin to search Find will recursively search all sub-directories. options
name, perm, type, user, group, size, inum, atime, mtime, ctime, newer Many more
example: find all the *.c* and *.h* files in and below the current directory
find . name *.[ch]*
UNIX file times

Three Unix file times
Change time (ctime)
a change modifies the attributes of the file
Modification time
a change modifies the contents of the file
Access time (atime)

the last time the file was read or written
Note:
Reading a file updates its access time but not its change time or modification time These time attributes can also be used in the find command
Generating calendar
Show a calendar for this month
cal
Show a calendar for a specific month

cal 12 2000
Show a calendar for a specific year

cal 2000
3/9/2012
Samia Riaz
56
Summary
*
match any number of characters
match one character
man command
read the online manual page for a command
whatis command
brief description of a command
apropos keyword
match commands with keyword in their man pages
3/9/2012
Samia Riaz
57
Topic Five
File system security (access rights) Changing access rights Processes and Jobs Listing suspended and background processes Killing a process
3/9/2012
Samia Riaz
58
File system security (access rights)

ls -l (l for long listing!) [diff b/w ls l & ls -lg]*
E.g -rwxrw-r-- 1 ee51ab beng95 2450 Sept29 11:52 file1
In the left-hand column is a 10 symbol string consisting of the symbols d, r, w, x, 1st d or The 9 remaining symbols indicate the permissions, or access rights, and are taken as three groups of 3.
The left group of 3 gives the file permissions for the user that owns the file (or directory) (ee51ab in the above example); The middle group gives the permissions for the group of people to whom the file (or directory) belongs (beng95 in the above example); The rightmost group gives the permissions for all others.

Access rights on files
r (or -), indicates read permission (or otherwise) w (or -), indicates write permission (or otherwise) x (or -), indicates execution permission
Access rights on directories

r allows users to list files in the directory; w means that users may delete files from the directory or move files into it; x means the right to access files in the directory. This implies that you may read files in the directory provided you have read permission on the individual files

E.g
-rwxrwxrwx -rw-------
a file that everyone can read, write and execute (and delete). a file that only the owner can read and write - no-one else can read or write and no-one has execution rights (e.g. your mailbox file).
3/9/2012
Samia Riaz
61
Changing access rights

Only the owner of a file can use chmod chmod (changing a file mode) E.g
chmod go-rwx biglist
Symbol u g Meaning user group
o
a r w x + -
other
all read write (and delete) execute (and access directory) add permission take away permission
62
3/9/2012
Samia Riaz
More on chmod . . .
chmod ijk <name> sets permissions for
owner (i) group (j) all others (k)
where each of i, j, and k are octal digits (0-7). Viewed bitwise, 4 = read, 2 = write, 1 = execute. Since 7=4+2+1 E.g
chmod 700 <file-name> sets read, write, and execute permission for the file owner and for no one else
Suppose that you have a subdirectory named mywork in your current working directory. Then
chmod 700 mywork ????
Processes and Jobs

A job is a program whose execution has been initiated by the user. At any moment, a job can be running or suspended. Foreground job:
a program which has control of the terminal
Background job:
runs concurrently with the parent shell and does not take control of the keyboard.
3/9/2012
Samia Riaz
64
Processes and Jobs

ps (process status) In general the shell does not return the UNIX prompt until the current process has finished executing Running background processes
Backgrounding a long process has the effect that the UNIX prompt is returned immediately Symbol used is & E.g sleep 10 &
[1] 6259
Backgrounding a current foreground process

E.g sleep 100 (the current process) [control] key and typing [z] (^Z will suspend the process) bg
3/9/2012
Samia Riaz
65
Listing suspended and background processes

jobs E.g (output)
[1] Suspended sleep 100 [2] Running netscape [3] Running nedit
foreground a suspended processes

fg jobnumber E.g. fg 1 * Typing fg with no job number??? *
3/9/2012
Samia Riaz
66
Killing a process
kill (terminate or signal a process) kill a job running in the foreground
^C
kill a suspended or background process

kill jobnumber
ps (process status) to find out the PID then use it to kill a process
kill PID_number
* If a process refuses to be killed????????*
3/9/2012
Samia Riaz
67
Summary
ls -lag chmod [options] file
list access rights for all files change access rights for named file run command in background kill the job running in the foreground suspend the job running in the foreground background the suspended job list current jobs foreground job number 1 kill job number 1 list current processes kill process number 26152
Samia Riaz 68
command &
^C ^Z bg jobs fg 1 kill 1 ps kill 26152
3/9/2012
Work to do
1. 2. Quota
Built in kernel utility that keeps you from exceeding values for disk space set by the Gurus who run your system how they are partitioned, how big they are, how much of that is used, and available disk usage expresses itself in Kbytes du -s to get a summary of all the files you own, underneath that point
df

3.
Du

4. 5. 6. 7. 8.
3/9/2012
compress gzip File History awk

Samia Riaz 69

Systems Programming: Samia Riaz

Cargado por

Información del documento

Descripción original:

Título original

Derechos de autor

Formatos disponibles

Compartir este documento

Compartir o incrustar documentos

Opciones para compartir

¿Le pareció útil este documento?

¿Este contenido es inapropiado?

Copyright:

Formatos disponibles

Systems Programming: Samia Riaz

Cargado por

Copyright:

Formatos disponibles

Systems Programming

Samia Riaz samiariaz@gmail.com

Unix Systems Programming: Communication, Concurrency, and Threads by

Kay A. Robbins, Steven Robbins

BASIC UNIX IDEAS

BASIC UNIX IDEAS

Mail and Message Facilities Interpreters DBMS Word Processors FTP

Copyleft instead of Copyright

Library Files /lib

Redirectors, Pipes, Filters etc

Converted to binary language by shell

uderstand the Request

Some Linux Commands

who rm {filename} mail mail {username}

pwd mv {f1} {f2}

grep {w} {file}

tail -|+ {linenumber} {filename}

Linux Commands (cont..)

Output of First Command

Take input from Ist command for this command

Introduction to Shell Scripting

Unix File System Structure

File and Directory Names

Should not contain spaces or the following:

Filenames and pathnames have limits on lengths

Extension of the file concept

Some Standard Directories

Lectures Out line

Listing Files & Directories

Changing to a different directory

The directories . and ..

Navigating the File System

list files and directories

list all files and directories

mv [-fi] source... target_dir

Removing files and directories

Displaying the contents of a file on the screen

Searching the contents of a file

Displaying the contents of a file on the screen

to see what kind of file the shell is. (1)

send on only the front end (2)

tail send on only the tail end (3)

display the first few lines of a file

The out put would be

<command> > <file-name>

<command> >> <file-name>

Redirecting the Output

Redirecting the Input

More efficient method is

Wildcards (meta char )

Wildcards (meta char )

UNIX file times

Access time (atime)

Show a calendar for a specific month

Show a calendar for a specific year

match any number of characters

match one character

read the online manual page for a command

brief description of a command

match commands with keyword in their man pages

File system security (access rights)

File system security (access rights)