INTRODUCTION

In this 21st century, computers play a dominating part in our lives. The facilities provided by these machines are really breathtaking and overwhelming and internet is one of the most significant one. Today communications and gathering information on each and every aspect has a brand new meaning. I think every-one has heard of or rather they know what computer hardware or software is? But did everyone ever wonder what an operating system is? An operating system (the OS) is a life giver to a computer- providing it with basic intelligence to work. Technically speaking, it is a program that is loaded into the computers memory when the system is booted, and it remains there. It interacts with two agencies: Applications that run under its auspices. Users who pass commands to it using a command language interpreter. This too is a program; it translates commands to instructions that can be understood by the operating system. In a way an operating system runs programs and controls all resources that the program requires. It also provides services for the user to handle mail, files and processes. Besides DOS and WINDOWS, theres an operating system called UNIX. It arrived in the year 1969 much before Dos and Windows and has stayed long enough to give us the internet. It has practically everything an operating system should have, and several features that other operating system never had. Beginners with some experience in Windows think of UNIX in terms of it, quite oblivious to the fact that the similarities are only superficial. UNIX is a giant operating system and is way ahead of other systems in sheer power. Even UNIX introduced a number of profound and diverse concepts previously unknown to computing. But some people demand that UNIX is not user friendly. This is quite a bit true but it in fact challenges you to unravel its mysteries. In other words we can tell that UNIX system doesnt offer fixed a set of services. Here you have to use your imagination in devising improvised tools from the existing ones. This is what makes UNIX so challenging and exciting. Now let us throw some light on LINUX operating system. Linux was a phenomenon waiting to happen. The computer industry suffered from a rift. In the 1980s and 1990s, people had to choose between inexpensive, market-driven PC operating systems from Microsoft and expensive, technology-driven operating systems such as UNIX. Free software was being created all over the world, but lacked a common platform to rally around. Linux has become that common platform. LINUX is a free operating system with many similarities to UNIX. It is the result of initial work done by Linus Torvalds, a student of the University of Helsinki. He tried to improve the Minix operating system and succeeded in improving it. He uploaded his improved version of the Minix operating system onto the internet in 1991. Soon, programmers throughout the world began downloading Linux and started modifying and

fine-tuning it. This process of modification and improvement is an ongoing one. This is the major factor that has made Linux stand out among the prominent operating systems in the world today. Linus was responsible only for creating the system kernel. The kernel is the core of the operating system, which is responsible carrying out the system tasks. LINUX is now being referred to as a collection of software, which includes the kernel, making it a complete operating system. LINUX has been catching the attention of almost everyone. Its leap from freeware to the enterprise platform has been a great achievement for Linux. A new breed of computer professionals is discovering a miracle called Linux. First of all let us look at the reasons for the sudden upsurge in the popularity of Linux. Free: Linux is available for free on the internet. We dont have to pay anything to download it. A lot of distributions of the Linux are available for free download. A Linux distribution comes to us along with software that is worth thousands of dollars, at a minimal cost or no cost whatsoever. Open Source: Open source means we get not only the executables but also the source code. The user can access the source code and make improvements with it. Well Documented: The HOW-Tos, as they are called, are extensive and very simple. They are not written by programmers but by common people like us. So they are very easy to understand. They can guide us from the simplest of things to the most customized installations. Customizable: Linux users have the option of opting for using other software other than those bundled with the installation kit. These other software may either be commercial software or free software available off the net. Hardware: Linux runs on almost any hardware, be it 386, 486, Pentium MMX, Pentium I I, Sparc, Dec Alpha or even Motorola 68000 series. Multi-Tasking: Linux is a multitasking system. What this means is, a single user can run multiple programs at the same time. Each task is called a process. Multi-User: This means that more than one person can use the system at the same time. The multi-user concept stems directly from the multitasking aspect. Multiple Virtual Consoles: We have multiple virtual terminals on the server. This means that there are different screens each of which represents a separate terminal, enabling different users to log in at the same time on the same machine. TCP/IP Networking: TCP/IP networking is built into the kernel itself. Linux is among the best operating systems in terms of networking. It includes programs like telnet, ftp, rlogin, rsh, and other such programs. High Level Security: One of the main advantages of Linux is that it provides a very high level of security by using user authentication. It also stores password in an encrypted form. Linux also includes file system security that enhances the existing security. GUI: Linux these days has a very good Graphical User Interface (GUI) KDE, Window Maker and Afterstep all providing excellent GUI. Multiple Processor Support: Linux supports multiple processors.

 Web Server: Linux can be used to run a web server such as Apache to serve application protocols such as HTTP or FTP. Stable Operating System: Linux is a complete operating system that is stable. This means that the malfunctioning of an application is not likely to bring the system down.

LINUX ARCHTECTURE

The Linux operating system is categorized into three levels. They are the Kernel, the shell and the various tools and applications as shown in the figure below. The kernel carries out interactions with the hardware and co-ordinates system critical tasks. The shell is a program comprising of a command interpreter. It interprets the user requests and passes them on for execution. The tools and applications add enhanced features and capabilities to the Linux operating system.

Directory Hierarchy

Some of the Linux directories that may interest you include the following: /bin-Contains common Linux user commands, such as ls, sort, date, and chmod. /boot-Contains bootable Linux kernel and boot loader configuration file (GRUB). /dev-Contains files representing access points to devices on your system. These include terminal devices (tty*), floppy disks (fd*), hard disk (hd*), RAM (ram*), cd-rom (cd*). /etc-Contains several system configuration files. /home-Contains directories assigned to each user with a login account. /mnt-Provides a location for mounting devices such as remote file systems and removable media. /proc-Provides a mechanism for the kernel to send information to processes. /root-Represents root users home directory. /sbin-Contains administrative commands and daemon processes. /tmp-Contains temporary files used by applications. /usr-Contains user documentation, games, graphical files, libraries, and a variety of other user and other administrative commands and files. /var-Contains directories of data used by various applications. /opt-Typically contains extra and third party software. /net- Standard mount point for entire remote file systems.

BASIC COMMANDS

Commands have the following syntax: o command options arguments Each item is separated by a space Options modify a command's behavior o Single-letter options usually preceded by Can be passed as -a -b -c or -abc o Full-word options usually preceded by - Example: --help Arguments are filenames or other data needed by the command Multiple commands can be separated by ;

How to see date time & calendar?

date - display date and time cal - display calendar

How to edit date and time?

#date -s mm/dd/yyyy #date s hh:mm:ss

Linux Help tools

The whatis command:

Displays short descriptions of commands Uses a database that is updated nightly Often not available immediately after install

# whatis cal cal (1) - displays a calendar

# makewhatis (This command is used to built the database forcefully that is maintained by whatis)

The Man command: Provides documentation for commands Almost every command has a man "page" Pages are grouped into "chapters" Collectively referred to as the Linux Manual # man <chapter> command Manual sections: 1) 2) 3) 4) 5) 6) 7) 8) User commands System calls Library calls Special files File formats Games Miscellaneous Administrative commands

The info Command: Similar to man, but often more in-depth Run info without args to list all page info pages are structured like a web site Each page is divided into "nodes" Links to nodes are preceded by * # info <command>

The Help command: Displays usage summary and argument list. Used by most, but not all commands. # date --help

How to make a directory? # mkdir <directory-name> How to delete a directory? # rmdir <directory-name> (the above command removes only empty directories) # rm r <directory-name> (the above command removes a directory and its contents) How to create a file? # cat > <file-name> (After creating a file with cat command u will get an empty space for writing some text, write some text and then save the file by pressing ctrl-d) How to create an empty file? # touch <file-name> How to display the contents of the file created by cat command? # cat <options> <file-name> -b (number each line of output) -s (squeeze multiple adjacent blank lines into a single blank line) -A (Show all characters, including the control and non-printing characters)

How to remove a file? # rm <options> <file-name> -i (interactive) -r (recursive) -f (forceful)

# rm -r <file-name> # rm i <file-name> # rm -f <file-name> # rm ri <file-name>

# rm -rf <file-name>

How to copy a file? # cp <options> <source> <destination> -i (interactive copy, ask before overwriting a file) -r (recursive, recursively copy an entire directory tree) -p (preserve permissions, ownerships and time-stamps) -a ( archive, copies files and directories recursively while preserving permission)

How to list directory contents? # ls # ls a (includes so called hidden files and directories whose name begin with a dot) # ls -l (long listing ,for some more details) # ls -R(rescurses through subdirectories, listing their contents too) # ls -d(this command list directory name and not their contents) # ls -C # ls -c How to move a file or a directory? # mv <options> <source> <destination> -i (interactive) -f (forceful)

How to rename a file? # mv <old-filename> <new-filename>

How to display the terminal id in which you are logged in? # tty How to see the present working directory? # pwd How to display which are the users logged in and which terminal they are occupying? #who How to see who you are? #whoami How to change directory? cd changes directories to an absolute or relative path cd /home/Anindya/projects cd Anindya/projects To a directory one level up cd . . to your home directory cd to your previous working directory cd ~

GETTING STARTED WITH VI EDITOR

The vi Editor
An editor is a program used to edit files. Files can be of different types such as text files, graphics files or others, depending on what they contain. A text editor is a program that is used to edit files that contain text only. For example, a letter or source code of a program. Linux offers editors such as pico, vi, and many others. The vi editor is one of the most powerful and the earliest editors available in Linux and UNIX environments. It offers many powerful features that are not available in other editors. Its features are so extensive that very few people are even aware of all the available capabilities.

The vi Editor
There are three basic modes of operation in vi. The three modes are: Command mode. Insert or input mode. The last line mode. At any time, vi will be in one of these modes. The command mode is one in which user is able to execute commands, for example, to add text, to delete text. This can be compared to the menu commands of notepad. In the input mode or insert mode, the user can enter the data or edit the data. The last line mode is one in which the user is able to enter special commands which typically start with the : (colon). When vi is invoked, it comes up in command mode. For the user to be able to enter any text, the user must switch to input mode. The user can come back to the command mode by pressing the Escape key.

The vi Functions
The functions of the vi editor include: Inserting and deleting text. Reading and writing files.

 Searching for text. Copying and moving text.

LIST OF VI COMMANDS
Commands to Insert Text Command i I o O a A Purpose Invokes insert mode and inserts before character. Same as i but insertion is at the beginning of the line. Opens blank line before the current line. Same as o but puts blank line after current line. Same as i but appends after cursor. Same as a but appends at the end of the line.

Commands for Deleting Text Command x nx dw ndw d0 d$ or D dd ndd u Purpose Deletes one character at the cursor position. Deletes the next n characters from the cursor position. Deletes one word at the cursor position. Deletes n words at the cursor position. Deletes from the beginning of the line till the cursor position. Deletes from the cursor position till the end of the line. Deletes one line at the cursor position. Deletes the text n lines from the cursor position. Undoing a change.

Commands to Replace Existing Text Command r R s S Purpose Replace a single character at the current cursor position. Invoke insert mode and go on replacing till <Esc> is pressed. Deletes the current character and inserts character until <Esc> is pressed. Removes all the characters in the current line and allows the user to enter data.

Commands for Saving and Exiting Command :q :q! :w :wq :x :ZZ Purpose Quit vi. Quit vi without saving changes. Save and continue working in vi. Save the changes and exit from vi. Save and exit from vi. Save the changes and exit from vi.

Commands to Copy Text Command yy nyy dd ndd p P Purpose Copy a line into temporary storage. Copy n number of lines into temporary storage. Move a line into temporary storage. Move n lines into temporary storage. Move the text from temporary storage to the line below the current position of the cursor. Move the text from temporary storage to the line above the current position of the cursor.

Commands for Pattern Searching Command /g ?g / or n ? or N Purpose Search for a pattern g forwards. Search for pattern g backwards. Repeat the last search forward. Repeat the last search backward.

Commands for Scrolling Command Ctrl-f Ctrl-b Ctrl-u Ctrl-d Ctrl-L Purpose Scrolls a screen forward. Scrolls a screen backward. Scrolls half a screen backward. Scrolls half a screen forward. Refreshes the screen.

Basic Cursor Movement

Command k j h l w b

Purpose Up one line Down one line Left one character Right one character Right one word Left one word

Setting Basic Options in VI Command :set :set all :set sm :se nosm :se ai :se noai :se nu :se nonu :se ic :se noic :se textwidth=65 :se textwidth=0 :se wrapmargin=15 :se wrapmargin=0 Purpose List a small number of configuration items Lists the vast panoply of configuration items This option causes cursor to jump momentarily to the matching left curly brace to right curly brace Turns off the above mentioned behavior Causes new line to inherit the indentation level of the previous line Turns off the above mentioned behavior Causes line numbers to display Turns off the above mentioned behavior Causes searches to be case-insensitive Turns off the above mentioned behavior Causes to wrap when text width exceeds 65 characters Turns off the behavior Causes text to wrap when it reaches 15 characters from the right margin. Turns the above mentioned behavior off

More on Cursor Movement Command e $ ^ + G :n ) ( } { H M L % Purpose Move to the end of current word Move to end of current line Move to beginning of current line Move to beginning of next line Move to beginning of previous line Go to last line of the file Go to line with this number Move to the next sentence Move to the previous sentence Move to the next paragraph Move to the previous paragraph Move to the top line of the screen Move to the middle line of the screen Move to the last line of the screen Move to matching brackets ( { [ ] } )

Some Advanced Commands Command :r newfile :r !date :1,20w xfile :.,$w yfile :1,20w >> zfile :n :N :1,20y :put :1,5d Purpose Inserts new file contents into buffer Inserts result of date into buffer Writes lines 1-20 of buffer to xfile Writes current line through end of buffer to yfile Appends lines 1,20 to zfile Switches to next open file Switches to previous open file Copies 1 to 20 lines of a file Pastes the copied lines below the current cursor position Deletes lines numbered 1 to 5

STANDARD I/O AND PIPES

Linux provides three I/O channels to the processes: Standard input- Keyboard by default. Standard output- terminal window by default. Standard error- terminal window by default.

Redirecting Input and output Shell redirection operators allow I/O channels to be redirected to/from a file. Pipes allow output channels to be redirected to the input of other program.

Common Redirection operators Command > file Direct standard output of command to file. Command >> file Append standard output of command to file. Command < file Send file as input to command. Command 2>- Redirect error message from command to file. Command 2>>- Append error messages from command to file. Piping command 1 | command 2- Pipe the standard output of command 1 into the standard input of command 2.

REDIRECTING OUTPUT
To demonstrate I/O redirection we will use the find command $ find /etc -name passwd This command will search for all the lines named passwd in /etc and its subdirectories. By default both the standard output and standard error are displayed on the screen. Example: $ find /etc -name passwd /etc/passwd find: /etc/default: Permission denied /etc/pam.d/passwd

Redirecting Standard Output Redirect Standard output with >

Example: $ find /etc name passwd > findresult find: /etc/passwd: Permission denied $ cat findresult /etc/passwd /etc/pam.d/passwd But the error message is still displayed on the screen.

Overwriting Vs. Appending If the target exists, the file will be overwritten To append data to an existing file use >> to redirect instead of >

Example: Redirect output and create/overwrite the output file $ find /etc -name passwd > output find: /etc/default: Permission denied View the output file $ cat Redirect output and append to the output file $ find /etc -name passwd >> output find: /etc/default : Permission denied find: /etc/cups/certs: Permission denied View the output file $ cat output /etc/passwd /etc/pam.d/passwd /etc/passwd /etc/pam.d/passwd

Redirect output and overwrite the output file $ find /etc name passwd > output View the output file $ cat output /etc/passwd /etc/pam.d/passwd

Redirecting Standard Error

$ find /etc name passwd 2> finderrors /etc/passwd /etc/pam.d/passwd View the file $ cat finderrors find: /etc/default: Permission denied find: /etc/cups/certs : Permission denied Redirecting some more stderr to the file $find /tmp name passwd 2>> finderrors View the file $ cat finderrors find: /etc/default: Permission denied find: /tmp/orbit-root: Permission denied

Redirecting both Standard output and Error $ find / -name passwd 2> errs > results Redirect both stderr and stdout to one file $ find /etc name passwd > alloutput 2>&1 View the output file $ cat alloutput /etc/passwd Find: /etc/default: Permission denied /etc/pam.d/passwd

Redirecting Input $ cat filename.txt Hello, World! $ cat < filename.txt Hello, World! Using Pipes $ ls /usr/lib | less $ ls C | tr a-z A-Z $ cat file.txt | xargs rm f $ ls l | less

USERS GROUPS AND PERMISSIONS Users are of two types: Normal user Root User or Super User or System Administrator How to create a user? # adduser <username> Or # useradd <username> The lists of options that can be used with useradd command are:

How to set Password to your user account? # passwd <username> How to delete a user without deleting its home directory? # userdel <username> How to delete a user along with its home directory? # userdel r <username>

How to create a group? # groupadd <groupname> How to set passwd to your group? # gpasswd <groupname> How to insert a user to a group? Inserting a user as a primary member of a group # usermod g <groupname> <username> Inserting a user as a secondary member of a group # usermod G <groupname> <username> How to delete a group? # groupdel <groupname> How to change the group of a file or a directory? # chgrp <groupname> <filename> How to change ownership of a file or a directory? # chown <username> :<groupname> <filename>

Permissions There are three types of permission: Read Write Execute

Default Permission Default permission for directories is 777 minus umask Default permission for files is the directory defaults execute permission (i.e. 666 minus umask)

Non-privileged users umask is 002 So a non-privileged users file will have permission 664 after subtracting the umask and the directory permission of a non-privileged user will be 775 after subtracting the umask. Roots umask is 022 So roots file will have permission 644 after subtracting the umask and the directory permission root will be 755 after subtracting the umask. To whom we are assigning the permissions: Users(u): users who owns the file Group(g): users in the files group Other(o): Other users Changing Permission- Symbolic Method To change access modes: chmod [-R] mode file Where mode is: u, g or o for user, group and other. + or for grant or deny. r, w, x for read, write and execute. Examples: ugo+r: Grant read access to user, group and other users o-wx: Deny write and execute to others. Examples # chmod u+w <filename or directory name>: grant write access to owner of the file or directory. # chmod u+w, go-w <filename or directory name>: grant write access to owner and denies it to the group and others. # chmod u=rw <filename or directory name>: sets user permissions to read and write, with execute turned off, regardless of the current permission. # chmod +r <filename>: makes the file world readable Changing Permission- Numeric Method Uses a three-digit mode number first digit specifies owners permissions second digit species group permissions third digit represents others permissions

Permissions are calculated by adding 4 for read 2 for write 1 for execute Examples # chmod 664 file: grants read/write to the owner and group and read-only to others. # chmod 660 file: grants read/write to the owner and group and no permission for others. # chmod 600 file: grants read/write permission to the owner and no permission for group and others. # chmod 755 dir: grants full permission to the owner, read and execute to group and other. #chmod 770 dir: grants full permission to the owner and group and no permission to others. # chmod 700 dir: grants full permission to the owner and no permission to group and others. User and Group Information Files Authentication information is stored in plain text files: /etc/passwd: This file contains seven colon delimited fields. They are as followsuser name, password placeholder, uid number, gid number of the users primary group, GECOS field, home directory, and shell to be run when a user logs in. /etc/group: This file contains four colon delimited field. They are as follows: group name, group password placeholder, gid number, and a comma separated list of group members. /etc/shadow: This file is referenced when someone logs in. The file contains a mapping of a user name to a password. /etc/gshadow: This file contains the information about group password.

Changing Your Identity To change your password, run passwd $ passwd To start a new shell as a different user: su su su <username>

su - <username> User Information commands

Find out who you are $ whoami Find out the group you belong to $ groups $ id Find out who is logged in $ users $ who $w Login/reboot history $ last Special Permission Special permissions: a fourth permission set (in addition to user/group/other) Applicable in four cases: suid for an executable (S, s or 4) sgid for an executable (S or s or 2) sgid for a directory (S or s or 2) sticky bit for a directory (T or t or 1) Special permission for executables suid: command run with permission of the owner of the command, not executor or the command sgid: command runs with group affiliation of the group of the command Examples: $ chmod 4551 passwd $ chmod 2551 same-gnome Special permission for directories Sticky bit: files in directories with the sticky bit set can only be removed by the owner and root, regardless of the write permission of the directory. sgid: Files created in directories with the sgid bit set have group affiliations of the group of the directory. Examples: $ chmod 2551 <dir-name> $ chmod 1551 <dir-name>

LINUX FILESYSTEM IN DEPTH A Linux file system makes no difference between a file and a directory, since a directory is just a file containing names of other files. Programs, services, texts, images and so on are all files. Input and output devices and generally all devices are considered to be files according to the system. Most files are just simple files called regular files, they contain normal data for example text files, executable files or programs, input for or output from a program and so on. While it is reasonably safe to suppose that everything you encounter on a Linux system is a file, there are some exceptions. Directories: files that are list of other files. Special files: the mechanism used for input and output. Most special files are in /dev. Links: a system to make a file or directory visible in multiple parts of the systems file tree. Sockets: a special file type, similar to TCP/IP sockets providing inter-process networking protected by the file systems access control. Named pipes: acts more or less like sockets and form a way for processes to communicate with each other, without using network socket semantics.

The table given below gives an overview of the characters determining the file types: Symbol d l c b s p Meaning Regular file Directory Soft Link file Character Special file Block Special file Socket Named pipe

Files can also be identified with the help of their colors. The table given below gives an overview of the colors determining the file types: Color Blue Red White Pink Cyan Yellow Green Flashing Red File Type Directories Compressed Archives Text Files Images Links Devices Executables Broken links

Partitions and File system Disk drives are divided into partitions. Partitions are formatted with filesystem. A filesystem is a data structure written to the media that allow users to store and access files. Default filesystem in Linux is Third Extended Linux Filesystem (ext3) or Second Extended Linux Filesystem (ext2).

ext2 or ext3 filesystem keeps a list of files in the filesystem in a table called an inode table. An individual entry in the inode table is called an inode. The inode is referenced by its number, the inode number, which is unique within a filesystem. The inode contains metadata about files. Among the data stored in the inode is: The file type File permission Link count: the number of file names associated with the inode number. The user ID number of the file owner. The group ID number of the associated group. Time stamp, including last access time, last modification time, and the inode change time. Location of the data on the hard disk. Other metadata about the file.

Directories We know that a directory is a container for files and other subdirectories. In fact a directory is a mapping between the file names that humans use to reference files and inode numbers used by the computer to reference files. When a file name is being referenced by a command or application, Linux references the directory in which the file resides, determines the inode number associated with the file name, looks up the inode information in the inode table, if the user has permission, and returns the contents of the file. cp and inodes When a file is copied to a new name in the same directory, the directory and the inode table gets a new entry. $ ls li penguin 246674 -rw-rw-r-1 digby digby 26 Sep 25 20:56 penguin

$ cp penguin tux $ ls li tux

246575

-rw-rw-r

digby digby 26

Sep 25 20:56 tux

mv and inodes When a file is moved, the underlying file, either as inode entry or data on the hard disk, does not move. What moves is the entry in a directory. When a file is moved within the same filesystem, only the directories itself are affected: $ ls li tux 246575 -rw-rw-r-1 digby digby 26 Sep 25 20:56 tux

$ mv tux fedora $ ls li fedora 246575 -rw-rw-r-1 digby digby 26 Sep 25 20:56 tux

The inode number remains the same. The data on the filesystem is not moved. The inode is not changed, except that the inode change time is updated. rm and inodes The rm command has a broad effect. The entry in the directory is effectively deleted; the inode number is made available; the block locations that the file was using are placed on the free list. The underlying data is actually not removed, however. The data will be overwritten eventually when the blocks are reused by some other file, but the data is otherwise unmodified. Links A link is nothing more than a way of matching two or more file names to the same set of file data. There are two ways to achieve this: Hard link: Associate two or more files with the same inode. Hard links share the same data blocks on the hard disk, while they continue to behave as independent files. Soft link or symbolic link: A file that is a pointer to another file. A symbolic link contains the path to the target file instead of a physical location on the hard disk. Since inodes are not used in this system, soft links can span across partitions. How to create a soft link file? $ ln -s <filename> <linkname>

$ ln s package package1 $ ls li package package1 30338 33276 -rw-rr-root root 1729 lrwxrwxrwx digby digby 8 Sep 25 11:43 package Sep 26 11:05 package1->package

The content of the symbolic link is the name of the file that it references. The size of the symbolic link is always the number of the characters in the path name. How to create a hard link? $ ln <filename> <linkname>

Checking Free Space

df- Reports disk space usage Reports total kilobytes, kilobytes used, kilobytes free per filesystem. -h: displays sizes in easier to read units. $ df h du- Reports disk space usage Reports kilobytes used per directory. Includes subtotals for each subdirectory. -s option reports only single directory summary Also takes h option

Removable Media Mounting means making a foreign filesystem look like a part of the main tree. Before accessing a media must be mounted. Before removing a media must be unmounted. By default non-root users may only mount certain devices. Mountpoints are usually under /media directory. Mounting CDs and DVDs Automatically mounted under GNOME/KDE Otherwise must be manually mounted. # mount /media/cdrom # mount /media/cdrecorder

Eject command unmounts and ejects the disk Otherwise must be manually unmounted

# umount /media/cdrom # umount /media/cdrecorder Mounting USB Media Detected by the kernel as SCSI devices. /dev/sdax or /dev/sdbx or similar. Automatically mounted in GNOME/KDE Icon created in Computer Window Mounted under /media/Device ID Device ID is built into device by vendor. Mounting Floppy Disks Must be manually mounted or unmounted # mount /media/floppy # umount /media/floppy DOS floppies can be accessed with the help of mtools Mounts and unmounts device transparently Uses DOS naming conventions # mdir a: # mcopy /home/file.txt a: Formatting Floppy Disks Two types of format needed to prepare a floppy A low level format # fdformat /dev/fd0H1440 A filesystem format, one of: # mkfs t ext2 /dev/fd0 # mke2fs /dev/fd0 # mkfs t vfat /dev/fd0 # mformat a:

File Archiving Why Archive files? Archiving places many files into one target file. Easier for backup, store and transfer. Tar is a standard Linux Archiving command. Creating an Archive # tar cvf archive_name files # tar cvf home.tar /home c: create a new archive v: display verbose message f: archive_name is the name of a new file Options do not need a leading dash Inspecting Archives # tar tf archive_name.tar # tar tvf archive_name.tar First command displays a list of all files in the archive Second command displays a long listing of each file in the archive.

Extracting an Archive # tar xvf archive_name.tar Extract to the current directory by default. Target can be specified with C dir

File Compression Why do we compress file? Results in smaller file size Text files can be compressed over 75% Binary files usually dont compress much, if any. tar archives are often compressed Compression Utilities gzip, gunzip bzip2, bunzip2

gzip or gunzip Standard Linux compression utility Over 75% compression for text files bzip2 or bunzip2 Newer Linux compression utility Generally achieves better compression than gzip

How to compress and decompress a file using the compression utilities? $ gzip v <filename> $ gzip d <filename.gz> $ gunzip <filename> $ gunzip <filename.gz> $ bzip2 v <filename> $ bzip2 d <filename.bz2> $ bunzip2 <filename> $ bunzip2 <filename.bz2>

INTRODUCTION TO STRING PROCESSING Head Displays first few lines (default 10 lines) of text in a file.

# head <filename> Uses n or -lines parameter to change number of lines displayed

# head n 15 <filename> Tail Displays the last few lines (default 10 lines) of text in a file.

# tail <filename> Uses n or -lines to change number of lines displayed.

# tail n 15 <filename> Use f to follow the end of a text file as it changes

# tail f <filename> Word Count Count words, lines, bytes and characters Can act upon a file or STDIN Uses l for only line count Uses w only for word count Uses c for only byte count Uses m for character count (not displayed) # wc <filename> # wc l <filename> # wc w <filename> Sort

Sorts text to stdout- original file unchanged

# sort <options> file(s) Common options

 Uniq

-r performs a reverse sort -n performs a numeric sort -f ignores case of characters in string -u removes duplicate lines in output -t c uses c as a field separator -k x sorts by c-delimited field x

Removes successive duplicate lines in a file Can use in conjunction with sort to remove all duplicates Sort u achieves the same effect Use c to count number of occurrences of duplicate data

Cut Displays specific column of the field data $ cut f4 results.dat -f specifies field or column -d specifies field delimiter $ cut f3 d: /etc/passwd -c cuts by characters

$ cut c2-5 /usr/share/dict/words Paste Paste combines files horizontally. It takes a line from each file and pastes them together to standard output separated by a tab. Use the d option to change the output delimiter. $ paste d: file.txt file2.txt > file3.txt tr (Character Translator) $ tr a-z A-Z > file3.txt Version Comparison with diff Compares two files for differences $ diff file1.txt file2.txt 33c33 < x = y+2; ------> x = y+4;

33c33 indicates lines where files differ. < indicates line in the first file, > indicates line in the second file. Spell Checking with Spell Interactive spell-checker Easy way to check spelling in a file $ aspell check <filename> $ aspell l < filename aspell l will non-interactively list the misspelled words in a file read from standard input. Look Look command is a quick spelling dictionary lookup. It comes in handy when you need the spelling of a word of which you know the first few letters. $ look exer

FINDING AND PROCESSING FILES Basic find Examples $ find name a.txt Search for files whose names are a.txt $ find iname a.txt Case-insensitive search for files named a.txt, A.txt etc. $ find user anindya group anindya Searches for files which are owned by the user anindya and the group anindya $ find / -name .txt In the above example find command will search for the files that are names .txt, not files that contained in their names the string .txt. / indicates the searching will be carried out in every directory. $ find / -name *.txt The above command will find all files on the system that have .txt as the end of their name. Find and Logical Operators Criteria are ANDed together by default Can be ORd or negated with o and not Parentheses can be used to determine logic order, but must be escaped in bash

$ find user anindya not group anindya $ find user anindya o -group anindya $ find not \(-user anindya o -user arya\) Find and Permissions Can match ownership by names or id Can match octal or symbolic permissions $ find / -user anindya o uid 500 $ find perm 755 matches if mode is exactly 755

$ find perm +222 matches if anyone can write $ find perm -222 matches if everyone can write $ find perm +o+w matches if the other members can write $ find not perm ug+r matches the files where neither the user nor the group have read access Find and Numeric Criteria $ find size 1024k matches files which are exactly 1 megabyte $ find size +1024k matches files which are more than 1 megabyte $ find size -1024k matches files which are less than 1 megabyte Find and Access Time -atime when file was last read -mtime when file data last changed -ctime when file data or metadata was last changed Value given is in days

$ find / -atime 5 $ find / -atime -5 $ find / -atime +5 $ find / -mtime 5 $ find / -mtime -5 $ find / -mtime +5 $ find / -ctime 5 $ find / -ctime -5 $ find / -ctime +5 The Gnome Search Tool Actions -> Search for Files. Graphical tool for searching by name, content, owner/group, size, modification time.

THE BASH SHELL Bash Introduction Bourne Again Shell Successor to sh, the original Unix shell Developed for the GNU project The de facto standard Linux shell Backward compatible with Bourne shell (sh)- the original UNIX shell Command Line Shortcuts (File Globbing) When typing commands, it is often necessary to issue the same command on one or more than one file at the same time. The use of wildcards or metacharacters, allow one pattern to expand to multiple filenames by a process called globbing. Globbing is wildcard expression: * - matches zero or more characters. ?- matches any single character. [a-z]- matches a range of characters. [^a-z]- matches all except the range. Examples: If a directory contains the following files: joshua.txt, James.txt, alex.txt, Angelo.txt, gonk.mp3, zonk.mp3 Type the following commands: $ rm *mp3 $ rm gonk.mp3 zonk.mp3 The result is that all files in the directory that have names ending in mp3 In addition to wildcards, ranges of characters can be specified within straight-braces (?[? and ?]?). Red Hat Linux uses UTF-8 encoding of characters, which means that each capital letter comes directly after the corresponding lower-case letter. As a result a command like: $ ls [a-j]*.txt Would list alex.txt, Angelo.txt and joshua.txt but not James.txt. echo can be used to test the expansion of metacharacters before using them in a destructive command like rm. $ echo ?o*

Command Line Shortcuts (The Tab Key) Type tab to complete the command lines. For the command names it will complete a command name. For an argument, it will complete a file name. Examples: $ xte<Tab> $ xterm $ ls myf<Tab> $ ls myfile.txt Command Line Shortcuts ( History) bash stores a history of commands youve entered, which can be used to repeat commands. Use history commands to see the list of remembered commands. $ history In addition to basic command recall with the arrow keys, the bash history mechanism supports a variety of advanced ways of retrieving commands from the history list. !!- repeats the last command !char- repeats last command that started with char !num- repeats a command by its number in history output. !?abc- repeats last command that contains abc !-n- repeats a command entered n commands back. Command Line Expansion (The tilde) The tilde (~) expansion feature is borrowed from the C shell and makes it easy to reference files and directories inside your or someone elses home directory. Tilde ( ~ ) May refer to your home directory.

$ cat ~/.bash_profile May refer to another users home directory.

$ ls ~julie/public_html

Command Line Expansion (Variable and string) Parameter/Variable: $ Substitute the value of a variable in a command line $ cd $HOME/public_html Curly braces: {} A string is created for every pattern inside the braces regardless if any file exists.

$ rm hello.{ c, o} Curly braces expand to create string patterns they describe. Commas separate the various string patterns and an initial comma means that the null string is one of the requested patterns. For example: $ echo {a, b} ab $ echo x{a, b} xa xb $ cp file.txt{,-save} Curly braces are useful for generating patterned strings. For example: $ mkdir p work/{inbox,outbox,pending}/{normal,urgent,important} $ ls work inbox outbox pending $ ls work/inbox important normal urgent Command Line Expansion (Command and Math) Command output - `` or $( ) $ echo Hostname : `hostname` Arithmetic - $[ ] Substitute the result of arithmetic expression in a command line

$ echo Area : $[ $x * $y] $ echo Area: `expr $x \* $y`

Command Editing Tricks Ctrl-a moves to beginning of line Ctrl-e moves to end of line Ctrl-u deletes to beginning of line Ctrl-k deletes to end of line Ctrl-arrow moves left or right by word GNOME-terminal Applications -> System Tools-> Terminal Graphical terminal emulator that supports multiple tabbed shells Ctrl-Shift-t creates a new tab Ctrl-Pgup/PgDn switches to next/prev tab Ctrl-Shift-c copies selected text Ctrl-Shift-v pastes text to the prompt.

CONFIGURING THE BASH SHELL The shell is configured through a variety of mechanism: Local Variables Aliases and functions The set and shopt command The shell can also configure other commands or applications through environment variables. Variables A variable is a label that has a value. Set with VAR=VALUE Used to configure the shell or other programs Two Types: local and environment Local variables are used only by the shell Environment variables are passed onto other commands Display variables with set and env set- displays all variables env- displays environment variables $ HI=hello $ echo $HI hello The PS1 Local Variable The PS1 variable sets the prompt. The prompt can vary each time it is displayed by using special escaped sequences. These include: \d the date \h short hostname \t the current time \u user name \w the current working directory \! the history number of the current command \$ - shows $ if you are a non-privileged user and a # if privileged user. $ PS1=\u@\h:\w <\!>\$ digby@kennel: /tmp <500>$ Aliases Aliases let you create shortcuts to commands. Use alias by itself to see all set aliases. Use alias followed by a alias name to see alias value.

$ alias c=clear $ alias $c The above created alias will be lost if you log out and log back in. To ensure that the new alias available each time the user logs in, type the following commands as given below: $ vi .bashrc Locate the line that contains the text: # User specific aliases and functions. Add your alias to the line immediately below: $ alias c=clear Save the file and exit. Test your change by logging out, logging back in on the terminal. Other Shell Configuration Command Set and shopt are less common but powerful commands to configure elements of the shell. The set commands can configure many different aspects of the shell. Examples: set b : Reports termination of background commands immediately, rather than waiting for next prompt. set u : Unsets variables generate an error set o noclobber : Prevents overwriting of files with > and >& operators set +o noclobber : Enables overwriting of files.

Common Environment Variables HOME Path to users home directory LANG Identification of default language rules to use PWD Users current working directory EDITOR Default editor programs should invoke for text editing LESS Options to pass to the less command The TERM Environment Variable TERM environment variable sets the terminal type reset command used to reset a terminal should the screen become corrupted. The PATH Environment Variable The PATH environment variable is a colon separated list of locations where commands can be found

which command showing location in the PATH of an executable. The which command searches through the directories listed in the PATH environment variable looking for a matching executable. When it finds one, it prints the path to the executable.

$ echo $PATH /usr/local/bin: /usr/bin: /bin: /usr/X11R6/bin: /home/digby/bin $ which less /usr/bin/less Shell Startup Scripts Most of the configuration items that we have seen in this unit are only valid in the given shell. But typically, we will want settings to be established every time we start a shell, rather than having to type in all of our variables, aliases, and other commands on a per shell basis. To accomplish this, we use startup scripts, scripts that run when a shell is created. The system administrator sets up some startup scripts, but individual users can control startup by editing scripts in their home directories. Login Shell A critical concept to understand when it comes to startup scripts in the idea of the login shell. A login shell is a shell that someone started by logging in the system. A non-login shell is a shell started up in some other way, perhaps by a user or a program issuing the bash command. Startup Scripts: Order of Execution Login shells /etc/profile /etc/profile.d ~/.bash_profile ~/.bashrc /etc/bashrc Non-login shells ~/.bashrc /etc/bashrc /etc/profile.d

/etc/profile The /etc/profile shell script is the first startup script to run when a login shell is started. It only runs for login shells, non-login shells do not invoke this script. This script will set a series of variables including PATH, USER, LOGNAME, MAIL, HOSTNAME, HISTSIZE and INPUTRC. It will also run scripts in the /etc/profile.d directory. /etc/profile.d The /etc/profile.d directory contains additional setup shell scripts. Two copies of each script are listed in this directory, a Bourne shell style script with a .sh suffix, and a C shell style script with a .csh suffix. These do such things as set an alias for ls so that it displays colors by default. The script in this directory are controlled by the system administrator, they cannot be modified by non-privileged users. ~/.bash_profile and ~/.bashrc The ~/.bash_profile and ~./bashrc files are controlled by the individual user. The user may put variables, aliases, or any other startup command in these files. The ~/.bash_profile file is only called by login shells. The ~/.bashrc file is called by both the login shells and the non-login shells. ~./bash_logout The seldom used ~/.bash_logout file runs when a user logs out. Although most users do not use this file, it may be handy to do such housekeeping chores as making backup files, deleting temporary files or just displaying the logout date and time.

STRING PROCESSING USING REGULAR EXPRESSION Regular expressions are a pattern matching engine. Regular expressions are as abbreviated regex. Unlike the shells file name generation pattern matching system, regular expressions are designed for power over ease of use. Among the tools that use regular expressions are: grep analyze the contents of files a line at a time, returning lines that match a pattern. sed, the stream editor returns the contents of a file or stream of data, performing the specified search and replace instruction. less, this uses regular expressions in search commands. vi, which uses regular expressions for searches or search and replace. awk, a data oriented programming language. Regular expressions are greedy. That is if a regular expression can match a smaller string or a bigger string it will match the largest string possible. Regular expressions can be divided into three broad categories: wildcard characters, modifiers and anchors. Wildcard Characters Wildcard characters stand for another single character. . any single character [abc123] any single character in the set. [0-9] any single character in the range [^abc123] any single character not in the set [^0-9] any single character not in the set. Modifiers Modifiers determine the number of the previous character. * Zero or more of the previous character. \+ one or more of the previous character. \? Zero or one of the previous character. \ {i \} exactly i of the previous character. \ {i, \} i or more of the previous character. \ {i, j\} i to j of the previous character. Anchors Anchors match the beginning or end of a line or word. ^ the beginning of a line $ the end of a line \< the beginning of a word \> the end of a word

grep Prints line of a file where a pattern is matched. Options used with the grep command are: -v: return lines that do not contain the pattern. -n: precede return lines with line number. -c: only return a count of lines with the matching pattern. -l: only return the name of files, starting with the named directory. -r: performs a recursive search of files, starting with the named directory. -i: performs a case in-sensitive search. Examples: $ grep john /etc/passwd. $ ls | grep .c $ grep ^g /etc/passwd $ grep v ^g /etc/passwd $ grep bash$ /etc/passwd $ grep c fish /root/package $ grep ^.$ /root/package $ grep ^.{16 \}$ /root/package Sed (Stream Editor) Reads a file or stream of data, writes out the data, performing search and replace instruction. Uses regular expressions in search string. Stream Editing with regular expression Consider a file called cats containing the following words: cat catalog concatenate polecat Cat Now try out the following commands given below and see the output that they generate. $ sed s/cat/dog/cats $ sed s/[Cc]at/dog/ cats

$ sed s/\<[Cc]at/dog/ cats $ sed s/[Cc]at\>/dog/ cats $ sed s/\<[Cc]at\>dog/ cats $ sed s/\<[Cc]at\>/& and dog/ cats In the above written commands the cats file will sent to standard output with the string cat being replaced by the string dog. By default, sed makes a maximum of one change per line. To instruct sed to make multiple changes per line, the g command standing for global, should be appended at the end of the search and replace pattern. $ sed s/[Cc]at/dog/g cats

PROCESS A Process can be defined as program in execution. Creation of a process Traditionally in UNIX like operating system process began through a process called fork and exec. That is when one command starts another, the child process first forks, the kernel copying over pages of memory from the parent process to a new location for the child process. The child than execs, executing the new command overwriting the data. For example when shell executes the ls command the kernel would duplicate the shells pages of memory and then execute the ls command, overwriting the duplicate data. This was wasteful for system resources. LINUX replaced this model for process initialization with three other methods, the most commonly used being copy-on-write. With copy-on-write, when one process starts a subprocess, the original process does not duplicate its pages of memory when it forks, but rather it points the child to the same pages in memory as the original process, but making the memory area as read-only for both parent and child. When either process wishes to write the data, as will happen when the child execs, a copyon-write fault will occur and the kernel will create new pages in memory for that process. Manually execing Processes It is also possible for a command to exec without forking. For example when logging into the X window system, if the .xsession file exists, the .xsession file will execute the commands in a shell. The last command will be something like: exec metacity This means the shell in which the .xsession command was running will be replaced in memory with the metacity command. Were the command to fork and exec, the shell that ran .xsession would remain on the system performing no useful function until the metacity command exited. Process Ancestry init is the first process started at boot time- always has PID 1. Except init every process has a parent. A process can be both parent and a child at the same time. Viewing Process Ancestry $ pstree

Process States The following process states are defined in Linux: Runnable (R): Process is on the run queue. It is waiting for its turn to run or it is executing. Sleeping (S): Process is not executing, nor it is ready to run. It is waiting for an event to occur or a signal to arrive to wake it up. Stopped (T): Process is not executing because it has been stopped. Uninterruptible Sleep (D): Process is sleeping and cannot be woken up until an event occurs. It cannot be woken up by any signal. Typically the result of an I/O operation. Zombie (Z): Just before a process dies, it sends a signal to its parent and waits for an acknowledgement before terminating. Even if the parent process does not immediately acknowledge this signal, all resources except for the process identity number (PID) are released. Zombie processes are cleared from the system during the next system reboot and do not adversely affect system performance.

Viewing Processes The ps command provides several styles of output depending on the options used. Without options it displays information about processes specific to the active terminal. Some of the options that can be used with ps command are: -a: displays all processes, not including processes not controlled by a terminal. -x: includes processes not controlled by a terminal, such as daemon processes. -l: displays a long listing, which includes more information about a process. -u: displays the user name of the process. $ ps alx Sending Signals to Processes Signals can be specified by their name such as KILL, or by their number, such as 9. kill command can send many signals, but processes only respond to the signals they have been programmed to recognize. Default signal is TERM. Syntax: kill[signal] pid(s) kill[signal] %jobid Example: $ kill 3428

$ kill -9 3428 $ kill TERM 3428 $ kill KILL 3428 In addition to kill there is a killall command that can be used to send a signal to a group of commands, such as all getty processes. It should be used with caution since killing some processes may have a detrimental effect on the system. Heres an example that sends a KILL signal to all running processes named galleon-bin: $ killall KILL galleon-bin Terminating Processes Processes ordinarily terminate on their own when they have completed their task. Interactive applications may need the user to issue a quit command. Many processes can terminated with Ctrl-c, which sends an interrupt (INT) signal to the process. The process is shutdown cleanly, that is child processes are terminate at first and any pending I/O operations are completed. The same is true if a process is sent to terminate (TERM) signal via the kill command. If a process will not respond to a TERM signal, the KILL signal can be used. However, the processes may not be ended cleanly. The KILL signal should be used only if a process will not respond to a Ctrl-c or TERM signal. Using KILL signal on a routine basis may cause zombie processes and lost data. Altering Process Scheduling Priority Every running process has a scheduling priority. A ranking among running processes determining which should get the attention of the processor. The formula for calculating this priority is complex, but users can affect the priority by setting the niceness value, one element of this complex formula. The niceness value is a number ranging from -20 (highest priority) to 19 (lowest priority). It defaults to a value zero. Syntax: nice n priority programname Non-privileged users may not set niceness value to less than zero, that is they may not request a higher than normal priority for their processes. This is a function reserved for the superuser. Altering priorities of running programs Syntax: renice priority p PID

$ renice -15 p 3426 The p option is not strictly necessary, however if u wish to change the priority of an entire process group, use the g option. Also useful if the u option, which can be used to modify the priority of all of the processes of a particular user: $ renice 15 u student Interactive Process Management Tools Display real-time process information Allowing sorting, killing and renicening Command-line: top GUI: gnome-system-monitor

Running a Process in the Foreground When a process is started from a terminals command-line, it is normally running in the foreground. The process can be stopped, restarted in the background, or terminated. This is called job-control. Running a Process in the Background Running a command in the background allows another process to run concurrently on the same terminal. Launch a program as a background process by appending an & (ampersand to the end of a command. $ firefox & When a process is started in the background a new bash subshell is created. The bash program is then replaced with the command being executed. Background processes can be managed like any other process. Suspending a Process Foreground jobs can be suspended temporarily halted without being killed. Suspend a foreground process with Ctrl-z. Suspended jobs can be resumed in the foreground (fg) and can also be resumed in the background (bg). Listing Background and Suspended Jobs Jobs display all processes running in the background or that are suspended. $ jobs [1]+ Stopped [2]Running

man bash find / -name joe > output 2>&1 &

The number in the brackets is a job number, used to kill jobs or bring them back to the foreground. Job numbers are referred with %. The plus or minus sign next to the job number tells which job is the default, that is the job that is referenced if no job number is given. When a command is suspended or backgrounded it can be brought back to the foreground with fg. Suspended jobs can be resumed in the background with bg. Syntax: fg %jobnumber bg %jobnumber Scheduling Periodic Processes