Chown, chmod, Pipe operator, command

redirectio
-Jatin Maheshwari
 chown command in Linux with Examples
Different users in the operating system have ownership and permission to ensure that the files are
secure and put restrictions on who can modify the contents of the files. In Linux there are different
users who use the system:

• Each user has some properties associated with them, such as a user ID and a home directory. We
can add users into a group to make the process of managing users easier.
• A group can have zero or more users. A specified user can be associated with a “default group”. It
can also be a member of other groups on the system as well.
Ownership and Permissions: To protect and secure files and directory in Linux we use permissions to
control what a user can do with a file or directory. Linux uses three types of permissions:
• Read: This permission allows the user to read files and in directories, it lets the user read
directories and subdirectories stores in it.
• Write: This permission allows a user to modify and delete a file. Also it allows a user to modify
its contents (create, delete and rename files in it) for the directories. Unless the execute
permission is not given to directories changes does do affect them.
• Execute: The write permission on a file allows it to get executed. For example, if we have a file
named php.sh so unless we don’t give it execute permission it won’t run.
Types of file Permissions:
• User: These type of file permission affect the owner of the file.
• Group: These type of file permission affect the group which owns the file. Instead of the group
permissions, the user permissions will apply if the owner user is in this group.
• Other: These type of file permission affect all other users on the system.
Note: To view the permissions we use:
ls -l

chown command is used to change the file Owner or group.

Whenever you want to change ownership you can use chown command.
Syntax:
chown [OPTION]… [OWNER][:[GROUP]] FILE…

chown [OPTION]… –reference=RFILE FILE…

Example: To change owner of the file:

chown owner_name file_name

In our case we have files as follows:

Now if I use file1.txt in my case, to change ownership I will use the following syntax:
chown master file1.txt

where the master is another user in the system. Assume that if you are user named user1 and you want
to change ownership to root (where your current directory is user1). use “sudo” before syntax.
sudo chown root file1.txt

Options:
• -c: Reports when a file change is made.
Example:
chown -c master file1.txt

• -v: It is used to show the verbose information for every file processed.
Example:
chown -v master file1.txt

• -f: It supresses most of the error messages. When you are not permitted to change group
permissions and shows error, this option forcefully/silently changes the ownership.
Examples:
1. To Change group ownership In our case I am using group1 as a group in the system. To
change ownership we will use
chown :group1 file1.txt

You can see that the group permissions changed to group1 from root, if you use -v option it will
report that. We just need to add a “:” to change group.

2. To change the owner as well as group: Again taking master as user and group1 as a group in the
system
chown master:group1 greek1

Here, greek1 is a file.

3. To change the owner from particular ownership only: Suppose we want to change ownership
from master to root where current owner must be master only.
chown --from=master root greek1

4. To change group from a particular group:

chown --from=:group1 root greek1

Here, the group of greek1 is changed to root.

5. To copy ownership of one file to another:

chown --reference=greek1 greek2

6. To change ownership of multiple files:

chown master:group greek2 greek3
 chmod command in Linux

In Unix-like operating systems, the chmod command is used to change the access mode of a file.
The name is an abbreviation of change mode.
Syntax :
chmod [reference][operator][mode] file...

The references are used to distinguish the users to whom the permissions apply i.e. they are list of
letters that specifies whom to give permissions. The references are represented by one or more of the
following letters:

Reference Class Description

u owner file's owner

g group users who are members of

the file's group

o others users who are neither the

file's owner nor members of

the file's group

a all All three of the above, same as ugo

The operator is used to specify how the modes of a file should be adjusted. The following operators are
accepted:

Operator Description

+ Adds the specified modes to the

 specified classes

- Removes the specified modes from

the specified classes

= The modes specified are to be made

the exact modes for the specified

classes

Note : Putting blank space(s) around operator would make the command fail.
The modes indicate which permissions are to be granted or removed from the specified classes. There
are three basic modes which correspond to the basic permissions:

r Permission to read the file.

w Permission to write (or delete) the file.

x Permission to execute the file, or, in

the case of a directory, search it.

Types of permissions which we will be changing using chmod command :

In linux terminal, to see all the permissions to different files, type ls -l command which lists the files in
the working directory in long format. The figure below shows an example to use ls -l and its output :

Let us take a look at above figure. To make things easy to understand some columns and rows are
eliminated and extra spaces are added to the permissions column to make it easier to read as shown
below:

- rw- rw- r-- mik mik assgn1_client.c

- rw- rw- r-- mik mik assgn1_server.c

d rwx rwx r-x mik mik EXAM

- rw- rw- r-- mik mik raw.c

- rwx r-x r-x mik mik header.sh

... so on...

•The very first column represents the type of the file i.e. is it a normal file or a
directory where d represents a directory and – represents a normal file.
•The first set three letters after the file type tell what the Owner of the file, have permissions to
do. For example: In assgn1_client.c, has owner’s permission as rw-, which means the owner mik
can only read(r) and write(w) the file but cannot execute(x).
•Note: The 3rd and 4th columns represents the name of the owner of the file and the group to
which the owner belongs respectively.
•The next three letters after the user’s permission are the group’s permissions.
For example: header.sh has group permissions as r-x, which means Other people in the mik group
can not write(w) the header.sh script but can only read(r) or execute(x) it.
•Note that when a directory has the x set, this takes the special meaning of “permitted to search
this directory”.
•The last three letters in the permissions column tell us what the “others” may do. The general
practice is to protect the files from external access so that others can’t write any files or
directories. They may read(r) or execute(x) it. For example: The assgn1_client.c has others
permission as r- – which means it can only be read by other(external) access but cannot be written
or executed by them.
Now, let us see how chmod command can be used to change the access mode of a file.

Example 1 :
Let’s change the assgn1_client.c permission so that the owner cannot write(w) in the file but can only
read it.

BEFORE: -rw-rw-r-- mik mik assgn1_client.c

COMMAND: chmod u=r assgn1_client.c

AFTER: -r--rw-r-- mik mik assgn1_client.c

Example 2 :
Let’s restrict the permission such that the user cannot search the directory EXAM.

BEFORE: drwxrwxr-x mik mik EXAM

COMMAND: chmod u=rw EXAM

AFTER: drw-rwxr-x mik mik EXAM

After applying the chmod u=rw EXAM command, the user (owner) cannot change the directory. If the
user tries to change the directory, then it shows the message “Permission denied” as shown in the figure
below :

octal notations

First of all it is very essential to know about ugo and rwx manner:
• Understanding of attribute which can be out put by ls-l:
• Applying Permission:

• Using Octal number:

Hence Following work same like:
chmod +rwx [file_name]
chmod 777 [file_name]
And
chmod 775 [file_name]
chmod ug+rwx,o=rx [file_name]
 Piping in Unix or Linux
A pipe is a form of redirection (transfer of standard output to some other destination) that is used in
Linux and other Unix-like operating systems to send the output of one command/program/process to
another command/program/process for further processing. The Unix/Linux systems allow stdout of a
command to be connected to stdin of another command. You can make it do so by using the pipe
character ‘|’.
Pipe is used to combine two or more commands, and in this, the output of one command acts as
input to another command, and this command’s output may act as input to the next command
and so on. It can also be visualized as a temporary connection between two or more commands/
programs/ processes. The command line programs that do the further processing are referred to as
filters.

This direct connection between commands/ programs/ processes allows them to operate simultaneously
and permits data to be transferred between them continuously rather than having to pass it through
temporary text files or through the display screen.
Pipes are unidirectional i.e data flows from left to right through the pipeline.
Syntax :
command_1 | command_2 | command_3 | .... | command_N

Example :
1. Listing all files and directories and give it as input to more command.
$ ls -l | more

The more command takes the output of $ ls -l as its input. The net effect of this command is that the
output of ls -l is displayed one screen at a time. The pipe acts as a container which takes the output of ls
-l and gives it to more as input. This command does not use a disk to connect standard output of ls -l to
the standard input of more because pipe is implemented in the main memory.
In terms of I/O redirection operators, the above command is equivalent to the following command
sequence.
$ ls -l -> temp

more -> temp (or more temp)

[contents of temp]

rm temp

Output of the above two commands is same.

2. Use sort and uniq command to sort a file and print unique values.
$ sort record.txt | uniq

This will sort the given file and print the unique values only.

3. Use head and tail to print lines in a particular range in a file.

$ cat sample2.txt | head -7 | tail -5

This command select first 7 lines and last 5 lines from the file and print those lines which are common
to both of them.

4. Use ls and find to list and print all lines matching a particular pattern in matching files.
$ ls -l | find ./ -type f -name "*.txt" -exec grep "program" {} \;

This command select files with .txt extension in the given directory and search for pattern like
“program” in the above example and print those ine which have program in them.

5. Use cat, grep, tee and wc command to read the particular entry from user and store in a file
and print line count.
$ cat result.txt | grep "Rajat Dua" | tee file2.txt | wc -l

This command select Rajat Dua and store them in file2.txt and print total number of lines
matching Rajat Dua
 Output Redirection
The '>' symbol is used for output (STDOUT) redirection.

Example:
ls -al > listings
Here the output of command ls -al is re-directed to file "listings" instead of your screen.

Note: Use the correct file name while redirecting command output to a file. If there is an existing file
with the same name, the redirected command will delete the contents of that file and then it may be
overwritten."
If you do not want a file to be overwritten but want to add more content to an existing file, then you
should use '>>' operator.

You can redirect standard output, to not just files, but also devices!
$ cat music.mp3 > /dev/audio
The cat command reads the file music.mp3 and sends the output to /dev/audio which is the audio
device. If the sound configurations in your PC are correct, this command will play the file music.mp3

Input redirection
The '<' symbol is used for input(STDIN) redirection

Example: The mail program in Linux can help you send emails from the Terminal.
You can type the contents of the email using the standard device keyboard. But if you want to attach a
File to email you can use the input re-direction operator in the following format.
Mail -s "Subject" to-address < Filename

This would attach the file with the email, and it would be sent to the recipient.
The above examples were simple. Let's look at some advance re-direction techniques which make use
of File Descriptors.

Error Redirection
Whenever you execute a program/command at the terminal, 3 files are always open, viz., standard
input, standard output, standard error.

These files are always present whenever a program is run. As explained before a file descriptor, is
associated with each of these files.
File File Descriptor
 Standard Input STDIN 0
Standard Output STDOUT 1
Standard Error STDERR 2
By default, error stream is displayed on the screen. Error redirection is routing the errors to a file other
than the screen.
Why Error Redirection?
Error re-direction is one of the very popular features of Unix/Linux.
Frequent UNIX users will reckon that many commands give you massive amounts of errors.
•For instance, while searching for files, one typically gets permission denied errors. These
errors usually do not help the person searching for a particular file.
•While executing shell scripts, you often do NOT want error messages cluttering up the normal
program output.
The solution is to re-direct the error messages to a file.
Example 1
$ myprogram 2>errorsfile

Above we are executing a program names myprogram.

The file descriptor for standard error is 2.
Using "2>" we re-direct the error output to a file named "errorfile"
Thus, program output is not cluttered with errors.
Example 2
Here is another example which uses find statement -
find . -name 'my*' 2>error.log
Using the "find" command, we are searching the "." current directory for a file with "name" starting
with "my"

Example 3 Let's see a more complex example,

Server Administrators frequently, list directories and store both error and standard output into a file,
which can be processed later. Here is the command.
ls Documents ABC> dirlist 2>&1
Here,
•which writes the output from one file to the input of another file. 2>&1 means that STDERR
redirects to the target of STDOUT (which is the file dirlist)
•We are redirecting error output to standard output which in turn is being re-directed to file
dirlist. Hence, both the output is written to file dirlist

Summary
•Each file in Linux has a corresponding File Descriptor associated with it
•The keyboard is the standard input device while your screen is the standard output device
•">" is the output redirection operator. ">>" appends output to an existing file
•"<" is the input redirection operator
•">&"re-directs output of one file to another.
•You can re-direct error using its corresponding File Descriptor 2.
mkdir, rmdir, touch, cat, tail , more, rm, mv,
cp, cd, wget, links
-Teertha
mkdir :
mkdir command in Linux allows the user to create directories (also referred to as folders in some
operating systems ).

This command can create multiple directories at once as well as set the permissions for the directories.

Syntax: mkdir [options...] [directories ...]

Options:

--version: It displays the version number, some information regarding the license and exits.

Syntax: mkdir --version

--help: It displays the help related information and exits.

Syntax: mkdir --help

-v or --verbose: It displays a message for every directory created.

Syntax: mkdir -v [directories]

-p: A flag which enables the command to create parent directories as necessary. If the directories
exist, no error is specified.

Syntax: mkdir -p [directories]

-m: This option is used to set the file modes, i.e. permissions, etc. for the created directories. The
syntax of the mode is the same as the chmod command.

Syntax: mkdir -m a=rwx [directories]

rmdir
rmdir command is used remove empty directories from the filesystem in Linux.

The rmdir command removes each and every directory specified in the command line only if these
directories are empty.

So if the specified directory has some directories or files in it then this cannot be removed by rmdir
command.

Syntax: rmdir [-p] [-v | –verbose] [–ignore-fail-on-non-empty] directories …

Options:

rmdir -p: In this option each of the directory argument is treated as a pathname of which all
components will be removed, if they are already empty, starting from the last component.

rmdir -v, –verbose: This option displays verbose information for every directory being processed.

rmdir –ignore-fail-on-non-empty: This option do not report a failure which occurs solely
because a directory is non-empty. Normally, when rmdir is being instructed to remove a non-
empty directory, it simply reports an error. This option consists of all those error messages.

rmdir –version: This option is used to display the version information and exit.

touch

The touch command is a standard command used in UNIX/Linux operating system which is used to
create, change and modify timestamps of a file. Basically, there are two different commands to create a
file in the Linux system which is as follows:

cat command: It is used to create the file with content.

touch command: It is used to create a file without any content. The file created using touch command
is empty. This command can be used when the user doesn’t have data to store at the time of file
creation.

Touch command Syntax to create a new file: You can create a single file at a time using touch
command.

Syntax : touch file_name

Touch command to create multiple files: Touch command can be used to create the multiple numbers
of files at the same time. These files would be empty while creation.

Syntax: touch File1_name File2_name File3_name

options:

touch -a: This command is used to change access time only. To change or update the last access or
modification times of a file touch -a command is used.
 Syntax: touch -a fileName

touch -c : This command is used to check whether a file is created or not. If not created then don’t
create it. This command avoids creating files.

Syntax: touch -c fileName

touch -c-d : This is used to update access and modification time.

Syntax: touch -c-d fileName

touch -m : This is used to change the modification time only. It only updates last modification time.

Syntax: touch -m fileName

touch -r : This command is used to use the timestamp of another file. Here Doc2 file is updated with
the time stamp of File 1.

Syntax: touch -r second_file_name first_file_name

touch -t : This is used to create a file using a specified time.

Syntax: touch -t YYMMDDHHMM fileName

cat

Cat(concatenate) command is very frequently used in Linux. It reads data from the file and gives their
content as output. It helps us to create, view, concatenate files. So let us see some frequently used cat
commands.

1) To view a single file

Command: $cat filename

Output : It will show content of given filename

2) To view multiple files

Command: $cat file1 file2

Output :This will show the content of file1 and file2.

3) To view contents of a file preceding with line numbers.

Command: $cat -n filename

Output: It will show content with line number

4) Create a file

Command: $ cat >newfile

Output: Will create and a file named newfile

5) Copy the contents of one file to another file.

Command: $cat [filename-whose-contents-is-to-be-copied] > [destination-filename]

Output: The content will be copied in destination file

6) Cat command can suppress repeated empty lines in output

Command: $cat -s geeks.txt

Output Will suppress repeated empty lines in output

7) Cat command can append the contents of one file to the end of another file.

Command: $cat file1 >> file2

Output :Will append the contents of one file to the end of another file

8) Cat command can display content in reverse order using tac command.

Command: $tac filename

Output: Will display content in reverse order

9) Cat command can highlight the end of line.

Command: $cat -E "filename"

Output :Will highlight the end of line

10) If you want to use the -v, -E and -T option together, then instead of writing -vET in the
command, you can just use the -A command line option.

Command : $cat -A "filename"

Tail:

The tail command, as the name implies, print the last N number of data of the given input.

By default it prints the last 10 lines of the specified files.

If more than one file name is provided then data from each file is precedes by its file name.

Syntax:
tail [OPTION]... [FILE]...
1. -n num: Prints the last ‘num’ lines instead of last 10 lines. numis mandatory to be
specified in command otherwise it displays an error. This command can also be written
as without symbolizing ‘n’ character but ‘-‘ sign is mandatory.’

$ tail -n 3 state.txt

$ tail -3 state.txt

Tail command also comes with an ‘+’ option which is not present in the head
command. With this option tail command prints the data starting from specified line
number of the file instead of end. For command: tail +n file_name, data will start
printing from line number ‘n’ till the end of the file specified.

2. -c num: Prints the last ‘num’ bytes from the file specified. Newline count as a single
character, so if tail prints out a newline, it will count it as a byte. In this option it is
mandatory to write -cfollowed by positive or negative num depends upon the
requirement. By +num, it display all the data after skipping numbytes from starting of
the specified file and by -num, it display the last num bytes from the file specified.
Note: Without positive or negative sign before num, command will display the
last num bytes from the file specified.

With negative num

$ tail -c -6 state.txt
$ tail -c 6 state.txt
 With positive num
$ tail -c +263 state.txt

3. -q: It is used if more than 1 file is given. Because of this command, data from each file is not
precedes by its file name.

4. -f: This option is mainly used by system administration to monitor the growth of the log
files written by many Unix program as they are running. This option shows the last ten
lines of a file and will update when new lines are added. As new lines are written to the
log, the console will update with the new lines. The prompt doesn’t return even after
work is over so, we have to use the interrupt key to abort this command. In general, the
applications writes error messages to log files. You can use the -foption to check for
the error messages as and when they appear in the log file.

Syntax: $ tail -f logfile

5. -v: By using this option, data from the specified file is always preceded by its file name.
6. –version: This option is used to display the version of tail which is currently running on
your system.

More

more command is used to view the text files in the command prompt, displaying one screen at a
time in case the file is large (For example log files). The more command also allows the user do
scroll up and down through the page. The syntax along with options and command is as follows.
Another application of more is to use it with some other command after a pipe. When the output is
large, we can use more command to see output one by one.

Syntax:
more [-options] [-num] [+/pattern] [+linenum] [file_name]
 [-options]: any option that you want to use in order to change the way the file is displayed.
Choose any one from the followings: (-d, -l, -f, -p, -c, -s, -u)
 [-num]: type the number of lines that you want to display per screen.
 [+/pattern]: replace the pattern with any string that you want to find in the text file.
 [+linenum]: use the line number from where you want to start displaying the text content.
 [file_name]: name of the file containing the text that you want to display on the screen.
While viewing the text file use these controls:
Enter key: to scroll down line by line.
Space bar: To go to the next page.
b key: To go to back one page.

Options:
-d : Use this command in order to help the user to navigate. It displays “[Press space to continue, ‘q’ to
quit.]” and displays “[Press ‘h’ for instructions.]” when wrong key is pressed.

-f : This option does not wrap the long lines and displays them as such.
-p : This option clears the screen and then displays the text.

-c : This command is used to display the pages on the same area by overlapping the previously displayed text.

-s : This option squeezes multiple blank lines into one single blank line.

-u : This option omits the underlines.

+/pattern : This option is used to search the the string inside your text document. You can view all the instances
by navigating through the result.

+num : This option displays the text after the specified number of lines of the document.

rm

rm stands for remove here. rm command is used to remove objects such as files, directories,
symbolic links and so on from the file system like UNIX. To be more precise, rm removes
references to objects from the filesystem, where those objects might have had multiple references
(for example, a file with two different names). By default, it does not remove directories.

This command normally works silently and you should be very careful while running rm command
because once you delete the files then you are not able to recover the contents of files and
directories.
Syntax:
rm [OPTION]... FILE...

Options:

1. -i (Interactive Deletion): Like in cp, the -i option makes the command ask the user for
confirmation before removing each file, you have to press y for confirm deletion, any other key
leaves the file un-deleted.

2. -f (Force Deletion): rm prompts for confirmation removal if a file is write protected. The -f option
 overrides this minor protection and removes the file forcefully.
3. -r (Recursive Deletion): With -r(or -R) option rm command performs a tree-walk and will delete all the
files and sub-directories recursively of the parent directory. At each stage it deletes everything it finds.
Normally, rm wouldn’t delete the directories but when used with this option, it will delete.

4. –version: This option is used to display the version of rmwhich is currently running on your system.

mv:
mv stands for move. mv is used to move one or more files or directories from one place to another in file system
like UNIX. It has two distinct functions:
(i) It rename a file or folder.
(ii) It moves group of files to different directory.
No additional space is consumed on a disk during renaming. This command normally works silently means no
prompt for confirmation.
Syntax: mv [Option] source destination

mv a.txt geek.txt

If the destination file doesn’t exist, it will be created. In the above command mv simply replaces the
source filename in the directory with the destination filename(new name). If the destination file exist,
then it will be overwrite and the source file will be deleted.

Options:
1. -i (Interactive): Like in cp, the -i option makes the command ask the user for confirmation before
moving a file that would overwrite an existing file, you have to press y for confirm moving, any other
key leaves the file as it is. This option doesn’t work if the file doesn’t exist, it simply rename it or move
it to new location.
2. -f (Force): mv prompts for confirmation overwriting the destination file if a file is write protected.
The -f option overrides this minor protection and overwrite the destination file forcefully and delete the
source file.
3. -n (no-clobber): With -n option, mv prevent an existing file from being overwritten.
4. -b(backup): With this option it is easier to take a backup of an existing file that will be overwritten as a
result of the mv command. This will create a backup file with the tilde character(~) appended to it.
5. –version: This option is used to display the version of mv which is currently running on your system.

cp:
cp stands for copy. This command is used to copy files or group of files or directory. It creates an exact image of
a file on a disk with different file name. cp command require at least two filenames in its arguments.

Syntax:
cp [OPTION] Source Destination
cp [OPTION] Source Directory
cp [OPTION] Source-1 Source-2 Source-3 Source-n Directory

1. Two file names : If the command contains two file names, then it copy the contents of 1st
file to the 2nd file. If the 2nd file doesn’t exist, then first it creates one and content is copied to
it. But if it existed then it is simply overwritten without any warning. So be careful when you
choose destination file name.
cp Src_file Dest_file

2. One or more arguments : If the command has one or more arguments, specifying file names and
following those arguments, an argument specifying directory name then this command copies each
source file to the destination directory with the same name, created if not existed but if already existed
then it will be overwritten, so be careful !!.

cp Src_file1 Src_file2 Src_file3 Dest_directory

3. Two directory names : If the command contains two directory names, cp copies all files of the
source directory to the destination directory, creating any files or directories needed. This mode
of operation requires an additional option, typically R, to indicate the recursive copying of
directories.
cp -R Src_directory Dest_directory

In the above command, cp behavior depend upon whether Dest_directory is exist or not. If
the Dest_directory doesn’t exist, cp creates it and copies content of Src_directoryrecursively as
it is. But if Dest_directory exists then copy of Src_directory becomes sub-directory
under Dest_directory.

Options:

1. -i(interactive): i stands for Interactive copying. With this option system first warns the user before
overwriting the destination file. cp prompts for a response, if you press y then it overwrites the file and
with any other option leave it uncopied.
 2. -b(backup): With this option cp command creates the backup of the destination file in the same folder
with the different name and in different format.
3. -f(force): If the system is unable to open destination file for writing operation because the user doesn’t
have writing permission for this file then by using -f option with cp command, destination file is deleted
first and then copying of content is done from source to destination file.
4. -r or -R: Copying directory structure. With this option cpcommand shows its recursive behavior by
copying the entire directory structure recursively.
5. -p(preserve): With -p option cp preserves the following characteristics of each source file in the
corresponding destination file: the time of the last data modification and the time of the last access, the
ownership (only if it has permissions to do this), and the file permission-bits.

cd:
cd command in linux known as change directory command. It is used to change current working
directory.
Syntax: cd [directory]
To move inside a subdirectory : to move inside a subdirectory in linux we use
cd [directory_name]

cd /: this command is used to change directory to the root directory, The root directory is the
first directory in your filesystem hierarchy.

cd dir_1/dir_2/dir_3: This command is used to move inside a directory from a directory

cd ~ : this command is used to change directory to the home directory.

cd : this commad also work same as cd ~ command.
cd .. : this command is used to move to the parent directory of current directory, or the directory
one level up from the current directory. “..” represents parent directory.
cd “dir name”: This command is used to navigate to a directory with white spaces.Instead of
using double quotes we can use single quotes then also this command will work.

Wget
Wget is the non-interactive network downloader which is used to download files from the server even
when the user has not logged on to the system and it can work in the background without hindering the
current process.
  GNU wget is a free utility for non-interactive download of files from the Web. It
supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies.
 wget is non-interactive, meaning that it can work in the background, while the user is
not logged on. This allows you to start a retrieval and disconnect from the system,
letting wget finish the work. By contrast, most of the Web browsers require constant
user’s presence, which can be a great hindrance when transferring a lot of data.
 wget can follow links in HTML and XHTML pages and create local versions of remote
web sites, fully recreating the directory structure of the original site. This is sometimes
referred to as recursive downloading. While doing that, wget respects the Robot
Exclusion Standard (/robots.txt). wget can be instructed to convert the links in
downloaded HTML files to the local files for offline viewing.
 wget has been designed for robustness over slow or unstable network connections; if a
download fails due to a network problem, it will keep retrying until the whole file has
been retrieved. If the server supports resuming, it will instruct the server to continue the
download from where it left off.
Syntax : wget [option] [URL]

1. To simply download a webpage:

wget http://example.com/sample.php
2. To download the file in background
wget -b http://www.example.com/samplepage.php
3. To overwrite the log wile of the wget command
wget http://www.example.com/filename.txt -o /path/filename.txt
4. To resume a partially downloaded file
wget -c http://example.com/samplefile.tar.gz
5. To try a given number of times
wget --tries=10 http://example.com/samplefile.tar.gz

Options :

1. -v / –version : This is used to display the version of the wget available on your system.
Syntax : wget -v
2. -h / –help : This is used to print a help message displaying all the possible options of the line command that is
available with the wget command line options
Syntax :wget -h [URL]
3. -o logfile : This option is used to direct all the messages generated by the system to the logfile specified by the
option and when the process is completed all the messages thus generated are available in the log file. If no log
file has been specified then the output messages are redirected to the default log file i.e. wget -log
Syntax: wget -o logfile [URL]
4. -b / –background : This option is used to send a process to the background as soon as the process has started
so that other processes can be carried out. If no output file is specified via the -o option, output is redirected to
wget-log by default.
Syntax: wget -b [URL]
5. -a : This option is used to append the output messages to the current output log file without overwriting the
file as in -o option the output log file is overwritten but by using this option the log of the previous command is
saved and the current log is written after that of the previous ones.
Syntax :wget -a logfile [URL]
6. -i : This option is used to read URLs from file. If -i is specified as file, URLs are read from the standard
input.If this function is used, no URLs need be present on the command line. If there are URLs both on the
command line and in an input file, those on the command lines will be the first ones to be retrieved. The file
need not be an HTML document if the URLs are just listed sequentially.
Syntax :wget -i inputfile
7. -t number / –tries=number : This option is used to set number of retries to a specified number of times.
Specify 0 or inf for infinite retrying. The default is to retry 20 times, with the exception of fatal errors like
connection refused or link not found, which are not retried once the error has occurred.
Syntax : wget -t number [URL]
8. -c : This option is used to resume a partially downloaded file if the resume capability of the file is yes
otherwise the downloading of the file cannot be resume if the resume capability of the given file is no or not
specified.
Syntax :wget -c [URL]
9. -w : This option is used to set the system to wait the specified number of seconds between the retrievals. Use
of this option is recommended, as it lightens the server load by making the requests less frequent. Instead of in
seconds, the time can be specified in minutes using the m suffix, in hours using h suffix, or in days using d
suffix. Specifying a large value for this option is useful if the network or the destination host is down, so that
wget can wait long enough to reasonably expect the network error to be fixed before the retry.
Syntax :wget -w number in seconds [URL]
10. -r : this option is used to turn on the recursive retrieving of the link specified in case of fatal errors also. This
option is a recursive call to the given link in the command line.
 Syntax: wget -r [URL]

Links
A link in UNIX is a pointer to a file. Like pointers in any programming languages, links in UNIX are
pointers pointing to a file or a directory. Creating links is a kind of shortcuts to access a file. Links
allow more than one file name to refer to the same file, elsewhere.

There are two types of links :

1. Soft Link or Symbolic links

2. Hard Links
These links behave differently when the source of the link (what is being linked to) is moved or
removed. Symbolic links are not updated (they merely contain a string which is the pathname of its
target); hard links always refer to the source, even if moved or removed.

For example, if we have a file a.txt. If we create a hard link to the file and then delete the file, we can
still access the file using hard link. But if we create a soft link of the file and then delete the file, we
can’t access the file through soft link and soft link becomes dangling. Basically hard link increases
reference count of a location while soft links work as a shortcut (like in Windows)

1. Hard Links
Each hard linked file is assigned the same Inode value as the original, therefore they reference the same
physical file location. Hard links more flexible and remain linked even if the original or linked files are
moved throughout the file system, although hard links are unable to cross different file systems.

ls -l command shows all the links with the link column shows number of links.

Links have actual file contents

Removing any link, just reduces the link count, but doesn’t affect other links.

We cannot create a hard link for a directory to avoid recursive loops.

If original file is removed then the link will still show the content of the file.

Command to create a hard link is:

$ ln [original filename] [link name]

2. Soft Links

A soft link is similar to the file shortcut feature which is used in Windows Operating systems. Each soft
linked file contains a separate Inode value that points to the original file. As similar to hard links, any
changes to the data in either file is reflected in the other. Soft links can be linked across different file
systems, although if the original file is deleted or moved, the soft linked file will not work correctly
(called hanging link).

ls -l command shows all links with first column value l? and the link points to original file.

Soft Link contains the path for original file and not the contents.

Removing soft link doesn’t affect anything but removing original file, the link becomes “dangling” link
which points to non existent file.

A soft link can link to a directory.

Link across filesystems: If you want to link files across the filesystems, you can only use symlinks/soft
links.

Command to create a Soft link is:

$ ln -s [original filename] [link name]

ls , who , top , ps
-Lasya
ls is the list command in Linux
ls It will show the full list and content of the directory.

ls ~ It gives the contents of home directory.

ls ../ It give the contents of parent directory.

ls --version It checks the version of ls command.

ls -a In Linux, hidden files start with . (dot) symbol and they are not visible in the
regular directory. The (ls -a) command will enlist the whole list of the current directory
including the hidden files.

ls -l It will show the list in a long list format.We will be able to view the file details
like type ,size , modified date etc.

Here, as you can see the list in long list format.

Columns above indicate specific things:

Column 1 indicates information regarding file permission.

Column 2 indicates the number of links to the file.

Column 3 & 4 indicates the owner and group information.

Column 5 indicayes size of the file in bytes.

Column 6 shows th date and time on which the file was recently modified.

Column 7 shows the file or directory name.

ls -lh

This command will show you the file sizes in human readable format. Size of the file is
very difficult to read when displayed in terms of byte. The (ls -lh)command will give
you the data in terms of Mb, Gb, Tb, etc.

ls -d */

It is used to display only subdirectories.

ls -g or ls -lG

With this you can exclude column of group information and owner.

ls -r It is used to print the list in reverse order.

ls -R It will display the content of the sub-directories in a recursive manner.

ls -ltr

It will sort the list by displaying recently modified filed at top and in reverse output
order.

who command in Linux

who command is used to find out the following information :

1. Time of last system boot

2. Current run level of the system

3. List of logged in users and more.

Description : The who command is used to get information about currently logged in
user on to system.

Syntax : $who [options] [filename] The who command displays the following
information for each user currently logged in to the system if no option is provided :

Login name of the users

Terminal line numbers

Login time of the users in to system

Remote host name of the user

who -a To display all details of current logged in user

whoami To display system’s username

who -r To display current run level of the system.

A runlevel is one of the modes that a Unix -based operating system will run in. Each
runlevel has a certain number of services stopped or started, giving the user control over
the behavior of the machine.

Conventionally, seven runlevels exist, numbered from zero to six.Servers with GUI and
desktop start at runlevel 5. Linux servers which lack GUI start at runlevel 3.

who -d To show details of all dead processes

who -b To show time of the system when it booted last time
who -l To show system login process details
who -q all the login names and number of users logged on.

top command in Linux

top command is used to show the Linux processes. It provides a dynamic real-time view
of the running system.

Usually, this command shows the summary information of the system and the list of
processes or threads which are currently managed by the Linux Kernel.

As soon as you will run this command it will open an interactive command mode where
the top half portion will contain the statistics of processes and resource usage.

And Lower half contains a list of the currently running processes. Pressing q will simply
exit the command mode.
PID: Shows task’s unique process id.
PR: Stands for priority of the task.
SHR: Represents the amount of shared memory used by a task.
VIRT: Total virtual memory used by the task.
USER: User name of owner of task.
%CPU: Represents the CPU usage.
TIME+: CPU Time, the same as ‘TIME’, but reflecting more granularity through
hundredths of a second.
SHR: Represents the Shared Memory size (kb) used by a task.
NI: Represents a Nice Value of task. A Negative nice value implies higher priority, and
positive Nice value means lower priority.
%MEM: Shows the Memory usage of task.

top -u Display Specific User Process

top -b Batch Mode : Send output from top to file or any other programs.
top-c Command Line : The below command starts top with last closed state.
top-d Delay time : It tells delay time between screen updates.

ps command in Linux

As we all know Linux is a multitasking and multi-user systems. So, it allows multiple
processes to operate simultaneously without interfering with each other. Process is one
of the important fundamental concept of the Linux OS. A process is an executing
instance of a program and carry out different tasks within the operating system.

Linux provides us a utility called ps for viewing information related with the processes
on a system which stands as abbreviation for “Process Status”
Syntax –ps [options]
ps Simple process selection : Shows the processes for the current shell
Result contains four columns of information.
Where,
PID – the unique process ID
TTY – terminal type that the user is logged into
TIME – amount of CPU in minutes and seconds that the process has been running
CMD – name of the command that launched the process.

ps -A or ps -e View Processes : View all the running processes use either of the
following option with ps

ps -a View Processes not associated with a terminal : View all processes except both
session leaders and processes not associated with a terminal.

ps -T View all processes associated with this terminal

ps -r View all the running processes
ps -x View all processes owned by you
FTP, sftp, scp, vi editor commands, PING,
TRACEROUTE
-Rakesh
 SCP
SCP (secure copy) is a command line utility that allows you to securely copy files and directories between two
locations.

With scp, you can copy a file or directory:

From your local system to a remote system.

From a remote system to your local system.

Between two remote systems from your local system.

SCP Command Syntax:

scp provides a number of options that control every aspect of its behavior. The most widely used options are:

-P Specifies the remote host ssh port.

-p Preserves files modification and access times.

-q Use this option if you want to suppress the progress meter and non-error messages.

-C. This option will force scp to compresses the data as it is sent to the destination machine.

-r This option will tell scp to recursively copy directories.

Copy a Local File to a Remote System with the scp Command:

scp file.txt remote_username@10.10.0.2:/remote/directory

Copy a Remote File to a Local System using the scp Command:

scp remote_username@10.10.0.2:/remote/file.txt /local/directory

Copy a File Between Two Remote Systems using the scp Command:
scp user1@host1.com:/files/file.txt user2@host2.com:/files

SFTP
SFTP, which stands for SSH File Transfer Protocol, or Secure File Transfer Protocol, is a separate protocol
packaged with SSH that works in a similar way over a secure connection. The advantage is the ability to
leverage a secure connection to transfer files and traverse the filesystem on both the local and remote system.

We can establish an SSH connection and then open up an SFTP session using that connection by
issuing the following command:

sftp sammy@your_server_ip_or_remote_hostname

The most useful command to learn first is the help command. This gives you access to a summary of
the SFTP help. You can call it by typing either of these in the prompt:

help

This will display a list of the available commands:

Available commands:

bye Quit sftp

cd path Change remote directory to 'path'

chgrp grp path Change group of file 'path' to 'grp'

chmod mode path Change permissions of file 'path' to 'mode'

chown own path Change owner of file 'path' to 'own'

df [-hi] [path] Display statistics for current directory

exit Quit sftp

get [-Ppr] remote [local] Download file

help Display this help text

lcd path Change local directory to 'path'

Transferring Remote Files to the Local System:

If we would like download files from our remote host, we can do so by issuing the following
command:

get remoteFile

Msg:Fetching /home/demouser/remoteFile to remoteFile

/home/demouser/remoteFile

We can copy the remote file to a different name by specifying the name afterwards:
get remoteFile localFile
Transferring Local Files to the Remote System:
put localFile

Msg:Uploading localFile to /home/demouser/localFile

localFile

Simple File Manipulations with SFTP:

SFTP allows you to perform the type of basic file maintenance that is useful when working with file hierarchies.

For instance, you can change the owner of a file on the remote system with:

chown userID file

Notice how, unlike the system "chmod" command, the SFTP command does not accept usernames, but instead
uses UIDs. Unfortunately, there is no easy way to know the appropriate UID from within the SFTP interface.

An involved work around could be accomplished with:

get /etc/passwd

!less passwd

root:x:0:0:root:/root:/bin/bash

daemon:x:1:1:daemon:/usr/sbin:/bin/sh

bin:x:2:2:bin:/bin:/bin/sh

sys:x:3:3:sys:/dev:/bin/sh

sync:x:4:65534:sync:/bin:/bin/sync

games:x:5:60:games:/usr/games:/bin/sh

man:x:6:12:man:/var/cache/man:/bin/sh

Notice how instead of giving the "!" command by itself, we've used it as a prefix for a local shell command.

The UID will be in the third column of the file, as delineated by colon characters.

Similarly, we can change the group owner of a file with:

chgrp groupID file

Again, there is no easy way to get a listing of the remote system's groups. We can work around it with the
following command:

get /etc/group

!less group

root:x:0:

daemon:x:1:

bin:x:2:

sys:x:3:

adm:x:4:

tty:x:5:

disk:x:6:

lp:x:7:

The third column holds the ID of the group associated with name in the first column.

FTP
Step 1: Establishing an FTP connection:
ftp domain.com

Step 2: Login with User and Password:

The terminal will return a message like this:

230 Login successful.

Remote system type is UNIX.

Using binary mode to transfer files.

ftp>

Step 3: Downloading files with FTP:

get file

Step 4:Uploading files with FTP:

put file
When the file that you want to upload is not in the local directory, you can use the absolute path
starting with "/" as well:

put /path/file

To upload several files we can use the mput command similar to the mget example from above:

mput *.xls

Step 6: Closing the FTP connection

Once we have done the FTP work, we should close the connection for security reasons. There are three
commands that we can use to close the connection:

bye

exit

quit

vi editor commands
What is the VI editor?

The VI editor is the most popular and classic text editor in the Linux family. Below, are some reasons which
make it a widely used editor –

It is available in almost all Linux Distributions

It works the same across different platforms and Distributions

It is user-friendly. Hence, millions of Linux users love it and use it for their editing needs

Command mode:

The vi editor opens in this mode, and it only understands commands

In this mode, you can, move the cursor and cut, copy, paste the text

This mode also saves the changes you have made to the file

Commands are case sensitive. You should use the right letter case

Insert mode:

This mode is for inserting text in the file.

You can switch to the Insert mode from the command mode by pressing 'i' on the keyboard
 Once you are in Insert mode, any key would be taken as an input for the file on which you are currently
working.

To return to the command mode and save the changes you have made you need to press the Esc key

Starting the vi editor:

To launch the VI Editor -Open the Terminal (CLI) and type

vi <filename_NEW> or <filename_EXISTING>

vi Editing commands:

Note: You should be in the "command mode" to execute these commands. VI editor is case-sensitive so make
sure you type the commands in the right letter-case.

Keystrokes Action

i Insert at cursor (goes into insert mode)

a Write after cursor (goes into insert mode)

A Write at the end of line (goes into insert mode)

ESC Terminate insert mode

u Undo last change

U Undo all changes to the entire line

o Open a new line (goes into insert mode)

dd Delete line

3dd Delete 3 lines.

D Delete contents of line after the cursor

C Delete contents of a line after the cursor and insert new text. Press ESC key

end insertion

dw Delete word

4dw Delete 4 words

cw Change word

x Delete character at the cursor

r Replace character

R Overwrite characters from cursor onward

 s Substitute one character under cursor continue to insert

S Substitute entire line and begin to insert at the beginning of the line

~ Change case of individual character

Make sure you press the right command otherwise you will end up making undesirable changes to the file. You
can also enter the insert mode by pressing a, A, o, as required.

Moving within a file:

You need to be in the command mode to move within a file. The default keys for navigation are mentioned
below else; You can also use the arrow keys on the keyboard.

Keystroke Use

k Move cursor up

j Move cursor down

h Move cursor left

l Move cursor right

Saving and Closing the file:

You should be in the command mode to exit the editor and save changes to the file.

Keystroke Use

Shift+zz Save the file and quit

:w Save the file but keep it open

:q Quit without saving

:wq Save the file and quit

PING
The ping command is usually used as a simple way to verify that a computer can communicate over the network
with another computer or network device.It is of type ICMP.

Ping Command Options

Item Explanation

-t Using this option will ping the target until you force it to stop by using

Ctrl-C.
 -a This ping command option will resolve, if possible, the hostname of an

IP address target.

-n count This option sets the number of ICMP Echo Requests to send, from 1 to

4294967295. The ping command will send 4 by default if -n isn't used.

-l size Use this option to set the size, in bytes.

-f Use this ping command option to prevent ICMP Echo Requests

from being fragmented by routers between you and target.

-i TTL This option sets the Time to Live (TTL) value, the maximum of which

is 255.

Ping Command Examples:

ping -n 5 -l 1500 www.google.com

In this example, the ping command is used to ping the hostname www.google.com. The -n option tells the ping
command to send 5 ICMP Echo Requests instead of the default of 4, and the -l option sets the packet size for
each request to 1500 bytes instead of the default of 32 bytes.

The result displayed in the Command Prompt window will look something like this:

Reply from 172.217.1.142: bytes=1500 time=30ms TTL=54

Reply from 172.217.1.142: bytes=1500 time=30ms TTL=54

Reply from 172.217.1.142: bytes=1500 time=29ms TTL=54

Reply from 172.217.1.142: bytes=1500 time=30ms TTL=54

Reply from 172.217.1.142: bytes=1500 time=31ms TTL=54

Ping statistics for 172.217.1.142: Packets: Sent = 5, Received = 5, Lost = 0 (0% loss),

Approximate round trip times in milli-seconds:

Minimum = 29ms, Maximum = 31ms, Average = 30ms

 TRACEROUTE
Tracert Command Options:

Item Description

-d This option prevents tracert from resolving IP addresses to hostnames,

resulting in much faster results.

-h MaxHops This tracert option specifies the maximum number of hops in the by

search for the target. If you do not specify MaxHops, and a target has

not been found 30 hops, tracert will stop looking.

-w TimeOut You can specify the time, in milliseconds, to allow each reply before

timeout using this tracert option.

-4 This option forces tracert to use IPv4 only.

-6 This option forces tracert to use IPv6 only.

target This is the destination, either an IP address or hostname.

/? Use the help switch with the tracert command to show detailed help

about the command's several options

Command:

tracert www.google.com

With the tracert command shown above, we're asking tracert to show us the path from the local computer all the
way to the network device with the hostname www.google.com.

Tracing route to www.l.google.com [209.85.225.104]

over a maximum of 30 hops:

1 <1 ms <1 ms <1 ms 10.1.0.1

2 35 ms 19 ms 29 ms 98.245.140.1

3 11 ms 27 ms 9 ms te-0-3.dnv.comcast.net [68.85.105.201]

13 81 ms 76 ms 75 ms 209.85.241.37

14 84 ms 91 ms 87 ms 209.85.248.102
15 76 ms 112 ms 76 ms iy-f104.1e100.net [209.85.225.104]

Trace complete.

In this example, we can see that tracert identified fifteen network devices including our router at 10.1.0.1 and all
the way through to the target of www.google.com, which we now know uses the public IP address of
209.85.225.104, one of Google's many IP addresses.
Sed, grep , awk, find,
-Kiran
 SED
SED command in UNIX is stands for stream editor and it can perform lot’s of function on file like,
searching, find and replace, insertion or deletion.

SED is a powerful text stream editor. Can do insertion, deletion, search and replace(substitution).

SED command in unix supports regular expression which allows it perform complex pattern matching.

Syntax:

sed OPTIONS... [SCRIPT] [INPUTFILE...]

Example:

Consider the below text file as an input.

$cat > geekfile.txt

unix is great os. unix is opensource. unix is free os.

learn operating system.

unix linux which one you choose.

unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

Sample Commands:

1.Replacing or substituting string : Sed command is mostly used to replace the text in a file. The below simple
sed command replaces the word “unix” with “linux” in the file.

$sed 's/unix/linux/' geekfile.txt

Output :

linux is great os. unix is opensource. unix is free os.

learn operating system.

linux linux which one you choose.

linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

2.Replacing the nth occurrence of a pattern in a line : Use the /1, /2 etc flags to replace the first, second
occurrence of a pattern in a line. The below command replaces the second occurrence of the word “unix” with
“linux” in a line.

$sed 's/unix/linux/2' geekfile.txt

Output:
unix is great os. linux is opensource. unix is free os.

learn operating system.

unix linux which one you choose.

unix is easy to learn.linux is a multiuser os.Learn unix .unix is a powerful.

3.Replacing all the occurrence of the pattern in a line : The substitute flag /g (global replacement) specifies
the sed command to replace all the occurrences of the string in the line.

$sed 's/unix/linux/g' geekfile.txt

Output :

linux is great os. linux is opensource. linux is free os.

learn operating system.

linux linux which one you choose.

linux is easy to learn.linux is a multiuser os.Learn linux .linux is a powerful.

4.Replacing from nth occurrence to all occurrences in a line : Use the combination of /1, /2 etc and /g to
replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the
third, fourth, fifth… “unix” word with “linux” word in a line.

$sed 's/unix/linux/3g' geekfile.txt

Output:

unix is great os. unix is opensource. linux is free os.

learn operating system.

unix linux which one you choose.

unix is easy to learn.unix is a multiuser os.Learn linux .linux is a powerful.

5.Parenthesize first character of each word : This sed example prints the first character of every word in
paranthesis.

$ echo "Welcome To The Geek Stuff" | sed 's/\(\b[A-Z]\)/\(\1\)/g'

Output:

(W)elcome (T)o (T)he (G)eek (S)tuff

6.Replacing string on a specific line number : You can restrict the sed command to replace the string on a
specific line number. An example is

$sed '3 s/unix/linux/' geekfile.txt

Output:
unix is great os. unix is opensource. unix is free os.

learn operating system.

linux linux which one you choose.

unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

The above sed command replaces the string only on the third line.

7.Duplicating the replaced line with /p flag : The /p print flag prints the replaced line twice on the terminal. If
a line does not have the search pattern and is not replaced, then the /p prints that line only once.

$sed 's/unix/linux/p' geekfile.txt

Output:

linux is great os. unix is opensource. unix is free os.

linux is great os. unix is opensource. unix is free os.

learn operating system.

linux linux which one you choose.

linux linux which one you choose.

linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

8.Printing only the replaced lines : Use the -n option along with the /p print flag to display only the replaced
lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only
one time.

$sed -n 's/unix/linux/p' geekfile.txt

Output:

linux is great os. unix is opensource. unix is free os.

linux linux which one you choose.

linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

If you use -n alone without /p, then the sed does not print anything.

9.Replacing string on a range of lines : You can specify a range of line numbers to the sed command for
replacing a string.

$sed '1,3 s/unix/linux/' geekfile.txt

Output:

linux is great os. unix is opensource. unix is free os.

learn operating system.

linux linux which one you choose.

unix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

Here the sed command replaces the lines with range from 1 to 3. Another example is

$sed '2,$ s/unix/linux/' geekfile.txt

Output:

unix is great os. unix is opensource. unix is free os.

learn operating system.

linux linux which one you choose.

linux is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful

Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the
file.

10.Deleting lines from a particular file : SED command can also be used for deleting lines from a particular
file. SED command is used for performing deletion operation without even opening the file

Examples:

1. To Delete a particular line say n in this example

Syntax :

$ sed 'nd' filename.txt

Example:

$ sed '5d' filename.txt

2. To Delete a last line

Syntax:

$ sed '$d' filename.txt

3. To Delete line from range x to y

Syntax:
$ sed 'x,yd' filename.txt

Example:

$ sed '3,6d' filename.txt

5. To Delete from nth to last line

Syntax:

$ sed 'nth,$d' filename.txt

Example:

$ sed '12,$d' filename.txt

6. To Delete pattern matching line

Syntax:

$ sed '/pattern/d' filename.txt

Example:

$ sed '/abc/d' filename.txt

grep
grep stands for globally search for regular expression and print out.The grep filter searches a file for a particular
pattern of characters, and displays all lines that contain that pattern.

Syntax:

grep [options] pattern [files]

Options Description

-c : This prints only a count of the lines that match a pattern

-h : Display the matched lines, but do not display the filenames.

-i : Ignores, case for matching

-l : Displays list of a filenames only.

-n : Display the matched lines and their line numbers.

-v : This prints out all the lines that do not matches the pattern

-e exp : Specifies expression with this option. Can use multiple times.
-f file : Takes patterns from file, one per line.

-E : Treats pattern as an extended regular expression (ERE)

-w : Match whole word

-o : Print only the matched parts of a matching line,

with each such part on a separate output line.

Sample Commands:

Consider the below file as an input.

$cat > geekfile.txt

unix is great os. unix is opensource. unix is free os.

learn operating system.

Unix linux which one you choose.

uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

1. Case insensitive search : The -i option enables to search for a string case insensitively in the give file. It
matches the words like “UNIX”, “Unix”, “unix”.

$grep -i "UNix" geekfile.txt

Output:

unix is great os. unix is opensource. unix is free os.

Unix linux which one you choose.

uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

2. Displaying the count of number of matches : We can find the number of lines that matches the given
string/pattern

$grep -c "unix" geekfile.txt

Output:

3. Display the file names that matches the pattern : We can just display the files that contains the given
string/pattern.

$grep -l "unix" *

or

$grep -l "unix" f1.txt f2.txt f3.xt f4.txt

Output:

geekfile.txt

4. Checking for the whole words in a file : By default, grep matches the given string/pattern even if it found as a
substring in a file. The -w option to grep makes it match only the whole words.

$ grep -w "unix" geekfile.txt

Output:

unix is great os. unix is opensource. unix is free os.

uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

5. Displaying only the matched pattern : By default, grep displays the entire line which has the matched string.
We can make the grep to display only the matched string by using the -o option.

$ grep -o "unix" geekfile.txt

Output:

unix

unix

unix

unix

unix

unix

6. Show line number while displaying the output using grep -n : To show the line number of file with the line
matched.

$ grep -n "unix" geekfile.txt

Output:

1:unix is great os. unix is opensource. unix is free os.

4:uNix is easy to learn.unix is a multiuser os.Learn unix .unix is a powerful.

7. Inverting the pattern match : You can display the lines that are not matched with the specified search sting
pattern using the -v option.

$ grep -v "unix" geekfile.txt

Output:

learn operating system.

Unix linux which one you choose.

8. Matching the lines that start with a string : The ^ regular expression pattern specifies the start of a line. This
can be used in grep to match the lines which start with the given string or pattern.

$ grep "^unix" geekfile.txt

Output:

unix is great os. unix is opensource. unix is free os.

9. Matching the lines that end with a string : The $ regular expression pattern specifies the end of a line. This can
be used in grep to match the lines which end with the given string or pattern.

$ grep "os$" geekfile.txt

10.Specifies expression with -e option. Can use multiple times :

$grep –e "Agarwal" –e "Aggarwal" –e "Agrawal" geekfile.txt

11. -f file option Takes patterns from file, one per line:

$cat pattern.txt

Agarwal

Aggarwal

Agrawal

$grep –f pattern.txt geekfile.txt

awk
Awk is a scripting language used for manipulating data and generating reports.The awk command programming
language requires no compiling, and allows the user to use variables, numeric functions, string functions, and
logical operators.

1. AWK Operations:
(a) Scans a file line by line

(b) Splits each input line into fields

(c) Compares input line/fields to pattern

(d) Performs action(s) on matched lines

Syntax:
awk options 'selection _criteria {action }' input-file > output-file
Example:
Consider the following text file as the input file for all cases below.

$cat > employee.txt

ajay manager account 45000

sunil clerk account 25000

varun manager sales 50000

amit manager account 47000

tarun peon sales 15000

deepak clerk sales 23000

sunil peon sales 13000

satvik director purchase 80000

1. Default behavior of Awk : By default Awk prints every line of data from the specified file.

$ awk '{print}' employee.txt

Output:

ajay manager account 45000

sunil clerk account 25000

varun manager sales 50000

amit manager account 47000

tarun peon sales 15000

deepak clerk sales 23000

sunil peon sales 13000

satvik director purchase 80000

In the above example, no pattern is given. So the actions are applicable to all the lines. Action print without any
argument prints the whole line by default, so it prints all the lines of the file without failure.

2. Print the lines which matches with the given pattern.

$ awk '/manager/ {print}' employee.txt

Output:

ajay manager account 45000

varun manager sales 50000

amit manager account 47000

In the above example, the awk command prints all the line which matches with the ‘manager’.

3. Spliting a Line Into Fields :

For each record i.e line, the awk command splits the record delimited by whitespace character by default and
stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4 respectively. Also, $0
represents the whole line.

$ awk '{print $1,$4}' employee.txt

Output:

ajay 45000

sunil 25000

varun 50000

amit 47000

tarun 15000

deepak 23000

sunil 13000

satvik 80000

In the above example, $1 and $4 represents Name and Salary fields respectively.

4.Use of NR built-in variables (Display Line Number)

$ awk '{print NR,$0}' employee.txt

Output:

1 ajay manager account 45000

2 sunil clerk account 25000

3 varun manager sales 50000

4 amit manager account 47000

5 tarun peon sales 15000

6 deepak clerk sales 23000

7 sunil peon sales 13000

8 satvik director purchase 80000

In the above example, the awk command with NR prints all the lines along with the line number.

5.Use of NF built-in variables (Display Last Field):

$ awk '{print $1,$NF}' employee.txt

Output:

ajay 45000

sunil 25000

varun 50000

amit 47000

tarun 15000

deepak 23000

sunil 13000

satvik 80000

In the above example $1 represents Name and $NF represents Salary. We can get the Salary using $NF , where
$NF represents last field.

For the given text file:

$cat > geeksforgeeks.txt

A B C

Tarun A12 1

Man B6 2
Praveen M42 3

6) To print the first item along with the row number(NR) separated with ” – “ from each line in
geeksforgeeks.txt:

$ awk '{print NR "- " $1 }' geeksforgeeks.txt

1 - Tarun

2 – Manav

3 - Praveen

7) To return the second row/item from geeksforgeeks.txt:

$ awk '{print $2}' geeksforgeeks.txt

A12

B6

M42

8) To print any non empty line if present:

$ awk 'NF > 0' geeksforgeeks.txt

9) To find the length of the longest line present in the file:

$ awk '{ if (length($0) > max) max = length($0) } END { print max }' geeksforgeeks.txt

13

10) To count the lines in a file:

$ awk 'END { print NR }' geeksforgeeks.txt

11) Printing lines with more than 10 characters:

$ awk 'length($0) > 10' geeksforgeeks.txt

Tarun A12 1

Praveen M42 3

12) To find/check for any string in any column:

$ awk '{ if($3 == "B6") print $0;}' geeksforgeeks.txt

13) To print the squares of first numbers from 1 to n say 6:

$ awk 'BEGIN { for(i=1;i<=6;i++) print "square of", i, "is",i*i; }'

square of 1 is 1

square of 2 is 4

square of 3 is 9

square of 4 is 16

square of 5 is 25

square of 6 is 36

14.Another use of NR built-in variables (Display Line From 3 to 6):

$ awk 'NR==3, NR==6 {print NR,$0}' employee.txt

Output:

3 varun manager sales 50000

4 amit manager account 47000

5 tarun peon sales 15000

6 deepak clerk sales 23000

find
find command in unix is used to find files and directories and perform subsequent operations on them. It
supports searching by file, folder, name, creation date, modification date, owner and permissions

Syntax :

$ find [where to start searching from]

[expression determines what to find] [-options] [what to find]

1. Search a file with specific name.

$ find ./GFG -name sample.txt

It will search for sample.txt in GFG directory.

2. Search a file with pattern.

$ find ./GFG -name *.txt

It will give all files which have ‘.txt’ at the end.

3. How to find and delete a file with confirmation.

$ find ./GFG -name sample.txt -exec rm -i {} \;

When this command is entered, a prompt will come for confirmation, if you want to delete sample.txt or not. if
you enter ‘Y/y’ it will delete the file.

4. Search for empty files and directories.

$ find ./GFG -empty

This command find all empty folders and files in the entered directory or sub-directories.

5. Search for file with entered permissions.

$ find ./GFG -perm 664

This command find all the files in the GFG directory or sub-directory with the given

6. Search text within multiple files.

$ find ./ -type f -name "*.txt" -exec grep 'Geek' {} \;

This command print lines which have ‘Geek’ in them and ‘-type f’ specifies the input type is a file.
Gzip, gunzip, tar
-Aashi
 File compression commands
Gzip –
- Each single file is compressed into a single file. The compressed file consists of a GNU zip
header and deflated data.
- Compressed file id added with a suffix .gz
- With no arguments gzip compress the standard input and writes the compressed file to standard
output.
Syntax: gzip [options] [filename]
Options:
-f : Sometimes the file cannot be compressed like if a file is there, myfile.gz. As this file already
have .gz extension, it will show error while compressing.
For these scenarios force gzip is used

-k : If we want to compress the file and keep original file also.

-L : Display the gzip license.

-r : Compress evey file in a folder and its subfolders. It traverse the directory structure and compress
each file in that folder structure.
-[1-9] : Allow to change the compression level. Default level is 6.
-1/--fast: smaller compression but fast speed.
-9/--best: maximum compression but slow speed.

-v : Shows the % reduction for each file compressed or decompressed.

-V : Display version number and compilation options then quit.

-d : Decompress the the file.

-q : Supress all warnings.

-c : Concatenate input files before compressing them.

Gunzip –
Used to compress or expand a file or a list of files in Linux.
Syntax: gunzip [options] [archive name/filename]
Options:
-f : Decopress forcefully.

-k : Keep both the uncompressed and the original file.

-l : Information of compressed for uncompressed file.

-L : Show software license and exit

-r: Uncompress all the files within the folder and subfolder recursively.

-v: Get verbose information.

-V : Version number

-a : Decompress the file ignoring the ASCII option

-h : display the help information

tar command in Linux

The Linux ‘tar’ stands for tape archive, is used to create Archive and extract the Archive files.

tar command in Linux is one of the important command which provides archiving

functionality in Linux. We can use Linux tar command to create compressed or

uncompressed Archive files and also maintain and modify them.

Syntax:

tar [options] [archive-file] [file or directory to be archived]

Options:

-c : Creates Archive

-x : Extract the archive

-f : creates archive with given filename

-t : displays or lists files in archived file

-u : archives and adds to an existing archive file

-v : Displays Verbose Information

-A : Concatenates the archive files

-z : zip, tells tar command that create tar file using gzip

-j : filter archive tar file using tbzip

-W : Verify a archive file

-r : update or add file or directory in already existed .tar file

What is an Archive file?

An Archive file is a file that is composed of one or more files along with metadata. Archive

files are used to collect multiple data files together into a single file for easier portability and

storage, or simply to compress files to use less storage space.

Examples:

1. Creating an uncompressed tar Archive using option -cvf : This command creates a tar file

called file.tar which is the Archive of all .c files in current directory.

$ tar cvf file.tar *.c

Output :

os2.c

os3.c

os4.c

2. Extracting files from Archive using option -xvf : This command extracts files from

Archives.

$ tar xvf file.tar

Output :

os2.c

os3.c

os4.c

3. gzip compression on the tar Archive, using option -z : This command creates a tar file

called file.tar.gz which is the Archive of .c files.

$ tar cvzf file.tar.gz *.c

4. Extracting a gzip tar Archive *.tar.gz using option -xvzf : This command extracts files from

tar archived file.tar.gz files.

$ tar xvzf file.tar.gz

5. Creating compressed tar archive file in Linux using option -j : This command compresses

and creates archive file less than the size of the gzip. Both compress and decompress takes

more time then gzip.

$ tar cvfj file.tar.tbz example.cpp

Output :

$tar cvfj file.tar.tbz example.cpp

example.cpp

$tar tvf file.tar.tbz

-rwxrwxrwx root/root 94 2017-09-17 02:47 example.cpp

6. Untar single tar file or specified directory in Linux : This command will Untar a file in

current directory or in a specified directory using -C option.

$ tar xvfj file.tar

or

$ tar xvfj file.tar -C path of file in directoy

7. Untar multiple .tar, .tar.gz, .tar.tbz file in Linux : This command will extract or untar multiple

files from the tar, tar.gz and tar.bz2 archive file. For example the above command will extract

“fileA” “fileB” from the archive files.

$ tar xvf file.tar "fileA" "fileB"

or

$ tar zxvf file1.tar.gz "fileA" "fileB"

or

$ tar jxvf file2.tar.tbz "fileA" "fileB"

8. Check size of existing tar, tar.gz, tar.tbz file in Linux : The above command will display the

size of archive file in Kilobytes(KB).

$ tar czf file.tar | wc -c

or

$ tar czf file1.tar.gz | wc -c

or

$ tar czf file2.tar.tbz | wc -c

9. Update existing tar file in Linux

$ tar rvf file.tar *.c

Output :

os1.c

10. list the contents and specify the tarfile using option -tf : This command will list the entire

list of archived file. We can also list for specific content in a tarfile

$ tar tf file.tar

Output :

example.cpp

11. Applying pipe to through ‘grep command’ to find what we are looking for : This command

will list only for the mentioned text or image in grep from archived file.

$ tar tvf file.tar | grep "text to find"

or

$ tar tvf file.tar | grep "filename.file extension"

12. We can pass a file name as an argument to search a tarfile : This command views the

archived files along with their details.

$ tar tvf file.tar filename

13. Viewing the Archive using option -tvf

$ tar tvf file.tar

Output :

-rwxrwxrwx root/root 191 2017-09-17 02:20 os2.c

-rwxrwxrwx root/root 218 2017-09-17 02:20 os3.c

-rwxrwxrwx root/root 493 2017-09-17 02:20 os4.c

What are wildcards in Linux

Alternatively referred to as a ‘wild character’ or ‘wildcard character’, a wildcard is a symbol

used to replace or represent one or more characters. Wildcards are typically either an asterisk

(*), which represents one or more characters or question mark (?),which represents a single

character.

Example :

14. To search for an image in .png format : This will extract only files with the extension .png

from the archive file.tar. The –wildcards option tells tar to interpret wildcards in the name of

the files

to be extracted; the filename (*.png) is enclosed in single-quotes to protect the wildcard (*)

from being expanded incorrectly by the shell.

$ tar tvf file.tar --wildcards '*.png'

Note: In above commands ” * ” is used in place of file name to take all the files present in that

particular directory.