5.2.1. grep

The grep program searches a file or files for lines that have a certain pattern. The syntax is:

grep "pattern" file(s)

The name "grep" derives from the ed (a Unix line editor) command g/re/p, which means "globally search for a regular expression and print all lines containing it." A regular expression is either some plain text (a word, for example) and/or special characters used for pattern matching. When you learn more about regular expressions, you can use them to specify complex patterns of text.

The simplest use of grep is to look for a pattern consisting of a single word. It can be used in a pipe so that only those lines of the input files containing a given string are sent to the standard output. But let's start with an example reading from files: searching all files in the working directory for a word--say, Unix. We'll use the wildcard * to quickly give grep all filenames in the directory.

$ grep "Unix" *
ch01:Unix is a flexible and powerful operating system
ch01:When the Unix designers started work, little did
ch05:What can we do with Unix?
$

When grep searches multiple files, it shows the filename where it finds each matching line of text. Alternatively, if you don't give grep a filename to read, it reads its standard input; that's the way all filter programs work:

$ ls -l | grep "Aug"
-rw-rw-rw-   1 john  doc     11008 Aug  6 14:10 ch02
-rw-rw-rw-   1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--   1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

First, the example runs ls -l to list your directory. The standard output of ls -l is piped to grep, which only outputs lines that contain the string Aug (that is, files that were last modified in August). Because the standard output of grep isn't redirected, those lines go to the terminal screen.

grep options let you modify the search. Table 5-1 lists some of the options.

Table 5-1. Some grep options

Option	Description
-v	Print all lines that do not match pattern.
-n	Print the matched line and its line number.
-l	Print only the names of files with matching lines (lowercase letter "L").
-c	Print only the count of matching lines.
-i	Match either upper- or lowercase.

Next, let's use a regular expression that tells grep to find lines with carol, followed by zero or more other characters (abbreviated in a regular expression as ".*"),[15] then followed by Aug:

[15] Note that the regular expression for "zero or more characters," ".*", is different than the corresponding filename wildcard "*". See Section 4.2 in Chapter 4. We can't cover regular expressions in enough depth here to explain the difference--though more-detailed books do. As a rule of thumb, remember that the first argument to grep is a regular expression; other arguments, if any, are filenames that can use wildcards.

$ ls -l | grep "carol.*Aug"
-rw-rw-r--   1 carol doc      1605 Aug 23 07:35 macros
$

For more about regular expressions, see the references in Section 8.1 (Chapter 8).

5.2.2. sort

The sort program arranges lines of text alphabetically or numerically. The following example sorts the lines in the food file (from Section 4.5 in Chapter 4) alphabetically. sort doesn't modify the file itself; it reads the file and writes the sorted text to the standard output.

$ sort food
Afghani Cuisine
Bangkok Wok
Big Apple Deli
Isle of Java
Mandalay
Sushi and Sashimi
Sweet Tooth
Tio Pepe's Peppers

By default, sort arranges lines of text alphabetically. Many options control the sorting, and Table 5-2 lists some of them.

Table 5-2. Some sort options

Option	Description
-n	Sort numerically (example: 10 sorts after 2), ignore blanks and tabs.
-r	Reverse the sorting order.
-f	Sort upper- and lowercase together.
+x	Ignore first x fields when sorting.

More than two commands may be linked up into a pipe. Taking a previous pipe example using grep, we can further sort the files modified in August by order of size. The following pipe uses the commands ls, grep, and sort:

$ ls -l | grep "Aug" | sort +4n
-rw-rw-r--  1 carol doc      1605 Aug 23 07:35 macros
-rw-rw-r--  1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-rw-  1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-rw-  1 john  doc     11008 Aug  6 14:10 ch02
$

This pipe sorts all files in your directory modified in August by order of size, and prints them to the terminal screen. The sort option +4n skips four fields (fields are separated by blanks), then sorts the lines in numeric order. So, the output of ls, filtered by grep, is sorted by the file size (this is the fifth column, starting with 1605). Both grep and sort are used here as filters to modify the output of the ls -l command. If you wanted to email this listing to someone, you could add a final pipe to the mail program. Or you could print the listing by piping the sort output to your printer command (either lp or lpr).

5.2.3. Piping to a Pager

The less program, which you saw in Section 3.2 in Chapter 3, can also be used as a filter. A long output normally zips by you on the screen, but if you run text through less, the display stops after each screenful of text.

Let's assume that you have a long directory listing. (If you want to try this example and need a directory with lots of files, use cd first to change to a system directory such as /bin or /usr/bin.) To make it easier to read the sorted listing, pipe the output through less:

$ ls -l | grep "Aug" | sort +4n | less
-rw-rw-r--  1 carol doc      1605 Aug 23 07:35 macros
-rw-rw-r--  1 john  doc      2488 Aug 15 10:51 intro
-rw-rw-rw-  1 john  doc      8515 Aug  6 15:30 ch07
-rw-rw-r--  1 john  doc     14827 Aug  9 12:40 ch03
	.
	.
	.
-rw-rw-rw-  1 john  doc     16867 Aug  6 15:56 ch05
:

less reads a screenful of text from the pipe (consisting of lines sorted by order of file size), then prints a colon (:) prompt. At the prompt, you can type a less command to move through the sorted text. less reads more text from the pipe and shows it to you, as well as saves a copy of what it has read, so you can go backwards to reread previous text if you want to. (The simpler pager programs more and pg generally can't back up while reading from a pipe.) When you're done seeing the sorted text, the q command quits less.

5.2.3.1. Exercise: redirecting input/output

In the following exercises you redirect output, create a simple pipe, and use filters to modify output.

Redirect output to a file.	Enter who > users
Email that file to yourself. (Replace username with your own username.)	Enter mail username < users
Sort output of a program.	Enter who | sort
Append sorted output to a file.	Enter who | sort >> users
Display output to screen.	Enter less users (or more users or pg users)
Display long output to screen.	Enter ls -l /bin | less (or more or pg)
Format and print a file with pr.	Enter pr users | lp or pr users | lpr