Linux System Administration: A User's Guide

Next year, Addison Wesley Longman will be publishing my new book, Linux System Administration: A User's Guide (Copyright 2001, ISBN 0-201-71934-7). Since the focus of this issue is system administration, the kind folks at Linux Journal have provided me with an opportunity to give everyone a sneak peek at what is to come.

So, let me set the scene. It is a dark and stormy night (I've always wanted to write that), and a lone sysadmin is working late looking for ways to get home and still get his work done. This is a snippet from Chapter 15, a chapter I call “Creative Laziness”. Along with a sizable crowd of others over the years, I have been preaching, I mean speculating, that creative laziness can be a wonderful tool. The kind of lazy person I admire works hard to find easier ways to do things and is always looking for the simpler, more elegant solution to any problem. If I may quote one of the greats of science fiction, Robert A. Heinlein, “Progress is made by lazy men looking for easier ways to do things.”

The introduction to this chapter covers a variety of automation tools before arriving at this point.The hour has just struck 23:00. The pizza is long gone, and the cafeteria is out of coffee. Lights. Camera. Action....

At first glance, it would seem that you are out of luck if what you want to automate requires human intervention. There are things that need a human: picking from a menu option, entering passwords or making decisions based on the information presented. Interactive applications require a user's reaction, don't they? The answer, for the cleverly lazy system administrator, is not always—thanks to a little program called expect.

While I had heard of expect sometime before, I discovered how useful this language was only a few years ago. My partner and I were developing a web-based system that required regular updates from the main computer's database, a database that would not allow command line scripting. The data we needed required the execution of an SQL statement that could only be entered via the vendor's menu interface. That SQL statement would then generate the data file we needed for the web interface. The whole process hinged on writing something that mimicked a user sitting at a terminal and entering information as the various prompts were presented to him or her. expect, a software suite/language based on Tcl, was the answer to this dilemma. Later, expect would make it possible to stretch this web tool well beyond what we, ahem, expected at the time.

Still wondering whether you need it? Remember that laziness discussion at the beginning of the chapter? Well, pretend that you are working late and the last thing you need to do before leaving is to log on to your remote site, make sure that a specific application has been completed (it is always done by 3:00 a.m.), and then download the file that application generates back to your local site. It is now 10:00 p.m. and you would much rather go home than wait there for the magic moment when the file is ready. You could just launch an at job that starts ncftp for the download, but you don't know the file name since the output name changes at each run. You find the name by logging into the menu system and checking the completion log. I am purposely making this complicated to demonstrate that there are instances that are hard to automate with a simple shell script.

The basic format of an expect script is this:

#!/usr/local/bin/expect
# Comments on this script (name, what it does,
# optional)
spawn some_command
set response myanswer
expect "Some prompt . . . ."
send $response\r
close

Here's what happens. The “spawn” keyword tells expect to begin some program. This could be a shell (spawn /bin/bash) or any kind of command through which the session will take place. With the “set ” keyword, I am setting the variable response to some predetermined response. The language's namesake, the “expect” keyword does exactly what it sounds like it does. It scans the output of whatever command we invoked with spawn, searching for matching text. Then, “send” responds to the expected text with the first variable, $response. Let's do something real now.

I run an Apache web server on my system with OpenSSL extensions for secure transactions. Starting Apache with the OpenSSL extensions running requires me to enter a security passphrase in order for it to start up, because the private key files on the server are encrypted (see Chapter 26, “Building a Secure Web Server”). This is all fine if I am here to enter the passphrase, but what happens if the server goes down when I am not there? It hasn't happened for months, but these things happen and we do go on holidays sometimes. It could be something as crazy as me adding an SCSI card for my new tape drive. I might have forgotten (it has happened) to restart the web server with OpenSSL running. What then? To get around this problem, I wrote the simple expect script you see below:

#!/usr/bin/expect
# Routine: startapachessl
# Purpose: Start web server with OpenSSL active
#
log_file -a /tmp/expectlog
#log_user 0
spawn /bin/bash
sleep .2
send "usr/local/apache/bin/apachectl startssl\r"
expect "Enter pass phrase*"
sleep .2
send "mysecretphrasegoeshere\r"
sleep .2
close

When the system restarts, whether I am there or not, this script will restart my Apache web server with OpenSSL running. Looking at the script, you'll notice a couple of interesting things. For instance, the log_file parameter is new. What this does is define a log file for the execution of this script. Whether the file is written to or not is defined by the log_user parameter. If set to “1”, then logging will take place. I tend to use log_user when I am still testing the script, but you may decide you want to capture the output at all times. Notice as well that I am spawning a bash shell to execute the script that starts my server. Then, there are the sleep statements. In all cases, I have the shell wait one-fifth of a second before continuing . Finally, the close statement tells the spawned process that there is nothing more to come. At this time, expect terminates and returns to the process that spawned it.

There is no doubt that you could program these functions with other languages, but expect makes it easy. What you will find as you go along is that not every tool is perfect for every job. For quick and dirty automation of interactive applications, nothing beats expect. Fully exploring expect would require a book of its own (in fact, there is one). What I am trying to do is give you a taste of what you can do with it rather than explain every aspect of the language. Before I let you run off to do your own exploring, let me take these examples one step further.

We all know that changing passwords on a regular basis is as good a thing as choosing good passwords (see Chapter 6), and it is a fairly easy thing to have users do when they log in, but it is somewhat more difficult if they do not have a login account. I'm talking about e-mail—only users, the ones whom you allow POP3 mail pickup (or web-based e-mail) but no actual command prompt access. A number of offices have precisely this kind of setup for their Linux system—it serves as an e-mail or Internet gateway and allows no logins. So, how do you allow users to change their passwords when they aren't allowed to log in? After all, changing passwords is an interactive activity as the following dialog will attest:

[root@myhost] # passwd
New UNIX password:

Even more complicated is that in order for non-root users to change their passwords, they must first enter their old password, so the dialog is even more complex. What now?

You could create a web-based form whereby a user could enter all that information up front (see Figure 1).

Figure 1. A Password Change Webform

A Perl script behind the form would extract the variables and pass them to an expect script that does the rest. If you are curious about this little web application or would like to use it, feel free to download it from my web site. In the meantime, have a look at this segment from the application:

#!/usr/bin/expect
# Routine: psdcmd
# Purpose: to change a user's password with expect
log_user 1
set uservar [lindex $argv 0]
set currpassword [lindex $argv 1]
set newpassword [lindex $argv 2]
set renewpassword [lindex $argv 3]
#
# log_file -a /tmp/expectlog
# send_user "Spawning passwd command with uservar.\n"
spawn su -l -c "passwd" $uservar
expect "Password:"
sleep .1
send "$currpassword\r"
sleep .1
#
expect {
        "(current) UNIX password:" {send
"$currpassword\r"}
        "su: incorrect password" {exit 0}
        }
sleep .1
expect {
        "su: incorrect password" {exit 0}
                "New UNIX password:" {send
"$newpassword\r"}
}
sleep .1
expect {
        "BAD PASSWORD:" {exit 0}
                "Retype new UNIX password:" {send
"$renewpassword\r"}
}
sleep .1
expect {
        "su: incorrect password" {exit 0}
                "New UNIX password:" {exit 0}
}
#End of password change routine

The set varname [lindex $argv num] construct represents arguments passed to the expect routine. Notice that our “spawn” parameter calls does an su to the username in order to change the password. By default, CGI scripts on your web server execute as some unprivileged user like “nobody” or “www”, so we need to change our effective user in order to change the password. Incidentally, you want to keep the default user for your Apache server as non-root. The alternative constitutes a potentially horrific security weakness.

There is one other new item in the script. Look at the send_user parameter. This is essentially a print statement. I left it in the sample script because I wanted to show you a clever way of debugging your expect scripts. Every programmer has inserted debug statements into his or her code to monitor how things were going. This is the same idea in this case. You can use send_user as a means of communicating with the outside world in the course of the script's execution. Since I capture the output of the expect script via my Perl script, I will see these messages as well. By the way, the Perl script calls the routine in this way (the entire command is on one line):

$return_code =3D `./psdcmd "$username" "$currpassword" "$newpassword" "$renewpassword" `;

As you can see, the expect script is called with the username, current password, the new password and the new password repeated. We could simply have passed the new password twice, but we wanted to keep the verification aspect of the password change routine to be as close to what the user would experience at the command line. More to the point, it also is a good idea to force the user to confirm the password before changing it.

Now that you have had your introduction to scripting with expect, I am going to make the process almost impossibly easy. Rather than manually creating an expect script, how about letting a program do that for you, too? When you install expect, you will also install a cool little program called autoexpect. Simply put, autoexpect will watch whatever you are doing in an interactive session and create the expect script for you. Here is the format of the command:

autoexpect -f script_outputfile command_string

For instance, let's imagine that we wanted to log in to a remote system that is behind a firewall, essentially a two-step login process. After we log in to the firewall, we then execute a login (telnet, ssh, etc.) to yet another system on the internal network, then execute a standard menu program. We would like to have this whole process of logging in twice and starting this menu automated for us. From the command prompt, we would then type this command:

autoexpect -f superlogin.script telnet firewall.mycompany.com

There is still one other thing you will want to add. At the end of your new expect script, add this command: interact

This tells expect to return control to you after it has done its work. Without it, expect closes the spawned process, and all you've managed to do is log in and log out very quickly.

In no way do I intend this to be the definitive reference on expect. I do, however, hope that this little introduction (indeed, this whole chapter) will serve to whet your appetite and inspire your imagination to explore other ways of developing constructive laziness. After all, we all have other work to do.

What's all this on your screen about a magic cloak?