What's GNU

Last month we described the origins of Plan 9, the sam editor, and the 9term terminal emulator. Well, what about the shell to run inside the window? Here too, the Plan 9 authors took the opportunity to rethink the issue of just how should a shell work. The Plan 9 shell is called rc, because it “runs commands”.

Although in many ways the Bourne shell is a simple, elegant, high-level programming language, it has a serious flaw, in that it was designed to be much like a macro-processing language. Input text is scanned, rescanned, and rescanned again, as each stage of processing is performed. (This is carried to an almost absurd length in the Korn shell, with something like eleven different processing stages.) This leads to rather complicated and baroque quoting rules, with the need for nested escape sequences.

In rc, the input text is scanned and parsed exactly once. The language has a real yacc-based grammar, making it clear what everything means. The quoting rules are very simple. Quoted text must be enclosed between single quotes. To get a single quote inside a quoted string, double it (as in FORTRAN). An explicit operator is used to provide string concatenation, and variables can be lists of strings, not just single strings.

The syntax is closer to that of C or awk, instead of Bourne's Algol 68. This leads to less clutter, avoiding unnecessary keywords and semi-colons. It is much more like C than the fabled csh is.

rc provides shell functions, and signal handlers are written as functions with special names (sighup, sigterm, etc.), instead of using strings. I/O redirection is also more powerful, with a notation for hooking up file descriptors besides 0 and 1 to the input and output ends of a pipe.

A freely distributable clone of rc is available. It was written by Byron Rakitzis, and implements the language described in the rc paper, with some extensions. The beauty of rc is that it is small and fast, and shell programs can be quite elegant. It also runs on just about any kind of Unix system.

When using rc with 9term, it is conventional to set the primary prompt to be just a single semi-colon, and the secondary prompt to be empty. This allows you to snarf entire commands, including the prompt, and resend them. The semi-colon is treated as a simple null statement. The use of double-clicking to select the whole line, and the default saved action of the menus make sending and resending the same line over again extremely simple; most of the work can be done with just the mouse.

The Resources sidebar lists the ftp location of the rc shell. There is also a mailing list of people who use rc.

es is the “extensible shell”. Paul Haahr and Byron Rakitzis thought it would be interesting to try and combine some of the capabilities of functional languages with those of Unix shells. Many internal capabilities of the shell (such as I/O redirection and setting up pipelines) are available as built-in functions in the language, and program fragments can be passed around as arguments to functions.

es provides first class functions, lexical scope, an exception system, and rich return values (i.e. functions can return values other than just numbers). Most of this is beyond the scope of this article to explain. es is described in a paper in the Winter 1993 Usenix Conference Proceedings. It helps to read this paper, and also to go through the archives of the mailing list to see how the language evolved. For the full details on es, you'll need to read the paper, the man page, and the file initial.es in the es distribution. It is a good idea to also look at the sample .esrc file, too.

Basically, the idea behind es is to take the primitive operations that a shell does, such as forking processes, creating pipes, and setting up I/O redirections, and make them available as functions that a user program can call directly. In turn, traditional shell syntax is built on top of these primitive operations.

Lexical scoping allows you to save the definition of an operation, and then replace it with your own operation on top of the previous one. Here is an example from the paper on es. This code implements a pipeline profiler. It saves the definition of %pipe, which creates pipes, and provides a new one that times each component of the pipeline, using the old %pipe to actually create the pipeline. (es is the prompt from es used for examples in the paper. The default prompt is a semi-colon.)

es > let (pipe = $fn-%pipe) {
        {
                fn %pipe first out in rest {
                {
                        if (~ $#out 0) {
                                time first
                        } {
                                $pipe { time $first } $out
$in { %pipe $rest }
                        }
                }
es> cat paper9  tr -cs a-zA-Z0-9 '\012' | sort |
 uniq -c | sort | -nr  sed 6q
213 the
150 a
120 to
115 of
109 is
 96 and
 2r    0.3u   0.2s      cat paper9
 2r    0.3u   0.2s      tr -cs a-zA-Z0-9 \012
 2r    0.5u   0.2s      sort
 2r    0.4u   0.2s      uniq -c
 3r    0.2u   0.1s      sed 6q
 3r    0.6u   0.2s      sort -nr

This is a simple example, yet it illustrates some of the power available in es. es really deserves a column on its own. For more information, see the above sources and the mailing list archive.

The sidebar lists the ftp location for es, and a mailing list is also available.

The tools we've seen so far, notably sam and 9term, are built on top of X Windows, and work with any window manager. For some time, I ran them using mwm.

In the fall of 1993, I obtained a version of gwm, the Generic Window Manager, with WOOL (Windows Object Oriented Lisp) code that implemented an interface very similar to that of the original Bell Labs Blit terminal. This provides a simple, clean interface, similar to that used on Plan 9 (8½ can be considered a further evolutionary step from the Blit). This code was written by John Mackin at the University of Sydney. The resources sidebar shows where you can get this code, if you're interested. This code works, but it is large and slow.

However, a new window manager recently became available, 9wm. 9wm implements the window management policies of 8½, under X windows. Written by David Hogan at the University of Sydney, it uses raw Xlib (not a pretty sight), and is completely ICCCM compliant. 9wm is also small, and very fast. To quote from the README file:

9wm is “click to type”. This means you have to move the mouse into a particular window and then click button one. That window becomes the current window, indicated by a thick black border. Other windows have a thin black border. This behavior is identical to sam's.

The 9wm menu (accessed through button 3 on the root window) consists of five items:

What is perhaps most noticable about 9wm (and 8½) is that there are no icons. Instead, to remove a window from the screen, you select Hide from the main menu. The cursor becomes a target. You move the target to the window to be hidden, and then click button 3. Clicking any other button cancels the operation.

When a window is hidden, it disappears from the screen completely, not even leaving an icon. Instead, a new item appears at the bottom of the button 3 9wm menu, with the name of the window. To open the window again, you simply select the window's name from the menu.

As with the other programs, the 9wm menu remembers what you did last time, so that the next time you pop up the menu, the previous selection is already highlighted

And now, my own small contribution to the picture. The GWM Blit emulation, which I used for quite awhile, understood that it was built on top of X, and when you selected New, it gave you a menu of hosts (that you defined in a configuration file) on which to start remote xterms. This was nice, and I found it missing under 9wm.

(In Plan 9, this is not an issue; the multiple hosts in the network are very tightly integrated, but in X with Unix, it is a problem.)

What I wanted was a simple program to which you could give menu items and associated commands, and this program would pop up a window that was nothing but a menu. Selecting an item would run a command. The program would be long lived, leaving the menu up permanently. A program close to this exists, xmenu. Unfortunately, xmenu goes away after executing the command, and is not well behaved when interacting with 9wm.

Inspired by 9wm, starting with its menu code, and with help from David Hogan, I wrote 9menu. 9menu pops up a window containing the list of items, and executes the corresponding command when a button is pressed.

9menu allows you to write your own menus and customize the behavior to suit you, without the headaches of a .twmrc or .mwmrc file. It is real easy to have one item spawn another 9menu, giving a similar effect to pull-right menus.

Here are two I use it: one for remote systems, the other for programs I may want to run. Being lazy, I have xterm in both. I use a shell script named rxterm that knows about the remote hosts I will want to open windows on, and whether they can start a 9term or an xterm. (This is left over from the GWM Blit code, and is mostly for convenience.) These examples are from my .xinitrc. The -geometry strings are to get 9wm to place the windows even though they start out iconified.

9menu -geometry 67x136-4+477 -iconic -popdown -label Remotes \
        'solaria:rxterm solaria.cc.gatech.edu' \
        'burdell:rxterm burdell.cc.gatech.edu' \
        'chrome:rxterm chrome.cc.gatech.edu' \
        'xterm:rxterm xterm' \
        exit &
9menu -geometry 103x102-3+624 -iconic -popdown -label 'X programs' \
        'xterm:rxterm xterm' \
        xtetris xlock '9wm restart' '9wm exit' exit &

I start the programs using -iconic so that they'll be automatically hidden and part of the 9wm menu. The -popdown option causes the menu to automatically iconify itself after an item is selected, since I find this to be the most convenient way for me to work: pop up the menu, select an item, and then go on with what I want to do without the menu hanging around. Although not nearly as large scale a program as sam, 9term, or 9wm, I find that 9menu completes the package for me.

I have been using this environment for almost two years, and find it to be clean, elegant, and easy to use. Initially, I started by using rc, and then sam when it became available in early 1993. Shortly after that, I started beta-testing 9term, in particular getting it to work correctly under SunOS. In the fall of 1993, the GWM Blit code became available, and I switched to that, using it for almost a year. In the spring of 1994, I started beta-testing 9wm, which was finally released at the end of 1994. I switched to es in January of 1993 after reading about it and hearing the presentation at the winter Usenix.

The research group at Bell Labs is well known for applying the “small is beautiful” principle to software design. This was initially true of Unix, and has been re-applied to distributed systems, shells, and user interfaces with Plan 9.

The interface is simple, consistent, easy to use, and very clean. All the programs described in this column behave the same way, in terms of what the buttons do, which window is current, and how the menus remember the previous operation.

An important point that I have not emphasized so far, is that all the programs use pop-up menus. I find this to be an enormous convenience, particularly compared to systems like Windows or the Macintosh, where you must move the mouse to the menu bar to pull down a particular menu. Pop-up menus save an incredible amount of otherwise useless mouse motion, leading to a system that is much easier to use.

My first exposure to window systems was long ago, on a Blit terminal. The interface was simple, clean and elegant. Ever since then, I had been searching for an X windows environment that matched the Blit's elegance. Now, with the combination of sam, 9term, 9wm and rc or es, I feel that I have finally found that environment, and I'm very happy. What's even nicer is that all of these programs are fast, and I have the broad range of X applications available to me also (xoj, anyone?). This latter point is unfortunately not true of the only other alternative, mgr (which I used until 9term became available.

All the programs described here can be made to compile under Linux. I don't have a Linux system of my own (believe or not!), but for a while I borrowed one, and was able to bring up all of these programs. Unfortunately, the system was a laptop, with too small a screen to make using X worthwhile. sam comes up fine, using the Make.solaris makefile as a starting point. 9wm also compiled just fine. 9term took a little bit of work, but it did compile and run. After asking on the mailing lists, I learned that 9term does not (yet) work quite correctly under Linux. This may be fixed by the time you read this column, though. Two people to contact for information about porting 9term to Linux are Pete Fenelon (pete@minster.york.ac.uk), and Markus Friedl (msfriedl@faui01.informatik.uni-erlangen.de). rc and es, both compile and run under Linux, but with some work. For rc, you have to generate the sigmsgs.c file by hand, based on /usr/src/linux/include/sys/signal.h. There is one other bug, reported by Jeremy Fitzhardinge, which is that rc uses ints for the array of additional groups, while Linux uses gid_ts, which are shorts. es requires similar changes for the signal handling, but these are actually documented in the Makefile.

The combination of 9term and 9wm provides a very close emulation of the elegant Plan 9 user interface. sam is a powerful, easy to use editor. rc is a simple, clean shell, and es is a nifty shell with lots of promise. It is worth reading the papers describing each of these components. The complete combination proves once again that “small is beautiful.”

Thanks to Chris Siebenmann and Daniel Ehrlich, maintainers of the various mailing lists, for their help, as well as to the members of the lists who responded to my questions about Linux. Thanks to Bob Flandrena, Paul Haahr, and Miriam Robbins for their comments.

These papers are all available in Postscript as part of the Plan 9 documentation.

This paper is available for ftp along with the es source code.