Testing Applications with Xnee

Xnee can record user actions during a session and then replay those actions. By recording sessions when testing a program, Xnee automatically can test the program for you later. These test sessions can be replayed before every release, or every night, to ensure the quality of your program. Is it as easy as it sounds? Almost.

Xnee doesn't test only GUIs. You also can use Xnee to test command-line programs by making a few test scripts that test all the options of a command-line program and analyze the results. Xnee also has been used to test how much traffic is being sent over a large network with numerous thin clients. Support for distributing events to multiple displays has been added if you want to test the same cases on multiple machines at the same time. Besides testing programs, Xnee also is used to demonstrate programs. In this case, Xnee acts as a patient demonstrator, doing the same job over and over again without complaining.

In 1997, Henric Johansson and I wrote our master's thesis on recording and replaying X events. We implemented a nonfree recorder and replayer for a Swedish company for its internal purposes. After finding a job, I often lacked a free test program for X11, so I decided to implement one on my own, using the experience gained from the thesis. The Xnee Project started in the summer of 1999 and was licensed under GPL from the start. In November 2002, version 1.0 was released, and by the end of February 2003, Xnee was dubbed a GNU package.

Before we go on with Xnee, this short introduction to X explains a lot of the terms used in this article. X is a window-based user interface system for various platforms. The X server is a program that handles all the hardware and actually does the drawing on the screen. On GNU/Linux systems, XFree86 is the most-used X server. X programs are known as clients; examples are xterm and Galeon. The clients communicate with an X server using the X protocol.

In this article we concentrate on the packets used to send information between the X server and its clients. These packets are called Event, Request, Reply and Error and are referred to here as protocol data. The following list shows the X11 protocol data description:

The most interesting thing here is every time the user interacts with the computer using the mouse or the keyboard the X server sends the appropriate client one or more events. Some of these events result directly from user input. These events are referred to as device events. The device events are ButtonPress, ButtonRelease, MotionNotify, KeyPress and KeyRelease. If we could record all of these events during a session, we would have a complete description of all the actions the user performed. If we had a robot that could read these events if they were printed to a file or on paper, the robot then could interact with the system as the user did when recording, and we would have ourselves a test robot. Or, even better, if we had support for faking those events, we would have a test robot made of software. Fortunately, support exists for both recording and replaying in X.

To record X protocol data we can use the extensions RECORD or XTrap. There are other ways to accomplish recording, such as sniffing the X socket, but we'll focus on RECORD as it's what Xnee uses. To replay events, we can use both the XTest extension and the RECORD extension. During replay, the RECORD extension is used to synchronize what's happening when replaying with what happened when the session was recorded.

The RECORD extension sends copies of the data sent between the clients and the server to the client requesting it. Using the RECORD extension, Xnee can record all protocol data the user wants and save it to a file to replay later.

The XTest extension can reproduce or fake all device events. This extension lets Xnee fake user actions, such as moving the pointer, pressing and releasing a key or pressing and releasing a button. No other data can be replayed.

Xnee is a GNU package, and the sources can be found at the Xnee home page. Download the latest source; as of this writing, the latest version is 1.0.6. Extract the package, and then configure, build and install it:

tar zxvf xnee-1.0.6.tar.gz
cd xnee-1.0.6
./configure
make
make install

RPM packages are available at the home page, and Xnee also is available in the FreeBSD ports tree. Xnee comes with a user manual and a developer manual in various formats. The TeX sources to the manuals are distributed with Xnee and covered under GNU FDL. Instead of building the documents yourself, you can download the format of your choice (PDF, HTML, INFO and TXT) from the Xnee home page. As of this writing, the Xnee documentation version is 1.0.4. Extract the documents:

tar zxvf xnee-doc-1.0.4.tar.gz

When running Xnee, make sure the RECORD extension is enabled. On XFree86 make sure the RECORD module is loaded. Open the XFree86 configuration file, typically /etc/X11/XF86Config-4, and look in the Module section. The following line should be included:

Load  "record"

See the Xnee FAQ for more information about this.

We don't go into any details about Xnee in this section, but rather begin slowly with a simple example. Start Xnee with the --all-events option. This sets up the recording of a few events. The option is not useful when doing serious Xneeing. It is intended to simplify your introduction to using Xnee:

xnee --all-events

When moving the mouse or pressing the buttons or keys, Xnee prints information about the action. We move on to record a simple session that we replay immediately. To record 20 mouse motions, start Xnee like this:

xnee --record --out session1.xnr \
--device-event-range MotionNotify --loops 20

The options on the command line mean use recording mode (--record), save output in a file called session1.xnr (--out session1.xnr), record the device-event MotionNotify (--device-event-range MotionNotify) and record 20 events (--loops 20).

To replay this event, start Xnee like this:

xnee --replay --file session1.xnr

The options on the command line mean use replay mode (--replay), and read data to replay from file session1.xnr (--file session1.xnr).

Xnee uses ranges to explain what is to be recorded. Ranges have a start value and a stop value. The following data can be recorded: core-requests, device-event, delivered-event, error, reply, extension requests and extension replies. We don't describe the above data in this article. If you want to read more, see the RECORD extension documents. When, for example, you want to record the device event MotionNotify, use:

--device-event-range MotionNotify

To record the events from KeyPress up to MotionNotify and CreateNotify, use:

--device-event-range KeyPress-MotionNotify,\
CreateNotify

You can use the number corresponding to the event name instead of the name itself if you want shorter command lines. To find the number for the data you want to record, use Xnee's --print-data-name option:

xnee --print-data-name

You can stop recording by setting the number of the data to record (--loops option), or you can interrupt Xnee by sending a TERM signal (pressing Ctrl-C in the terminal window where you started Xnee). Alternatively, you can dedicate a modifier and key combination that won't be used to do anything else during recording. Setting the modifier and key is done with the --stop-key option. To set up Xnee so it stops recording when Ctrl-Alt-A are pressed, add the following to the command-line option:

--stop-key Control+Alt,a

But why even bother to record data other than device events when you can't replay it? Xnee uses that other data to synchronize, which is where things get complicated. Think of recording a session when using Galeon or any other Web browser. When recording, everything goes well and the network is up and running. But when replaying the Galeon session, you can't reach the Internet. If not for synchronization, Xnee might replay user events such as clicking on a link on the Web page. If Galeon could not load the page, it is not useful to continue the replay until the network is up and the page can be loaded.

When recording other data, we can use it to synchronize the session. For example, if we record the data sent when displaying the Web page in the Galeon window, we can wait for the same data to be sent when replaying. This ensures that the Web page is loaded before we go on and replay the coming events. In this example, we skip a lot of the X protocol data sent when recording in order to keep it simple (see Table 1). When replaying this simple session, Xnee uses the same events (see Table 2).

Although synchronization is needed, finding the right data to use for synchronization may be difficult. Xnee solves this with plugin files that specify what should be recorded for a range of applications. These plugins are named after the applications they are intended to test. If you want to test a browser you've written, it would be a good idea to use the Galeon plugin. Sometimes, though, no plugins are suitable for your program, and you need to find the right protocol data to synchronize. The following example hopefully makes it easier for you in the future. We chose gnumeric as a program for which the right options need to be found. First, launch gnumeric. Then start Xnee in a terminal emulator with the following options:

xnee  --delivered-event-range \
EnterNotify-MappingNotify --human-printout \
--loops 1000

This generates a lot of useless events that fill the screen, so stop Xnee. Filter out those useless events by excluding them when setting ranges:

xnee --delivered-event-range \
EnterNotify-KeymapNotify,VisibilityNotify- \
MappingNotify --human-printout  --loops -1

This looks better. Now, start recording a session with Xnee with the following options:

@cx:xnee  --delivered-event-range \
EnterNotify-KeymapNotify, \
VisibilityNotify-CirculateRequest, \
SelectionClear-MappingNotify --loops \
1000 --out session1.xnr

Type some stuff into the gnumeric spreadsheet and use the menus to insert today's date or other input. When you're done, go to the terminal and press Ctrl-C to stop recording. It is now time to replay your session. Set gnumeric in the same state it was in when you recorded. Launch Xnee in replay mode like this:

xnee  --replay -f session1.xnr

Xnee sometimes pauses when replaying the session. This happens if the protocol data isn't sent in the same order as it was recorded. Xnee pauses execution for a while in order to wait for the expected data (as read from file) to be sent by the server. Eventually, a timeout expires and Xnee tries to continue. If Xnee can't synchronize between the recorded data traffic and the data traffic as sent when replaying, it bails out.

Xnee supports giving record options through plugins. When you have found the settings for your applications, save them in a plugin file. The syntax of a plugin file is similar to the command-line options. The easiest way to crate a new plugin is to copy an old one, fill in your settings and then rename it to some appropriate name. Xnee is distributed with plugins for different clients. If you want to send a plugin file for your application to Xnee, please do. The Xnee home page has instructions for how to contribute.

If you have a program that creates windows for user feedback, you have to make sure these windows pop up at the same location. Xnee records all device events with coordinates referring to the root window, not the window that was created.

To ease recording, make scripts that start Xnee with the right settings for a specific purpose. You can add a launcher to your panel or add a menu item to your window manager menu.

Xnee has seen a lot of activity lately, mainly due to feedback from Xnee users. We hope you consider Xnee for your project. Happy testing and happy hacking.