COAS: A Flexible Approach to System Administration Tools

COAS stands for Caldera Open Administration System. It will be incorporated as the main configuration tool in future versions of the OpenLinux distribution.

For those who have never used OpenLinux, the tool we have been using for quite a while is called LISA (Linux Installation and System Administration), which is basically one huge shell script using a modified version of the dialog tool to interact with the user. When we felt it was time to move on to something new, we of course looked at what was already available. The only viable option at that time seemed to be LinuxConf, which had quite a ways to go before it would become useful. Since that time it has become much better, but because we had already started work on COAS, we decided to stick with it. Of course, we believe our concept is better.

The source code to COAS is released under the GNU General Public License. We feel our work might be useful to the Linux community as a whole and we want to invite interested programmers, administrators and users to participate in its development by offering comments contributing patches or even modules.

The main idea behind COAS is not to provide just another administration tool, but an entire framework for writing one. From the start, we wanted it to be a modular application where assumptions about such things as system data representation, file locations and dependencies are separated as much as possible from user dialogs and vice versa. Ambitious as this goal may appear, our main interest was the ability to easily adapt the tool to changes in the underlying platform and in porting it to other Linux platforms.

I like to call this vertical modularity, because it breaks up the task of system administration into three layers. At the lowest level are native system data files, such as /etc/passwd, /etc/hosts or files that define the IP address for a particular network interface.

On top of that, COAS implements an internal representation as a kind of database. If this term made you jump in your seat and shout, “Oh no, Mr. Bill, not a Linux Registry!”, please be assured that this is definitely not what we want it to be. COAS is supposed to be vi-administrator friendly. We want users to be able to switch between COAS and vi (or Emacs) administration, because even though we hope COAS will be useful for everyday tasks, it cannot cover each and every feature of a system component. (Consider the configuration monster incarnate, sendmail--you can spend as much time writing configuration software for it as Eric Allman keeps churning out new features.)

The native system files will remain the primary source of information. The COAS data model is strictly a run-time representation of system data that attempts to hide the on-disk representation from the upper layers. For instance, an administration module for the BIND server should not have to bother about where DNS zone files are located and how they are to be parsed; all it needs is the list of DNS zones this server is a primary or secondary name server for and the records they contain.

Having an abstract data representation also allows for alternate data access mechanisms. For example, our database engine can store a change log of an administration session to a file, which could then be distributed to other machines, thus allowing for bulk updates. Also, there's the vague idea that COAS might one day support remote access via LDAP or SNMP.

The top-most layer is the user interaction code. This code drives the dialog with the user and controls what information is displayed to the user at what time. It uses a standard set of dialogs, provides on-line help, etc. We decided to use a scripting language, Python, at this layer in order to allow for rapid prototyping. In addition to this, wrapping all lower-level functionality in Python classes and functions provides an additional level of insulation that restricts the number of tricks a programmer can pull. This may seem like a disadvantage to the hackers among you, but it is truly a big plus when it comes to code maintenance.

You may have guessed from my choice of the term “vertical modularity” that there is also a horizontal one, and so there is. Consider the following scenario: a security problem or other misfeature requires you to update a component of your system, such as the BIND name server. Alas, the update is from version 4.9 to version 8.2, which uses an entirely different configuration file format. We could now ask you to install an all-new version of our administration tool in order to accommodate the new configuration file format. On one hand, that is costly in terms of bandwidth. On the other hand, making sure the tool operates properly with all possible combinations of updates applied or not applied would be rather time-consuming for us. The ideal solution would be to package the DNS server administration module alongside our BIND update.

We are attempting to accomplish the following: COAS lets you rip out an entire module, including the data model definition, Python code, message catalogs and so on, and replace it with a different version. We have nicknamed these CLAMs, which is short for Caldera Linux Administration Module (we invented the acronym first and then decided on its meaning, in case you were wondering).

Let's take a closer look at the internal data representation. All information is stored in a tree, with each node having an individual name. For instance, the node containing the password of the root user is named system.accounts.users.0.password. If you're familiar with SNMP, think of the way SNMP variables are named.

Nodes can have different types; e.g., system is a directory, users is a list and password is a scalar. Scalar nodes can have various constraints attached to them; for instance, a string may be required to match a regular expression or contain only values from a predefined set of choices. You can also attach your own parsing and representation functions (written in C) to a scalar type, creating custom types that do such things as convert date strings, e.g., Jun 12 or tomorrow, to UNIX time.

All this information is provided by the so-called schema. The schema acts as a sort of blueprint for the data model in much the same way an SNMP MIB definition describes the types and organization of entities for SNMP.

For instance, the definition of the mouse parameters might look like this:

MODULE   "PERIPHERALS/MOUSE"
MSGCATALOG "peripherals/mouse"
TYPEDEF DevicePathName  STRING MATCHES
    "/dev/[a-z0-9]*"
TYPEDEF MouseProtocol  STRING IN CHOICE {
    "Busmouse", "MouseSystems",
    "Mouseman", "Microsoft",
    ...
}
device         RECORD {
    model      STRING
    protocol    MouseProtocol
    deviceFile   DevicePathName DEFAULT
        "/dev/mouse"
    emulate3btn   BOOLEAN DEFAULT "false"
}

This creates a record named mouse containing five scalar nodes. For instance, model is a plain string variable, while deviceFile is a special string type whose definition is shown above the record. The first two lines contain some syntactic sugar that need not concern us at the moment.

%Those funky strings (|":MOUSE_EMULATE_NONE:"|)
%are tags for the COAS message catalogs.

When accessing data items, COAS instantiates the portions of the instance tree from the schema definition and populates the data by invoking so-called “mappers”. These mappers are responsible for parsing and writing back system files, locking them if necessary. Usually, they are written in C++ and kept in shared libraries loaded on demand. The most recent release also supports mappers written in Python.

In the case of the mouse device, there is no standard location where this information is stored. On a Red Hat box, for example, it is kept in /etc/sysconfig/mouse, a file which contains a list of shell variable assignments. COAS already has a general-purpose mapper for this type of file (it turns out that about 80% of all system files are quite close to four or five standard formats), so all that is left is defining the mapping. This is done by the so-called platform repository, where we might enter code like this:

peripherals.mouse.device {
    mapper     builtin.sysconfig
    path      /etc/sysconfig/mouse
    relation    MOUSETYPE:model:\
            PROTO:protocol:\
            DEVICE:deviceFile:\
            XEMU3:emulate3btn(map=/no=false,yes=true/)
}

The mapper keyword associates the mapper specified with the mouse device node. When accessing the device node, the first time, COAS detects this and invokes the mapper in order to populate the tree below the mouse device node. The mapper retrieves the path parameter and reads the file specified. The relation parameter tells the mapper which shell variables within the file correspond to which data model nodes.

The same thing happens when you have modified a protocol (e.g., the mouse) and invoke the device node's commit function. The data engine will invoke the mapper in order to write the data back to the file. Again, the mapper will use the specified relation to determine which data model values will be assigned to which shell variables. Note that in an act of vi-administrator friendliness, the mapper does not touch shell variables it does not know about and tries to preserve comments as well as it can.

The platform information is usually installed by merging it with the main COAS platform definition, which resides in /usr/lib/coas/repository.

Having written and installed the above files, you can already display and modify the mouse configuration using COAS. For example, COAS comes with small utilities such as coas dump and coas change that let you dump portions of the data tree or modify individual nodes. You can even write Python scripts that perform more complex operations on your data.

However, the ultimate goal (at least for us) is a Python module that interacts with the user, guiding him through the administration task. The module sits on top of the database engine and operates exclusively on the abstract data representation. It displays data to the user, selects which items are to be edited, provides on-line help, etc.

Why Python? Well, a very early prototype used Tcl as the scripting language, but for various reasons it didn't work too well. In contrast to Tcl, Python has fairly good object support and at least as good an extension mechanism. The other candidate was Perl, but we decided against it because it is so easy to write horrible code in Perl.

Communication with the user happens via an abstract user interface API written in C++, which currently supports a curses and a Qt front-end. Work on an extended Qt front-end that takes advantage of features provided by KDE is in progress. Of course, in order to be able to use this API from Python, a Python wrapper is provided.

The user interface provides a limited but useful set of dialogs: notice/question dialogs consisting of a text and a few buttons; list dialogs (single- and multi-selection, with or without check boxes, etc); prompt dialogs (containing edit fields for one or more scalar values); and table dialogs (which display data in a table, allowing in-place editing).

Listing 1

For instance, a minimal module for editing the mouse configuration would look like Listing 1 (some of the Python fluff, such as import statements, is not shown). For those not familiar with Python, this code defines the class Mouse, derived from the CLAM class defined in module clam. The {__init__} method is Python's way of declaring a constructor.

The method run is invoked by COAS. The first thing it does is look up the data model node for the mouse device. As described above, this step will trigger the parsing of the configuration file into the internal data representation.

Next, a prompt dialog is created and three edit fields are added for the mouse's model, protocol and device file. The last few lines are the somewhat standard dialog loop. Depending on whether the user terminates the dialog by pressing the Okay or Cancel button, either the commit or cancel method (inherited from the CLAM base class) is invoked, which displays a small question dialog along the lines of “Do you really want to save/cancel?”

The first thing that will probably strike you as odd about this example is that it has no label strings anywhere. Nevertheless, the dialog is supposed to have a title, edit fields are supposed to have a label to their left, etc.

The answer is that COAS generates NLS strings for you out of the information it has. For instance, when creating the prompt dialog, we inconspicuously passed the string mouse into the function. As a consequence, COAS will create tags such as |":MOUSE_TITLE:"| for the dialog's title and attempt to look it up in the module's message catalog. (The message catalog name was specified in the class constructor.) Likewise, for the protocol edit field, it will generate the tag |":MOUSE_PROTOCOL_LABEL:"|. All you need to do is write the message catalog, mapping these funky strings to intelligible English (or French, German, etc.) and install the file.

Looking at the sample code above, you may also have thought: I understand where they put the data in the dialog, but how do they put it back into the data model?

This is the interesting part about the data editing process. If you have ever programmed Motif, you know how tedious it can be to extract the value to be edited from the data model, put it into the dialog and write the resulting value back to the data model when the user hits the OK button.

The approach taken by COAS is to tie data model nodes into the dialog directly and let the dialog select an appropriate widget type (string, combo box, toggle button, spin button, etc.). When the user provides a new value, the dialog will automatically check the value's syntax against data model constraints and write it back into the data model.

In our example, the dialog would create a simple string edit field for deviceName, a pop-up list for protocol (since it is limited to a set of choices) and a toggle button for emulation.

What's more, this mechanism offers you easy-to-use context help for each input field, bound to the f2 key. Adding this type of help to a data item is as easy as adding the HELP attribute to the data definition in the schema file:

device         RECORD {
    model      STRING HELP "HELP_MODEL"
    protocol    MouseProtocol HELP "HELP_PROTOCOL"
    ...
}

These help messages will be looked up in the message catalog associated with the schema file (remember the MSGCATALOG keyword in the schema file?) and displayed in a pop-up dialog whenever the user presses f2.

Of course, every scheme you devise has a drawback. In this case, it is how to cancel changes made during the execution of the dialog. When the user presses the Cancel button, he wants all changes to go away.

This is where the marker object comes into play. The data node's getMarker method obtains a marker for the node's change log (called a journal in COAS lingo). When the user requests a discard of all changes, the CLAM base class invokes self.mouse.cancel(marker), which reverts all changes made after the marker object was obtained.

I have to admit that the above example, in its simplicity, is a bit deceptive. What I'm showing here is the simplest version of a dialog. In fact, what you see here is just a glorious interface to the configuration file because it does not offer the user any help or guidance. A good dialog would automatically choose the appropriate device file when a selection is made (e.g., a bus mouse) and keep the user from enabling three-button emulation for mice that already have three buttons. As a consequence, your average COAS module will have a lot more than those 20-odd lines in the example above.

However, the greatest advantage COAS offers in this context is that it relieves you of the usual hassle when working with a GUI and lets you concentrate on the data flow instead.

Dependency Model

If this article has piqued your interest and you would like to take a closer look at COAS, you can find out more about it on http://www.coas.org/ and http://developer.coas.org/. If you want to participate in the development of COAS, don't hesitate to contact me.

All listings referred to in this article are available by anonymous download in the file ftp.linuxjournal.com/pub/lj/listings/issue58/3019.tgz.