Aap is a flexible tool that can do what make does and much more. Learn how to write portable recipes for maintaining your Web sites and building programs.
Many people use a Makefile and shell scripts to automate tasks where a change to a file requires an action to be taken. You edit a file and then invoke make in the hope that all actions necessary to effect the changes are done. Often you need to tweak the Makefile to get it right and end up using touch to work around a missing dependency. As a result, when other people look at your carefully tuned Makefile, they have a hard time understanding how it works.
These tasks can be done more easily and reliably with Aap than with make. For example, Aap has built-in Internet support. Downloading and uploading is taken care of without the need to specify the commands or to keep timestamp files. Reliability is achieved by figuring out dependencies automatically and using signatures instead of timestamps. With Aap it is simpler to specify the work you want done, and you make fewer mistakes. You still can fall back on using shell commands where you want them. This article presents two examples of Aap in action, maintaining a Web site and building a program. Aap can do much more, of course, but these subjects should be sufficient to get you started.
To use Aap you need Python version 1.5 or later. In the unlikely event that you do not have Python on your system, download it from Python.org, or install it from your Linux distribution CD or update system.
Installing Aap can be done in four simple steps. Start by downloading the latest Aap zip archive (see the on-line Resources section). Then, unpack the archive in a temporary directory (unzip aap-1.053.zip). If you are root, run ./aap install. If you are a normal user, install the archive in your home directory with ./aap install PREFIX=$HOME. For more information about downloading and installing Aap, see Resources. Aap is distributed as open-source software under the GNU GPL.
With the current Aap software in place, let's look at using it in a basic example task. You have designed a simple Web site with HTML files and images. The files are on your local computer, and you need to upload them to the Web server. Listing 1 shows the Aap script, called a recipe, that does this.
Store the recipe under the name main.aap. This file is what Makefile is to make—the default file to be executed. Running aap without arguments executes the main.aap recipe in the current directory.
Comments in an Aap recipe start with # and continue until the end of the line, as in a Makefile or shell script. The first effective line in the recipe is an assignment; the list of files to be uploaded is assigned to the Files variable. There are no back slashes nor punctuation to mark the end of the assignment. Aap recognizes command continuation by the amount of indentation used. This may appear strange at first, but you quickly get used to it. This method avoids the usual mistakes with punctuation and enforces a layout that is easy to read. Aap allows you to use either tabs or spaces to indent lines.
The :attr line is an Aap command. All Aap commands start with a colon to make them easy to recognize. This command adds the publish attribute to its arguments. The publish attribute tells Aap where to upload the files when they are published. The method used here is scp://, secure copy. Other supported methods are rsync:// and ftp://. The last argument of :attr is $Files, the value of the Files variable. The attribute is attached to each item in $Files.
When you run Aap without an argument, it updates the target all. The final line specifies that the default target all depends on publish. This is a special target, which tells Aap to upload all items that have a publish attribute.
You now can edit the HTML files, add pictures and view them locally. Once you are satisfied, execute aap. Aap figures out which files have changed and uploads them. Signatures (checksums) are used; thus, if you restore an old version of a file it still works properly. If you were using make, you would have to touch the restored file to update its timestamp.
If you want to try this example, but you don't have a server to upload to, you can use the publish attribute file:/tmp/html/%file%. With it, Aap creates the /tmp/html directory, if necessary.
A word of warning: Aap does not delete files on the server no longer in use. Then again, neither does make. You have to delete these files manually. Hopefully, automatic deletion will be added to Aap soon.
A wild card was used to select the images: images/*.png. This is convenient, but it has the danger of including images you do not want uploaded. Explicitly naming each file avoids this trap, but then you might forget an image. Being that this is a common issue, Aap provides a function to extract the image filenames from the HTML files. Listing 2 shows how this is done; the Python function get_html_images is invoked, and the back ticks enclose a Python expression. Aap evaluates the expression and puts the result, the image filenames, in its place. The get_html_images() function has limited capabilities, however. It works only for plain HTML files with images that have a relative pathname.
Most HTML files consist of a header, title, main contents and footer. Obviously, you don't want to type the common parts each time. A simple solution is to concatenate a number of files. Listing 3 shows the recipe that implements this. Five parts are used: header, title, middle, contents and footer. The title and contents are different for each page, but the other three parts are the same.
The main difference between Listing 2 and Listing 3 is the added :rule command in Listing 3. It specifies that a target (the HTML file) depends on five source files (the parts) and lists the command to build the target from the sources. The % character is used instead of the name of the file, similar to a * wild card. All % characters in the rule stand for the same name. Thus, for index.html the % stands for index. The sources then include index_title.part and index.part.
Below the :rule line comes the indented block of statements that are executed when the target of the rule needs to be updated. So, the recipe has two levels: the commands at the top level are executed when reading the recipe, and the command block of the rule is executed later, when needed.
The :cat command concatenates files, the same as the UNIX cat command. It actually can do much more, such as read files from a specified URL. In a rule, $source stands for the whole list of source files.
The HTML files need to be generated before obtaining the list of image files they contain. To get this right, the :update command is invoked before calling get_html_images(). The HTML files are updated using the defined rule. This is at the top level of the recipe, so it always is done when Aap reads the recipe.
Now that you have so many files, how does Aap keep track of what needs to be done? Aap works with dependencies, the same as make does. It starts with the target you specify on the command line. When no target is given, all is assumed. Aap then locates those dependencies and rules in which this target appears before the colon. The colon basically means depends on; after the colon are the source files on which the target depends. Each of these source files then is inspected, and Aap finds rules where they appear as a target. This continues recursively until no more rules are found. The result is a tree of dependencies. Aap then executes commands for those dependencies that need to be built, starting at the end of the tree (depth first). This sounds complicated, doesn't it? Because Aap takes care of this, you only need to make sure you specify the sources on which each target depends. Aap figures out what needs to be done.
As a nice addition, let's add a timestamp to the HTML file, so you can see on the Web site when the page was last generated. Put the string @TIMESTAMP@ somewhere in the file footer.part. Listing 4 shows the rule in which this string is replaced with the current date. The rest of the recipe is as shown in Listing 3. The :eval command evaluates a Python expression, and string.replace is a standard Python function for replacing one string with another. This way, you can use any Python expression to filter text. The HTML page is piped through the :eval command, as with a shell.
The first time the new rule is used, all HTML files are updated. That is because Aap remembers a signature for the commands. Thus, you don't need to worry about forcefully generating the files after changing the commands in the recipe.
When making small changes to a Web page, it is a waste of bandwidth to upload the whole file each time. A good way to upload efficiently is to use rsync. It uploads only those parts of a file that have been changed. Aap uses rsync when it finds rsync:// in the publish attribute. By default, rsync is used over an SSH connection. You can change this by setting the $RSYNC variable.
rsync is not a standard command. If it does not exist on the system, you encounter a nice feature of Aap—you are offered the choice to install rsync:
% aap
Aap: Uploading ['index.html'] to
rsync://my.server.net/html/index.html
Cannot find package "rsync"!
1. Let Aap attempt installing the package
2. Retry (install it yourself first)
q. Quit
Choice:
Aap has a mechanism to install a package when it is needed by downloading a recipe from the Aap Web site that specifies how the package is to be installed. The downloading features of Aap come in handy here. How the package is installed depends on your system; not all systems are supported yet. After rsync has been installed, Aap starts uploading the files.
Aap includes support for building a program from C and C++ code. Here is the one-line recipe that builds the program called myprog from four C source files:
:program myprog : main.c common.c various.c args.c
Despite the simplicity of the recipe, Aap takes care of several issues:
Dependencies are figured out automatically. You don't need to specify the included header files or do a make depend.
This recipe works on most systems without modification. Aap finds a compiler and linker to use and figures out the arguments they need.
The object files are stored in a separate build directory for each system. You can build several versions without cleaning up.
Aap creates a log file, AAPDIR/log, that contains details about what happened. If your build fails and the output scrolls off the screen, you don't need to repeat the build command with the output redirected.
A few default targets are added automatically: aap install installs the program, and aap clean deletes generated files.
It would be possible to do the same work with make, with the help of a few extra tools. But the Makefile would be much longer and not portable; it also would require more effort to maintain.
Now let's build a program in two variants, a release and a debug version. Aap includes support for variants. All you need to do is specify what variants you want to build and what is different between them. Listing 5 shows the recipe.
The first line of the :variant command specifies the variable name used to select the variant to be built. You can set this variable on the command line; aap Build=debug builds the debug version. Without an argument, the release variant is built, because it is mentioned first.
The amount of indentation identifies the other parts of the :variant command. The possible values have less indentation; the commands used for each value have a bit more. You are forced to align the parts, which makes them easier to read.
The release variant sets the OPTIMIZE variable. This is a number in the range of zero to nine that indicates the amount of optimizing to be done. It automatically is turned into the right argument for the compiler being used. The debug variant sets DEBUG to yes. The default value is no. The Target variable holds the name of the resulting program. The two variants use a different name, so both programs can exist.
A nice advantage of using variants this way is that object files for each variant are stored automatically in a separate build directory. When switching between the two variants you should notice that Aap does not rebuild all the files.
For languages other than C and C++ you need to import a language module. A few standard modules are included with Aap. For example, this is how to build from D sources; D is a new programming language:
:import d :program myprog : main.d common.d various.d args.d
The :import d command is used to load the support for the D language. Otherwise this process is similar to building from C sources.
You can write a module yourself to add support for a language. Because Aap is open source, you are encouraged to submit the module to be included in the Aap distribution. Until that happens, drop the file in the Aap modules directory; this works as a plugin.
Building a KDE application involves working with a lot of tools, including using Qt Designer to create dialogs, generating header files from user-interface descriptions and generating interprocess communication code. Nevertheless, a recipe for building a KDE application can be as simple as this:
:import kde
:program logger : main.cpp
logwidget.ui
dcop.h {filetype = skel}
{var_OBJSUF = _skel.o}
Of the three input files, main.cpp can be compiled directly. The Qt Designer file logwidget.ui first needs to be processed by uic to generate an include file; then moc must be used. Aap recognizes the .ui suffix and takes care of all of this. Handling this kind of multistep compilation, from ui to h to moc to object file, is a useful feature in Aap. Doing the same thing in a Makefile requires far more explicit rules.
The dcop.h file contains special KDE items but has a normal suffix. It cannot be recognized automatically. Therefore the filetype attribute is specified explicitly. The :program command also needs to know the name of the object file, which is specified with the var_OBJSUF attribute. You do not need to specify explicitly the KDE tools being used; the complexity is hidden in the KDE module. This is considerably less complex than using automake.
So far, you have used high-level Aap commands to specify quickly what needs to be done. For nonstandard tasks, you need to spell out the dependencies and commands. This mostly works like a Makefile. Besides shell commands, you can use portable Aap commands. If that is not enough, you can add a Python script.
Listing 6 shows what a low-level recipe looks like. Every dependency is given explicitly here—all depends on hello, hello is compiled from hello.c and hello.c is generated from scratch.
Because the build commands in a recipe are Aap commands, you need to use :sys to execute a shell (system) command. In the example, :sys cc executes the C compiler. Obviously, this works only on systems with the cc command. Using shell commands reduces the portability of a recipe.
The hello.c file is generated with :print commands. The first line uses >! $target to overwrite an existing hello.c file. Without the exclamation mark, you receive an error message if the file already exists. This line also contains $(#), which escapes the special meaning of the # character to start a comment. Likewise, $(<) and $(>) are used to get < and > characters instead of redirection.
The hello.c file is generated when it doesn't already exist; no source file dependency is specified. The file can be generated in another situation as well—if you change one of the :print commands, because it changes the signature of the build commands. When the build commands change, Aap knows that the target must be rebuilt.
The file is generated with Aap commands; no shell commands need to be used. This part of the recipe therefore can work on any system. But the number of Aap commands is limited. When you need more functionality and also require portability, you can use Python scripting.
All flow control in Aap recipes is done with Python, and Listing 7 illustrates an example of a recipe that applies patches to Vim. A loop is used to generate a list of patch filenames, starting with vim-6.2.001 and counting up to the last patch number, specified with LASTPATCH. Each of the patch files is to be downloaded and applied. The $* in done/$*Patches is used for rc-style variable expansion; done/ is prepended to every item in Patches.
Normally, you don't need to use much Python in your recipe, but it is good to know that complicated tasks are possible to accomplish when they arise.
We already mentioned that Aap can install rsync for you if it cannot be found on your system. The package install mechanism also can be invoked directly. For example, to install Agide use the command aap --install agide. Agide is the A-A-P GUI IDE, another part of the A-A-P Project. You can use it to build and debug programs with Vim and gdb. It still is in an early stage of development, but it is good enough to develop and debug C programs.
Several packages currently are available, and more will be added over time. A list of the current packages can be found on www.a-a-p.org/packages.html. Aap itself also can be installed. Updating to the latest version can be done with aap --install aap. This command overwrites any existing Aap version. If your system has a package manager, you probably should use that instead.
You now have an idea of the tasks that you can automate with Aap. When you start experimenting you can find a lot of help in the comprehensive documentation. You can find it on the Aap Web site in several forms (see Resources). These pages explain many things that could not be included in this article, such as using CVS for version control, automatic configuration and so on.
Resources for this article: /article/7458.
