Using the APM application to install, distribute and remove packages specifically used in database applications.
One of the core ideas of software engineering is to divide a large project into separate modules. Modularization makes it easier to customize a system for your own specific needs, allowing you to write new modules and remove unnecessary ones. Using modules also makes it easier to distribute the work among many different programmers. A quick review of the available Linux, Apache, Perl and Python modules freely available on the Internet makes this point very clear.
OpenACS 4 (Open Architecture Community System), the toolkit for creating on-line communities that was initially examined here last month, dramatically improves on earlier versions in a number of ways. But perhaps the most important change is the division of functionality into modules, which are called “packages” in the OpenACS world. Because each package is self-contained, and because it is possible to connect any package with any URL, OpenACS 4 has made it easier than ever to create flexible community web sites.
This month, we take an initial look at OpenACS packages, including how we can install and use them. (This article assumes that you already have installed PostgreSQL, AOLserver and the core OpenACS functionality, as described in the last two installments of At the Forge.) Since most OpenACS sites use some of the functionality that comes with the built-in applications, rather than write everything from scratch, installing packages is something every OpenACS administrator needs to know how to do soon after installing the core system.
Consider the following simple CGI program written in Perl:
#!/usr/bin/perl use strict; use warnings; use CGI; my $query = new CGI; print $query->header(); print $query->start_html(-title => "Testing"); print "<p>This is some text</p>\n"; print $query->end_html();
If I install this program as test.pl in my web server's CGI directory, others can see the results of its execution by retrieving www.lerner.co.il/cgi-bin/test.pl. If I want this program to be available under a number of different names, I can copy it; the name that I choose will be reflected in the URL.
Things get a bit trickier if my server-side application consists of several CGI programs rather than a single program. If I want to have several copies of such an application suite running on my system, I must copy all of the program files. In many cases, it'll be easier to place all of the files in a directory, then copy the directory and all of its contents each time I want the application to run somewhere else.
Making such copies carries potential synchronization problems: if I fix a problem in one copy of a program, I will have to make the same change to every copy of the program. I can resolve some of these problems with CVS, but I also could eliminate this issue by keeping only one copy of my program on the filesystem. Then I could configure the web server (either Apache or AOLserver) to treat one or more URLs as requests for my program.
Now consider what happens if this application suite takes advantage of a relational database. Installing the application is no longer as simple as copying files or configuring the HTTP server. Now, we also need to have some way of resolving potential conflicts and confusion between the copies of a single application, such that the forums at /foo/bboard don't get confused with /bar/bboard in the database. If and when we remove our application from the system, we also will need a way to remove the database tables it used.
In OpenACS, the solution to this problem is APM, the ArsDigita Package Manager. APM was originally written by ArsDigita, a now-defunct consulting company that wrote the predecessor to OpenACS. ACS worked only with an Oracle database server, whereas OpenACS works with both Oracle and PostgreSQL.
APM handles a number of different issues inherent in server-side applications that use a database, including version control, scripts for table creation and removal and database independence. APM also has been designed to allow each copy of an application to have independent configuration variables and to be associated with one or more separate URLs.
An APM really is nothing more than a .tar.gz file with an .apm extension. The file is typically named like this: packagename-0.5d.apm—where packagename is the unique name associated with the package. This example package contains development version 0.5. Opening a package with tar -zxvf reveals a standard file and directory structure:
packagename.info, an XML file describing the contents of the package. This file, normally created automatically by the OpenACS APM application, tells OpenACS which files are associated with the package and which configuration parameters are available for the user. It also indicates whether the application is a singleton (i.e., provides services for the rest of the system) or an application (i.e., can be run from a particular URL).
The sql directory is where the table-creation (and table-destruction) scripts are located. Originally, when ACS supported only Oracle, this directory normally would contain two files: packagename-create.sql and packagename-drop.sql. The APM installer would run the create script when the package was installed and the drop script when it was removed. (The create script often runs INSERTs as well, seeding database tables with standard data for later use.)
Now that OpenACS supports PostgreSQL as well as Oracle, this directory structure has changed somewhat. Within the sql directory are oracle and postgresql directories that have parallel scripts for creating and dropping the tables. Each installed copy of OpenACS knows which databases it supports (based on the value of a variable in AOLserver's nsd.tcl configuration file), and thus chooses the most appropriate script.
The tcl directory contains Tcl files containing procedure definitions. These procedures are loaded into AOLserver at startup time, giving them a speed advantage over those defined inside of .tcl (or .adp) pages elsewhere in the OpenACS system.
The www directory contains what we normally expect to be associated with a web application. This is where we put our .tcl and .adp pages, as well as any graphics and auxiliary files associated with the application. OpenACS's query dispatcher, which makes it possible for server-side programs to support multiple database servers, works with XML files with an .xql extension; these also go in the www directory.
Because of how the OpenACS templating system works, it's not unusual for a single web page to use three files: a .tcl file for setting variables, an .xql file that defines the SQL query used to retrieve rows from the database and an .adp file that is responsible for turning the information into HTML.
APMs also may contain a number of other files, such as database upgrade and migration scripts (for those users who are upgrading from a previous version of the package), regression tests (to ensure that the package works correctly), administration facilities (under www/admin) and HTML-formatted package documentation (under www/doc).
The first step in working with an APM package is to load it, which normally means copying it into the filesystem where your OpenACS system resides. If your OpenACS system is under /web/atf/, then all packages go under /web/atf/packages. (For this reason, each package needs a unique name; many OpenACS developers use an Emacs-style package naming convention, in which the package name is preceded by the developer or client name. This helps to avoid conflicts between the foo-attributes and the bar-attributes packages.) Copy the package's entire directory structure into /web/atf/packages, making sure the files and directories are readable (and writable) by the user ID under which AOLserver operates.
An easier and more reliable way to install APMs is to click on the load packages link within the package manager. OpenACS will ask you for the URL of the APM or the name of the directory in which one or more packages reside. OpenACS then will find all of the .apm files located there, unpack and load them into the system.
Once a package has been loaded into the filesystem, you must install its data model and register it with the system. This is done through the web-based package manager, which normally is found under the URL /acs-admin/apm on an OpenACS system. Typically, only the site administrator has access to the package manager.
The main package manager screen shows all of the packages that are loaded in the system and indicates whether each has been installed, superseded by a newer version or is yet to be installed. Using the menu options at the top of the page, you can ask to see different subsets of the packages on the system, including only those that you are personally responsible for developing and managing.
The package manager is the main way in which you create, modify, update and install packages:
You can install the data model for a package and register the package with OpenACS. The next time OpenACS restarts, files in the package's tcl subdirectory will be loaded into AOLserver's memory. Your package can be connected to a URL via the OpenACS site map, as we will soon see.
You also can use the package manager to create a new OpenACS package. Indeed, the first step in creating a new OpenACS package is to use the package manager to set up a new directory and .info file.
You can examine any package currently loaded into the system and retrieve a list of parameters, files or any other information associated with the package.
You can modify a package, changing parameters, files and other information associated with the package.
To install a new package, click on the install packages link at the bottom of the page. The package manager will scan the packages directory for any new packages, allowing you to choose which packages should be installed. (When you first install OpenACS, no packages are installed, so this is a very long list.) Each package can depend on one or more other packages. If you try to install a package without its prerequisite dependencies already installed, the package manager will require confirmation before continuing.
The installer lets you decide whether you want to install a package's data model or also enable the package for use on the site. Personally, I always enable any package I install, but I'm sure there are reasons why you might not want to do this. After checking the appropriate boxes, click on the Next button, which installs the data model. Following this, you must restart the AOLserver, because many packages depend on Tcl libraries loaded into AOLserver at startup. Therefore, the packages will not work until the server has been restarted.
Once you have restarted AOLserver, go to the package manager and look at the list of enabled packages. All of the packages you loaded should be visible on this list. From here, you can modify the packages, load more packages or begin to turn the loaded packages into an actual web site. This process is known as mounting and is performed using the OpenACS site map. In one of the more confusing parts of OpenACS administration, the site map is not part of the site-wide administration page; rather, it is part of the main site administration page. In other words, you manage packages under /acs-admin/apm, but you manage the site map under /admin/site-map. There are some good reasons for this, but it tends to confuse people more than it helps them.
The site map tells OpenACS how to connect a URL to an application. For example, you might want the OpenACS bboard package to be under the /forum URL or the /bboard URL. In some cases, you actually might want to have it in both places. The site map allows you to do this by clicking the mouse.
To connect a package to a URL for the first time, click on the New subfolder link to the right of the / path. You are asked to name the URL under which the new application should be mounted. To install the bboard package under /forum, you would enter forum (without the leading slash).
If you stop here, you can treat the new subfolder as nothing more than a folder in which new folders and/or static documents are placed. But you also can click the New application link associated with this folder, choosing an installed application package and giving it a human-readable name that will be used in titlebars and headlines. So while you might put the forums under the /bboard URL, you might want to give it the name Discussion forums. This title cannot be changed all that easily, so give this process some thought.
The new application link creates a new package instance and then attaches it to the directory you have created. To make an alias to this application, you can create a new subfolder and then use the mount link. It took me a while to understand that mount lets you connect to existing application instances, while new application creates an entirely new instance. This makes more sense when you consider that unmounting an application (using the unmount link to the right of the pathname) does not delete it but makes it unreachable. To delete an instance of an application completely, you must click on the unmounted application link from the site map, and then click on the delete link next to the unmounted application.
Each instance of an application has its own permissions and its own parameters. I have found parameters to be a particularly useful part of OpenACS, in that they let me create an application once and use it many times, but give each instance its own configuration. Following the appropriate links on the site map, you can view or change the parameters associated with a package instance.
OpenACS packages, distributed as .apm files and managed with the APM application within CVS, make it possible to create and distribute web/database applications. Once imported, an APM package can be instantiated multiple times, each with its own permissions and associated parameters. As we will see next month, you also can use APM to create your own web/database application packages and distribute them easily to coworkers and other community members.
