Sympa 6 mailing list software

List Master


Mailing lists are more complex than they appear at first glance - especially if you expect a degree of user-friendliness. Sympa 6.0 breaks the mold with a simple web-based management portal.

By Matthias Warkus

Sharpshot, Fotolia

Mailing lists are often the lowest common denominator for groupware. If a group is too large or too decentralized for a conventional groupware solution, a mailing list is often the best way to stay connected.

Although Yahoo, Google, and other Internet companies offer public list services, users who want to reduce dependence and keep control of their data sometimes prefer to maintain their own list servers. By the standards of today's Internet, many of these list service systems are decidedly unfriendly. The web interface often is not much more than a wrapper around an existing command-line interface. The underlying assumption is that users rarely need to subscribe to and list owners rarely need to send to more than one list. Each list has its own homepage and separate passwords for subscribers and owners.

The situation is a bit more complicated in the real world: at universities, in software projects, and in large enterprises, users tend to belong to a number of groups. A few list admins who are responsible for managing most of the lists would really benefit from the ability to assign granular privileges and delegate some of the list management work to less technology-minded users.

Since 1997, a group of French universities has been working on a list service tool that handles multiple lists, subscribers, privilege owners, and admin roles. The Sympa list manager is accessible through a browser window (see Figure 1). Users can manage the lists of which they are members, whether as subscribers, moderators, or owners. If necessary, Sympa will even use LDAP to retrieve login information from a directory service. If you subscribe to, say, ten lists, and manage two of them, you will only need one password instead of a dozen.

Sympa emphasizes simplicity and clarity, which makes it popular with large institutions such as universities, as well as international organizations such as UNESCO and HP.

Figure 1: Sympa lets you access your mailing lists through a simple web interface.

Getting Started

A Sympa tarball is available for download at the project website [1]. With the exception of some minor C wrappers, Sympa is written in Perl and uses an impressive number of modules. Distribution packages for Version 6.0 are currently not available for any distribution except Mandriva.

Admins can install Sympa from the source code by entering the ./configure command. The installation is simpler than for Sympa 5, and the installer will even semi-automatically retrieve the required Perl modules from CPAN. If you are updating from the previous version, you will not only need to run sympa.pl --upgrade after the install but will also need to perform an irreversible migration to the new password encryption mechanism by running sympa.pl --md5_encode_password. (It is a good idea to save the database first.) Although the directory structure changes in Sympa 6, it remains unconventional. If you are unhappy with it, you can either pass your paths in to configure as options or use the --enable-fhs switch.

Sympa normally runs on a separate sympa user account. If you need to become root to perform the install, note that the script will not automatically reset the ownership and permissions for the files you install: this means some manual work to adapt the permissions.

Roles in Sympa

Sympa identifies five different user roles: list masters, privileged owners, standard owners, moderators, and subscribers. Moderators are allowed to moderate messages in their own lists. Owners configure their own lists and name other owners or moderators if they are privileged owners. List masters have unrestricted privileges, although scenarios can reassign these privileges.

A list always has at least one owner, although this need not be a privileged owner. If and only if a list does not have a moderator, the owners assume this responsibility.

Well Attached

The list server needs an SQL database. Although Sympa users have successfully deployed Sympa with Oracle or PostgreSQL, the program is best adapted to MySQL. If you use any other back end, you need to create the tables manually; in MySQL, Sympa will set up the database structure. Because Sympa 6 temporarily stores outgoing mail in the database, you need to set the buffer size sufficient to store messages of the maximum permitted size.

The web interface runs in the form of an fcgid script on an Apache web server with the use of Perl's Template Toolkit (TT2, [2]). The layout resides in a CSS style sheet. Theoretically, it is possible to dress Sympa in any kind of clothes, but practically, any changes beyond minor alterations of the style sheet will mean a considerable amount of work. Instead of redesigning the presentation layer, you can call Sympa as a web service from within your own applications. The SOAP server supplied with the distribution is easy to control and address. French educational institutes use the software as a back end for group management or an authentication service for web portals.

Setting the Scene

Sympa is event-driven: Nearly anything the server does is a response to some kind of action in the web interface or some kind of incoming mail. Sympa currently differentiates between 17 events, the most important of these being adding and deleting list subscriptions, subscribing to and unsubscribing from lists, and of course, sending mail via a list (Figure 3). Sympa's response to an event is decided by evaluating a small configuration file, the so-called scenario. Scenarios are typically configured per list by the owner or list master, although some, such as requesting a new list, are decided globally by the list master.

Figure 3: Sympa 6 shows a log of actions and the target address.

A scenario has a slightly unusual syntax: Each line comprises a condition, an arrow, and an action (Listings 1 and 2). Sympa processes the file top down until a condition applies, then it executes the assigned action and stops evaluating.

Each event is assigned to a user (i.e., to an email address); this can be the address in the From header of the mail that triggered the event, or it can be the address under which the user triggering an action is logged in. A condition can compare the user address with a regular expression or check to ensure that the address belongs to a member of a privileged group (subscriber, moderator, etc.) Sympa supports several built-in conditions, and if you like, you can write a new condition as a Perl plugin.

The most important actions following the arrow are do_it and reject, both of which are accompanied by an argument. The do_it action allows any event for which the software has called the scenario to happen - such as adding a subscriber or sending a message. The reject action prevents the event from occurring and returns the template file specified in a parameter as an error message. Depending on whether the user triggered the event by mail or via the web, the message will be mailed to you or displayed in a pop-up window. To make sure something happens, the last condition has to be defined to be true(). Sympa will then perform the assigned action, which will typically be to create a rejection message.

Listing 1: send.private
01 title.gettext restricted to subscribers
02
03 is_subscriber([listname],[sender]) smtp,md5 -> do_it
04 is_editor([listname],[sender]) smtp,md5 -> do_it
05 is_owner([listname],[sender]) smtp,md5 -> do_it
06 true() smtp,md5,smime -> reject(reason='send_subscriber')
Listing 2: del.auth
01 title.gettext deletion performed only by list owners, need authentication
02
03 is_owner([listname],[sender]) smtp -> request_auth
04 is_listmaster([sender]) smtp -> request_auth
05 is_owner([listname],[sender]) md5,smime -> do_it
06 is_listmaster([sender]) md5,smime -> do_it
07 true() smtp,md5,smime -> reject(reason='del_owner')

Certificate-Based Access

The scenario in Listing 1 sends a message to the list if the sender is a subscriber, moderator, or owner of the list. In any other case, the last true() condition will send a rejection message.

Scenarios with a list of legitimation modes to the left of the arrow can be fairly complex. Sympa supports the keywords smtp, md5, and smime. smtp specifies legitimation of the user address based exclusively on the From header in the mail; md5 designates legitimation by logging in to the web interface (for historical reasons); and finally smime denotes legitimation by an S/MIME signature in an incoming message or by an X.509 signature in a browser certificate.

In Listing 2, the first two lines of conditions apply if the owner or list master requests the deletion of an address from the list by unsigned mail (smtp). The commands trigger a request_auth action, and Sympa will not delete the entry until the user has confirmed the one-time ticket. Lines 3 and 4 express the same conditions, but with a legitimation mode of md5, smime, and the do_it action. Sympa complies with delete requests by signed mail or via the web interface without requiring confirmation. The true() condition rejects other requests.

This scenario-based approach makes it possible to model privileges flexibly. Marburg University relies on this approach to make sure that lists are only requested by authorized staff, that scripts for dispatching newsletter lists are legitimate, or that users in some list groups can post between groups. By linking various scenarios, lists can behave like web forums, or confidential lists can be created that require a digital signature for any action.

As Sympa uses a search path for scenarios, it is up to the admin to overwrite or ignore the default scenarios, or to add their own without changing the default configuration directory.

Feature Rich

Upgrading from Sympa 5 should not concern a seasoned admin, and password security is a persuasive argument in favor of the change. If you are currently using a different tool as your list server, you might even consider migrating, although you will discover architectural differences between Sympa and Mailman or Majordomo. Sympa 6 offers easier installation and other improvements. The portal-style web interface, which has always been a defining feature, has improved but still looks slightly amateurish. LDAP and SOAP offer many opportunities for extending and customizing your implementation. If you are looking for more, you can always try the alpha version of Sympa 6.1 [3].

New Features in Sympa 6

The latest version of Sympa comes with a number of significant enhancements. Most of Sympa 6.0's new features are under the hood. Owners of large installations will appreciate the fact that the software does not transmit outgoing messages directly but stores them in a database. The bulk.pl daemon picks them up from the database and sends them on their way. The mail content and target addresses reside in separate tables, and thus Sympa only stores a message once regardless of the number of recipients.

The bulk.pl script supports parallelization and will launch other processes if necessary. It is possible to run multiple instances on different servers in a server cluster if necessary; however, Sympa can easily handle millions of subscribers out of the box.

Sympa gives admins the ability to specify the date and time when a waiting message should be dispatched. It will then deliver high-volume messages at times of minimal load. A new dispatcher mechanism in the recent Sympa 6 release will continue to deliver where it left off in case of a system crash or restart.

Security conscious users will be happy to hear that Sympa no longer sends passwords in the clear. A new user who requests a password, or a user who has forgotten a password, will simply receive an email message with a one-time URL. Clicking the URL takes the user to a page where they can set a new password. And at long last Sympa now encrypts passwords irreversibly in the user database.

Although enhanced password security is probably the most important change, the most visible change for Sympa users will be the new web interface design (Figure 2), which includes the new logo, pastel colors, and rounded corners.

The new interface not only looks different, it also organizes the administrative options in a more intuitive way, while at the same time offering context-based help in the form of buttons next to the option headings. The options for configuring the web interface design are still just gadgets right now; you can't actually change anything apart from the style sheet colors. However, the developers do intend to move the configuration data to the database in the near future.

Besides the typical wagon load of bug fixes, the new version adds a couple of features: for example, you can now copy lists, and moderators can define and select multiple templates for rejection messages if they moderate multiple lists (see the box titled "Roles in Sympa" for details of the privilege model). The developers have improved localization; in particular, the French convention of inserting blanks before punctuation marks is no longer hard-coded.

Admins will be interested to hear that Sympa 6.1, which has already been announced, will be capable of processing templates in outgoing mails depending on the recipient. This feature, which has been on many users' wish list for years, lets you add a Click to unsubscribe link to the footer of a message.

Sympa also allows you to personally address the recipient of a customer newsletter. This feature lets Sympa serve as a mailing engine with an SQL back end; the software has always been able to map subscribers to free data fields. In Sympa 6.1, users will also be able to suspend all subscriptions at a single stroke, reinstate them later, and unsubscribe via a generated list. This is not currently the case for people who feed the software with recipients from LDAP sources, SQL databases, text files, or other lists. In future, the program will also manage list blocks.

Figure 2: Sympa 6 users can select an application field at the top and the lists themselves in the tabs below. The software will display details or operations as needed in the left-hand column if the user is logged on.
INFO
[1] Sympa: http://www.sympa.org
[2] Perl TT2: http://template-toolkit.org
[3] Demo server: http://demo.sympa.org
THE AUTHOR

Matthias Warkus is a post-graduate student of Philosophy at Philipps-Universitšt Marburg. He works for the university data center, where he manages a major production installation of Sympa 6. He has also published developer manuals for Gnome.