When it comes to e-mail, what do small organizations need? Why? And, how do you make it happen?
Every month, more and more small businesses and families depend on e-mail to stay in business or to remain in fast and cheap contact with distant relatives. Both groups often need to enforce common rules, such as parental control or acceptable usage policies. This article is the first in a series devoted entirely to building such an e-mail system with free software, and it is divided in three parts. The first sums up how small groups use e-mail today and why it is often a suboptimal solution. The second describes the overall architecture. The last part introduces the main characteristics of the central piece of the system, the SMTP server.
Centralized configuration and logging, automatic blocking of spam and viruses, access to the same mailboxes from any e-mail client or from the Web—these are just a few of the services that both small business owners and responsible parents want to have under complete control. Equally important is having constant e-mail addresses, totally independent from the current Internet access provider. A secure and usable Webmail interface as well as consolidation in one mailbox of old accounts from previous ISPs or college days also are a must.
Most groups manage e-mail in one of two ways. The first way is to make every group member choose any of the big “free e-mail for the masses” providers (for example Gmail, Yahoo and so on) and make do with what those providers offer. This route is the easiest one, but sometimes its only real benefit may be constant, if generic, addresses. With such providers, there is little or nothing that can be centrally monitored, configured or controlled, unless the resident geek manually sets and always keeps all the computers in sync—including every time a user changes computer or e-mail client. Besides these practical issues, the reliability and privacy concerns of a whole world going with two or three e-mail providers are equally obvious.
The second option is to buy professional, dedicated e-mail hosting from a specialized provider. Note that I said “dedicated e-mail hosting”, not “vanilla e-mail services from generic Web hosting companies”. Generic Web hosts provide suboptimal spam filtering, at best. Dedicated e-mail hosting usually offers top-notch spam filtering, reliable e-mail forwarding and so on, because it is a separate business that requires different know-how and configurations from Web hosting. All providers of this kind can handle e-mail for any domain, so they are flexible enough to make many users happy.
If a dedicated e-mail host provides such good service, why look beyond that? May small businesses and families want complete control of their e-mail features. Other deal-breakers may be insufficient bandwidth or disk space, not sophisticated enough spam filtering and privacy concerns. Please note that privacy here doesn't merely mean “a trustworthy provider that never will read your messages”. In order to nail down spam senders with the smallest possible effort, some providers, including several in the “dedicated” category, do not read your messages but add the IP address of the computer from which you are writing to each e-mail you send. This makes it impossible to send messages like “I can't come to the office for the 30th day in a row; I've flown to Fiji” from home, because everybody able to read and understand e-mail headers will see immediately that the message came from your home Internet connection. For some users, this reason alone may be enough to exclude certain providers.
The only way to set up e-mail for your small community exactly like you want and keep it going with the smallest possible effort is to configure your own state-of-the-art e-mail server using only free software. And, that is what this series of articles is all about.
Unfortunately, the majority of on-line tutorials on this subject fall into two categories: the ultimate single-user e-mail server or the perfect Linux/BSD e-mail platform for Fortune-500 corporations. Although the first category is too little even for small groups, tutorials in the second class often imply that you will be happy to learn, deploy and maintain a bunch of other servers and databases just to provide Webmail, authentication and so on to five to ten users.
With only a few users, it is more efficient to stick to configuration files, limiting the number of third-party components as much as possible. Whenever possible, this series follows that approach.
This first article describes only the general architecture of the system and the essential capabilities of the SMTP server that will be the basis for the whole environment. In the next articles, I'll cover how to receive e-mail for several domains, how to relay messages for authorized users and how to filter spam and viruses at the server level. After that, I will discuss how to do as much advanced e-mail sorting and processing as possible on the server. Next, I'll show how to serve the received messages to desktop e-mail clients or through a Web interface. Along the way, you also will learn when and how to use a backup MX server, how to back up mailboxes, how to keep everything under control with automatic reporting tools and several other cool tricks.
The advantage of a centralized e-mail environment is that there is very little that can be done from a graphical interface or messed up by end users. The whole point of going through this hassle is to have e-mail configured and working always as you prefer—no matter how often you or other group members change, test or mix different clients, operating systems and locations (home, office, Internet kiosks and so on). The right way to make this happen, however, is to have as much as possible done on one server, with tools that often are controllable only by text configuration files. This task is absolutely possible without programming, as long as one is willing to spend some time tweaking configuration files and reading the right documentation.
The hardware can be an old computer stored in a closet or a Virtual Private Server (VPS) in a data center. The first solution saves both money and the environment (hardware pollutes!). The second solution is much more reliable, as downtime due to blackouts or broken hardware almost disappears. Because the configuration and tips described in this series are the same for both cases, I simply ignore that issue when dealing with setup.
To make things easier, only configuration of binary packages available for your chosen distribution are considered. You may not get the latest-and-greatest version of everything, but the convenience of not compiling anything and having automatic updates is too great to be refused, unless you are a full-time paid administrator. Therefore, I assume you already have a Linux box and know how to install binary packages on it.
The system requires the components shown in Figure 1. Only the desktop e-mail client runs on a computer other than the e-mail server.
Figure 1. System Requirements
The SMTP server is the piece that uses the Simple Mail Transport Protocol to accept legitimate e-mail from the outside world or to ship all the messages from authorized users, no matter how they were sent, to their final destinations. This server is assisted by several dæmons that help block spam, viruses or otherwise unwanted connections. Outgoing messages from external clients are accepted and relayed only after an authentication procedure.
After it has accepted a message, the SMTP server hands it to a local delivery agent (LDA) that places it into one or more mailboxes, according to predefined filtering criteria. The single messages are then passed to Webmail or desktop clients by an IMAP (Internet Mail Access Protocol) server. I focus on IMAP because it works the same way for both classes of clients, and it also is possible to move messages from the server to your personal computer with the POP3 protocol. Most IMAP servers also support POP3. Please note that any Webmail software requires a Web server (usually Apache) and PHP in order to run.
Our example e-mail box will serve about ten users in two different domains: myfamily.net and mybusiness.com. The actual applications used in this series are Postfix (SMTP), Dovecot (IMAP/POP3), Squirrelmail (Webmail) and procmail (LDA). They are not the only FOSS products suited for this task, of course, but I have found them to be an excellent combination in terms of features, documentation, great on-line support and ease of installation and configuration. Most of the concepts and tricks can be ported to other products easily, however.
Let's start looking inside our chosen SMTP server, Postfix. The anatomy of Postfix already has been described in another Linux Journal article (see Resources), so I summarize only the main points here. Postfix is actually a suite of many programs, some interacting with user processes, some running as dæmons, all controlled by a master dæmon. Usage of all the dæmons is specified, one logical line per dæmon, in a file called master.cf. Logical lines can extend on more lines of the file. Local e-mail delivery, for example, can be handled by a Postfix dæmon that can write to mbox and Maildir mailboxes directly or by an external local delivery agent, such as procmail. Listing 1 shows a snippet of a typical master.cf file.
Each line starts with the dæmon name (service) and the way it talks with Postfix (Internet/UNIX sockets or FIFOs). The next parameters show whether it's private to the mail system and whether it runs with or without privileges into a chroot (that is, into a restricted filesystem). Wakeup and maxproc define the wake-up interval and the maximum number of processes that may execute this service simultaneously. The line ends with the actual command that invokes the dæmon, with all its options. We'll look in detail at some dæmons in the upcoming articles. For now, it's enough to know that one or more -v switches enable verbose logging and that a - value means use the default value for that field.
Postfix has hundreds of configuration variables, but don't worry. To build a working system, you need to set only about 20 of them. To get an idea of Postfix's capabilities and the way it works, see Listing 2. It is an excerpt, by no means complete (or working), of a real main.cf file. More specifically, it shows only the main options that must be set to a different value than the default or the ones that are particularly important for spam fighting and not making an open relay. We will build a complete main.cf file in upcoming articles in this series.
Section 1 contains the full name of the VPS box. Section 2 defines from where we accept mail and for which destinations. The first law of each SMTP server is never to become a spambot by relaying messages of unknown origin or useless bounce notifications to the Internet. This is what Section 3 is for. As it is, it means that only messages originated on the server will be sent outside. Section 4 says that we accept e-mail for the two myfamily.net and mybusiness.com domains, but only for the users in the /etc/postfix/vmailbox map, and that we use procmail to store incoming messages into the /var/mail/mail_storage folder.
When proper SMTP filtering rules are available (as shown in Figure 1), an SMTP server can recognize and refuse a lot of spam as soon as it is contacted, without even downloading the whole message. This is what sections 5 and 6 do. Besides the official documentation, their content is described in full detail in the Postfix anti-UCE cheat sheet (see Resources). Note that the server actually will work only if you set DNS properly, but this, as well as the exact meaning of all those variables, is something I'll discuss in future articles. For now, let's finish by introducing Postfix maps.
Postfix can store strings and filtering expressions in many indexed or linear maps. These structures are used to store information on how to deal with clients, senders and single messages. Indexed maps are binary databases built from regular text files. External relational databases also can be used as indexed maps. Linear maps are regular text files read from top to bottom until a matching record is found, so the order of the record matters.
This article is an introduction to the topic of this series, rather than a practical tutorial in order to clarify some general concepts. But, don't worry; we'll get to the practical concerns in the next article. In the meantime, take a quick look the documentation listed in the Resources and keep it handy for when you read “Work the Mail, Part II”. You also are welcome to write me directly about any specific e-mail tip you would like to find in this series.
