Linux laptop users, try the mail solution that combines the advantages of fast local mail folders and a server-based IMAP repository.
E-mail is for many people the single-most important feature of the Internet. We read e-mail at home, at work, while traveling and on many different computers. But it's difficult to see the same mail from all of those places. If you delete a message from home, it may not show up as deleted when you look at the same account from work. Worse, you might be able to view a given message on only one machine. And if you sometimes want to download mail to your laptop and read it without any Internet connection, things get even more complex.
Some people try to solve these problems by using IMAP in their mail clients. But IMAP can be slow and poorly supported; especially on a slow connection, it tends to make mail reading unpleasant. I recently faced exactly this situation—I was a very annoyed programmer. Many programs come about because a programmer somewhere was annoyed. Thus, I wrote OfflineIMAP.
OfflineIMAP is designed to let you read the same mailbox from many different computers, with or without an active Internet connection. It performs a bi-directional sync, which means that any changes you make eventually are reflected on all your machines. In its most common form, OfflineIMAP works by connecting to an IMAP server and synchronizing your folders to a series of Maildir folders on your local machine. Despite its name, OfflineIMAP is useful even if you never read mail off-line.
OfflineIMAP installation is easy. Visit the OfflineIMAP home page at quux.org/devel/offlineimap, and download the .deb or the tar.gz file. Debian users simply can run dpkg -i offlineimap.deb to install it, and then use apt-get -f install to fix any missing dependencies. If you're not running Debian, make sure you have Python 2.2 or above installed. If you do not have Python already, check with your distribution or visit www.python.org to download it.
When you're ready to install OfflineIMAP, run tar -zxvf offlineimap_4.0.2.tar.gz to unpack the source. Change into the new directory and then, as root, run python setup.py install. If you get stuck, the OfflineIMAP manual contains some more installation hints.
OfflineIMAP configuration is done in the ~/.offlineimaprc file. That file has three different sections: general, which controls overall behavior of OfflineIMAP; repository, which describes a place where mail is stored; and account, which describes how two repositories are synchronized together. A basic, simple setup requires only a small configuration file. Here is an example:
[general] accounts = MyMail [Account MyMail] localrepository = MyMailLocal remoterepository = MyMailRemote [Repository MyMailLocal] type = Maildir localfolders = ~/MyMail [Repository MyMailRemote] type = IMAP remotehost = hostname.example.com remoteuser = my-username-goes-here ssl = yes
This example defines one account, MyMail. The MyMail account is synchronized from the hostname.example.com server to the ~/MyMail directory on your local machine. All remote folders are copied. If your IMAP provider does not support SSL encryption, delete the ssl = yes line. Now, run offlineimap. You are asked for your password, and then it synchronizes your mailboxes once and exits.
If you're connected to the Internet while you read your mail, you can have OfflineIMAP continually keep your local tree synced up with the server. To do this, simply add an autorefresh line to your account section. For instance, you might modify your account section to look like this:
[Account MyMail] localrepository = MyMailLocal remoterepository = MyMailRemote autorefresh = 5
When you run OfflineIMAP now, it synchronizes your mailbox like before. But when it's done, instead of exiting, it keeps running, synchronizing your mail every five minutes.
OfflineIMAP is quite capable of synchronizing multiple accounts. For instance, you might want to be able to read mail from both your work e-mail and your home e-mail. To do this, add one account and two repository sections for each account, making sure to use unique names. Then, add the account to the accounts list in the general section. Separate the names by commas.
On the local side, you should make sure that each account synchronizes into a different directory. Otherwise, confusion and corruption may occur.
OfflineIMAP's defaults, as illustrated with the examples above, are quite conservative. It tries to work with as many IMAP servers as possible right out of the box, so the advanced features that occasionally cause trouble are disabled by default.
If you have many mail folders or get a lot of mail in each folder, the synchronization process can be slow. This is especially true if you are using a high-latency Internet connection, such as a modem or satellite. To speed things up, OfflineIMAP is capable of establishing multiple connections to your server at once. It then is able to perform tasks in parallel. For instance, OfflineIMAP might download three messages and synchronize two folders simultaneously.
OfflineIMAP offers several configuration options. First, you should add a line such as maxsyncaccounts = 5 to your general section. This enables OfflineIMAP to synchronize multiple accounts simultaneously, which is almost always a good thing. Second, in the repository section for the remote part of each account, you can control how much parallelism to use. For instance, you might add a line saying maxconnections = 3 to the MyMailRemote repository section in our example. This allows OfflineIMAP to establish up to three connections to the server.
If you are performing continuous syncs with the autorefresh option described above, there's another source for delay. Each time OfflineIMAP starts syncing an account, it connects to the server. When it's done with that particular sync, it disconnects. Establishing these connections can be slow in many cases. OfflineIMAP provides an option to keep the connections open even between syncs. The problem is that some servers disconnect clients that are idle for a long time. To combat that problem, OfflineIMAP also can send little bits of traffic every so often to make sure the timers don't expire. To take advantage of these features, add lines like these to the remote repository section:
holdconnectionopen = true keepalive = 60
OfflineIMAP ships with many different user interfaces. The two most common are Tk.Blinkenlights and Curses.Blinkenlights. The former presents a small graphical window on OfflineIMAP's progress on your X desktop. The latter runs in a terminal and provides a nice monitor of progress (see Figures 1 and 2).
Figure 1. The Tk.Blinkenlights GUI Interface for OfflineIMAP
Figure 2. The Curses.Blinkenlights Interface, Running in a Terminal Window
With the Tk.Blinkenlights interface, you can click on the Sync immediately button to force the account to synchronize right away. You can do the same thing in the Curses.Blinkenlights interface by pressing the number next to the account name. Both interfaces display a log of current activities. You also get a mesmerizing display of flashing status lights so you won't get bored while watching the synchronization happen.
The TTY.TTYUI interface can run without any Curses support—it uses no color or terminal controls. It can read password input, but it provides no other capabilities for interaction.
Noninteractive.Basic is a user interface designed never to receive any input from the user. It can, however, display status messages. If you need a password in order to log in to a remote server, add a line such as remotepass = mypassword to the remote repository section of the configuration file.
Finally, Noninteractive.Quiet goes one step further and does not output status messages. Some people like to run OfflineIMAP from a cron job, and Noninteractive.Quiet is good for that.
You can specify which user interface should be used in one of two ways. First, you can use the -u option on the OfflineIMAP command line. For instance, you might run offlineimap -u Curses.Blinkenlights. Alternatively, you can add a ui line to your general section, like this:
ui = Tk.Blinkenlights, Curses.Blinkenlights,
TTY.TTYUI
With this configuration, OfflineIMAP first tries the Tk.Blinkenlights interface. If your Python doesn't support Tk, or if you are not running under X, it then tries the Curses.Blinkenlights interface. If that too fails, the TTY.TTYUI interface is tried. If even that does not work, OfflineIMAP aborts with an error.
By default, OfflineIMAP asks your remote IMAP server which folders are available to you and synchronizes all of them. A folderfilter option can be added to your remote repository section to restrict what is brought over. The folderfilter option is a tremendously powerful option. Unlike the other options you've seen so far, folderfilter actually expects to be handed a Python function. The function takes one argument and should return true if that folder is to be included.
Python provides a feature called lambda that lets you create simple functions on the fly. You thus can construct some complex rules. Here are a few examples. You can specify a set of folders you want to synchronize. You can use the Python in operator to test whether the folder in question is in the list, like this:
folderfilter = lambda foldername: foldername in
['INBOX', 'Sent Mail',
'Received']
This code synchronizes only the three named folders. Notice the indentation on the second and third lines—if you indent them, the configuration parser treats them as part of a single statement.
You also can specify folders to exclude:
folderfilter = lambda foldername: foldername not in
['Spam', 'Junk']
In this example, all folders except Spam and Junk are synchronized.
You also can use regular expressions, such as:
folderfilter = lambda foldername:
not re.search('(^Trash$|Del)', foldername)
Sometimes, you may want to alter the folder names before storing the folders on the local side. OfflineIMAP provides an option called nametrans, also specified in the remote repository section, to do exactly that. Some IMAP servers, such as Courier, add “INBOX.” to the start of all folders, which can be annoying. The nametrans feature lets you get rid of that. Here's an example:
nametrans = lambda foldername:
re.sub('^INBOX\.', '', foldername)
Like folderfilter, nametrans takes a Python expression. This expression receives a folder name as an argument and should return the new and improved folder name. In this example, any folder whose name starts with INBOX. gets the leading INBOX. removed. It's important to remove not only the leading INBOX; the folder INBOX itself does exist, so you'd wind up with an empty folder name (and that's a bad thing).
It's also important to be careful with your nametrans rules. You must make sure that nametrans returns a different value for each folder. If it returns the same thing for two different folders, bad things can happen.
In case you're wondering, nametrans does not change folderfilter. That is, your folderfilter rules operate on the folder names before nametrans takes effect.
Some mail readers don't support Maildir hierarchies well. For them, OfflineIMAP introduced a new feature: the ability to synchronize two IMAP servers directly. The idea is this: you install an IMAP server on your local machines. Your mail reader, which already may have slow IMAP support, is fairly speedy in accessing an IMAP server located on your own machine. The mail reader never needs to know that OfflineIMAP is sticking the messages in the folders.
To make this happen, you need to make a few simple changes to your local repository section. First, change the type from Maildir to IMAP. Secondly, remove the localfolders and other Maildir information and instead specify IMAP configurations, such as remotehost and remoteuser. Finally, delete your ~/.offlineimap directory to make sure that none of the old status information lingers around.
Certain options still are supported only in the remote section—nametrans and folderfilter are two examples—but the options relating to the connection itself are supported in both places. You can, in fact, have your local IMAP server on a machine that is remote to you.
OfflineIMAP is a powerful mail solution. I've introduced you to the basics of OfflineIMAP in this article, but there still is more you can do with it. To learn more, check out the OfflineIMAP home page and the example configuration files. If you're a Python programmer, you'll find some nice hooks for Python code as well.
