Browser synchronization and more with Mozilla Weave

Weaving the Web


Mozilla Weave for Firefox synchronizes your critical browsing information between multiple machines.

By Nathan Willis

Twwx, photocase.com

The Firefox extension site [1] lists dozens of bookmark synchronization tools. Some sync bookmarks with a commercial service, like the Web bookmarks site Delicious; others sync with private storage, such as a WebDAV or FTP server that you maintain. Weave gives you an option - you can set up a free account at the newly launched services.mozilla.com, or you can run your own server using the open source Weave server code. Your initial choice does not handcuff you, either - migrating between services.mozilla.com and a private server is as simple as entering your new account credentials and performing a sync.

Furthermore, your privacy is protected no matter where you store your remote data. The Weave extension encrypts everything on the client side with public-key encryption before it is transmitted, and your key is never sent over the network. You can use Weave between multiple machines, including not just Firefox on desktops and laptops, but the mobile browser Fennec as well.

Weave is an ongoing project from Mozilla's Mozilla Labs [2] test bed site, and although the current release focuses on synchronization, in the future, Weave will be able to do much more. The underlying engine supports machine-readable microformats, which makes real data mash-ups possible, and developers can write add-ons with the help of the Weave API.

Getting Weave

The Weave homepage [3] is where you will find the "Getting Started" information and a link to the latest release. The current version is numbered 0.3 and is the first Weave build open to the public, without having to request participation in the beta testers' trial.

To use Weave, you must run the latest betas of Firefox 3.5. From Firefox, you can visit the Weave page and click and install the extension just as you would any other Firefox add-on. Once you restart the browser, you will see Weave's Celtic knot icon sitting in the status bar at the bottom of the screen. Right-clicking on the icon brings up the menu. Then select Sign In....

The setup procedure requires you to agree to the Weave terms of service and create an account at services.mozilla.com. When you are configuring subsequent computers, you will skip the account creation step and simply sign in with your existing credentials. To create your account, supply an email address and choose a username and password - the setup wizard is polite enough to check automatically for whether the username you select is available as you type, so you can quickly and easily pick one that is free (Figure 1).

Figure 1: The account creation wizard helps you seamlessly create an account for the services.mozilla.com sync server.

After you have chosen your account information, you will then be asked to choose an encryption passphrase. This passphrase will be used to encrypt your bookmarks, tabs, and other data before it is relayed to the Weave server. It is a separate secret from the password you selected in the account creation step - that password is only used to sign in to the server and, by necessity, is used for authentication across the network. Your encryption passphrase is never sent.

Weave also asks you to name each computer on which you install the extension. Currently, you cannot view or control advanced settings on a per-computer basis, but this type of functionality could be added in the future.

Once your account and encryption passphrase are set up, Weave performs an initial sync: itemizing, encrypting, and uploading your data to your account at services.mozilla.com (Figure 2). Depending on how much history you save and how many bookmarks you use, this could take several minutes. Subsequent connections to the sync server only transmit incremental changes, which are much smaller.

Figure 2: Once your account is verified, Weave performs an initial one-way sync, encrypting and uploading all of the sync data from your browser.

In Sync

If you use Firefox on just one computer, Weave can serve as an off-site backup solution, ensuring that you do not irretrievably lose your bookmarks or saved passwords. To keep multiple Firefox installations in sync, all you need to do is install Weave on each of them. During initial setup, simply choose to enter your existing account details (username, password, and encryption passphrase) rather than create a new account.

If you do nothing else, Weave will connect to the server once every five minutes and exchange encrypted updates to the data it is tracking for your account. By exploring the Weave preferences, you can get more out of it (and better understand its inner workings). Open the Preferences panel by right-clicking Weave's menu status bar icon (Figure 3) or from Firefox's Tools menu.

Figure 3: The Weave status bar icon hides a context menu from which you can sign in with your server account, open the extension's preferences, or view the log.

The Weave Preferences dialog has five tabs: Account, Data, Clients, Add-ons, and Advanced. Account allows you to sign in and out of your connection to the server. Advanced allows you to change the server URL if you are running your own server and use debugging tools, such as viewing the activity log. Add-ons is currently unused. Clients shows a list of the computers associated with your account (Figure 4).

Figure 4: Weave's Clients tab in the Firefox Preferences dialog allows you to keep track of how many browser instances you have set up in conjunction with your account. Because you assign the client names yourself, choose them carefully.

The Data tab is where the critical settings are (i.e., which data to sync). Although Weave 0.3 syncs only four data types (bookmarks, browsing history, tabs, and saved passwords), you can see from the grayed-out entries in the Data tab (Figure 5) that many more are in the works, including cookies, saved form data, search plugins, and extensions.

Figure 5: The Data tab of the Preferences panel shows not only which data types you have selected to synchronize but provides a window into what data types could be supported in future versions of Weave.

Before you sync two computers, it is a good idea to visit the Data tab and uncheck items that you do not want to share. For example, you might want to keep a different set of bookmarks on your office desktop, or you might not care to synchronize tabs.

By default, Weave attempts a two-way sync between the local data and the data saved on the server. When multiple sets of data are on separate computers, Weave reconciles them by combining them into one set that represents the union of all of the computers' data - hopefully without duplicates, although the sync engine can occasionally get confused.

In some cases, pooling together all of the data might not be the behavior you want. For example, when setting up a new computer, you might want to download your existing bookmarks and passwords, overwriting the defaults installed out of the box. Weave lets you do just that. On the Data tab of the Weave Preferences panel, you can use the Sync Now... button to initiate a sync manually. The Sync Now tool features a drop-down menu for choosing between two-way sync, a download-only sync that overwrites locally stored data, and an upload-only sync that overwrites the stored server data.

In version 0.3, the Sync Now tool also has a non-functional drop-down menu for selecting what data types to sync - although the menu is disabled, you can emulate the same behavior by un-checking anything you do not want to sync from the main Weave Preferences panel before initiating the sync.

Unless you need to perform a manual sync, you might never know Weave is running. The status bar icon indicates whether you are signed in (Figure 6), and the incremental syncs once every five minutes are completely unobtrusive. In my initial tests, performed on Ubuntu 8.10, the only real hiccup was the occasional bug in the Firefox beta itself. That is hardly Weave's fault, of course, and I could detect no discernible slowdowns or interference attributable to Weave.

Figure 6: Weave's status bar icon indicates its status. "Sign in" is displayed when you are not logged in to the Weave synchronization server, your username is displayed when you are logged in, and the icon changes from the Weave logo to a spinning sync symbol whenever a sync is in progress.

Additionally, I never found Weave to lose or overwrite a bookmark or saved password. It is a bit more difficult to track changes mentally in browser histories between multiple machines, but Weave did successfully sync the distinctive test pages I visited just to observe its performance. As an added bonus, I like that Weave makes tabs from other synchronized computers available as a History sub-menu; it helps, but without getting in the way of the local browser history.

Weave for the Fennec Mobile Browser

Weave is straightforward to use between desktops and laptops because you can run the same version of Firefox, even on different operating systems. The Weave team wants to bring the same experience to mobile devices, beginning with the mobile browser Fennec [4].

So far, official builds of Fennec are provided only for Nokia's Maemo-based Internet tablets. If you have an N800 or N810, you can download the latest Fennec Debian package or add the Mozilla repository to your tablet's Application Manager.

The same Weave client extension works on Fennec as well as Firefox, so installation is a snap. Just visit the Weave homepage on Mozilla Labs and click on the download link. Unlike Firefox, however, Fennec does not support the creation of a new services.mozilla.com account, so you must have a working Weave account set up through Firefox before you begin.

After you install Weave, restart Fennec, then drag the screen to the right to reveal the button menu. Now press the setup button (the one shaped like a gear). From setup, press the slider button to open Fennec's Preferences. Weave's preferences are in the Privacy & Security section. Press the Details button, and when prompted, provide your services.mozilla.com username, password, and encryption passphrase. Once Weave successfully authenticates you to the server, it will open up a Preference page from which you can select the data types to sync, change the client name assigned to your tablet, and alter your login credentials. The Sync Now... button is not yet fully implemented in Fennec, but basic data synchronization is already supported.

Behind the Scenes: Client-Server Communication

Once you have successfully synced your browsers a few times, you will probably wonder how the whole process works. Fortunately, the Mozilla project makes the Weave server source code publicly available and provides documentation on the API [5].

The Weave client uses https requests to communicate with the server for everything from account creation to data storage and retrieval. The data are stored at services.mozilla.com in essentially a highly structured WebDAV share. Each data type is in its own subdirectory, where snapshots and "deltas" (changes since the last snapshot) are kept in JavaScript Object Notation (JSON) files.

The Weave system is designed so that very little of the work is done server-side; this allows the server to scale up to many thousands of users. Instead, the client handles most of the heavy lifting, from encrypting and decrypting data to deciding how to reconcile changes between the server's snapshot and local data, depending on the kind of sync to be performed.

Mozilla developer (and early Weave user) Atul Varma took a look around the server's directory structure well before the v.0.3 release and shared his insights on his blog [6]. Unfortunately, for security reasons, you can no longer log in to services.mozilla.com and peek through your user directory as Varma did, but his is an interesting tour for those curious about the server setup. More details are available as reference material on the Mozilla wiki, although because the system is under development, the documentation is incomplete.

One of the more interesting aspects of Weave's design is its use of encryption. All of your data is stored on the server in encrypted form, but the system actually uses three encryption keys in a clever way. A secret, symmetric key is used to encrypt the data itself. Because it is symmetric, the same key is used to encrypt and decrypt the data. This secret key is in turn guarded by a public-private key pair. You and you alone have access to the private key, meaning that you can encrypt the data stored in your account.

The public key allows other Weave users to share data securely with you. The reverse situation is easier to explain. If you want to share your bookmarks with another Weave user, Weave makes a duplicate copy of the secret symmetric key used to encrypt the bookmark data. Rather than locking the duplicate key with your private key (as with the original), the duplicate key is locked with the other user's public key. That ensures that only the other user can access it.

The current encryption scheme uses a separate secret key for each type of data - bookmarks, passwords, tabs, and history. Likely you'll want to share only a subset your bookmarks or tabs with others, so the framework is in place in the Weave API to split stored data into several segments, each protected by a distinct key - this functionality is not exposed yet in the v.0.3 release.

Self-Serve

If you feel daring, you can download the Weave server code and set up your own server. The server is written in PHP, and requires PHP Data Object (PDO) and JSON support. Although Weave is based on the same ideas as WebDAV, it is important that you not enable WebDAV sharing on the location you plan to use as your Weave server - Weave and WebDAV would interfere with each other.

At the Mozilla wiki [5], you will find detailed instructions on setting up a Weave server. At this point in time, the installation process is not automated - you will need to modify the configuration files by hand. A Weave server can use a variety of storage back ends, including SQLite and MySQL. The server-side account creation method is not part of the basic Weave server, but Perl scripts are included in the release bundle, so you can create accounts. When your server is up and running, all you need to do on the client end is change the account credentials and server URL in each Weave client's preferences.

What's Next? Sharing and More

If you don't mind running the latest Firefox betas, Weave is already a winning addition to the daily browsing experience. Browser synchronization tools come and go. Over the years, I have used Firefox add-ons from at least three sources, and all either ceased to receive updates, didn't work across operating systems, or slowly broke for undiagnosed reasons. In a sense, Weave is merely the latest entrant into the browser sync contest, but it is special. First, it is a Mozilla-sponsored project, and second, it is extensible and has the potential to do more than preserve bookmarks and browser info.

As mentioned, Weave's server-side encryption scheme anticipates the addition of another major feature: sharing data between accounts. The simplest case is bookmark sharing, but considering the list of data types sketched out in the Weave roadmap, several are ripe for sharing as well - dictionaries, themes, and extensions, for example.

The same infrastructure that permits secure sharing between users could be used to share information with online services, just one way that Weave can integrate services into the browser. For example, you could sync bookmarks with a social networking site, said Weave's lead developer, Dan Mills, or notify your Dopplr friends automatically when you make travel arrangements that will bring you nearby. "Right now, you basically have to do that by hand," Mills said. "The ticket issuing companies and Dopplr are two separate silos that don't cooperate with each other. Part of what we are trying to do is raise the level of innovation on the services arena by making it so that when a brilliant entrepreneur has an idea for a service that ties in to the browser, they can execute on the area they know best."

"Creating an add-on is a sizable expense and effort from these organizations," Mills added. By building the synchronization and communication infrastructure into Weave, the service provider has less to do. Weave supports machine-readable microformats, and Mills says upcoming builds will integrate with Firefox's built-in microformat parser.

In the short term, the emphasis is on maintaining the simple and useful user experience. The broader Weave services, including third-party service integration, are still being fleshed out. In the meantime, you can take advantage of the sync platform - across machines, across operating systems, and on mobile devices.

INFO
[1] Firefox extensions: https://addons.mozilla.org
[2] Mozilla Labs: http://labs.mozilla.com
[3] Weave homepage: http://labs.mozilla.com/projects/weave/
[4] Fennec on the Weave wiki: https://wiki.mozilla.org/Labs/Weave/InstallWeaveFennec
[5] Weave Server code and documentation: https://wiki.mozilla.org/Labs/Weave/0.3/Setup/Server
[6] Atul Varma's blog entry about Weave data storage: http://www.toolness.com/wp/?p=41