The Sysadmin's Daily Grind: Zsync

Vive la Difference

By Charly Kühnast

Zsync - the name is reminiscent of Rsync, the Swiss Army knife of file transfers. The Rsync algorithm is at Zsync's core, but Zsync has a different purpose. Whereas Rsync synchronizes whole filesystem trees, Zsync focuses on downloading single, bulk files.

Imagine a company has tailored a Linux distribution to suit its own needs. They upload an ISO file of the distribution to their Intranet server. Whenever an administrator needs to set up a new PC, they just download the distribution from the server. This is something that occurs quite regularly, as the distribution is updated continually.

Any self-respecting admin would look to avoid downloading the whole ISO file, and would try to just download the delta to the previous version. This is exactly what Zsync does. Besides this, the tool uses HTTP and can thus work around proxies and firewalls. Another advantage of Zsync is that it doesn't place any load on the downloading server's CPU. Deltas are calculated client-side.

Practical Applications

A tarball is available from [2]. After unpacking, do the following to install:

configure && make &&
make install

Apache is also running on the machine. To put Zsync through its paces, all I need is a series of fat files with slightly differing content. I went for the fairly bulky, 200 MB logfile on my web server and copied it to a directory called /www/kuehnast/test, which is visible to Apache. Time to run Zsyncmake:

zsyncmake /www/kuehnast/test/
vhost_access_log

This command line creates a file called vhost_access_log.zsync. I can now start the download process on the client:

zsync http://kuehnast.com/test/
vhost_access_log.zsync

Of course, this step has to download all 217 MB because Zsync does not have a previous version to work with.

At this point I start to twiddle my thumbs waiting for Apache to write the file. If that happens to you, you can take a break and allow some new data to accumulate. Then repeat the procedure: copy the logfile to the webserver directory, run zsyncmake, and then do the following on the client:

zsync http://kuehnast.com/test/
vhost_access_log.zsync

Amazing! This time the file transfer volume is just 16 kb of changes since the last version (see Figure 1)

Figure 1: Zsync transfers only those parts of a large file that have changed, thus saving time and bandwidth.