By Klaus Knopper
Aufs [1] is a stacked filesystem similar to UnionFS [2]. One very common use for aufs is adding "temporary writing" capability to a filesystem residing on read-only media. I use aufs in Knoppix to join read-only data from the compressed Knoppix file (usually located on a CD or DVD) with a read-write filesystem on ramdisk or a USB flash drive. A stacked filesystem creates a stack of existing directories that are "transparent" to the user: Each access to a file is tried for each directory of the stack until it succeeds or the end of the stack is reached.
Even if you aren't building your own Live system, you still might have a practical use for aufs as a means of adding virtual read-write access to files stored on a read-only device. In this article, I describe how to add temporary write capability to a read-only device using aufs.
Aufs2 comes in two forms: a complete Git-based kernel with aufs2 included and a "standalone" version that contains only the changes that have to be patched into the kernel source.
Because checking out the complete aufs2-patched kernel is probably not the most common scenario and because compiling the ready-to-build version is no real challenge, I will describe the procedure for adding the standalone version of aufs2 to the vanilla kernel source, which you can always get from the Linux Kernel Archives [3].
The following examples are based on kernel version 2.6.29, which resides in a subdirectory of the current working directory and is called linux-2.6.29.
If you have never used git before, it might be missing from your development system. On Debian-based systems, the command for installing it is
aptitude install git-core
For other package management systems, consult your vendor documentation.
The command to check out aufs2 as standalone patch for kernel 2.6.29 is:
git clone http://git.c3sl.ufpr.br/pub/scm/aufs/aufs2-standalone.git aufs2-standalone.git
After this command, aufs2-standalone.git will be a local copy of the aufs2-standalone Git repository.
The command
git checkout origin/aufs2-29
checks out the aufs2 version for kernel 2.6.29 (note the 29 at the end of the command parameters) and defines it as the current working version. You might see a Git note or warning about origin/aufs2-29 not being a local branch, which you can safely ignore at this point.
Listing 1 shows the current working directory aufs2-standalone.git (ls -l). The directory fs includes the aufs2 source, and the include directory holds the necessary kernel include files for aufs2. Both must be copied into the kernel source directory.
| Listing 1: aufs2-standalone.git (ls -l ) |
01 -rw-r--r-- 1 knopper users 9222 15. Apr 00:51 aufs2-standalone.patch 02 -rw-r--r-- 1 knopper users 40296 15. Apr 00:51 ChangeLog 03 -rw-r--r-- 1 knopper users 661 15. Apr 00:47 config.mk 04 -rw-r--r-- 1 knopper users 17990 15. Apr 00:47 COPYING 05 drwxr-xr-x 4 knopper users 104 15. Apr 00:51 Documentation 06 -rw-r--r-- 1 knopper users 1481 15. Apr 00:51 ecryptfs.patch 07 drwxr-xr-x 3 knopper users 72 15. Apr 00:51 fs 08 drwxr-xr-x 3 knopper users 72 15. Apr 00:51 include 09 -rw-r--r-- 1 knopper users 617 15. Apr 00:47 Makefile |
If your kernel source is located one directory above in linux-2.6.29, enter:
cd ../linux-2.6.29
cp -a ../aufs2-standalone.git/{fs,include,Documentation} .
cp ../aufs2-standalone.git/config.mk fs/aufs/
The aufs2 README describes aufs2 installation differently, but I find that the method of copying all additional aufs2 source files directly to the kernel source tree is a convenient way for building ready-to-use Linux kernel image packages in Debian containing aufs2 without the need to create a modules/aufs subdirectory with a corresponding external module infrastructure.
In the aufs2-standalone.git directory (Listing 1), you might have noticed a file called aufs2-standalone.patch. This file contains changes that have to be made to various filesystem-related places of the original kernel source. The kernel's Makefile and configuration system also require that some changes be made to them so that aufs2 will be visible as a new filesystem in make menuconfig. From the current directory, linux-2.6.29, use patch as follows,
patch -p1 < ../aufs2-standalone.git/aufs2-standalone.patch
which produces the output shown in Listing 2.
If you see no lines to indicate that an error has resulted from the patch, then everything is now set up and ready to compile.
| Listing 2: Patching Files |
01 patching file fs/Kconfig 02 patching file fs/Makefile 03 patching file fs/namei.c 04 patching file fs/namespace.c 05 patching file fs/open.c 06 patching file fs/splice.c 07 patching file fs/super.c 08 patching file include/linux/Kbuild 09 patching file include/linux/lockdep.h 10 patching file include/linux/namei.h 11 patching file include/linux/splice.h 12 patching file security/device_cgroup.c 13 patching file security/security.c |
When you call the command
make menuconfig
to set up or change kernel options, you will find aufs2 beneath the File system option group, in the Miscellaneous filesystems submenu (see Figure 1).
If aufs is missing and everything else has succeeded so far, you should check to see whether the experimental features of the kernel have been enabled in the configuration.
The Maximum number of branches line shown in Figure 1 defines how many directories aufs can combine into one virtual directory. The Use inotify... line allows you to modify a branch directly and have the changes appear immediately in the mount point (which otherwise could give unexpected results, in that aufs does not monitor each file for changes until it is changed on the mount point). The NFS-exportable line enables some features that are needed for exporting directories via the NFS network filesystem, and the Ramfs as an aufs branch line is only needed if the initial ramdisk stays as the root filesystem after booting and is used as a writable branch for aufs.
The online help in the Documentation directory provides helpful details about each option.
| Complexities |
|
This very simple concept of stacking a writable filesystem with a read-only filesystem becomes complicated when you consider the need for making changes inside the stack. For example, if a file residing in a ready-only directory branch is deleted, how do you make it go away? In the case of file deletion, a new "hidden" file (so-called "whiteout") is created in the writable branch to tell aufs to act as if the original file ceased to exist. When writing to a file residing in an unwritable branch, a copy of the changed file must be created on the writable branch. Even more complicated cases occur when handling operations like concurrent file access, differing permissions, and access methods for diverging files on different branches. Therefore, handling files in a stacked directory tree is not as easy as you might think, and in fact, the aufs source code is about the same size and complexity as the source code of a regular Linux disk-based filesystem. |
| A New Aufs |
|
Recently, further development of aufs version 1 has stopped in favor of aufs2, which is a kernel extension for adding filesystem stacking capabilities directly into the kernel tree. At the same time, aufs author Junjiro Okajima switched from using CVS for source code management to Git, which is also used by the Linux kernel maintainers. According to the developer, the primary goals for creating aufs2 were to provide easier and wider review of the code and to make the source files simpler and smaller. Aufs1 consisted of several patch files that were highly dependent on kernel compile-time options and versions, whereas aufs2 just uses the kernel's own configuration system and compiles more easily. |
If you have enabled aufs as a module (with the m option), aufs.ko will be built when you build all kernel modules with the command:
make modules
Alternatively, you can build only the aufs module while skipping all others with:
make ./fs/aufs/aufs.ko
Because some exported filesystem functions changed when you patched the kernel, you might still need to recompile other modules, as well as the static kernel, so recompiling and installing the full kernel with all modules is recommended, followed by a reboot. After that, you can load the aufs module (if it is not statically compiled in) with:
modprobe aufs
If you mount a hard disk partition read-only
mount -r /dev/sdb1 /media/disk
and add a directory for writable data somewhere (/tmp should be sufficient)
mkdir /tmp/cow
you can now join the two directories as a virtual, writable directory. All the files will come from /media/disk first (the read-only branch), and behind the scenes, changed or newly created files will go to /tmp/cow:
mkdir /tmp/aufs mount -t aufs -o br:/tmp/cow=rw:/media/disk=ro none /tmp/aufs
Note that the list of directories associated with an aufs volume is NOT given as "device file" as usual in the mount command, but within the filesystem options (-o) as br:directory1:directory2: ..., wherein all directories but the first should have a suffix =ro to mark them as read-only for aufs. The first directory has =rw for being writable.
Also, it is possible to virtually join the contents of multiple directories as one - for example, if you want to combine a collection of multiple multimedia files in a single directory when they are actually spread across several disks and locations.
To add directories to the stack, use the command
mount -o remount,append:new_directory /tmp/aufs
or remove them with the del mount option. The full set of aufs options, including a description of how to insert, remove, or modify specific branches from an aufs directory stack, is outlined in aufs.5, which is an nroff-formatted man page. To read aufs.5 without installation, use:
nroff -man Documentation/filesystems/aufs/aufs.5 | less
One aufs mount option that can be useful in a Live system is noplink, which disables permanent hard links to files on different branches, thereby saving a lot of valuable space on the writable branch.
Aufs easily creates workarounds to provide virtual write access to read-only media or files residing in read-only filesystems, such as ISO9660 (which supports no write operation, whether the underlying block device is writable or not). Without aufs, then symlinks, bind-mounts, and filesystem restructuring is necessary to support writing to otherwise unwritable files. Setting up a chroot environment to record changes to the filesystem is easy with aufs, because it puts all changed files and a record of modifications to metadata on the writable branch. With this feature, you can create "incremental" installations by adding filesystem history step by step. Also, you can use existing mount points as targets, so the old directory data is "hidden" by the modified data.
Aufs2 users have long backed adding aufs2 to the main kernel. Because of attempts to add overlay capabilities into the generic virtual filesystem layer rather than a separate filesystem, aufs2 likely won't make it into the kernel soon.
| INFO |
|
[1] Aufs: http://aufs.sourceforge.net/
[2] UnionFS: http://www.filesystems.org/project-unionfs.html [3] Linux Kernel Archives: http://www.kernel.org/ [4] Wikipedia on aufs: http://en.wikipedia.org/wiki/AuFS |