Running Network Services under User-Mode Linux, Part I

In my May 2006 Paranoid Penguin column, I expounded on the virtues of Debian 3.1's excellent support for virtualization environments, including User-Mode Linux. In that same issue, in the article “User-Mode Linux”, Matthew Hoskins gave a quick-and-dirty recipe for test-driving User-Mode Linux using prebuilt UML kernels and root filesystem images.

Did those articles whet your appetite for a more comprehensive and security-focused look at using UML? If so, you're in luck; for the next couple of columns, we're going to dive into the User-Mode Linux experience and cover every step (including every command) for creating your very own User-Mode Linux containers for network services.

So, why are we doing this, and what do we hope to achieve? As I've said before in this space, virtualization is similar to the concept of the chroot (changed root) jail. It encapsulates a process or dæmon into a subset of the operating environment in which it resides, in a manner that makes it much harder for attackers to get at the underlying environment should they succeed in compromising that process or dæmon.

Whereas chrooting restricts a process to a subset of the host system's real filesystem, virtualization restricts the process to a complete virtual machine running within the host (real) machine. This includes a completely virtualized hard disk, memory and kernel, and even virtualized system devices, such as network and sound cards. In the case of User-Mode Linux, this is achieved by running a guest (virtual) kernel as a user-space process within the host (real) kernel.

Because both guest and host kernel are Linux kernels, virtualization in User-Mode Linux is fast and efficient. And, because the guest kernel does not need to run as root under the host kernel, even attackers who compromise some dæmon on the guest system and escalate their privileges to root (on the guest system) and somehow manage to gain shell access to the underlying host system will have achieved only unprivileged access to that host system.

This does not make it impossible to gain root access to the host system. If attackers do make it as far as shell access on the host, they may be able to escalate their privileges via some local privilege escalation vulnerability in the host's kernel or some user-space program on the host. (Remember: no vulnerability is strictly local on any networked system!) It does mean, however, that it's more difficult for attackers to get to the point of being able to exploit such a vulnerability, especially if it isn't also present on the guest (virtual) system.

This brings us to our design goals. The guest machine should be as bare-bones as possible with respect to installed software—both to minimize resource utilization and to minimize its potential for compromise (its attack surface). If, for example, the guest machine is to act as a DNS server, it should have basic network support, BIND (or some other DNS server package) and very little else. No X Window System and no Apache—nothing else not directly related to DNS services.

If you're really paranoid, you even can skip the Secure Shell dæmon and instead administer the system via a virtual serial console. (Though allowing SSH from only authorized IP addresses, such as that of the host system, might be a more reasonable middle ground.) You could also run User-Mode Linux under SELinux; however, that's beyond the scope of this series of articles.

If a single bastion server is to host multiple network services—for example, Apache and BIND—you could run two different guest systems on the same host: one containing only Apache and its dependent packages and another containing only BIND et al. In this way, a vulnerability in BIND would not lead directly to Web site defacement. Conversely, a poorly coded Web application would not necessarily lead to DNS tampering.

In summary, our two design principles will be to run one virtual machine per major network service and to make each virtual machine as minimal and secure as possible. The end result will (hopefully) be a very compartmentalized bastion server that places as much defensive abstraction as possible between attackers and total root compromise.

For the remainder of this series of articles, I use the example of a single guest system running BIND. Both guest and host system are based on Debian 3.1, because Debian is so popular for UML guests (it lends itself to stripped-down installations—a trait it shares with Slackware). However, most of what follows also applies to other Linux distributions on both host and guest.

Our tasks are:

First, you need to make sure you've got the right kind of kernel on your host system. You very likely may need to compile a new kernel.

On the one hand, some Linux distributions already have User-Mode Linux compiled into their default kernels. On the other hand, your distribution of choice may or may not also have the skas (separate kernel address space) patch compiled in as well. It is, in fact, somewhat unlikely that your default kernel has skas support. Although the Linux kernel source code has included UML support since version 2.6.9, the skas patch is still maintained separately (and Linus has resisted its inclusion).

The skas patch is important. It greatly improves UML performance and security by running the guest system's kernel in separate address space from its other processes (just like the host's kernel does). The User-Mode Linux Web site's skas page on SourceForge provides a more detailed explanation of why you need skas (see the on-line Resources).

To obtain kernel source code, your best bet may be simply to install your Linux distribution's kernel-source package. Take care, however, that your distribution provides a kernel version of 2.6.9 or higher, because UML support is included from 2.6.9 onward, and prior UML patches had security vulnerabilities.

Because Debian 3.1 still uses kernel version 2.6.8, I decided not to use the official Debian kernel packages and instead downloaded the 2.6.17 kernel from kernel.org. I did, however, install the kernel-package package, which provides tools for generating Debian packages from official kernel source.

Besides kernel source code, you need the skas patch, the latest version of which is available on Blaisorblade's site (see Resources). Be sure to download the patch version that corresponds to the kernel source code you're about to patch.

On my Debian host, I unpacked my official source code to /usr/src/linux-2.6.17.3, renamed the source code directory to /usr/src/linux-2.6.17.3-host and copied the skas patch tarball (skas-2.6.17-rc5-v9-pre9.patch.bz2) to /usr/src. I then changed ownership of the directory /usr/src/linux-2.6.17.3-host to a nonroot account. (Adhering to the principle of never being root unless you really need to, we're going to do most of this kernel build as an unprivileged user.)

Here are the commands I executed as root:

host:/usr/src/# tar -xjvf ./linux-2.6.17.3.tar.bz
host:/usr/src/# mv ./linux-2.6.17.3 ./linux-2.6.17.3-host
host:/usr/src/# chown mick ./linux-2.6.17.3
host:/usr/src/# su - mick

To apply the skas patch, I then navigated, as my nonroot user, to /usr/src/linux-2.6.17.3-host and ran the following command:

host:/usr/src/linux-2.6.17.3-host$ bunzip2 -c
 ↪../skas-2.6.17-rc5-v9 -pre9.patch.bz2 | patch -p1

Next, from the same directory, I issued the command make menuconfig. When setting up the kernel configuration for User-Mode Linux, the defaults generally are fine, though you should ensure that the configuration matches your host's hardware. In addition, it's probably prudent to double-check the following settings:

From this point on, the process is the same with any other kernel build: issue the commands make bzImage and make modules;. Then, become root and issue the commands make modules, make modules_install and make install. (Or in the case of Debian, use the make-kpkg command to achieve the same thing, and run dpkg to install the resulting kernel package.)

Once your new host kernel is installed, reboot your system. Your host system is now capable of running User-Mode Linux guest systems.

Okay, we've got UML host capabilities, but we still need a guest kernel to run. This process is somewhat simpler than the host-kernel build, because we don't need the skas patch.

First, navigate back to the directory in which your Linux kernel-source tarball resides, and unpack it a second time. Remember when we renamed the unzipped source code directory? This was so we could unpack the kernel tarball a second time. We need to build our host and guest kernels in separate source trees.

On my Debian test system, therefore, I unpacked the source tarball to /usr/src/linux-2.6.17.3, and this time, renamed it to /usr/src/linux-2.6.17.3-guest. Again, change ownership of this directory to a nonprivileged user, and change your working directory to it.

Again, at this point we can skip the step of applying the skas patch. Because we're going to compile our kernel for the special um (User-Mode Linux) architecture rather than for a real architecture like x86, I recommend you prepare your source code tree with the following three commands:

host:/usr/src/linux-2.6.17.3-guest$ make mrproper ARCH=um
host:/usr/src/linux-2.6.17.3-guest$ make defconfig ARCH=um
host:/usr/src/linux-2.6.17.3-guest$ make menuconfig ARCH=um

The make mrproper command clears out any configuration and object files in your source tree; make defconfig generates a fresh default configuration file appropriate to the um architecture; and make menuconfig, of course, gives you the opportunity to fine-tune this configuration file.

Pay particular attention to the following:

Once you've saved your new configuration file, you can compile the kernel with this command (without first becoming root; execute this as an unprivileged user):

host:/usr/src/linux-2.6.17.3-guest$ make linux ARCH=um

Note that I did not tell you to make a zipped or bzipped image. Remember, you're going to be running this kernel as though it were a user-space command, so it shouldn't be compressed. The finished kernel will be located in the top-level directory of your source tree (/usr/src/linux-2.6.17.3-guest in the above examples) and will be named linux—you'll probably want to rename it to something more descriptive, such as uml-guestkernel-2.6.17.3. You'll also probably want to move it to the directory from which you intend to run it—perhaps something like /usr/local/uml/.

By the way, don't be scared by the size of your guest kernel file. Most of that bulk is symbol information that will not be loaded into memory when you execute it.

Your host system now fully supports User-Mode Linux, and you've got a guest kernel image to run. The next step is to obtain or create a root filesystem image to use with the guest kernel. That's where we'll pick up again next time!

Resources for this article: /article/9260.