Zack's Kernel News



The Linux kernel mailing list comprises the core of Linux development activities. Traffic volumes are immense, often reaching ten thousand messages in a given week, and keeping up to date with the entire scope of development is a virtually impossible task for one person. One of the few brave souls to take on this task is Zack Brown.

Our regular monthly column keeps you abreast of the latest discussions and decisions, selected and summarized by Zack. Zack has been publishing a weekly online digest, the Kernel Traffic newsletter for over five years now. Even reading Kernel Traffic alone can be a time consuming task.

Linux Magazine now provides you with the quintessence of Linux Kernel activities, straight from the horse's mouth.

GIT

Last month, the direction of kernel version control was completely up in the air, with BitKeeper gone and Linus considering an array of options, none of which seemed entirely satisfactory. In the short span of a few weeks, the landscape has changed, and git (pronounced with a hard `g') has emerged not only as Linus's likely choice for ongoing kernel work, but also as a great advance in free revision control software, leaving everything else in the dust. Even existing versioning tools like arch and darcs are starting to integrate with git at their back end. Something very big is happening.

When Linus started work on git, he designed it as a filesystem with low-level commands to track the content of a given directory. As he described them, these commands could be thought of as the `system calls' for basic operations, not a true set of revision control features. It was his hope that a `real' versioning system would be layered on top of git, in the form of scripts, providing higher level commands, that would do more of what people expected.

Currently, Petr Baudis's Cogito project seems most likely to be the high-level git-based tool Linus ends up using for kernel development. But at this stage, anything can still happen.

One may ask, what is going on here? During the entire time Linus used BitKeeper, there were plenty of outraged free software evangelists who were also exceedingly good developers, working to create a free alternative to BitKeeper. Yet they didn't even come close. Now, days after BitKeeper exits the scene, a solid replacement is up and running. What happened? The answer is, a whole lot.

  1. None of the free alternatives had the benefit of knowing exactly what Linus wanted.
  1. Linus designed the git filesystem to be fast, and to allow distributed development. He did not distract himself with more complicated features that seemed to him superfluous.
  1. Everyone pitched in. When Linus announced that he was effectively taking a break from kernel development to work on git, hundreds of very talented developers followed him, working day and night to create an exoskeleton of tools to surround his design.

One of BitMover CEO Larry McVoy's arguments in favor of BitKeeper was that the open source development model was just not suited to creating a difficult tool like BitKeeper, with all its corner cases, ugly implementation details, and "boring" hard work. For years he lorded BitKeeper over the kernel developers as proof that the free software methods were simply insufficient for developing this kind of tool. And for years, no one was able to prove him wrong.

The advent of git has been a completely shocking phenomenon, far beyond anyone's expectation. If there is anything to learn from the history of git and BitKeeper, it will certainly be a good lesson.

Kernel Mentors

The Kernel Mentors project is an idea whose time has certainly come. Matt Mackall started the new `kernel mentors' mailing list with the purpose of training newcomers in the art of getting their code accepted into the main kernel development tree. As Matt puts it, the technical and social hurdles associated with getting code accepted into the kernel can be daunting.

While Matt is certainly correct, most of the difficulties are not arcane. The most obscure problems occur when old-timers give feedback on the "proper" way to implement a given portion of code. Newcomers often do not expect such a pointed critique of their code and do not see the logic in the requested changes - which often involve quite a bit of additional work.

Other difficulties associated with the kernel approval process, such as who to submit a given patch to and how to break the patch into small chunks, pose less frustrating problems, although some particularly large patches have been especially difficult to properly split, and especially frustrating for that reason.

Whatever else might be said, to the uninitiated, kernel development practices can seem opaque. The Kernel Mentors project is a welcome addition to the Linux development community. This project will hopefully shepherd many new developers safely across the craggy peaks.

SCSI Projects Merger

The linux-iscsi and open-iscsi projects have merged into a single iSCSI project, using the open-iscsi code base as their starting point.

Apparently, discussions between the linux-iscsi and open-iscsi development teams have been ongoing for some time, and both sides agreed that joining together into a single development project would give them the best chance of writing a really good SCSI stack for Linux.

After the two development teams settled the issue of which code base to use as a starting point, the next question they addressed was which version control system to use: open-iscsi's Subversion tree or linux-iscsi's CVS repository. It seems that, at least for the near term, the new combined development team will stick with open-iscsi's Subversion tree for now, but with all the recent upheaval surrounding BitKeeper and git, anything is possible.

New ConfigFS Filesystem

Joel Becker has created the ConfigFS filesystem, another interface into kernel space along with ioctls, ProcFS, udev, SysFS, and DebugFS. As Joel has said, "configfs is not intended to replace sysfs or procfs, merely to coexist with them." One of the main goals of the ConfigFS filesystem is to provide an interface to kernel space that is entirely clear and scriptable.

ConfigFS is still in an early stage of development, and some of its implementation details have yet to be fully fleshed out, such as at exactly what point changes to a ConfigFS filesystem actually modify the part of the kernel they influence.

It's not clear that there is a true need for yet another filesystem-based interface into the kernel. Apparently ConfigFS was inspired by the fact that none of the existing solutions, including the latest attempt, SysFS, are quite satisfactory.

Kernel.org Hardware Upgrade

Hewlett-Packard has donated a pair of Proliant DL585 quad Opteron servers, with 24 GB of RAM and 10 TB of disk space using a pair of MSA-30 arrays for each server, to kernel.org. The site announced the deployment of the first server on April 4, and the second server went live later in the week. By April 9, both the new servers were in full production.

kernel.org is also known as the Linux Kernel Archives. As you may already know, the kernel.org site is the primary location for the Linux kernel source, although, as a message on the home page states, the website has "much more than just kernels."

The upgrade comes in answer to ever-increasing bandwidth problems for the site, as with each new Linux release, kernel.org continues to be hammered by the entire world. Both new servers went into production with only minor glitches.