LJ Archive

Upfront

Various

Issue #109, May 2003

diff -u, LJ Index, and more.

diff -u: What's New in Kernel Development

The devfs filesystem work by Richard Gooch may be coming out of the kernel. At the end of December 2002, Adam J. Richter announced a patch to replace devfs with a new mechanism based on RamFS. The new system attempted to mimic devfs' behavior in many ways, though Adam did not intend to include all of the devfs functionality in the RamFS system. He wanted his implementation to be, in part, a cleanup of the devfs interface, so that features used only by few systems might be replaced with other methods. As a result of this restructuring, he managed to reduce the size of the code to one-fourth of what it had been. The devfs system always has been controversial, and Linus Torvalds' decision to include it in the official tree was even more so. Folks like Alexander Viro and others have firmly refused to use it on grounds that it simply wasn't coded well enough. Shortly before Adam announced his own work, Alexander had begun an invasive cleanup and restructuring of the devfs code. Richard, having struggled for years to produce devfs and make it available in timely patches, seems to have vanished entirely from the kernel mailing list.

The sysfs filesystem is intended to be a replacement for /proc and other methods of exposing kernel data to user space. It began as a tool for driver writers, but its use was broadened in 2002 to all parts of the kernel. Since then, there has been an ongoing effort to migrate a variety of other interfaces to sysfs. In January 2003, /proc/cpufreq came under the knife when Dominik Brodowski marked that interface deprecated in favor of a new sysfs interface in the cpufreq core code. Patrick Mochel also had a hand in this, making sure Dominik's work matched up with all the latest sysfs features. Later that month, Stanley Wang sent some code to Greg Kroah-Hartman to replace pcihpfs with a sysfs interface. In this case, however, sysfs was not up to the task as the needed hot-plugging code was not yet fully in place. No problem. Greg coded up the needed sysfs feature and sent it to Patrick.

One day in January 2003, Alan Cox happened to mention that the tty code in the 2.5 tree was badly broken and had been for a while, primarily as a result of locking changes in the kernel preemption code. This came as a surprise to many people, and some wondered why this was the first they'd heard of it, especially because the 2.5 tree was already in feature-freeze, headed for 2.6 or 3.0. Greg Kroah-Hartman looked at the problem and was horrified. He said it was not going to be easy to fix and was most likely something for the next development tree. But Alan said this wasn't an option, because the tty code was broken already and had to be fixed before the next stable series.

Traditionally, the Linux kernel has been compilable only with the GNU C compiler, and even then it often has been necessary to use a particular version of the compiler to compile particular versions of the kernel. The kernel always has depended on GCC extensions, and the relationship between kernel and compiler has been intertwined for years, like an old married couple. Therefore, various people were shocked to learn that the kernel also could be compiled with Intel's C++ compiler, icc. Apparently, Intel has had this as a goal for quite some time, and they've even submitted patches to Linus with the sole purpose of enabling their compiler to handle the kernel source tree.

It's always nice to learn that the feature you desire already has been implemented. According to the documentation (at least as of late January 2003), the only filesystem with quota support was ext2. However, apparently work has been going on behind the scenes, because ReiserFS, ext3, UFS and UDF now support quotas.

—Zack Brown

File access statistics:

www.hszk.bme.hu/~nm127/file_statistics

This utility scans any portion (or all) of the filesystem tree and provides fairly detailed statistics regarding the files on that system. If you happen to be running Debian or a Debian-based system, such as Knoppix, you can receive even more information on the associated dpkg files. This program uses the access times rather than creation or modification times to tell you how “old” or stale a file is. Chances are, files not accessed during the past five years are either historical archives or cruft. Requires: Perl.

—David A. Bandel

Football Manager: www.autismuk.freeserve.co.uk

Football Manager is a game where you are the manager of a soccer team. Graphics are crude, but the game is a lot of fun. It's a game of strategy where you buy and sell players and choose who will play the game this week. Once you've done your job, sit back for 30 seconds to watch a few shots at the goal and see who won. Then, see your team's rating rise or fall compared with other teams in the league. If I don't remove this game I'll never get any work done—it's more addictive than Adventure. Requires: libSDL, libm, libX11, libXext, libdl, libpthread, glibc.

—David A. Bandel

GNU Pilot Logbook Pro:

ftp.stampede.org/skibum/gplbp/gplbp.tar.gz

If you're a pilot, you know maintaining a logbook is not a big chore. But, when someone wants to know how many hours of which type you have, it becomes a little more difficult. This logbook is like the professional logbook for pilots with all the entries you'll need, plus two user-definable fields. With one click you can see all totals to date. And, by running a small script on the data file (you'll have to create that yourself), you can create a data file for 60 or 90 days back to see how your totals are for currency. Requires: libgtk, libgdk, libgmodule, libglib, libdl, libXi, libXext, libX11, libm, glibc, pilot's license and airplane (last two optional).

—David A. Bandel

Hardware Lister:

ezix.sourceforge.net/software/lshw.html

This hardware lister shows quite a bit of detail, including IRQ, module used and more for cards and other hardware. If you need a great quantity of detail on a system for an inventory, you might want to look at this program. About the only thing missing is the MAC address on the network cards, but that's easy enough to get. Requires: libstdc++, libm, libgcc_s, glibc.

save space.

—David A. Bandel

LJ Index—May 2003

  1. Bottom price in thousands of dollars of the new SGI Altix 3000 high-end Linux servers: 30

  2. Top price in millions of dollars of the new SGI Altix 3000 high-end Linux servers: 1

  3. Number of old SGI machines replaced by Dells running Linux at Sony Pictures Imageworks: 600

  4. Number of desktop Linux systems now selling at Sams Club's on-line store: 1

  5. Price of the desktop Linux systems now selling at Sams Club's on-line store: $297.95

  6. Number of different Linux systems (all Microtel) now selling at Wal-Mart's on-line store: 33

  7. Number of different Lindows-based systems: 15

  8. Number of different Mandrake-based systems: 9

  9. Number of different Lycoris-based systems: 9

  10. Bottom price for a Linux (Lindows) system at Wal-Mart's on-line store: $199.98

  11. Top price for a Linux (Mandrake) system at Wal-Mart's on-line store: $648.00

  12. Millions of dollars the Japanese government plans to spend on open-source Linux development for consumer electronics goods in the next fiscal year (starting April 1, 2003): 8.3

  13. Thousands of dollars the Japanese government plans to spend in the next fiscal year to study switching its own computers to Linux: 416

  14. Position of Running Linux among O'Reilly and Associates best-sellers: 1

  15. Number of Running Linux copies sold: 200,000

  16. Current minimum percentage of Linux server shipments, according to Meta Group: 15

  17. Current maximum percentage of Linux server shipments, according to Meta Group: 20

  18. Linux share of server shipments by 2006 or 2007, according to Meta Group: 45

  19. Multiple of performance improvement Reuters Market Data Service gets out of its new Red Hat Linux/HP/Intel systems over earlier proprietary platforms: 2-5

Sources

1-3: Los Angeles Times4-5: samsclub.com6-11: walmart.com12-13: Associated Press14-15: “Running Linux in a New World” by Russel J. T. Dyer (www.linuxjournal.com/article/6617)16-18: Meta Group, Inc.19: Wall Street & Technology

Magic Notebook:

www.jonathanscorner.com/etc/magic_notebook

Using Magic Notebook is like keeping notes in a notebook, except you use a web interface rather than a pen or pencil. This program can be accessed from anywhere you can reach your web server and can be run normally or encrypted. The notes are stored on your filesystem as HTML files, so if you don't want to use the web interface, the notes are still there. Requires: web server that can serve up CGI scripts, web browser.

—David A. Bandel

I work on a number of servers and don't install X on most of them, so I'm always looking for command-line programs that can replace X programs. Pebrot is a Python version of MSN Messenger that runs without X, like the UNIX talk program. This makes things easier if you find it necessary to run the program remotely or don't have X installed. Requires: Python.

—David A. Bandel

Server Status: www.the-den.org/status

This is yet another program that allows you to keep an eye on servers and their status. Although it does require X, it's clean, fast and simple. A number of such programs require SNMP; however, this requires only basic network services. You can leave it running on any system with Perl and Tk and see instantly if you have a problem with a critical service. It won't send you e-mail, but it automatically updates every 60 seconds (configurable) and is easy to read. Requires: Perl, Perl modules IO::Socket, Tk, Tk::Checkbutton, Tk::Menubutton, Tk::Optionmenu.

—David A. Bandel

They Said It

The problem with intellectual property law is that it tries to take something that is extremely difficult to define and put hard definitions around it. It's not a system that we want to try to embed in cyberspace in the early days of this development....We're creating the architecture, the foundation for the social space where everybody in humanity is going to gather. And if we jigger the foundation design to suit the purposes of organizations that will likely be dead in 15 years, how shortsighted is that?

—John Perry Barlow

It's hard to find successful adults now who don't claim to have been nerds in high school.

—Paul Graham

Linux servers are taking on new roles in enterprise computing, moving from the web-centric workloads, where they are already well established, and moving into application-serving and database-serving workloads. This move is being made possible as ISVs (independent software vendors) port more applications that formerly had been running only on UNIX servers and Windows servers to Linux servers. IDC expects that Linux servers will continue to evolve, both in “scale out” clustered configurations for technical and commercial computing and in “scale up” configurations for larger databases with a single-system image.

—Jean S. Bozman, research vice president of IDC's Global Enterprise Server Solutions Group

Ericsson Releasing TIPC to Open Source under GPL: One Step Ahead toward Building Carrier-Grade Linux Clusters

Ericsson released the source code for TIPC (Telecom Inter-Process Communication) to the Open Source community on February 3, 2003 under the GNU General Public License (GPL). TIPC is a specially designed protocol for intracluster communication and has been used as a part of Ericsson products for years, deployed at hundreds of sites around the globe. It is now ported to Linux and is implemented as a loadable kernel module.

TIPC is a useful toolbox for anyone wanting to develop or use carrier-grade Linux clusters. It provides the necessary infrastructure for cluster, network and software management functionality.

The functional addressing scheme seems to be unique, as is the subscription services and agile connection concept. The signaling link implementation, providing full load sharing and safe failover over any type of bearer is also an asset.

TIPC features include:

  • Full location transparency: TIPC provides a functional addressing scheme, hiding all aspects of the cluster's physical topology for the application programs. Mapping between functional and physical addresses is performed transparently and on the fly using a distributed, internal translation table.

  • Lightweight, “agile” connections: by avoiding any hidden protocol messages, the message exchange within a transaction, including connection setup, short data transfer and shutdown, can be tailor-made by the user and, hence, be made more efficient. An established connection will react to and report a problem to the application upon any kind of service failure.

  • Generic, adaptive, signaling link protocol: tasks that typically are implemented in the transport layer, such as retransmission, segmentation, bundling and continuity check, are pushed down to the signaling link layer. This makes the link layer more complex but provides better resource utilization and results in a more efficient stack. Signaling links are tightly supervised by a continuity check of configurable frequency and are able to detect and report link failures within a fraction of a second. Failover to redundant links in such cases is handled transparently and is disturbance-free. Signaling links are self-configuring, using a broadcast/multicast neighbour-detection protocol when possible.

  • Performance: TIPC transfers short (< 1KB) single messages between processors, 25-35% faster than TCP/IP and with comparable speed for larger messages. For intraprocessor messages, delivery speed is 75% better. Furthermore, by using the lightweight connection mechanism, a transaction can be performed by exchanging as few as two messages, to be compared with a minimum of nine in TCP/IP. Hence, short transactions, typical in telecom applications, can be performed in a fraction of the time of corresponding TCP transactions.

  • Quality of service: in-sequence, loss-free message delivery can be guaranteed in both connection-oriented and connectionless mode. In case of destination unavailability, nondelivered messages are returned to the sender along with an error code indicating the cause of the problem.

  • Subscription services: it is possible for application programs to subscribe for the availability/non-availability of functional and physical addresses. This means it is easy to keep track of both functional and topological changes in the cluster, as well as to synchronize the startup of distributed applications.

We are planning to write a full technical article on TIPC for LJ in the upcoming months; meanwhile, feel free to contact Jon Maloy (Jon.Maloy@Ericsson.com) to discuss any aspect of TIPC.

Resources

—Jon Maloy and Ibrahim Haddad

email: david@pananix.com

LJ Archive