Fire, Brimstone and Real-Time Linux

Though fewer than 10% (or less) of embedded Linux applications actually require real-time enhancements or add-ons, articles and discussions on that subject invariably spark passionate debate. For whatever reason, the topic is a magnet for what might be characterized best as a sort of religious fervor. So it is with some tiptoeing through a minefield that this month's embedded column begins with several topics related to real-time Linux. Hold on to your hats.

In their recently published research report “Linux's Future in the Embedded Systems Market”, Venture Development Corp. (VDC) concluded that the availability of real-time solutions for Linux are needed to accelerate the broad adoption of Linux in embedded designs.

The report analyzes the current size and future growth of the worldwide market for embedded Linux software solutions (www.vdc-corp.com). According to the study, the dominant factors favoring the use of Linux in embedded projects include the availability of source code, royalty-free options and reliability. On the other hand, VDC found “real-time limitations” to be the most common issue cited by embedded developers as inhibiting their adoption of Linux for future projects.

Here is a list, in ranked order, of what developers told VDC were their main concerns with respect to using Linux in embedded systems and devices:

The usual real-time debate erupted shortly after Embedded Linux Journal published the third article in a series by Kevin Dankwardt on real-time Linux technologies. Here's an outline of the sequence of reactions and responses from key players in the real-time Linux market, followed by a pointer to where you can read them all on-line:

You can access all the above, including the original three-part ELJ article by Kevin Dankwardt, from this summary page: www.linuxdevices.com/news/NS4265889552.html.

Clark Williams of Red Hat wrote a whitepaper titled “Linux Scheduler Latency”, in which he compares the performance of two popular methods for improving Linux kernel preemption latency—the preemption patch pioneered by MontaVista and the low-latency patch pioneered by Ingo Molnar—and discovers that the best approach might be a combination of the two (www.linuxdevices.com/articles/AT8906594941.html).

The ADEOS Project announced its first release of ADEOS, a hardware abstraction layer that allows a real-time kernel and a general-purpose kernel to coexist on one CPU. The announcement claims that “RTAI will eventually use ADEOS services, thus offering a real-time kernel based on a principle clearly different from the 5,995,745 US Patent”, aka the “RTLinux patent” (www.freesoftware.fsf.org/adeos).

Victor Yodaiken published a whitepaper that points out the disadvantages of dealing with the issue of “priority inversion” in real-time systems by means of a commonly used method known as “priority inheritance”. Priority inversion refers to the situation when a scheduled task must wait for a lower-priority task to complete. In the whitepaper, Yodaiken describes “the classical nightmare case” of priority inversion as being “when a low priority task owns a resource, a high priority task is blocked [and] waiting for the resource, and intermediate priority tasks keep preempting the low priority task so it cannot make progress toward releasing the resource.” Yodaiken says the technique of priority inheritance is intended to allow:

However, “priority inheritance is neither efficient nor reliable”, the paper argues, and its “implementations are either incomplete (and unreliable) or surprisingly complex and intrusive”, asserts Yodaiken (www.linuxdevices.com/articles/AT7168794919.html).

Red Hat discontinued development of its eCos open-source embedded operating system and is rumored to have discontinued many of its embedded Linux development efforts.

Asked whether Red Hat was still in the embedded market, Red Hat Chief Technology Officer Michael Tiemann replied, “Yes—but our strategy is to expand the scope of Linux to encompass the embedded space.” Expanding on this point, Tiemann said “the embedded world that [Red Hat is] most interested in needs a Linux platform that extends into the embedded space”, as opposed to a unique version of Linux tailored specifically to embedded devices.

These statements explain Red Hat's move away from products like eCos, an open-source embedded operating system that Red Hat inherited via its acquisition of Cygnus Solutions in late 1999, and µClinux, a version of Linux geared toward resource-constrained, “deeply embedded” devices that Red Hat got involved in via its mid-2000 acquisition of Wirespeed. In addition, Tiemann's reduced investment statements explain Red Hat's developing embedded-oriented tools, like the Embedded Linux Developers Suite (ELDS).

Tiemann elaborates on what he means by “a Linux platform that extends into the embedded space” in a guest editorial for LinuxDevices.com entitled, “How Linux will Revolutionize the Embedded Market” (www.linuxdevices.com/articles/AT7248149889.html). That editorial basically unfolds a strategy that treats the embedded market as a portion of a continuum—one that increasingly demands greater consistency of technology, APIs, middleware and tools, due to growing end-to-end application connectivity and interoperability.

The best approach, Tiemann argues, is to offer a solution that meets the needs of the entire range of requirements within a single platform, rather than providing a unique version of Linux specially tailored to embedded systems. Quoting from the conclusion,

Finally, here are links to three excellent on-line reviews of Craig Hollabaugh's well-received book, Embedded Linux:

Embedded Linux is published by Addison-Wesley Professional (ISBN: 0672322269) and is available at various on-line retailers. Amazon.com provides 43 sample pages of the book on their web site.