LJ Archive


diff -u: What's New in Kernel Development

Zack Brown

Issue #266, June 2016

An effort to speed up futex locks for real-time software has hit a snag, though possibly not a permanent one. Thomas Gleixner posted some patches to eliminate collisions between the hash values used to track futex state changes. These collisions didn't break the locks; they just slowed things down. Thomas' code would speed up the threads that used his code, but only if most other threads allowed the collisions to take place. His reasoning was that real-time applications could call his routines, while normal code could do things the old way.

There were various objections. One was that, as Linus Torvalds put it, “the last thing you want is programmers saying 'I'm so important that I want the special futex'. Because every single programmer thinks they are special and that their code is special. I know—because I'm special.”

But his main objection, first voiced by Ingo Molnar, was that Thomas' code introduced an ABI (application binary interface) change that was highly specialized, but that would have to be carried into the future and supported in the kernel on a permanent basis, long after any value it once possessed had disappeared.

The solution, proposed by Ingo, was to enable Thomas' code for all futex use by default. This would have the annoying quality of making the code less useful for real-time applications, because all other applications would see the same benefit. But, it had the benefit of actually being an improvement even if everyone used it equally, and also of avoiding the ABI change.

Binary interface changes are killers. I think Linus would staple his arm to the table before he'd accept a patch that causes existing compiled software to break. Application programmer interfaces (APIs) are different. Those can be deprecated and replaced gradually, as users update their source code to use the new procedures. It's a hassle and takes a long time, but it's doable. ABI changes? Not so much.

The nightmare holy grail of cgroup implementation continues at a breakneck pace. Trying to make a single piece of hardware look like it's actually multiple independent systems, and having that be secure, while letting all software run natively on the original hardware, turns out to be one of the more insane of the many world-changing things that have found Linux at their absolute center.

Adding features to cgroups is like trying to climb up a sheer cliff using only your teeth and the tips of your toes. The cadre of the insane includes Bandan Das, who recently submitted some patches to get workqueues running under cgroups.

Workqueues are primarily a coding construct that allows user code to set certain low-priority tasks in motion, memory allocations and whatnot, without having to wait for them to finish. Workqueues typically are handled by dedicated worker threads that plow through them as system load permits.

One of the features of Bandan's patches was to allow a given workqueue to be handled by a worker thread that was dedicated to the particular virtual server that was associated with that workqueue. This way resource accounting wouldn't break, and the underlying system could track properly which virtual server should be given how much RAM, CPU time and so on.

Bandan's patch didn't encounter the kind of byzantine security objections that often hit cgroup features, but it did exhibit some unexplained slowdowns. Once those are resolved and Bandan submits the code for actual inclusion in the kernel, the security hawks undoubtedly will descend from their watchful perches for feeding time.

Shuah Khan has been working on stopping media devices from periodically hanging the system under certain circumstances. Apparently, some of the existing media device code would allow users to release a device while it was still in use, after which bad things would happen, up to and including a full-on system crash. She posted some patches to implement a proper sequence of events of media device shutdown that also would account for the fact that any given media device may have multiple users and various pieces of software waiting to grab it.

During the course of discussion, Shuah, Takashi Iwai and Mauro Carvalho Chehab discussed how to organize the work so that it could be developed in an ongoing way and possibly made useful for more than just media devices. They planned to nail down the API so user code could start using it, then to begin to flesh out to fixes and features, and finally to put the whole thing into git, rather than have it be just a set of patches coming in through e-mail.

Jens Axboe hurled his keyboard across the room, ran to the window and threw it open and shouted into the dark of night, “Since the dawn of time, our background buffered writeback has sucked!!!!!!!” Enough was enough. He was sick of waiting for background writes to finish while managers and users screamed at him that their production systems had locked up. Not only that, but his Chrome was slower too.

He posted some patches to relegate background buffered writeback to its proper place in the universe—a background process that doesn't interfere with user activity.

Holger Hoffstätte loved the code and backported it to Linux 4.4.x, reporting wonderful smoothness. Dave Chinner, on the other hand, concocted a test suite that was able to expose some performance problems with Jens' code that made the system a bit worse than it had been before. He and Jens worked on reproducing and fixing that issue together.

Android Candy: More Life Gamification

Shawn Powers

Issue #266, June 2016

You might remember a couple months ago my mention of Habitica, which is a gamification of your daily to-do list. One of my friends on Twitter mentioned an app he uses on Android called Wokamon, which ties in with your FitBit (or any of several other “step-counter” devices). Based on how many steps you take in a day, your Wokamon grows and evolves into a larger, more powerful...well...Wokamon.

It's absurd, but it's still incredibly fun to see your steps make a little Tamagachi-like digital critter grow. The other aspect of the game is that tapping on the screen will earn you crystals, which can be used to purchase upgrades. Those upgrades make your Wokamon grow faster. You also can watch short ads to get a treasure chest full of crystals. That allows you to buy even more things and also gives the game's developers a little ad revenue for their efforts.

I have no idea whether Wokamon is something I'll keep playing with, because there doesn't seem to be much in the way of competition. Maybe that's just because I'm the only person I know in real life who uses it. Still, it's fun for now, and it has forced me to consider another project for the future—namely, how to create an automated “screen tapper” so I can earn crystals all night long while I sleep. I figure anything that helps me think of crazy future projects is worth my time. If it sounds interesting to you, just search for “Wokamon” in the Google Play store. It's free and oddly entertaining.

Non-Linux FOSS: Screenshotting for Fun and Profit!

Shawn Powers

Issue #266, June 2016

I do a lot of my day-job work on a Windows computer. Part of this involves taking screenshots for training purposes. For years, I've used the built-in “Snipping Tool” that comes with Windows, but I've always hated it.

The other day, I happened across an open-source tool for Windows that launches with a tap of the “Print Screen” key, and it offers a flexible, easy to use screenshot process. You can save the screenshot, but my favorite part is that it can be dumped directly into a program like Paint.NET (my favorite Windows graphics editor, and it's also open source).

If you are on Windows and wish you had a simple way to take a screenshot, check out Greenshot today. It's open source and works amazingly well: getgreenshot.org.

Ubuntu MATE, Not Just a Whim

Shawn Powers

Issue #266, June 2016

I've stated for years how much I dislike Ubuntu's Unity interface. Yes, it's become more polished through the years, but it's just not an interface that thinks the same way I do. That's likely because I'm old and inflexible, but nevertheless, I've done everything I could to avoid using Unity, which usually means switching to Xubuntu. I actually really like Xubuntu, and the Xfce interface is close enough to the GNOME 2 look, that I hardly miss the way my laptop used to look before Unity.

I wasn't alone in my disdain for Ubuntu's flagship desktop manager switch, and many folks either switched to Xubuntu or moved to another Debian/Ubuntu-based distro like Linux Mint. The MATE desktop started as a hack, in fact, because GNOME 3 and Unity were such drastic changes. I never really got into MATE, however, because I thought it was going to be nothing more than a hack and eventually would be unusable due to old GNOME 2 libraries phasing out and so forth.

I was wrong.

I'm very happy I was wrong, and with the advent of Ubuntu 16.04 LTS, I decided Ubuntu MATE (now officially in the Ubuntu family) was here to stay. It was the first version of 16.04 that I installed, and I've never looked back. Running Ubuntu MATE is like living in an alternate reality where GNOME 3 and Unity were never invented. Imagine if GNOME 2 was still king, and you could tweak your desktop to look exactly like it always looked when using GNOME. It's glorious.

Xubuntu, I still love you. You got me through a rough patch, and I'll forever be grateful. But I'm now an Ubuntu MATE man, and I have no plans of changing. And, if MATE 3.0 suddenly changes everything good about desktop computing again, I'll dust of my Xubuntu installer, and apologize for ever leaving!

Due to its bringing back of the GNOME 2 interface in a stable, updated way, Ubuntu MATE 16.04 gets this month's Editors' Choice award (and also my sincere thanks). You MATE folks are awesome.

Nightfall on Linux

Joey Bernard

Issue #266, June 2016

In my last few articles, I've looked at general astronomy programs that are helpful for many tasks you might need to do in your stargazing career. But, several specific jobs are more complicated and require specialized software to make relevant calculations, so in this article, let's look at Nightfall (www.hs.uni-hamburg.de/DE/Ins/Per/Wichmann/Nightfall.html).

Nightfall is a program that can handle calculations involving binary star systems. It can animate binary star systems, taking into account not only orbital speeds but also rotational motion and the changing shape of stars due to their close positions. You can model what it would look like and what kind of light curves you would register when observing a binary system. You even can take a set of actual observational data and find a best-fit model for the system you are studying.

Most distributions don't include a package for Nightfall, so you need to build it from source. There are several dependencies, so the instructions following assume that you are using a Debian-based distribution. If you are using something else, you should be able to find the comparable packages for your distribution of choice. To install the dependencies, run the command:

sudo apt-get install libgtk2.0-0 libgtk2.0-dev gnuplot

Nightfall also includes the ability to use OpenGL to handle 3D rendering of animations of the binary systems you want to model. If you want to use OpenGL, you also need to install:

sudo apt-get install libgl1-mesa-dev freeglut3-dev 
 ↪libgtkgl2.0-dev libjpeg62-dev

Once all of the requirements are installed, you can install Nightfall itself. You should change directory into a temporary or source directory where you can do the unpacking and build the code. Once you are there, download the latest version of Nightfall with:

wget http://www.la-samhna.de/nightfall/nightfall-1.88.tar.gz

Then, unpack it with:

tar xvzf nightfall-1.88.tar.gz

When you go to configure Nightfall, you probably will want to include the openmp option. This allows Nightfall to use the multiple CPUs you probably have in your machine to speed up the calculations involved. You can build and install Nightfall with:

./configure --enable-openmp make sudo make install

This installs Nightfall under the /usr/local directory. You then can start Nightfall with:

nightfall -U

The -U option is necessary to force the GUI to be used interactively.

Figure 1. The GUI lets you configure all kinds of options within Nightfall to set up your model.

Now that Nightfall is up and running, you will start to see just how much control you have over the model that is being simulated. The first tab is where you can set up the core parameters for your binary system model. You can set the mass ratio and the inclination of the two stars. You also can set the surface temperatures and the Roche lobe filling factors for each of the stars. The temperature helps define their luminosities, and the Roche lobe filling factors define the distortion of the stars.

Depending on the exact conditions, you may have an accretion disk of material around the central star. The second tab lets you set the type of accretion disk (that is, how it interacts with the central star), along with more physical parameters like the inner and outer diameters and the temperature.

The advanced tab allows you to set some less obvious parameters for your model, such as the eccentricity for the orbiting star and whether you also need to model the atmosphere.

Now you can click on the Compute button at the top of the window, and on most modern machines, it goes pretty quickly. Once the calculations are done, you can plot the output from the system you just modeled.

The Output menu item on the menu bar at the top gives you several options on how to display the calculated results. The first entry is PlotCurve, which draws the light curve as seen from a distant observer. Gnuplot is used to draw the actual plot of the visible amount of light that is seen.

Figure 2. Once the calculation is done, you can plot the observed light curve.

The second output option is to select ViewGeometry. This plot shows you how the stars themselves are distorted within their orbits.

Figure 3. You can plot the geometry of the stars within the binary system.

The StarView menu item presents a view of how the binary system would look from your observation point. A control box also pops up that allows you to change the inclination of the orbital plane and the phase of the orbit.

Figure 4. Selecting StarView lets you see the binary system from some distance away.

The RocheSlicer menu item provides a way to visualize just how distorted the stars become. A control box also pops up that lets you select various slices through the star system.

Figure 5. The RocheSlicer lets you investigate the distortions in the stars of your binary system.

The last menu item is the DataSheet option. This option pops up a new window with a text description of the results of all of the calculations that were made.

Figure 6. The DataSheet gives you the detailed numerical results of your simulation.

One of the keys in science is being able to reproduce your results, both experimental and computational. With this in mind, you can save the model you just ran and load it again later. These models are saved as configurations by clicking the menu item File→Save Configuration. You can load previous models by clicking the menu item File→Open config file. If you compiled and installed Nightfall using the instructions above, you will have a number of example configurations available that you can play with as well.

Nightfall is not only useful in determining what a given binary star system would look like, but you also can feed in your own observational data and calculate a best-fit model to those observations. You can load your observational data by clicking the menu item File→Open data file. Again, if you compiled and installed Nightfall following the instructions above, you will have several sample data files that you can load. You probably should click on the menu item File→Clear memory first. Once the data is loaded, select the Data Fitting tab in the main window. You then need to select the Mass and/or Separation buttons as parameters for the fitting. Then, select the FIT with tolerance button, using the tolerance you enter within the text box. You also can choose whether to use simulated annealing or Monte Carlo methods. You may want to do some research to see how active astronomers use these parameters to find best-fit models.

The last thing to notice is that almost everything you can do with the GUI is also doable with command-line options. This means you can generate many different models with different sets of parameters and generate the relevant plots and output data automatically. This way, you can farm out the work to some cluster of machines (but that's moving into the realm of “professional” astronomy and beyond the scope of this article).

When Birdcam Goes Mainstream

Shawn Powers

Issue #266, June 2016

If you read my articles on when I originally set up BirdCam a few years ago, you'll remember I did it with compatibility in mind. At the time of this writing, BirdCam (birds.brainofshawn.com) is simply an HTML page with the JavaScript language to refresh the images constantly, in order to create a low-fps video stream of sorts. One of the frustrations is that if I want to rearrange camera images or add a new camera (check out my Open-Source Classroom column in this issue), it means a lot of complex HTML coding. It also means the “stream” is less and less reliable, because it depends on multiple images refreshing several times a second. I want to do something more powerful.

Enter: YouTube.

My goal is to get BirdCam to stream a live video to YouTube Live, so it can be embedded on Web sites, viewed on mobile devices and even “tuned in” via Roku or Fire TV. I tried in vain to get ffmpeg and/or VLC to stream video to YouTube from the command line, but I couldn't ever get it working reliably. I haven't given up hope, but until then, I'm planning to play with Open Broadcaster Studio.

It's an open-source, cross-platform studio package that allows you to put multiple video streams, text boxes and still images into a single window that is encoded and streamed to YouTube. I couldn't get the OpenGL version required to work on my Ubuntu laptop, but the OS X version worked flawlessly in my test. With a little bit of tweaking, BirdCam 3.0 might be a legitimate 24/7 YouTube stream. If you want to play around with streaming live to YouTube, check out the awesome open-source studio software at obsproject.com.

They Said It

Broadly speaking, the short words are the best, and the old words best of all.

—Sir Winston Churchill

Brains, like hearts, go where they are appreciated.

—Robert S. McNamara

There are no secrets to success. It is the result of preparation, hard work, and learning from failure.

—Colin Powell

You cannot be mad at somebody who makes you laugh—it's as simple as that.

—Jay Leno

It is better to look ahead and prepare than to look back and regret.

—Jackie Joyner-Kersee

LJ Archive