UpFront

There's a slow effort underway to allow virtually any part of the kernel to be extracted into its own shared library, thus enabling users to use any alternative subsystem they please. There's a long history of this, going back to the debate between micro-kernels and monolithic kernels. Even Linus Torvalds, the proponent of the monolithic kernel, believes it's better to abstract features out of the kernel, so long as it can be done without sacrificing speed, stability and other core requirements.

Most recently, Hajime Tazaki extracted the entire networking stack from the kernel and converted it into a shared library. This wasn't in itself part of a more generalized attempt to do such things, but while no one objected to the idea, there was considerable debate over the right way to architect the extraction, and this led to the thought that Hajime's idea could be extended to other subsystems beyond the networking stack.

Ultimately, Richard Weinberger suggested that the portion of Hajime's code that stubbed out the networking stack so it could be linked with a shared library could be added to the kernel's testing code and used to stub out any arbitrary portion of the kernel.

As it turned out, Antti Kantee had been working on a similar type of thing for NetBSD for the past eight years, but he cautioned that the maintainability issues could rapidly get out of hand if the design didn't aggressively address maintainability from the start. And this, he felt, would require organizing the deeper kernel infrastructure around the need to stub out portions of the kernel to be turned into dynamic libraries. But at that point, he said, the code would have only a very low maintenance cost.

Antti's recommendations were met with some suspicion. Richard felt that the maintainability issues might go deeper even than Antti cautioned, due to the tremendous speed of Linux development relative to that of NetBSD. In the end, Richard suggested—and Hajime agreed—that the best approach would be for Hajime to maintain the code, now dubbed libOS, as a separate git tree himself, to get an exact measurement of how well it could keep pace with the rest of kernel development.

It's not clear whether Hajime's code ever will get into the kernel. It seems a lot of people like the ability it offers, but there are unanswered questions about how well those abilities could be sustained over time. It may take a year or more to get a better sense of these things, and until the kernel folks know more, they'll be unlikely to accept this code into the kernel.

Beata Michalska has been working on generic filesystem event notification—a kernel interface that any filesystem could use to alert the system to various events, such as being remounted read-only. Beata described four basic categories of events: warnings, errors, information and thresholds. Thresholds would include things like the amount of free space dropping below a set minimum. The kernel could respond to filesystem events by triggering any desired response.

In response to Beata's patch, Heinrich Schuchardt suggested that her code should be expanded to cover a range of filesystem scenarios that it didn't yet—for example, distributed filesystems like Lustre, remote filesystems like Samba, and any FUSE-based filesystems also should be supported, he said. He also suggested that thumb drives and any other automounted filesystem also should trigger an event in Beata's code, and the same for filesystems mounted under virtual machines.

Various other folks also offered technical suggestions on how Beata could improve her initial patch, and Beata invited more implementation suggestions for some of the features Heinrich had requested.

One benefit everyone seemed to agree on is that a generic notification system would be highly preferable to each filesystem implementing its own notifications independently. So there was a lot of enthusiasm for Beata's work, although the exact technical details probably will continue to be hammered out for some time.

Google has become the company that we love and can't live without, but at the same time, I think we all worry a little about just how much Google knows about us. With that caveat, it's hard to ignore Google's newest offering: Google Photos.

Using unlimited storage, automatic sorting and face/place/event recognition, Google is taking all the manual work of tagging and hiring googlebots to do the work for us. Is that creepy? Maybe a little. Honestly though, it's hard to scoff at unlimited storage of full-quality photos and videos. Plus, if you're like me, actually finding the time to sort and tag tens of thousands of photos is something you're going to get to “any day now”.

If you're not completely freaked out by Google having access to all your personal photos, give it a try today. It's free and seems to be legit. There's an Android app you can download from the Google Play Store, plus an iOS app and a Web application for using on computers. Head over to photos.google.com, and check it out!

The concept of PortableApps has been around for a long time. It's a great way to take your Windows apps from computer to computer using a USB drive and never worry about being without your favorite program. Honestly, remembering to carry around a USB drive can be a bit of a pain though. Also, USB drives are generally fairly slow, and working directly from the drive can be cumbersome.

Even with those limitations, the PortableApps.com system is an incredible way to keep the apps you want, with your preferences, exactly how you want them. The recent cloud changes are what sparked this mention. Although PortableApps still work great on USB drives, it's also added support for cloud-based services like Dropbox and Google Drive. If you want your programs to follow you around from computer to computer without needing to grab a Flash drive, the new syncing functionality means having all your apps and all your preferences on any computer running Dropbox!

I'm not sure if that makes them cloud apps, or TeleportingApps, or if it's just a really great idea—regardless of what you call the concept, it means fast and efficient computing on the various Windows machines you use on a daily basis. Download the completely free (beer and speech) installer today at portableapps.com.

I have covered several different software packages for doing scientific computation in Linux Journal, but I haven't spent as much time describing available libraries and the kind of work that can be done with those libraries. So in this article, I take a look at SymPy, the Python module that allows you to do symbolic mathematics, and specifically at a utility named GraviPy that is built upon SymPy. With this utility, you can do all sorts of computations that you'll want to do if you're researching general relativity.

You can install both modules using pip with the following command:

sudo pip install sympy
sudo pip install gravipy

Then, you can import everything from both modules with the Python statements:

from sympy import *
from gravipy import *

This way, you will have access to all of the functionality from these modules within the namespace of your own Python code.

When you do work in general relativity, you often need to look at very complicated equations. The GraviPy module includes a function, called init_printing(), that sets up everything you need for pretty-printed equations. You probably will want to call this near the top of your code so that your worksheet is more human-readable.

Let's start with the SymPy Python module. SymPy is designed to give you the ability to do symbolic mathematical computations. With it, you can do things like solve algebraic expressions, rearrange and simplify equations, and even perform symbolic derivatives and integrals. I've looked at SymPy in a previous issue of LJ, so here, I just focus on some of the core parts as a reminder.

The first necessary step is to define the symbolic variables that you are going to use in your calculations. Using the symbols command, you can define them with:

x,y,z = symbols('x y z')

This way, you define the variables that define the three space dimensions in Cartesian coordinates.

If you want to have all four space-time coordinates defined in spherical coordinates, you can use this:

t, r, theta, phi = symbols('t, r, \\theta, \phi')

Remember that you are feeding a string into the symbols function. This means you need to escape any special characters to get the results you are expecting.

These commands will give you the symbolic variables that you can use in expressions. But, general relativity has a special class of variables, called coordinates, that are used to define the space-time itself. There is a class, called Coordinates, that helps define this. Using the spherical coordinates above, you can create the coordinates with the statement:

x = Coordinates('\chi', [t, r, theta, phi])

The four space-time coordinates are stored in the object x. You can access the individual coordinates by using an index. In general relativity, there are two different ways of indexing variables: covariant and contravariant indexes. To look at the element values for the covariant version, you need to use positive index values. Negative index values will give you the contravariant versions. As an example, if you wanted to get the time variable, you would use the following:

x(-1)

Right away, you should notice that this implies that GraviPy uses 1-based indexing rather than 0-based indexing.

Now that you have a set of variables to define the space-time coordinates to be used, let's move on to actual general relativity.

The core part of general relativity is the metric. The metric defines how space-time is shaped. This space-time shape is what defines the gravitational force. The usual phrase that explains what is happening is that “matter tells space-time how to bend, and space-time tells matter how to move”. To define the metric within GraviPy, you need to start by creating a metric tensor object. You can do so with this statement:

Metric = diag(-(1-2*M/r), 1/(1-2*M/r), r**2, r**2*sin(theta)**2)

In this example, you'll notice a new variable, named M, which represents the mass of the matter that is creating this space-time distortion. If it is an item that will remain static, you don't need to do anything extra. But, there is no reason that it should remain static. If it is something that can change, you need to use the symbols command to define it. In its most general form, the metric tensor is a 4-by-4 matrix of elements. The above example is of a diagonal metric where only the non-zero elements are along the diagonal. Once you have this tensor object, you can define the metric based on it with the statement:

g = MetricTensor('g', x, Metric)

This example gives the function the metric definition and the coordinate system to be used.

To access the various elements, you can use indices that follow the same format as above. For example, you could use the following:

g(1, 1)  ->  2M/r-1

For both vectors and tensors, you can use a special index called All that will give you every possible value for the index in question. For example, you can get the entire list of coordinates with this:

x(-All)  ->  [t r theta phi]

You can get all of the elements of the metric tensor with the statement:

g(All, All)

Now that you have a set of coordinates and a metric tensor, there are a number of standard tensors that need to be calculated to help work out what the geometry of space-time actually looks like and how things like light beams travel through this geometry. The GraviPy module includes the tensor classes Christoffel, Riemann, Ricci, Einstein and Geodesic. The Christoffel, Riemann and Ricci tensors depend only on the metric. Therefore, they all have very similar forms to create new instances and get results out of them. For example, you can define the Christoffel tensor values with this statement:

Ga = Christoffel('Ga', g)

You can get individual elements with indices, just like with the metric tensor. But, some of these tensors can have a number of uninteresting zero entries. So, you can get the non-zero values with the statement:

Ga.components

This returns a dictionary where the keys are the coordinate sets for where this particular value is located, as well as the actual non-zero value at the point.

The Einstein tensor is the one used in the actual Einstein equations for space-time, and they are a little different. In order to calculate them, you first need to calculate the Ricci tensor with this statement:

Ri = Ricci('Ri', g)

Once you have this tensor, you can calculate the Einstein tensor with this:

G = Einstein('G', Ri)

Before I leave off, I should look at two techniques that are absolutely necessary to doing general relativity. The first is index contraction. This is where you end up summing values over two of the indices in a tensor. In Python, you can do this by explicitly summing with:

Rm = Riemann('Rm', g)
ricci = sum([Rm(i, All, k, All)*g(-i, -k) for i, k in 
 ↪list(variations(range(1, 5), 2, True))], zeros(4))

These two lines are equivalent to the above single creation of the Ricci tensor. In many cases, complicated calculations are not simplified automatically. This means that you need to do this simplification explicitly with the command:

ricci.simplify()

The other important technique is to calculate geodesics, which are equations that define how light beams travel in this space-time. You need to create a new variable to handle the world line parameter for these equations:

tau = symbols('\\tau')

Now, you can calculate the geodesics with this:

w = Geodesic('w', g, tau)

Again, you can use 1-based indices to access the various non-trivial equations for your space-time of interest.

With SymPy, you can do all sorts of symbolic calculations normally reserved for programs like Maple, Maxima or Mathematica. Building on these capabilities, GraviPy lets you play with space-time and do calculations to determine curvature and gravitational effects. There is not a great deal of information available on-line explaining what GraviPy can do. Your best option is to download the source files, which include a tutorial iPython notebook, even if you install GraviPy through pip. Now you can do your gravitational research all from the comfort of Python. If you are a Python fan, this module will let you do interesting research work in your favourite language.

Streaming video to multiple people always has been a challenge. Back when Kyle Rankin and I did “Linux Journal Live”, we'd use services like ustream or justin.tv in order to accommodate the bandwidth requirements. The problem with those services is that unless you pay significant money, the features are extremely limited. Fairly recently, Google has changed that with its “Hangouts on Air”.

(Image from Google)

The process works like any other Google Hangout video chat, except that while you're chatting, a YouTube live stream is taking place. It's possible to embed the live stream like any other YouTube video, and because it uses the Google networks, there's no need to worry about using too much bandwidth.

The process for starting, stopping and embedding Hangouts on Air still can be a bit cumbersome. Thankfully, as the product matures, its integration into YouTube gets better and better. I personally used Hangouts on Air to broadcast my kid's drama presentation to family members who couldn't make it. You even can keep the live stream as an archive video for folks who can't watch the stream live. Check it out today at plus.google.com/hangouts/onair.

I was chatting with Fred Richards on IRC the other day (flrichar on freenode) about sneaking around hotel firewalls. Occasionally, hotels will block things like the SSH port, hoping people don't abuse their network. Although I can respect their rationale, blocking an SSH port for a Linux user is like taking a mouse away from a Windows user! I mentioned that I used to have a remote server running SSH on port 443 so I still could get to my servers. (Port 443 is the HTTPS port, which is rarely blocked.)

I also mentioned that it was inconvenient to use port 443 for SSH, because it meant I couldn't host secure Web sites on that server. Fred graciously pointed me to sslh, which is an awesome little program that multiplexes (or maybe de-multiplexes?) network traffic based on the type of traffic it sees. In simple terms, it means that sslh will listen for incoming connections on a port like 443, and if it's a request for a Web page, it will send the request to Apache. If it's an SSH request, it sends it to the SSH dæmon. It also has support for OpenVPN traffic, XMPP traffic and tinc.

Conceptually the program is simple, but I never considered it would be something a simple open-source application could manage! I assumed it would require a hardware appliance and lots of horsepower. I'm happy to say I was very, very wrong. In fact, it's such an impressive piece of software, it gets this month's Editors' Choice award! If you'd like to reach your SSH server over port 443 while still hosting secure Web pages, check out sslh at www.rutschle.net/tech/sslh.shtml.

Abundance is, in large part, an attitude.

—Sue Patton Thoele

The trick is to make sure you don't die waiting for prosperity to come.

—Lee Iacocca

The power of illustrative anecdotes often lies not in how well they present reality, but in how well they reflect the core beliefs of their audience.

—David P. Mikkelson

Men for the sake of getting a living forget to live.

—Margaret Fuller

There are no secrets to success. It is the result of preparation, hard work, and learning from failure.

—Colin Powell