Paranoid Penguin

Recently, after receiving one of SUSE's regular security-update e-mail messages, it occurred to me that people who track security vulnerabilities for Linux distributions really are in the front lines of Linux security. I have the luxury of examining one command here, one architecture there, but these people track threats and vulnerabilities for entire operating systems.

And, for a moment, I was fearful. What if this occurs to my editors, who might reach the obvious conclusion that the Paranoid Penguin really ought to be authored by a real expert, like SUSE Security Team Lead Marcus Meissner, rather than this goofball Mick?

But, of course, if they fired me, I'd be forced to release those photos I took of them back in 2003 on the Linux Lunacy cruise, and nobody wants that to happen. Think of the disillusionment and embarrassment, not to say indigestion, this might cause! I, for one, do not wish to be party to such a vile media circus. (Unless that party features the TCP/IP Drinking Game, like at Def Con—good times! Then again, that's the kind of thinking that landed my editors in this pickle.)

So, you're stuck with me for the foreseeable future, gentle reader. But, take heart! This month you get both me and wisdom: Marcus Meissner graciously agreed to an interview, in which we talked about practically every major topic in Linux security I could imagine.

As you can see, Marcus has useful and interesting insights on all of these topics. Read, learn and enjoy.

MB: Thanks so much for taking the time to chat with Linux Journal! By way of introducing you to our readers, I'd like to start with a question or two about your background.

Marcus Meissner

MM: Thank you for interviewing me!

MB: Could you describe your duties at SUSE as Security Team Lead?

MM: My duties as Security Team Lead are mostly overseeing security-related work done for the SUSE Linux/OpenSUSE product lines. My team has three people and one trainee currently, so there still is time left for myself to be involved in the groundwork.

What my team does can be split into two parts: reactive work and proactive work.

Reactive work is what you know of as coordinating security updates for security incidents for the products currently under support. This involves reading mailing lists, watching the bug tracker, watching the internal update release processes and giving answers to packagers, QA and others wanting to know about the updates.

Proactive work consists mostly of reviewing security-related changes for future products, adding new security features for those and, of course, actively reviewing and auditing source code for flaws and fixing them.

My typical day is mostly lots of communication spread over the whole day: reading and answering e-mail, instant messages, phone calls and, of course, meetings. All this is done mostly to get the above-listed security tasks taken care of, so there are not specific phases of the day. Occasionally, I still do some programming and handling of the packages I maintain in the distribution.

MB: You started out at SUSE on its PowerPC Maintenance Team, right? Why did you go to the Security Team?

MM: Historically, I had been working in the security field already in my last year (2001) at Caldera. After coming to SUSE, there was no need for another security engineer there, so I had different tasks. I started out doing various non-PowerPC-related development my first year at SUSE, but gravitated in the end to the PowerPC Development Team.

In 2004, however, I wanted a bit more responsibility, and then SUSE started looking for a lead for the Security Team. So, the offer of interesting work, more responsibility and my first people management job got me into it in the end.

MB: Do you have any formal training in information security? (This is sort of a trick question: many of my most accomplished colleagues are mostly self-taught.)

MM: Portions of it I got from university, such as basics on cryptography, protection models like Bell-LaPadula and information leaks via side channels.

But, most of my information security training was on the job and by self-education, because it was not very high on the list of must-haves in the universities in the middle of the 1990s.

MB: How has your background in software development informed your work as a security professional?

MM: Lots of the low-level work of security in the Linux distribution area is actually bug fixing and helping others design and write good code to avoid bugs. So, yes, my background in software development was a must for this job.

MB: In my own day job at the bank, I frequently interact with software developers who have very little understanding of security or of secure coding. Although ignorant is the last word that I, being a humble nonprogrammer, would use in this context, it seems that security has not traditionally been an important part of programming culture and training. Have you found that to be the case yourself? Has this situation changed in recent years?

MM: I have found this to be the case, yes. In former times, it was mostly “get things working”. Now, it is a bit better. Everyone knows about viruses, worms, trojans and so forth. However, it is not optimal yet; the hard part is convincing people that their code might be exposed to such problems.

I am regularly hearing, “but then it will just be broken into the user account, not root” or “it will run behind the firewall here” as excuses. But, I see definite improvements.

MB: We seem to be stuck in an endless parade of security vulnerabilities in the full range of applications on which we depend—graphics editors, compression libraries, server dæmons, even security tools. I think some of us assumed that over time, as security awareness in the development and user communities increased, the rate of security bugs would decrease. Why, do you think, it hasn't? Or, is this rate, in fact, decreasing?

MM: This has several factors at play.

First, the classes of problems have changed due to security research over time.

The trivial exploit technologies, like stack and heap overflows and format string problems, are explored and can be captured by technology today, and programmers understand them.

But, integer overflows still are an uncharted sea for most—“Why is there a difference between 'signed' and 'unsigned'? Why is this multiplication bad?”—and are hard to find using today's compiler technology.

The new kind of applications on the Web has brought entirely new problems for things like SQL injection, command injection, cross-site scripting and others. Due to the startup boom, lots of those applications were not designed with security in mind. Here, the easy Web development languages are a bit at fault too—on one hand, allowing beginners to develop applications easily, but on the other hand, making it trivial to make mistakes.

Also, the increased integration of the network and Internet has brought more code up to the light, as in “can now be accessed from the network”. Just think of all the image libraries written with a command-line local user in mind that now run as a back end to Web galleries.

I personally see no reduction of security problems in the near future. At most, it will level off.

MB: It might seem non-intuitive for developers of non-networked desktop applications, like gPhoto, to have to worry about security. But, of course, such applications frequently are used as stepping-stones in attacks against other processes and for gaining access to system data or other local resources. Do you ever find yourself lecturing other developers on threat scenarios like that?

MM: Hmm, not for gPhoto or Wine at least, but for other distribution-related packages.

MB: What changes, especially improvements, have you seen lately in application architecture? What changes would you like to see?

MM: I see different languages being used, which bring different architecture concepts. The trend away from C toward Java, C#, Ruby and so on with stricter typing is helpful.

I would like to see people not re-inventing/reprogramming everything existing anew, but trying more to reuse existing, proven code.

MB: How do you think the race to Web-oriented application architectures has affected application and operating system security?

MM: It makes securing the process more difficult. As an OS vendor, we now are just one part of the chain—the secure base system used to browse the Web.

MB: Is it just me, or has the emergence of cross-site scripting (XSS) added a whole new dimension to the security landscape? I don't remember many pre-Web attack vectors that allowed you to use one person's server to attack other users. The older paradigm was that you attacked the system, not the users per se.

MM: Definitely, the advent of Web applications has brought new problems, and seeing that there still are dozens of XSS issues fixed per week shows that the problems are not contained yet.

And yes, when interacting between the network, the user is way more interactively involved than before. Consider phishing problems, for instance, which even though they fall under social engineering, do endanger system security.

MB: How has the emergence of Web services affected the SUSE Security Team's work? Is it easier to isolate and address bugs in Web applications, which are usually written in scripting languages, than in compiled applications?

MM: Fortunately, not much, as we do not include very many. Quite a lot of fixes went into the PHP core packages in the last few years, and the Web applications we include had their share of problems.

What we also shipped is an increasing rate of Web browser security fixes, as the browsers now get good reviews too. More than a hundred Mozilla bugs were fixed in the last few years, for instance.

As for isolating and addressing bugs, it's still at the same difficulty level. You need to understand the code and fix the problem. Also, as we usually back-port patches, we need to understand those, and there the language does not matter much. Plus, Web applications usually are not smaller than regular programs.

MB: In the Linux world, we've seen less malware (viruses, trojans and worms) than the Microsoft Windows world has been subject to. Why is this, do you think?

MM: First, Microsoft Windows just has more installations, and so it is a more valuable target; thus, it gets more research into exploitability.

Second, Windows has quite a high integration level. You can do lots of stuff from everything, and this was seen as good thing—easy embedding of document/image viewing and so on. Although on the one hand, this is a good thing, it also exposes a lot more code to the attackers.

Plus, the Windows software development community before the Internet was not really programming with security in mind, and so there were large holes. The same goes for reviewing the code; it was hard without source for externals.

It's something like a mix of all those things, I guess.

MB: My own opinion for several years has been that Linux isn't inherently more or less secure than Windows; their underlying security models are very similar. What are your thoughts on this?

MM: UNIX/Linux has, for example, the advantage that we separated (the concept of) the user from the administrator right from beginning, which Windows still has problems with.

Due to less integration, or integration at different levels, Linux has perhaps a better chance of resisting those attacks.

Linux also has less of a monoculture in programs and libraries, and it is also more rapidly changing than perhaps on Windows.

MB: What kind of potential do you see in mandatory access control (MAC) systems, like AppArmor and SELinux, in improving Linux security for the masses? To what extent do you think they're already helping?

MM: It's difficult to say. I have no experience with SELinux, but with AppArmor, I see a bit of acceptance issues in default settings, and then it does not catch everything.

MB: When SUSE incorporated Novell AppArmor into its general releases, this caused a bit of controversy. It seemed like some people involved with SELinux felt that this undermined their efforts. As a SUSE employee, I assume you're pro AppArmor, but what do you think about the controversy? Isn't it healthier for multiple MAC options to be available to people?

MM: There surely was controversy, but most of it seems to have died down now.

It is healthier to have more than one MAC system, especially in exploring the MAC problem from different angles.

That AppArmor was much more usable than SELinux also has caused lots of thinking and usability improvements in SELinux (think targeted policies, booleans and so on), and the other way around. AppArmor now can contain more things than in earlier times. We currently see both as solutions that even could co-exist to some degree.

Other new MAC approaches, like SMACK and so forth, also are appearing now.

MB: So, are there any plans for SUSE to support SELinux, as an alternative to AppArmor?

MM: I cannot say at this time, especially since partner requirements are still open for future products.

MB: When Linux virtualization first started to emerge into the mainstream a few years ago, it seemed to me that the whole concept of a hypervisor—an intelligence logically above the guest-OS kernel that manages system resources and monitors VM behavior—has a lot of security potential. Nowadays, I wonder whether I wasn't overly optimistic. The additional layer of abstraction might introduce other attack vectors. Your thoughts?

MM: Virtualization environments, unfortunately, were/are sold as security solutions, but the breakout possibilities are only now being investigated, and there likely was no formal containment design from the ground up.

Several ways also have been found for almost all virtualization technologies to break out of confinement.

So yes, I think its being used as security containers is overly optimistic.

MB: One of the most remarkable developments in Linux, it seems to me, is its rapid inroads in the embedded systems market. All kinds of consumer electronic devices are now Linux-powered. Does SUSE ever show up in this space? Do the particular challenges and ramifications of embedded operations figure into your team's work?

And, from a security perspective, how good of an idea is it to use a general-purpose operating system like Linux (or Windows) for embedded applications?

MM: We are not really showing in this space, even though we are working to bring the enterprise desktop more into the thin-client space. But, it's not the real embedded market.

What matters most for security in those devices is how they get updates and what security processes are there from their vendors. If the vendor just gives up support after six months for a device, but the device lives for five years or longer, it's bad. You have lots of unpatched devices out there.

MB: Your team, of course, digitally signs its communications. But 17 years after Phil Zimmerman gave us PGP, only a tiny percentage of ordinary users employ any kind of e-mail encryption. Any thoughts as to why, and what to do about it?

MM: It's too hard to use and, more so, too hard even to understand why to use cryptography. “Why does Aunt Emily need to encrypt letters to her niece Tina? Who cares about them anyway? And, how do I do it?”

MB: Maybe the real issue here is identity management. We haven't yet figured out any kind of universal identification on the Internet, which is part of the problem space that PGP, S/MIME, x.509 and LDAP are all supposed to address. But the paradox is that although such an identification infrastructure would greatly simplify all sorts of security problems—single sign-on, directory services, encryption and the like—the technology itself is very complicated.

MM: Yes, definitely. Perhaps a hardware solution might help here—something that Aunt Emily and niece Tina could physically exchange and so would physically grasp.

One could imagine doing premade USB tokens that can be torn off a strip and distributed for every family member involved, in a size that fits in regular letters. Or, using cell phones to pass encryption keys back and forth, as everyone owns cell phones now.

MB: Any time you talk about centralized identity management in the US, for which the logical starting point is the federal government, the discussion gets very strange very quickly. Americans are reluctant to trust their government not to abuse this information (which is perhaps strange given that they've got all sorts of information about us already). Are things different in Europe?

MM: They are better. People trust the government more, because they already hand out our passports and ID cards. But, with the current government trying to enter into our privacy more and more, I think even in Germany we will see more mistrust.

MB: We've amply filled this month's allotted space with a very wide-ranging discussion indeed. Thanks so much, Marcus, for a thoughtful and fun conversation!