Virtual Interview with Jeremy Allison and Andrew Tridgell

Andrew Tridgell and Jeremy Allison together have written the majority of the code in Samba, the popular SMB (Windows Networking) file server for UNIX and UNIX-like operating systems. Andrew, the creator of Samba, is currently a Ph.D. student at the Australian National University. Jeremy currently works for Whistle Communications Corporation, located in Mountain View, California. Whistle provides Windows file sharing services from the InterJet Internet appliance using Samba. Part of Jeremy's job at Whistle is to develop and maintain Samba to make sure it works properly on the InterJet. Before conducting this interview I had “met” both Jeremy and Andrew through e-mail when both provided a valuable commentary on my book Samba: Integrating UNIX and Windows.

Jeremy Allison

Andrew Tridgell

This interview is dubbed virtual because it didn't actually take place as printed. I interviewed Jeremy over the telephone and his comments are taken nearly verbatim from the tape of that conversation, though some changes were made in the interest of readability. Andrew's comments were added via e-mail after the fact.

John: The “history” file that accompanies Samba describes how Andrew wrote the precursor of Samba by reverse-engineering the DEC Pathworks protocol using packet dumps. Apparently he was motivated to do so because he had become used to using PC-NFS to access his home directory on a Sun machine, but couldn't run both the DEC Pathworks client and the PC-NFS client at the same time. How did you get involved with Samba, Jeremy?

Jeremy: At that time I was working for Vantive. I was porting Vantive products to NT on the quiet—they hadn't asked me to do it, but I knew it was something they would need. At the time there was no PC-NFS client for NT so I was actually doing the port using FTP. I would ftp the source files over, compile them, then ftp them back. This was a massive pain. Around this same time Andrew announced the SMB server 1.x—his second version. I downloaded it to take a look. It was Andrew's first networking program, I remember him saying, and it was a bit rough and ready.

Andrew: I didn't realize Jeremy looked at it so early. The first e-mails I got from Jeremy were around December 1993 at around version 1.5.20. That was after the nearly two-year gap when no work was done on the code by me or anyone else. Version 0.1 was released around December 1991.

Jeremy: I was actually porting a network service that worked exactly the same way that Samba does, so I started tightening up the networking code and adding some stuff. Andrew had implemented Samba just by sniffing the wire, because he's incredibly clever, where I can't do that sort of thing without a specification. Since I had the X-Open specification, I thought, why not extend this and implement Core Plus? So, I basically added the Core Plus protocol, which put in file locking and a whole bunch of other operations. That worked well enough for me.

Then Andrew leap-frogged me again and put in the LanMan 1 protocol. We kind of leap-frogged each other, but neither of us could figure out how to implement long file names, and we were stuck there for a while.

Then I was browsing in a book shop in Mountain View and I picked up a book on OS/2 programming, and it struck me that OS/2 already had to deal with this problem. Then I thought, this being Microsoft, what if they just linearized the long file name structures onto the wire? Well, worse things could happen, so I decided to try it. Luckily, I came down with the flu for a week, so I sat at home with my computer and it was one of those Zen programming moments where you just code it up, try it and it works—never before or since! Later on we found a few bugs, but for the most part it worked.

Andrew: Yeah, the specification just said “level-specific structures” to describe the structures. We didn't have the hang of the trans2 stuff which was the real stumbling block. Once Jeremy coded up trans2.c, things really got moving as we had the infrastructure needed to do all the “modern” SMB features such as long file names. We worked a lot with the OS/2 header files for a while and those really helped. Jeremy contributed the trans2 code in October 1994 for version 1.8.0. This was a giant step forward.

John: So, how accurate is the official X-Open spec, from your experience?

Jeremy: The actual Core specification is pretty accurate up to the point where you get to the OS/2 support. As soon as you go into the OS/2 areas of the specification, it just disappears, and it's sort of like “here be dragons” on a map.

Andrew: Yeah, it just says “level-specific structures” and leaves it at that. You do need to remember that the main client (Windows) didn't have long file names when the X-Open specification was written, so not putting it in is not too unreasonable, particularly as a full description would be huge.

John: So the protocol is just whatever they could get to work?

Jeremy: Yeah, basically. The protocol is really horrible. It sort of grew like a wart. You can tell by looking at it. So, what happened was, once we got the long file name support working, the next thing we wanted to get working was encrypted password support. We hated having to put plaintext passwords out. There was a “magic constant” that Microsoft actually refused to give out to anybody. It's used in the DES encryption for the LanMan password support. They refused to give it to us without an NDA (Non-Disclosure Agreement). We replied “We're not giving you an NDA, because we're publishing source.”

There it sat for a while until I went to a Microsoft NT conference up in Seattle and had a conversation with a, uhh, sort of mentally challenged Microsoft person who was basically saying “Why do you want to break open all our LanMan system's?”, and I said “I don't, I want to interoperate with them,” which I suppose to her was the same thing. So, she put me in touch with a Microsoft programmer, Richard Ward, who is in charge of some of their security stuff. I chatted with him, and he couldn't tell me what it was, but he said, if you forward me a message I'll put you in touch with somebody who knows more about it.

Now what happened—and this is really interesting—the programmer he put me in touch with basically said, “Well, what do you need to know?” I thought, well, if I ask him the question directly, I won't get the answer. He'll say, “I can't tell you that.” So, I phrased the question differently. The LanMan encryption is a two stage operation, and we needed the middle bit. I phrased the question in such a way that we could work backwards from what he told us. I asked, if we encrypt this password, what will the second stage be? Once he told us, it was trivial to reverse it and out popped the magic constant. And once I had the constant, I actually took a look at a hexdump of the Microsoft redirector, and the magic constant is contained in a field of zeros as plain as day.

Andrew: It actually took us a day or two to realize that the information he sent gave us what we needed. Jeremy and I both realized it independently within a few hours of each other. It was an unexpected bonus!

John: So it was easy to see the magic constant once you knew where to look? [Note: the magic constant is KGS!@#$%.]

Jeremy: Yeah, once you knew where to look, but it did confirm that we knew exactly what it was. So, it took a while getting the encrypted password stuff. I did all the SMB password stuff, which I was a bit nervous about because it's a setuid program and I never do like writing setuid programs. After that I kind of left Samba alone for quite a while, ended up moving to Cygnus to work on Kerberos. This was when Volker (another Samba Team member and author of the SMB file system for Linux) came on board, and when I finally came back to it, Volker was asking “who is this guy?”, because I hadn't worked on Samba for a while.

We went to a CIFS (Common Internet File System) conference and Roger Binns, who wrote VisionFS (another SMB file server for some UNIX systems written by SCO), said to me “your share-mode support is completely broken, did you know that?” And I replied, “Is it?” Right after that conversation, Herb Lewis at SGI started doing some benchmarks, even going to Utah to benchmark Samba against other programs such as the Syntax (VisionFS) code. Herb made the remark, “I can't get NetBench to run because I keep getting share-mode violations.” I remembered what Roger Binns had told me so I started looking at the share-mode code. The code was a tremendous piece of work, but it wasn't entirely safe—it only stored one share mode for each open file. We now have a completely correct share-mode implementation. So, in other words, we can have multiple share-mode entries for each open file. Once I put all of that code in—this is around 1.9.17, we could run NetBench. We got reasonable results—they were okay—probably as good as Syntax's.

At this time I realized that what we needed for performance was oplocks (“opportunistic locks”). About at the same time Herb, who's actually had a great effect on at least my parts of Samba code, started complaining that his browsing was broken—he couldn't make his browse lists be seen across subnets. Luke (Luke Kenneth Casson Leighton—another Samba Team member) had written all of that stuff and I had always thought it worked. Basically, I went and played on SGI's network, and I realized that there were things about the browsing that were broken. So that's when I started being involved with the browser stuff—I started fixing up Luke's code.

There have now been three implementations of the browser code. The first was a single-threaded completely blocking one that Andrew did which basically got it up and running. Then Luke took that and turned it into an event-driven model that was truly an amazing piece of work, very well done. But he was still learning the protocol at the same time. So, I hacked up some of that code section, and it sort of worked, but I realized that in order to make it completely robust, I had to keep the same event model and data structures but completely rewrite the code. So that was what I ended up doing. That effort was actually spurred by the fact that Whistle needed browsing to work correctly over semi-reliable links, i.e., links that would go down periodically. I began by not rewriting it—just fixing that particular bit, but I ended up completely rewriting it. It was one of those things where, you know, after a while you see you've completely rewritten something without meaning to.

Andrew: Actually the first version wasn't completely blocking—it used recursion to simulate some properties of threads (a really ugly idea). It worked but could easily run out of stack. Luke's implementation fixed that problem and also added a lot of new code.

Jeremy: So, then, I knew the oplock implementation was possible. I had a design for it, but something like six months had passed before I actually had time to sit down and do it. The oplock code is probably the most complicated part of Samba right now.

John: It's probably the most complicated part of Windows networking.

Jeremy: Well, yeah. That took a long time to get right. Typical with SMB, there were a lot of ugly implementation details that aren't in the specification. For example, when someone does an open that needs to break an oplock, do you check deny modes first? Now, logic would dictate that you do, because if the deny modes disallow you from opening the file then there's no need to check the oplock. It turns out if you do that everything breaks. My first implementation did that as an optimization because I thought, well, this will be really fast. It turns out if you do that a bunch of programs break because a lot of the Windows 95 redirector internals expect that a file could be opened twice and the second open would cause the oplock to break even though it's still going to get a share-mode violation.

John: Do you suppose the people that wrote that were optimizing for their own server code?

Jeremy: I don't know. Previously they had only tested against their own server. Probably there are a really weird bunch of hacks in there that work against their own server and that we have to emulate. Of course, whenever the redirector is broken and it doesn't work with Samba, people say that Samba is broken.

John: To change the conversation a little bit, are you paid by Whistle just to work on Samba or do you have other responsibilities?

Jeremy: I have other responsibilities, although getting Samba working has been a big part of my job. One of the issues right now is that Samba works well enough for Whistle, so I might actually have to work on some other programs. However, I really do love to work on extending Samba. My current plans for 1.9.19 are to do NT SMB, which will implement change notify and many other options. One of the things Whistle did that was actually very good is they drove me into doing dynamic internationalization support for Samba.

John: Any final comments?

Jeremy: One more thing that you should include—my wife is the product marketing manager for Network Appliance, a company that makes an SMB implementation. So, my wife and I are in direct competition along with Samba and Network Appliance, which is really amusing. She gives sales training which includes a competitive analysis of Samba. She says people always ask, “What kind of freaks write this stuff and then give it away?”

John: That reminds me of conversations where I'm trying to explain the GPL. It took me a while to fully internalize it and understand its implications, and explaining it is a long conversation—sort of like explaining a political theory.

Jeremy: Yes, it's a revolutionary idea. Like most revolutionary ideas it will end up changing the world, and it takes a while to get. I won't release software under any other license now. I did before, and I've always considered it a mistake and regretted making it. The pwdump stuff I released under the BSD license, and I wish I hadn't now. I see that code reused by other people, but because of the license I don't get to see how they're using it—they don't have to give back.

John: Andrew, why did you choose to release Samba under the GPL?

Andrew: I didn't at first! Server-0.1 and all versions prior to 1.5 were under a “if you want to use this for commercial purposes then contact me” license.

Version 1.5 came out after the 2-year break and after I had started using Linux. I was impressed with Linux and chose the GPL not because I understood it (I didn't) but because it was what Linux used. Now that I understand the GPL I'm very glad I chose it. It encourages just the sort of development effort that is needed for Samba.