Linux Powers The Spiderwick Chronicles

A Linux-based production pipeline is a perfect choice for a major motion picture like The Spiderwick Chronicles, with its many goblins and magical creatures. Hollywood has been the realm of Linux since 1997, when the movie Titanic proved that Linux can do big computer graphics jobs like rendering a sinking ocean liner. With an industry tradition of using UNIX-based operating systems for high-computation jobs, and due to the better, faster, cheaper nature of Linux, every major effects or animation movie today is produced using Linux. Visual effects facilities ILM and Tippett Studio each created visual effects for Spiderwick. Having multiple effects houses work on the same movie became common after 2003 when The Matrix Reloaded used a dozen effects houses.

The idea of switching Mac desktops to Linux is new in the film industry. The film industry routinely uses Macs running OS X for specialized tasks, such as art department concept artwork generated using Adobe Photoshop, picture editing with Final Cut Pro and sound editing with ProTools. When you scale past a few systems, the advantages of Linux for graphics become apparent, and Linux graphics PC desktops are the norm. The television series South Park is a notable exception, with Mac OS X desktops running Maya with a Linux renderfarm.

During the production of Spiderwick, Tippett Studio switched to Fedora Linux running on Macintosh desktops. “We currently have 119 Intel-based Apple Mac Pro workstations running Linux”, says Tippett Computer Graphics Supervisor Russell Darling. “We decided to go with Apple hardware running Linux for our primary artist workstations on The Spiderwick Chronicles, although it might have been considered a risky endeavor for a show in production. We initially had some problems with sound on Maya and a few other minor issues, but they were resolved. We got a patch from Autodesk that took care of everything.” Commercial Linux software vendors work closely with film studio clients.

Tippett chose Linux on Mac for many reasons. “There's the ability to run multiple operating systems, including Linux, OS X and Windows”, says Darling, and he continues, “The systems are fast! That makes for more productive artists. The hardware is quiet and energy-efficient. It's cost-effective, with a good cost per rendermark [a renderfarm performance benchmark]. It's standardized hardware. And, there's a good support plan. Although the majority of our workstations run Linux, we have a handful of other systems running to support specific software. We use the ability to boot in to other operating systems, but the ultimate goal is to move to a simultaneous multi-OS solution, such as Parallels.”

To beat traditional alternatives, the Apple Mac Pro workstations had to meet a specific set of Tippett requirements. They had to run Fedora FC4 and XFS. They also had to run tools that Tippett uses, such as Maya with sound and in-house and third-party plugins (MEL scripts), Apple Shake with in-house and third-party plugins, SyFlex, cMuscle, RealFlow, JET, Flipper, rtTools and cineSpace. Internally developed software uses Python, Perl and C/C++. The platform must render frames identical to existing hardware. And, it has to support necessary peripherals, especially tablets.

“The important thing with a fantasy genre is referencing nature”, says ILM Art Director Christian Alzmann. “The Byron plumage is based on a red-tailed hawk. We're always drawing reference from nature. I did the early design of the Sprites with a bee next to them for scale, with two bees flying in formation. Mulgarath is part man, part bull, part goat, part trees. The warthog is a mean aggressive character, so we got pointy with him. And, he's a lot more distorted. We also use scale cues, such as a Chiquita banana sticker or Pepsi bottle cap.”

“The Griffin has hair plus feathers and was rendered at 8k [images 8k pixels wide] to get detail”, says ILM Animation Supervisor Tim Harrington. To achieve that level of detail meant 25- to 30-hour renders.

“Spiderwick took 215 artists and 15 months”, says ILM Visual Effects Supervisor Tim Alexander. “It has 341 shots, 30 minutes, with 224 3-D shots.”

Industrial Light & Magic occupies the 865,000 square-foot Letterman Digital Arts Center on the 23-acre San Francisco Presidio campus. Its data network has more than 300 10GB ports and 1,500 1GB ports, with fibre to every artist's desktop. There are 600 miles of cable throughout the four buildings on the campus. A 13,500 square-foot data center houses a Linux renderfarm with 3,000 AMD processors and more than 100TB of storage. Proprietary render management tools add Linux desktop workstations to the renderfarm pool after hours, expanding the processing capacity to more than 5,000 processors.

As Creature Supervisor for The Spiderwick Chronicles, visual effects pioneer Phil Tippett oversaw the design and development of the film's fantasy characters. “Phil Tippett was on set with me every day”, says Director Mark Waters. “We were working on Charlotte's Web when Mark Canton gave us the script”, says Tippett Studio Visual Effects Supervisor Joel Friesch. “When we saw the creatures, we had to do it. It's based on real creatures, not fantasy. We wanted Hogsqueal. We created a bull goblin marquette [a detailed statuette] that gave Mark something he could hold. The bull goblin is based on toads. We brought in real toads and photographed them. We created movies good for the animators, showing how the eyes move and the throat. We created a test scene with a goblin scratching the back of his leg. That took one month of modeling and one month of animation.”

Figure 1. Tippett Studio's proprietary Creature Manager is used to maintain a library of creatures and animation cycles. The tool allows an artist to select and preview animation by pressing the larger creature button, then selecting a combination of an appropriate physical appearance for that creature from a predefined library and placing any number of selected creatures into a Maya scene.

Hand animation is a challenging laborious process. “One guy does blocking, like moving chess pieces”, says Tippett Studio Animation Supervisor Todd Labonte. “You get it approved. We watch it over and over. You can go blind. We play it back in mirror image in our player or play it backward.” Labonte demonstrates playing back a scene of goblins invading the house, shown in their Flipper playback software, which can display a mirror image or play in reverse to help catch animation inconsistencies. Flipper is used to view both QuickTime and image frame sequences of DPX, EXR or TIFF with synchronized AIF audio. Flipper predates commercial Linux flipbooks, such as FrameCycler. At older studios, like Tippett, it's common to find proprietary Linux tools created before commercial options were available. Tippett has a team of eight Linux programmers to maintain and develop tools.

Figure 2. Flipper is Tippett Studio's proprietary flipbook image viewing tool. It allows an artist to view a series of individual image files as a continuous sequence. It also can be synchronized with audio, which is important for character animation. The artist can view the audio waveform to help with lip synchronization, as seen in the lower part of the screenshot. The tool also has a number of image and pixel comparison and analysis features, as seen in the dialog on the upper left. Post-camera moves can be previewed with Flipper before they are actually applied in the composite stage.

“Creature Picklist is a GUI-based Maya plugin for creatures that allows animators to see visual representations of character, which they can select for their scene”, says Darling. “In the case of Spiderwick, 'Goblin kits' were created as combinations of variants and blendshapes. We have shots that have more than 100 goblins. That's too many to animate using traditional methods. The numbers are also too small to make a commercial crowd system, such as Massive, a viable solution. We developed our own system called Swarm. For the Spiderwick shots, we instanced around 150 goblins and managed animation clip data to animate them as particles.”

Figure 3. Tippett Studio's Picklist allows an animator to select creature variants from a library of different combinations of paint schemes and body parts.

Figure 4. A view of Tippett's Swarm crowd system in Maya. The scene is choreographed by defining paths and actions for creatures to follow.

Figure 5. Tippett Studio's Picklist allows an animator to select specific predefined poses for creatures, such as this bird.

Furocious is Tippett Studio's proprietary hair, fur and feather system. It's a collection of plugins, scripts and executables used to place guide geometry onto scalp surfaces, visualize fur as GL curves in Maya, and grow procedural primitives at render/expansion time by interpolating neighboring guides at predetermined follicle root locations.

“We currently have six RaveHD systems at the studio”, says Darling. “They're used to provide dailies in our screening rooms. We also use them for smaller reviews and on-demand playback for artists in special viewing rooms. And whenever we shoot with HD cameras on our stage, we use RaveHD to acquire the images and bring them on-line as individual digital frame files. We have been working with SpectSoft since 2002, before the RaveHD product really existed. RaveHD is an awesome system. It has a really nice design that allows it to integrate perfectly in our dailies pipeline—very reliable and easy to use”. The RaveHD box is a Linux embedded system that plays cinema-grade motion pictures at data rates that would choke a PC.

“I originally met [SpecSoft partners] Jason and Ramona Howard at a Linux Movies meeting in Berkeley, shortly after I joined Tippett Studio”, says Darling. “At that time, we were looking at developing a new dailies system. They had developed a Linux driver for the AJA Kona HD/SD card and had developed some DDR and editing tools. We were able to form a great relationship with SpectSoft where we provided specifications and requirements to them in order to help create a system that suited our purposes.” LinuxMovies.org is an association of Linux motion-picture technologists founded in 2002 [by Robin Rowe, author of this article].

Tippett uses more than 1,200 processors in its renderfarm. “Tippett Studio has its own shading library built around the RenderMan Shading Language. Our pipeline tools are also centered around the RIB interface. The most important consideration for renderfarm configuration is that all jobs submitted the night before must be finished by the next morning”, says Darling. “Each morning we do dailies to review the previous day's work. If the job is not finished, it can't be properly reviewed. There are occasional exceptions for special shots that may run long, but for the most part, we want everything to finish overnight.” The Tippett renderfarm is managed by a proprietary batch-scheduling software, so that each computer in the farm is working on only one frame at a time. “Our distributed rendering system Batch-o-matic has been in use at the studio for ten years”, says Darling.

Figure 6. Tippett Studio's renderfarm is managed by its proprietary Batch-o-matic system. In its GUI interface, CPUs are represented as circles with color-coded indications of their activity level and status. Information about an artist's render jobs is represented by color-coded squares indicating number of CPUs in use, as well as an overall status bar. More detailed information is shown for a specific job, with status for each task and frame.

JET is a proprietary Python-based system comprising software tools and scripts used to implement a visual effects and animation pipeline. “A visual effects and animation pipeline is an assembly line of software used to organize, automate and facilitate the creation of computer-generated imagery”, says Darling. “The JET tool is highly customizable, featuring XML-based user-interface templates that can be modified to suit specific types of artists or production needs. JET uses modular template chunks to perform each of the tasks in the pipeline, such as rendering or compositing. The templates are implemented as Python objects and are centrally located. JET is not only implemented entirely in Python, but it's also used to generate Python scripts automatically. These custom scripts form unique pipelines for each computer graphics job to run on the renderfarm.”

Figure 7. JET is Tippett Studio's proprietary tool for creating jobs that are to be submitted to the renderfarm. Each job comprises a series of chunks (as shown in the panel on the left) that perform specific tasks, such as making a RIB, growing fur, rendering with RenderMan and compositing with Shake. The primary work area on the right provides an interface for the selected chunk.

Tippett Studio uses Apple's Shake compositing software, which has been discontinued. “We took advantage of Apple's offer of selling the Shake source code to us”, says Darling. “We don't plan on modifying it, but it is good to have for an insurance policy. Shake is still very prominent in the visual effects industry. It's useful to be able to share Shake projects with other studios we're collaborating with. Shake has a really nice image-processing engine. In addition to standard compositing, we use it for all kinds of image-processing solutions in our pipeline. The product is very mature and feature-rich. It has support for plugins, which allows us to develop and enhance compositing nodes whenever we need something new. We don't have any current plans to switch compositing packages, but we're always keeping an eye on what's out there.”

“We use Photoshop CS as the painting interface for our texture painting, as well as for matte paintings”, says Darling. “It's used directly on Windows and Mac systems. Our goal is to use a system such as Parallels to permit artists to run tools outside their primary operating system. Deep Paint 3D is used to paint textures for 3-D models. The actual painting is done with Photoshop, but the interaction with the model is done in Deep Paint.” Deep Paint and Photoshop are both non-Linux commercial tools. Some studios, such as DreamWorks Animation, use Wine to run Photoshop on Linux. Another option is to run an open-source Linux paint package that supports industry-standard high-fidelity image formats DPX and OpenEXR (such as CinePaint). However, Deep Paint on Wine is untried, and there's no open-source option.

“When painting in Photoshop, it is essential to be able to view the image with the same type of color management used to produce the final rendered image”, says Darling. “Photoshop supports ICC profiles for color management. Since we use cineSpace in the rest of the pipeline, we had a tool created that would allow us to produce an ICC profile that matched the cineSpace profile. We collaborated with Joseph Goldstone, who is a member of the International Color Consortium (ICC) to create this tool.”

There are commercial Linux software tools available to Linux animators beyond big packages like Maya. “cMuscle is a system for simulating muscle movement”, says Darling. “We used it for muscle jiggle, skin jiggle, skin smoothing and sliding effects. We use SyFlex for the clothing worn by Red Cap and Hogsqueal. This system was augmented by some custom cloth/deformation software developed at Tippett Studio.”

Figure 8. Creature skin, muscles and skeletal structures—the green and yellow images represent soft body jiggle maps. Along with the muscle system, we also have what is called tension controls. These are blendshapes of various body parts that when fired give those areas a more flexed look, giving the illusion of the skin tightening.

Figure 9. Creature muscles (red) and basic skeletal system (white). For this creature, Tippett Studio's hair and fur system Furocious was used to grow fur that had the appearance and behavior of grass, which is one of the unique camouflaging features of this character.

Figure 10. Animation controls for the Troll.

“cineSpace allows us to have a specific, consistent look on all viewing devices that simulate our finaling medium—the look of the film that the director is using for final shots”, says Darling. “Since film is still the master format, this means a combination of a specific film stock and a specific lab that is used to process and print that film. A cineSpace profile allows us to produce an image digitally that has the same look and characteristics of that same image if it were printed to film. It is important that we are looking at the same thing that the director sees, as well as what will eventually be seen in the theater.”

“Tippett Studio's primary operating system is Linux”, says Darling. “Open-source software is very important to us. In addition to it being a part of our pipeline, we've contributed to a few projects over the years, such as Pixie. Open source is also something that can enable standards. An example of this would be the OpenEXR Project. We really needed a standardized high-dynamic-range image format that was supported in commercial and open-source CG software. The EXR image file format has now become a de facto standard image file format in our industry, simply because it was open-sourced.”

“In Spiderwick all unexplained phenomena in the world are due to fantastical creatures that are all around us”, says director Mark Waters. The movie is based on the best-selling series of children's books written by Holly Black and illustrated by Tony DiTerlizzi. The Spiderwick Chronicles is a family adventure fantasy set at the dilapidated Spiderwick Estate, constructed in Canada at Quebec's Cap-Ste-Jacques Nature park.

The movie opens to wide release on February 15th, 2008. There's also a Spiderwick video game in development.