This month Robin looks at how special effects are assembled with RAYZ in several new and upcoming motion pictures.
Last month we looked at Nothing Real's Shake, compositing software used on many major motion pictures to add special effects or to overlay animation. Nothing Real was recently acquired by Apple, which plans to use its fast scanline rendering technology in future Apple products. Shake's uncertain future on Linux may benefit competitor Silicon Grail's RAYZ.
Both RAYZ and Shake use an interactive scene-graph interface for defining how effects are assembled. While similar from a user-interface standpoint, RAYZ doesn't use the scanline rendering approach of Shake. “I think caching schemes are more important today than scanline algorithms for optimizing rendering”, says Silicon Grail President Ray Feeney. “Memory is much cheaper now. Scanlines can actually be slower than keeping more blocks in RAM—for example, with rotations.”
Hollywood effects company Flash Film Works used RAYZ on Linux for three recent Warner Bros. productions: the Arnold Schwarzenegger action film Collateral Damage, the Robin Williams dark comedy Death to Smoochy and the Eddie Murphy gangster sci-fi film Pluto Nash (to be released August 16, 2002). “RAYZ has the basic features a compositor should have, plus warp, morph and CineSpeed”, says Flash Film Works Technical Supervisor Dan Novy. RAYZ is a new codebase. Silicon Grail's previous compositor, Chalice, didn't support Linux. “Silicon Grail held discussion groups and polled Chalice users over a year for power-user input”, Novy says. “The RAYZ flipbook is just amazing. It can display 2k resolution in real time. Nodes will cache ahead as it is playing. I can work on the shot as it runs, do color-match changes in real time.”
“In Death to Smoochy RAYZ did a lot of compositing of characters in cars and boats”, notes Novy. “We used RAYZ to add muzzle flashes to guns and for street reflections. We did a lot of 3-D effects on top of 2-D plates.” In one scene, the RAYZ warp and morph feature was used to repair a piece of a physical set that broke during a take. Although other takes had the set intact, the director felt that the take with the set problem had the best performance.
“For Collateral Damage we did a lot of retiming”, says Novy. “Retiming 24fps into 36fps creates slow motion where none was shot.” The RAYZ software creates tweens, extrapolating the missing frames. “Retiming may not be flawless, but it's better than double framing. REALVIZ ReTimer is an alternative, but that isn't built-in and would have to be rendered out.” In Collateral Damage, a lot of compositing was done to place Schwarzenegger in front of explosions.
For Pluto Nash, RAYZ composited in moonscapes created in LightWave on Windows NT and blue-screened with Ultimatte. “The RAYZ net license Ultimatte feature is important to us. Digital Fusion, Combustion and After Effects require a dongle to use Ultimatte. You can't float to any machine like RAYZ.” For a 3-D theme-park ride film, Flash Film Works is developing a plugin to distort the pixels to create parallax. A smoke plume can be adjusted to match the inter-ocular distance and create a composite for each eye. Their plugins are in Perl and C++. Novy is considering a switch from LightWave to Houdini, a popular 3-D package for Linux.
After Kodak stopped making the Cineon compositor, Silicon Grail acquired the technology and incorporated it into RAYZ as Grain/Degrain, Sharpen and CineSpeed. “A unique thing about RAYZ is it can keep data in Cineon 10-bit log color format”, points out Novy. “Every other compositor makes you convert to 16-bit linear or to float. With RAYZ you can convert or keep it 10-bit all the way. It will do the internal math with logs.” Video monitors are 8-bit linear and need LUTs (colorspace lookup tables) to approximate film. “Because it was part of the Cineon technology incorporated into RAYZ, it can do full-monitor calibration to bring all our monitors into peak performance.”
“RAYZ runs on all of our operating systems and all our hardware”, notes Novy.
We're even using RAYZ on Alpha workstations, the 64-bit 21264. Back when we purchased these Alphas for Deep Blue Sea there was nothing faster, but a year and a half later they were in a corner doing nothing. Because of up-to-date Alpha support in RAYZ, we could resurrect this obsolete but fast hardware. We're moving toward Debian because Red Hat support for Alpha is lagging. I think Linux will dominate the motion picture industry like SGI did, but I am not sure about Red Hat.
Santa Monica-based Island Fever Productions is using RAYZ for visual effects in the 3-D, large-format film Haunted Lighthouse starring Lea Thompson. Due for summer 2002 release at Busch theme parks, the theaters showing the film will incorporate physical effects such as vibrating seats, water and odors.
“We're using RAYZ for all the compositing on this movie”, says Island Fever President and Visual Effects Supervisor Nick Ilyin. Compositing a large-format film shot in 65mm is especially demanding. The film is scanned at 4k resolution instead of the usual 2k typical with a 35mm film. “At 2k resolution performance is pretty reasonable, but at 4k RAYZ is handling four times the normal resolution of 35mm. Then, double that for 3-D. Our machines need a lot more memory to handle that. We use 2GB dual-processor workstations.” The 65mm scans are done at IMAGICA and delivered in Cineon DLT format at 4,096 × 1,536 resolution, stereoscopic. The finished 70mm 5-perf prints will be shown using two projectors and with Polaroid glasses for 3-D.
Island Fever Productions also uses Linux to create commercials, including three for Orkin showing cockroaches crawling on TV screens. Four commercials for Meijer Toy City feature funny animated penguins. “We created a penguin sculpture that was digitized and brought into Maya for 3-D character animation, then to Houdini for deformation and lighting and compositing, and then output to Pixar RenderMan”, says Ilyin. “We have written a lot of custom code to integrate that pipeline as well as a custom lighting package for Houdini.” Houdini's limited built-in compositor isn't powerful enough for projects like Haunted Lighthouse, which is why Island Fever recently moved to RAYZ for compositing.
Ilyin says Linux is definitely getting there, but wishes it were more dependable. He sometimes finds his renderfarm locked up, a problem that the industry generally attributes to issues being worked on in the Linux implementation of NFS. “We're hoping to have everything eventually in Linux, but support can be a problem. Red Hat support isn't enough.” Island Fever uses Linux Red Hat 7.1 and 7.2, SGI IRIX and Windows 2000.
Drew Perttula is establishing a RAYZ community web site at rayz.bigasterisk.com to offer tips, examples, discussions and code (plugin code and external utilities). As a freelance effects compositor, Perttula has worked on several independent films. “I stumbled upon the beta version of RAYZ back in April 2001”, says Perttula.
Since then I've been very active, with more than 600 feature and bug reports. Earlier versions of RAYZ seemed prematurely focused on optimization. Stability has gotten much better lately. In a few weeks the 2.2 version will be coming out with some exciting changes for plugin writers.
For indie film Less Miserables, Perttula is working on a background replacement for a scene shot on Mission Street in San Francisco. Where the street background has the wrong look, it is being adjusted to look more like Mission Street. “It's difficult because the actress' wardrobe presents very soft edges with feathers. Fortunately, I'm able to adapt a University of Washington friend's research to find edges, then import the result into RAYZ.”
Less Miserables is a comedy about what can go wrong on a motion picture set. Director Justin Lomba was turned down by HBO's Project Greenlight but produced the film anyway on a $20,000 budget. In one scene, the gag is that a grip shines a reflector directly into the lens of the camera, blinding the cameraman. “The reflector as shot is actually in a shadow, so we have to add the flare in post using RAYZ”, explains Perttula. “In another shot we're hoping to get a friend at PDI to model a 3-D camera crane that I'll composite in to make it look like a bigger movie set.”
Another shot for Less Miserables needs people added:
A lunchtime scene at a fountain in San Francisco should be filled with people, but the shot was actually done on a weekend when it was more deserted. If we had been called in earlier we might have simply had some friends walk in the background as extras. It is going to take some work to comp the crowds in afterward.
“Projects I worked on at PDI include indie Memories of You”, says Perttula.
People at PDI work on independent projects all the time, and I was brought in by my friend Marlon Montgomery. Director Shane Pollack wanted red rain and other supernatural effects, such as an actor crying a blue tear. That was fairly complex, with a couple hundred nodes in RAYZ and the rain from Maya and stock footage (from www.artbeats.com).
For his own project, Golden Memories, Perttula demonstrates a shot from a scene where a woman on vacation is feeding her fish by sending envelopes of fish food by mail. As it falls in the fish tank the envelope disintegrates. Golden Memories screened recently at ROBOTMEDIA, a monthly festival in Berkeley. “I'm taxing RAYZ by making each fragment of the envelope a block. I used Python and Pygame to create the blocks, so I don't need to manually create all the nodes in the RAYZ GUI.”
“RAYZ appeals to artists because it has a simpler interface than Shake”, says Perttula.
And for programmers, RAYZ has a much better documented API than Shake. With its text-based file formats you can easily create little RAYZ accessories. RAYZ doesn't have a statistics gatherer but I built one in ten minutes. I'm using Perl and Graphviz to create a pretty shot tree diagram of what went into RAYZ and what came out.
Perttula says that the ease of having Perl create an overview of how a whole project works is a testament to how easy it is to read RAYZ files and do interesting things with them.
To get RAYZ, download the license manager (license_manager.linux.intel.tar.gz) and the application (rayz-2.0.i386.rpm) separately from the Silicon Grail web site. Using alien we converted the RPM to install on our Debian system. The alien tool handles conversions between Red Hat RPM, Debian deb, Stampede slp and Slackware tgz formats.
alien -k rayz-2.0.i386.rpm dpkg-deb -c rayz_2.0-20020129_i386.deb dpkg -i rayz_2.0-20020129_i386.deb cd /usr tar xvfz /install/compositor/rayz/z/ license_manager.linux.intel.tar.gz grail/hostinfo grail/Install grail/grailadmin -r localhost grail/graild -l /var/log -k /usr/grail rayz
Using dpkg-deb -c reveals where files will be placed without extracting them. Many Linux programs fail to mention how to launch them, but dpkg-deb makes that easy to figure out. (Using dpkg-deb -Xe is an option if you ever want to simply extract a deb as though it were a tar file without installing it.) Since we see that the main directory is /usr/grail, we chose to install the license manager there, too. The hostinfo command displays your machine's ID number to send to Silicon Grail to get a license key. Rather than installing the license manager as a dæmon, we ran it manually.
The RAYZ development team consists of five developers, all working on Linux workstations. “We made a strategy decision three years ago to focus on Linux”, says RAYZ Product Manager Craig Zerouni. “It was clear to us then that Linux would be the place to be.” Windows and SGI IRIX versions of RAYZ are built from a single Linux-developed codebase of about a million lines of code. Tools used include gcc, g++, CVS and Jam. “We recently switched from make to Jam”, says Zerouni. “Our time for a recompile dropped from hours to minutes because Jam is smarter than make depends.” Head of R&D Kimball Thurston adds, “We can't use autoconf or tools like that cross-platform, but Jam works the same across all platforms.” Using Jam requires rewriting Makefiles into jamfiles. A familiar project that uses Jam is FreeType.
At $9,900 US per GUI and $2,900 US per render thread, RAYZ isn't cheap. However, Silicon Grail is cooperative with plugin developers and university users. Silicon Grail welcomes developers to become third-party plugin providers. “You don't have to be a studio to be involved with RAYZ”, says Zerouni.
Our plugin SDK, documentation and examples are included with the evaluation download. You can use all of RAYZ internal operations (for example, image rotation) as building blocks for your own code. If you just want to develop we can set up a three-month trial license beyond the one-month evaluation license available with the download. Depending on what you have in three months that may be extended.
Companies such as Martian Labs are creating RAYZ plugins, as well as The Foundry in the UK with Keylight and Photron with Primatte.