What's it like to work at Industrial Light & Magic? Ask this U of T grad

In Star Wars Episode V: The Empire Strikes Back, Luke Skywalker must trust himself to The Force. He is told by Master Yoda: “Do or do not. There is no try.” 

There is truth in these words for Noah Lockwood, PhD candidate in the department of computer science at the University of Toronto and a research and development engineer at Industrial Light & Magic (ILM), one of the world’s leading visual effects companies. 

“In R&D, some of the work is like academic research – trying and potentially failing,” he says. “However, one of the cores of the ILM spirit is that we always get the work done and finished on time. It’s something that’s really cool about this place, that’s also a little Jedi-like.” 

Lockwood is part of a R&D team that develops computer-generated imagery (CGI) tools used in films such as Star Wars Episode VII: The Force Awakens and Jurassic World. 

“Working at Industrial Light & Magic was a lifelong dream of mine,” says Lockwood, who completed two internships at ILM before joining the company. “What’s really fun about ILM is that everyone is excited about every film we work on and almost everyone is a Star Wars fan.”

Lockwood spoke with U of T writer Nina Haikara about his interest in computer graphics and the magic behind movies. 

Why did you choose to study computer graphics?
I was interested in computer graphics as someone who played video games and really enjoyed visual effects in film and television. I knew that computer graphics lay at the end of an undergraduate computer science program, after a lot of foundational work in computer science and math. When I was able to take the fourth-year graphics course, it was very inspiring – to finish a week’s worth of lectures and realize how it was done in a movie, or video game.

What is your thesis research about? 
My PhD research is on animation algorithms and changing how an animated character moves. If I have an animation of someone walking forward, I might want them to walk along a particular path instead. To create that new animation the normal way, I have to animate it by hand, bending every virtual joint, or I have to put someone in a motion capture suit.

But wouldn’t it be nice if I could take pre-existing motion data and edit it with a few clicks? 

I created an algorithm to do that, and then we’ve used it with different input technology including a touch sensitive tabletop – to walk with our fingers – to tell the system how the character should move. I’ve also done it on a mobile device, so moving the mobile device around controls the animation that way.

Is your research connected to your work as a visual effects R&D engineer?  
My research isn’t directly used at ILM but it’s related in the ideas of it, because my role as an R&D engineer is to make tools for artists, including animators. Technology work here is balanced between day-to-day, very short-term problems and developing long-term researched technology we can use repeatedly for very complex problems. 

Which films have you worked on? 
In technology, we often do work that supports everyone in the company, and all of the artists. It’s incredibly satisfying to see this work on screen that we’ve helped facilitate, like Jurassic World, and Star Wars Episode VII: The Force Awakens. 

For Jurassic World in particular, we developed a lot of new technology that was built on years of work at ILM so that we could do motion capture and show the virtual character in real-time.

If you’re doing animation the traditional way, it takes a very long time, and it’s very difficult to change – you have to throw it out and start from scratch. 

Jurassic World Velociraptor (Ilya S. Savenok/Getty Images)

The velociraptors in the original Jurassic Park were actually people in suits, live on set, working with the actors. For Jurassic World, we essentially put our performers in a virtual dinosaur suit. That was really important because there were four velociraptors, all interacting with each other. And because it’s seen in real-time, they can change it. We also did it for the Tyrannosaurus Rex and the Indominus Rex, who have a big fight. The performers were fighting and we could see the dinosaurs fighting at the same time on screen – it was really awesome. It was a fun way of doing very complex animation for these complex creatures that were really the stars of the show.

In Star Wars Episode VII: The Force Awakens, there’s a sequence in the middle of the film, where there are these big alien creatures called rathtars. They’re running around, grabbing things, and grabbing people with their tentacles – they’re scary! That was all animated by hand, but tentacles are something that are very difficult, traditionally, for animators to control because the virtual skeleton of a character is represented as a series of joints. If you’re doing something that can bend in the way that a tentacle can, representing it as a large number of joints is really difficult. 

So we saw some really cool technology from our friends at Pixar, using mathematical curves to represent tentacles. We were able to take the ideas from Pixar and adapt them for our needs, because visual effects are different than a feature-length animated film. We developed a new tool for tentacles, so they can easily wrap around things and stick to objects as the characters move around, and so on. We’re using it again in films that are currently in production. 

Have you met U of T computer science alumnus William “Bill” Reeves, Global Technology Supervisor at Pixar?
Unfortunately I have not met Bill Reeves, but I have heard him speak. He is, of course, a legend in computer graphics. He was at ILM, actually, when he invented a very powerful idea in computer graphics called Particle Systems, which was first used in the visual effects for Star Trek II: The Wrath of Khan. 

Visual effects and animation are different facets of filmmaking, and at ILM and Pixar, we’re able to learn from each other now that the walls have come down and we’re all part of one big company. It was like a homecoming. Pixar started as a part of Lucasfilm. 

Read more about William Reeves

How do you compare the technological advances through the different periods’ Star Wars films to today’s visual effects? 
One thing that is still really exciting about the original trilogy was that it was done practically on set and with miniatures and paintings since the digital technology didn’t exist yet. 

The popular opinion seems to be, in the prequel trilogy, a lot of the digital visual effects felt artificial. Jar Jar Binks, regardless of people’s opinion of the character, was a landmark achievement in terms of an extremely prominent, all-computer-generated character. Those films pushed the work that needed to be done on digital cameras and digital projection. There was a lot of CGI work done on those films that set a really high standard and influenced the industry a lot. 

The goal of visual effects is to make it seem like no work was done at all. Audiences have very discerning eyes, more so every year. In the first and second trilogies things were perhaps in one extreme or the other. From no computer graphics, to a lot of computer graphics. 

For the new Star Wars movies, that was very much kept in mind. There was more effort to do effects practically, to have real sets, instead of green screen. There’s still a tremendous amount of visual effects work done, much more than audiences realize, but it’s building on the stuff that’s already there. Like BB-8 – the beach-ball droid that rolls around – in many shots, there was a real guy there, puppeteering it, and the work is to paint that out, and augment it, rather than create everything from scratch using computers. 

How is CGI technology keeping pace with Hollywood? 
If you look at the Oscars, the nominees for best visual effects are massively more money-making than the nominees in any other category. The technology is important for Hollywood, financially.

The thing about visual effects that’s always impressed me, is that it takes an incredibly trained and skillful artist to know what to change in order to take the work from good to great. I’ve sat in on sessions where they’re reviewing a shot after dozens and dozens of changes and it’s still not quite right, and then, someone will point out this little thing that needs to be changed and everyone knows it will instantly look better. Every detail makes a big difference.

There are a lot of people who have transitioned from working in physical effects, painting or sculpting, to working digitally, because that artistic eye is most important. If that’s not there, then no amount of technology can make up for it. Our goal is to help the artists do that incredible work.

Yoda said to Luke in Star Wars Episode V: The Empire Strikes Back: “It is finished. No more training you require.” Will you feel like Jedi after defending your PhD this year? 

I try not to think about it too much. But it will be a big change. Feeling less like an apprentice and the start of being a master at something. It’s very exciting.