Where there’s smoke, there’s an Oscar

These explosions from the 2012 film Battleship were created using wavelet turbulence technology Universal Pictures

Although no Swiss were in the running for Oscar glory on Sunday, two behind-the-scenes heroes from Zurich – whose work has contributed to several blockbusters over the years – have been awarded a “Technical Oscar”.

Markus Gross, professor of computer graphics at the Federal Institute of technology in Zurich (ETHZ) and director of Disney Research Zurich, tells swissinfo.ch about mixing science, business and art – and why he didn’t patent his award-winning software.

On February 9, Gross, his former post-doc student Nils Thürey and two other scientists from Cornell University attended a ceremony in Beverly Hills where they received a Technical Achievement Award.

Markus Gross

1986 Diploma Degree in Computer Engineering, Saarland University, Germany

1989 PhD, Saarland University, Germany

1990-1994 Computer Graphics Center, Technical University Darmstadt, Germany

1994-1997 Assistant Professor of Computer Science, ETH Zurich

1997-present Full Professor of Computer Science, ETH Zurich

2004-2008 Head of the Institute of Computational Science

2008 Director of Disney Research Zurich

Their success is down to two words: wavelet turbulence. In 2008, they developed software that could calculate smoke and explosions in films quickly and recreate them realistically (see video).

Their software caught the attention of DreamWorks, the film studio co-founded by Steven Spielberg, which put it to use on Monsters vs Aliens in 2009. Since then, it’s become the industry standard and has been used in about 20 major Hollywood productions including Avatar, Battleship, Sherlock Holmes, Alice in Wonderland, Hugo, Kung Fu Panda, The Amazing Spider-Man and the forthcoming Iron Man 3 and Man of Steel. 

swissinfo.ch: You didn’t start your work on wavelet turbulence in order to win an Oscar. How did your work on this subject start?

M.G.: We are academics, researchers – and often researchers meet at conferences, develop ideas and then start collaborating on the ideas with a goal to publish a paper eventually, to come up with an innovation.

And this is what we did. We met at the annual ACM SIGRAPH conference [the leading conference for computer graphics], where the idea was born for this particular technology. We then worked out this idea, figured out the concept, did the implementations and finally submitted our research to the ACM SIGRAPH conference. It was accepted and published the subsequent year.

We then put the code online free for everybody to use. We never patented it or filed any IP [intellectual property] on it.

swissinfo.ch: Why not?

M.G.: We thought that if we made it available to the public and made it free for everybody to use, it would be picked up more quickly and it would migrate more quickly into applications.

Wavelet turbulence is a very good example of how essential open access to innovations is for the rapid transfer of technology into industry. From an academic perspective, this can make a lot of sense and you can get a lot of rewards for that – the Tech Oscar is an example. Of course, from an industrial perspective, things are different: there you might want to patent and protect it.

I also think that had we patented it, the effects studios, which are under a lot of financial pressure and operate at a very low margin, would not have picked it up. The burden of clarifying IPs, of paying special fees for licences and so forth would have been prohibitive. But with this open access and open source code, there was basically no hurdle for them, and it’s a technology that really provides a tangible improvement to the state of the art – it reduces costs and improves the visual quality. So they just picked up the code and integrated it into their own proprietary workflows. This is why it migrated so quickly into many Hollywood movies.

swissinfo.ch: How did the collaboration between ETHZ and Disney begin?

M.G.: This started four years ago. Disney Research is part of the company’s strategy to focus on technology. As part of this strategy, the decision was taken within Disney to create external research laboratories close to world-class research institutions. Since we at ETH had been working on visual computing and computer animation special effects for a long time, they identified us as one of two places to create these labs and build them up. The second place is at Carnegie Mellon University [Pennsylvania, United States].

swissinfo.ch: What are the advantages for both sides?

M.G.: For the ETH it’s wonderful. First of all we gain access to a spectrum of very interesting applied problems in the field of entertainment. We can work on novel technologies that in principle can be seen by billions of people in films or in theme parks. In particular for the ETH it’s interesting because we create IP a lot – we are filing patents and these patents belong to both partners, so the ETH gets a lot of patents. Also, Disney sponsors a significant number of PhD students at the ETH who work on Disney topics.

On the other hand, for Disney, the ETH provides an intellectual and physical infrastructure – the labs were designed to permit a high degree of permeability between the two institutions. So the senior researchers who we hire and work for Disney can get involved in graduate teaching, work with PhD students or co-supervise Masters students – it’s almost like a university environment, but at the same time we are very cautious in the way we protect intellectual property and so on.

So this high degree of permeability gives Disney access to a category of researchers which would otherwise be difficult to get in an industrial research lab.

Academy Award for Technical Achievement

“In recognition of the critical role played by science and technology in the moviemaking process, the Academy has, almost since its inception, honoured the achievements of pioneers in these fields whose work has advanced the motion picture industry.

First presented at the 4th Academy Awards ceremony in November 1931, Scientific & Technical Awards are conferred in recognition of original developments that result in significant improvements in motion picture production and exhibition.

Recent recipients of Scientific and Technical Oscars have included IMAX, for its method of filming and exhibiting large-format, wide-angle motion pictures; Avid Technology, for its non-linear editing system; and Horst Burbulla, for his invention and development of the Technocrane telescoping camera crane.”

(Source: AMPAS)

swissinfo.ch: As you say, billions of people have seen your work, but often the aim of good visual effects is not to be noticed. Doesn’t that bother you after all your hard work?

M.G.: That’s exactly the aim. We want to mimic reality with a lot of special effects. With all the physics simulations we’re doing, we really want to mimic physical effects. But in our case we want to create methods and effects that are as natural and as visually indistinguishable from reality as possible.

swissinfo.ch: Are you happy with the results on the screen? Do you go to the cinema often?

M.G.: Oh yes – maybe once a week. I watch two different genres: one is driven by special effects, so fantasy movies and sci-fi movies. Also lots of 3D movies because we also do a lot of work with stereoscopic 3D. But I watch these films essentially to judge the quality of the visual effects – it’s less about the story.

And then I like story-driven movies where the actors are in the foreground and which are not so heavy on visual effects, like Carnage or The King’s Speech – really good movies.

swissinfo.ch: What are you working on at the moment? What will you win your next Oscar for?

M.G.: Right now we’re focusing on facial animation – we’ve developed a very nice ‘markerless’ facial motion capture system that allows us to track and reconstruct facial geometry down to pore level, down to the micro-geometry. This is a fundamental technology that is essential for all sorts of visual effects on human faces – which is still some of the most difficult visual effects because of the high sensitivity of the human visual system with respect to human motion…

swissinfo.ch: Uncanny Valley…

M.G.: Yes, it’s the Uncanny Valley EffectExternal link – we’re very much focused on that. I hope that some years down the road this will all migrate into major productions and maybe be considered by the Academy for an award.

Oscar winners

Winners at the 85th Academy Awards, which took place on February 24, 2013, at the Dolby Theatre in Los Angeles, include:

Best Picture: Argo

Best Director: Ang Lee, Life of Pi

Best Actor: Daniel Day-Lewis, Lincoln

Best Actress: Jennifer Lawrence, Silver Linings Playbook

Best Animated Feature:Brave

Best Foreign Language Film:Amour (Austria)

Best Documentary:Searching for Sugar Man

Best Visual Effects:Life of Pi

No Swiss were in the running. Markus Gross, his former post-doc Nils Thürey, Cornell professor Doug James and his former researcher Theodore Kim received the Technical Achievement Award at a ceremony in Beverly Hills on February 9.

Certain film scenes that were generated using the award-winning software were screened at the main ceremony.