It’s two Oscars and counting for Yale computer scientist Theodore Kim

Chances are, if you’re a movie fan you’re familiar with Theodore Kim’s intricate pixel work.

From the Sorting Hat in “Harry Potter and the Sorcerer’s Stone,” to the dancers’ flowing skirts in “Coco,” Kim’s computer animation algorithms have brought a deeper, more textured realism to dozens of beloved films.

He now has two Academy Awards to prove it.

Kim, an associate professor of computer science in the Yale School of Engineering & Applied Science, co-leader of the Yale Computer Graphics Group, and former senior research scientist at Pixar Research, was in Los Angeles Feb. 24 to accept his second Oscar at the academy’s annual Scientific and Technical Awards ceremony. The live Oscars broadcast will be March 12.

Kim and his collaborators from Pixar Animation Studios — David Eberle, Fernando de Goes, and Audrey Wong — received certificates for the design and development of the Fizt2 elastic simulation system, an animation simulator that depicts the movement of soft materials such as cloth. Kim won his first Oscar in 2012 for the program Wavelet Turbulence, which deepened the realism of animated explosions and fires.

Kim recently spoke with Yale News about his achievements in computer animation.

What led you to study computer science and computer animation?

Theodore Kim: The first ‘Toy Story’ movie came out in 1995 when I was in high school, and it was a real moment where I said, “Wait, you can make a whole movie with computers? Can I do that?”

When I got to college, at Cornell, I wanted to take computer graphics classes. Usually that is pushed off until later in the course sequence, but I was pretty keen to do it. I took it my sophomore year. They also had a graduate-level class, so I took that, too.

This sort of led me, accidentally, to my first film industry job. I applied for a summer internship at the special effects studio Rhythm & Hues, and because I had taken a graduate-level class, they mistook me for a Ph.D. student. My work ended up in the first Harry Potter movie. They had just landed the Harry Potter contract — I don’t have a credit in the movie or anything like that — but the project my boss had given me was used on the Sorting Hat [a talking wizard’s hat that, when placed on students’ heads, “sorts” them into their different houses at Hogwarts School of Witchcraft and Wizardry].

Then I went the academia route, getting my Ph.D. at UNC-Chapel Hill, getting postdoctoral appointments, and then becoming a professor. I left a tenured position to become a senior research scientist at Pixar after my first Oscar and returned to academia when I came to Yale in 2019.

Do you remember seeing your finished work in a movie for the first time?

Kim: It was very cool, but it wasn’t on the big screen. My bosses from the internship made a recruiting trip to UNC-Chapel Hill, where I was in graduate school, and brought footage of the Sorting Hat. It was 48 hours before the movie was to be released, so they said, “OK everyone, this is top secret for the next two days. Here’s what Ted did over the summer!”

Your work seems like such a merger of art and science. Is that how you approach it?

Kim: I usually start by asking: What is the artistic result I want? From there, I can start asking: What is the math that will get me there? Sometimes you find that someone else has already done some of the work, for a different purpose, and you can deploy a piece of it for an artistic purpose. But at some point, you have to drill down into the math yourself. There’s no getting around that. There’s always more interesting math you can uncover, because you want your art to look better, and better, and better.

How is research different in academia as opposed to working in industry?

Kim: Research in industry is very different. If you’re working on a movie, research means that you have two weeks to figure out an answer to a very specific problem. Even at Pixar — where I was in the research group and had more time to develop things — the horizon is about a year and the motivating question is always, “How does this help us make a movie?”

In academia, if you’re only looking ahead one year, you’re not doing it right. You need to be looking forward five or ten years.

Do you see yourself as a role model for young people and persons of color who may want to embark on a career in computer animation?

Kim: As a scientist and teacher, it’s very important to me to expand representation and explore the possibilities of this field. It’s something that our discipline still struggles with. For example, only a dozen sitting professors have ever won a Sci-Tech Academy Award, and aside from me, they are all white men. Even more broadly, my colleague, Audrey Wong, whom I shared this year’s award with, is the first Asian-American woman to ever win a Sci-Tech Oscar. Lots of voices are still missing from this research.

Does having a technical insight into how movies are made change the way you watch films?

Kim: At a certain point, I really started to appreciate the techno-auteurs, directors like James Cameron or Steven Spielberg or Stephen Chow, who have an extremely precise understanding of what technology is good at and how it helps the storytelling — and also knowing when a practical special effect will help the story better. Watching some of these old movies, like Terminator 2, you can appreciate how Cameron knew exactly when to deploy the computer graphics and when not to. It really deepened my appreciation of these movies.

What computer animation innovations do you expect to see in the next few years?

Kim: Every year, my colleagues and I try to ask: What is the thing we haven’t seen yet in a story and what visuals haven’t we seen already? How can we tell those stories, make something that nobody has seen before? That is the driver for selecting problems and deciding what to work on next.