Amin Karbasi Wins Air Force's Young Investigator Award

Deep neural networks – a type of very complex architecture that can essentially “learn” by analyzing data – work incredibly well, playing a role in everything from Apple’s Siri to driverless cars. No one, however, is entirely sure of how they work.

As part of his project, which won the Young Investigator Award from the Air Force Office of Scientific Research (AFOSR), Amin Karbasi is trying to figure that out. The $450,000, three-year project involves creating rigorous mathematical formulas that will help define and measure and interpret the data in big data. Today, the amount of existing data far exceeds what Claude Shannon (the father of information age) imagined when he developed his Information Theory. Making sense of it will take some time.

“Everyone talks about big data, but it’s not clear what’s in it, so how do we measure it?” said Karbasi, assistant professor of electrical engineering & computer science. “Is it all garbage, or is there some interesting pattern?”

Part of the project entails deep neural networks, which have become increasingly prominent in computer science. It’s an old architecture but it has recently received a great deal of attention due to its success in artificial intelligence. Karbasi, who is a faculty member at the Yale Institute for Network Science (YINS), said there are a lot of guesses about why deep neural networks work, but no definitive answers.

“If you go to a machine learning conference, probably 80% of the people are talking about deep neural networks, and they’re talking about the amazing applications they can have,” he said. “But unfortunately, little is known about neural networks and it’s not very clear what they do.”

One theory holds that deep neural networks work because they can process a huge amount of data. For instance, a network is fed numerous pictures of dogs and cats, and it learns all the variants of the two animals. They can then identify new pictures of dogs and cats by mapping them back to previous pictures they’ve processed.

Karbasi is approaching the question from a different direction, positing that a small, but representative, amount of data could suffice for deep neural networks to work effectively. That way, a system could gain in computation but no information would be essentially lost.

“We’re trying to come up with some empirical and rigorous studies to show that even with a small amount of data, it can perform very well. But the data needs to be chosen in a smart way. To find an optimal way of finding informative data, we need to combine ideas from computational geometry, non-convex optimization, and high dimensional statstics.”

The Young Investigator Research Program awarded approximately $19.1 million in grants to 43 scientists and engineers from 37 research institutions and small businesses. The awards are made through a competitive process and recognize scientists “who show exceptional ability and promise for conducting basic research.” This year, the AFOSR received over 285 proposals.