Much of modern AI runs on multiplication. Neural networks behind everything from generated answers to photo organization and song recommendations perform millions or billions of operations that multiply inputs by learned weights. Lizy K. John thinks that’s more work than the job requires.

John, a professor of electrical and computer engineering at the University of Texas at Austin, has spent the past five years working on a class of models called weightless neural networks. Instead of repeatedly multiplying inputs by weights, these networks pass binary inputs through interconnected lookup tables—closer to consulting a collection of stored answers than solving the same arithmetic problem repeatedly. Depending on the task, she says, the networks can be less than a thousandth the size or 1,000 times as fast as conventional alternatives while maintaining comparable accuracy.

Her team’s work has so far focused on small, specific problems: medical sensors, activity tracking, keyword spotting. But she thinks the same approach could eventually reach much bigger targets, including the transformer models behind today’s chatbots.

What made you walk away from weights in the first place? Was there a specific moment that pushed you toward lookups instead?

Lizy K. John: A friend casually invited me to a weekly meeting a few years ago to talk about using lookups instead of weights, a technique that wasn’t new. K. had built a commercial product around it in the ’80s for pattern recognition, and then it just disappeared. Professors Felipe M. G. Franca and Priscila M. V. Lima at the Federal University of Rio de Janiero kept working on it for years, just one research group, so not as much work as in the conventional neural networks.

Source link