Limiting excessive frontier model use makes financial sense, executives at the Evident AI Symposium said.

NEW YORK – Model routing is emerging as a key tool for enterprises looking to curb artificial intelligence costs amid major investments into the technology’s adoption.

Bank of America plans to double its AI budget next year with a focus on triaging more advanced dialogue with its AI-enabled financial assistant Erica, said Hari Gopalkrishnan, the bank’s chief technology and information officer. Bank of America is also working to improve the AI tool’s institutional memory and ability to conduct more complex transactions.

More advanced capabilities will need to be facilitated by more advanced AI models, but that’s not the case for every interaction, Gopalkrishnan said during the Evident AI Symposium last week. The bank’s AI routing layer, Orchestra, uses a small, classifier model to route queries to the most financially effective model, he said.

Managing token spend and routing to lower cost models through Orchestra is an important and effective cost management tactic for the bank, he said.

“We didn’t want to be beholden to any given model,” he said. “We wanted to be flexible, to have the existing small language models, the existing fine-tuned models. We expected fully – and it turned out – that there are simpler classifier models that do very easy decision things that you don’t need a large language model for.”

Smaller, fine-tuned models have also benefited Capital One, said Prem Natarajan, the bank’s chief enterprise AI officer and chief scientist.

Source link