ElevenLabs launched Eleven Realtime on Tuesday, introducing two managed voice-agent models, Mini and Max, that integrate speech recognition, reasoning, turn-taking, and voice generation. Both use a cascaded architecture, but developers can’t select or independently configure the underlying models.

With this launch, the company is effectively taking over the voice-agent harness, including turn detection, interruption handling, and orchestration.

With this launch, the company is effectively taking over the voice-agent harness, including turn detection, interruption handling, and orchestration. That reduces the work required to build and maintain a voice-agent stack, but limits developers’ ability to independently troubleshoot, optimize, or replace its underlying components.

Developers can configure prompts, voices, tools, knowledge bases, and turn-taking settings, including eagerness, timeouts, and interruption handling, through ElevenAgents.

The alternative is Custom mode, which lets developers select the underlying language and speech models and configure the pipeline themselves. Realtime trades that flexibility for a managed implementation. Developers retain control over agent behavior, but not the individual models responsible for it.

Artificial Analysis generated the performance figures by evaluating both models on speech reasoning, conversational dynamics, and human preference. On Big Bench Audio, a benchmark of 1,000 spoken reasoning questions, Max scored 98.8%, compared with 75.9% for Mini.

ElevenLabs recommends Max for agents that make frequent or complex tool calls.

The gap was even more apparent in tool use. 3%. ElevenLabs recommends Max for agents that make frequent or complex tool calls. Developers building agents to handle bookings, customer accounts, or other multi-step workflows will want to test that difference for themselves.

Source link