AssemblyAI published a Python tutorial for a real-time voice agent that combines streaming speech recognition, LLM Gateway and text-to-speech.
AssemblyAI has published a tutorial showing how to build a real-time voice agent in Python that listens, reasons, calls functions and speaks back using three files and one API key. The guide centers on tool calling, and it uses Universal-3.6 Pro Realtime for streaming speech-to-text, an LLM through LLM Gateway, and the AssemblyAI Text-to-Speech API for spoken replies, according to the company’s Oct. 29, 2026 post.
The tutorial says the agent can listen through a microphone, decide when to invoke Python functions, speak the result and stop speaking when interrupted. AssemblyAI frames that as a production-style loop in which the script owns the orchestration: microphone audio goes to Universal-3.6 Pro Realtime, turn events trigger an LLM request with tools, tool results are fed back into the model, and the final response is streamed to text-to-speech.
AssemblyAI argues that the listening layer is a key differentiator for voice agents. In the company’s English voice-agent benchmark, which it describes as 12,460 scripted caller scenarios, Universal-3.6 Pro Realtime posted a 5.19% word error rate versus 7.78% for ElevenLabs Scribe v2, 8.64% for Deepgram Nova-3 and 13.50% for Deepgram Flux EN. For structured values, AssemblyAI says the model recorded a 14.4% entity error rate on the same benchmark, compared with 30.1% for Deepgram Flux EN. These are vendor-reported results from AssemblyAI, not independently verified measurements in this material.
The post also highlights agent_context, a parameter that lets the speech-to-text model ingest the agent’s own last line.
Source link
