AssemblyAI published a build guide showing how its streaming speech-to-text, LLM Gateway and text-to-speech services can power one voice agent loop.
AssemblyAI has published a hands-on guide for building a voice agent on its own stack, combining streaming speech recognition, LLM access and text-to-speech on one API key. The company says the approach is meant to replace the older pattern of wiring together separate vendors for listening, reasoning and speaking.
In the Oct. 29, 2026 guide, AssemblyAI says a voice agent can be assembled from Universal-3.6 Pro Realtime for streaming speech-to-text, LLM Gateway for tool-calling language model requests, and AssemblyAI Text-to-Speech for the spoken response. The example uses standard JSON over WebSocket or HTTP, and the company says the loop can be written directly or handled through orchestrators such as LiveKit or Pipecat.
The guide also lays out the protocol details it says developers need to manage, including authentication with a raw API key in the Authorization header, 16 kHz 16-bit mono PCM input for speech recognition, and 24 kHz output from the text-to-speech service. AssemblyAI says Universal-3.6 Pro Realtime sends Turn messages with built-in end-of-turn detection, while the TTS side supports Cancel for interruption handling.
AssemblyAI cites its English voice-agent benchmark, which it says includes 12,460 scripted scenarios. 1% for Deepgram Flux EN. The company says the entity-error metric matters most when agents must capture names, phone numbers or email addresses correctly.
Source link
