AssemblyAI says voice agents should be tested on worst-case telephony audio, peak concurrency, and billable session handling before launch.

AssemblyAI has published a technical guide on how to load test a voice agent before launch, arguing that demo performance on a single quiet-room call does not reflect production conditions. In the post, published on 2026-10-20, the company says teams should replay their hardest real calls through the full pipeline at projected peak concurrency and measure entity accuracy, P95 latency, rejected connections and cost per completed task.

The article says load testing should cover the full voice stack — speech-to-text, the LLM, text-to-speech, tool calls and telephony — and should separate concurrency, throughput and session duration, because each fails differently. AssemblyAI says a callback campaign that opens hundreds of calls in a short window can hit a different limit from a long-running concurrency load, and that most teams test only the first case. The guide also recommends four profiles: steady state, linear ramp, spike and soak.

AssemblyAI’s examples focus on production-like inputs rather than clean audio. The post says teams should build a corpus from their worst calls, including 8 kHz telephony compression, entity-heavy content such as account numbers and confirmation codes, real accents, background noise, interruptions and false starts. It also warns that human transcription labels can contain omissions or cleanup that distort word-error metrics, and says entity accuracy should be weighted above word accuracy when scoring voice workflows.

The article says the test should prove that sessions close cleanly and that ramp-up stays inside the provider’s limits.

Source link