AssemblyAI says transcript summarization quality varies by voicemail, short calls, support calls and meetings, with cost and latency trade-offs.

AssemblyAI says there is no single best large language model for summarizing transcripts, arguing that the right choice changes with call length, cost and latency. In a post published Oct. 28, 2026, the company said its Voice Standard Bench ranks models on real speech-to-text output across 30-second voicemails, 2-minute calls, 15-minute support calls and 60-minute meetings.

AssemblyAI said transcript summarization is not the same as document summarization because spoken input is disfluent, repetitive and often multi-speaker, and because transcript length can vary by two orders of magnitude within the same product. The company said short inputs can be handled well by many models, while coverage and faithfulness become more important as calls get longer.

On 30-second voicemails, AssemblyAI said 41 of the 52 models on its Summarization tab score 90 or higher, with differences driven mainly by speed, price and whether the model pads the answer. At 15 minutes, it said only 10 models score 90 or higher, and at 60 minutes the spread between the top and bottom rankings is widest.

The Voice Standard Bench is open source and tests Gateway models on summarization, information extraction, text correction and dictation cleanup, according to AssemblyAI. For summarization, each model receives the same prompt: write a plain-prose summary a busy person could read in place of the call, covering the topics discussed, then decisions, then action items, and keeping the length scaled to the call.

Source link