AssemblyAI releases Voice Standard Bench, scoring over 50 language models on transcription post-processing tasks across quality, speed and cost metrics, with open-source framework available.

AssemblyAI released Voice Standard Bench on October 13, 2026, a public benchmark that scores more than 50 language models on tasks commonly performed after speech-to-text transcription, according to a company blog post published that day. The benchmark evaluates summarization, information extraction, text correction and dictation across four transcript lengths—approximately 30 seconds, 2 minutes, 15 minutes and 60 minutes—reporting quality, response time and cost per hour of audio side by side.

The models tested are accessible through AssemblyAI's LLM Gateway, which the company states now supports over 50 models using the same API key customers use for its Speech-to-Text service. The catalog includes frontier models from major providers as well as open-source models hosted on AssemblyAI's own infrastructure, some of which the company describes as difficult to find hosted elsewhere.

According to the October 13 blog post, Voice Standard Bench tests models on real Speech-to-Text output, including filler words, false starts and transcription errors, at production-typical lengths. Two judge models—Claude Opus 5.5 and GPT-5.6 Sol—score each response against fixed rubrics for each task, with metrics normalized to a 0–100 scale. Dictation tasks are scored mechanically against known-correct output without a judge model. AssemblyAI reports that researchers on its team reviewed hundreds of sample scores to refine the grading prompts.

The company states that the benchmark also tracks format failures, refusals, timeouts and error rates for each model and task combination.

Source link