Voice Standard Bench compared more than 50 Gateway models on JSON extraction from transcripts, with rankings varying by transcript length.

Voice Standard Bench has published a transcript-specific comparison of large language models for structured data extraction, arguing that spoken calls pose different problems from PDFs or web pages. In the October 28, 2026 results cited by the post, the top model changes by transcript length, while schema validity and groundedness remain central to the scoring.

The post, published by AssemblyAI on October 27, 2026 and labeled current as of October 28, 2026, says the benchmark evaluates more than 50 models on information extraction from speech-to-text output. It focuses on a fixed JSON shape for participants, action items, decisions and named entities, and it requires models to output only the object, with every key present. If a field has nothing to report, the benchmark uses an empty array or the string "unspecified" rather than a guess or a missing key.

According to the methodology described in the post, scoring begins with a mechanical schema check. Outputs that fail to return valid JSON with all required keys receive the worst rubric score and are also counted separately as schema failures. Valid outputs are then judged on groundedness and completeness by Claude Opus 5.5 and GPT-5.6 Sol, with the two judges tracked separately and then combined. The benchmark runs at four transcript lengths: 30 seconds, 2 minutes, 15 minutes and 60 minutes, using 10 real transcripts per bucket.

Source link