Most benchmarks focus on short-form English transcription (<30s), and overlook other important tasks, such as (1) multilingual performance and (2) model throughput, which can a be deciding factor for long-form audio like meetings and podcasts.

Over the past two years, the Open ASR Leaderboard has become a standard for comparing open and closed-source models on both accuracy and efficiency. Recently, multilingual and long-form transcription tracks have been added to the leaderboard 🎉

As of 21 Nov 2025, the Open ASR Leaderboard compares 60+ open and closed-source models from 18 organizations, across 11 datasets.

In a recent preprint, we dive into the technical setup and highlight some key trends in modern ASR. Here are the big takeaways 👇

Models combining Conformer encoders with large language model (LLM) decoders currently lead in English transcription accuracy. For example, NVIDIA’s Canary-Qwen-2.5B, IBM’s Granite-Speech-3.3-8B, and Microsoft’s Phi-4-Multimodal-Instruct achieve the lowest word error rates (WER), showing that integrating LLM reasoning can significantly boost ASR accuracy.

💡 Pro-tip: NVIDIA introduced Fast Conformer, a 2x faster variant of the Conformer, that is used in their Canary and Parakeet suite of models.

While highly accurate, these LLM decoders tend to be slower than simpler approaches. On the Open ASR Leaderboard, efficiency is measured using inverse real-time factor (RTFx), where higher is better.

For even faster inference, CTC and TDT decoders deliver 10–100× faster throughput , albeit with slightly higher error rates. This makes them ideal for real-time , offline , or batch transcription tasks (such as meetings, lectures, or podcasts).

Source link