AMD says EAGLE-4 speculative decoding runs on Instinct MI355X GPUs in SGLang and ATOM, with benchmark gains over plain decoding and built-in MTP.
AMD said its EAGLE-4 drafter now runs on Instinct MI355X GPUs from day one in both SGLang and ATOM, positioning the latest speculative-decoding system for lower-latency text generation on the company’s data-center accelerator.
In a technical article published on 20 October 2026, AMD said EAGLE-4 uses the same checkpoint and launch flags as on other hardware and can run out of the box on the MI355X. The company said it measured EAGLE-Harness cost tables on the GPU so tree sizes fit the device, added tree verification for Qwen3.8’s linear-attention layers, and checked that outputs matched plain decoding up to BF16 rounding at near ties.
AMD’s benchmark table for Qwen/Qwen3.8-27B in BF16 on one MI355X GPU showed one-request throughput of 289 tokens per second in SGLang and 309 tokens per second in ATOM for EAGLE-4, versus 82 tokens per second for plain decoding and 178 tokens per second for the model’s built-in MTP in SGLang. AMD said that amounts to 3.5x plain decoding and 1.6x built-in MTP in its SGLang one-request test, with about 4.7 tokens accepted per check for EAGLE-4 versus about 2.9 for built-in MTP.
The company said the gains held as load increased. On one MI355X GPU, AMD reported EAGLE-4 at 941 tokens per second with four concurrent conversations, 2,167 at 16, and 2,784 at 32 in SGLang; ATOM posted 976, 2,309 and 3,243 tokens per second at those same concurrency levels.
Source link
