Microsoft plans to break free from NVIDIA and deploy its next-generation Maia 300 AI accelerator in 2027, targeting 40% more tokens per megawatt than leading GPU-based systems. The chip will power Microsoft’s expanding AI infrastructure, with a 96-accelerator rack configuration designed to deliver up to seven times the token throughput of its predecessor.

The announcement marks another step in Microsoft’s effort to build custom silicon for large-scale AI inference. Its Maia 200 accelerator already handles production workloads, including Microsoft Copilot. Microsoft expects GitHub Copilot and Microsoft Foundry to adopt the hardware by the end of 2026.

Maia 300 takes a system-level approach to inference efficiency. Its architecture targets memory-intensive workloads, long-context reasoning, and AI agents that execute complex tasks across multiple steps.

Maia 300 pairs eight HBM4 memory stacks with 18 terabytes per second of peak bandwidth. The system operates within a 950-watt thermal design power envelope, addressing the growing memory demands of large AI models.

Microsoft has also adopted a multi-chiplet design that separates compute, memory controllers, and input/output functions. Compute chiplets use TSMC’s N2P manufacturing process, which aims to improve performance per watt.

The accelerator also supports enhanced MXFP4 low-precision computing. This approach reduces memory requirements for model calculations and key-value cache storage. Those savings can help systems accommodate longer context windows and serve more requests within the same power budget.

Software plays an important role in the design. Microsoft’s software-defined dataflow architecture aims to keep data close to compute resources, reducing unnecessary transfers. AI-assisted kernel optimization also helps improve hardware utilization across different workloads.

Source link