Microsoft unveils Maia 300 inference accelerator for 2027 deployment
Microsoft says its multichiplet Maia 300 will enter production deployments in 2027, with 288GB of HBM4 per accelerator.
By Dominic Okoye · Staff Writer
· 3 min read
Microsoft on Oct. 12 unveiled the Microsoft Maia 300 accelerator, the third generation of its in-house Azure Maia chips for AI inference workloads. The company plans to start production deployments in 2027 and says it intends to make Maia its primary inference system across Microsoft products and services over time.
Maia 300 is part of Microsoft’s effort to build more of the infrastructure behind Copilot, its cloud AI services and internal model work. Reuters reported that Microsoft is planning potentially to deploy more than 1 million Maia 300 units next year for internal AI services, a plan rather than a committed shipment figure. Microsoft has also described multi-gigawatt Maia deployments beginning in 2027.
The announcement arrives as cloud operators and AI companies develop custom accelerators that could reduce their reliance on Nvidia hardware. Microsoft did not disclose a like-for-like workload comparison with competing systems.
How does Microsoft Maia 300 work?
Maia 300 is optimized for inference, the work of running AI models to produce responses and carry out tasks. Its design centers on reducing movement of data between compute and memory, which Microsoft identifies as a constraint in large inference systems, especially for long-context and decode-heavy workloads.
The accelerator splits functions across multiple chiplets. Its compute chiplets use TSMC’s N2P process, while separate dies handle memory-control and input/output functions. Microsoft says this lets it use different manufacturing technologies according to each function’s performance, bandwidth, connectivity and cost requirements.
Each accelerator has eight HBM4 memory stacks with 288GB of capacity, 18 TB/s of peak memory bandwidth and a 950-watt thermal design power. Microsoft also uses a locality-aware, non-uniform memory architecture across the chiplets. Software can place data nearer to the compute resources that need it, while software-managed SRAM, DMA engines and low-precision data formats such as MXFP4 are intended to reduce traffic to high-bandwidth memory.
Microsoft says those choices are aimed at long-context reasoning, latency-sensitive conversations and multi-step agentic workloads. The company expects Maia 300 to produce 40% more tokens per megawatt than unnamed leading GPU-based systems. Reuters reported that Microsoft did not identify the systems used for that comparison, and the supplied reporting provides no independent benchmark or like-for-like workload result.
What changes from Maia 200?
Memory is the clearest generational change. Maia 200, which Microsoft introduced in January and says is already handling production workloads, has 216GB of HBM3e and 7 TB/s of memory bandwidth. Maia 300’s 288GB amounts to 33% more capacity, while its 18 TB/s is about 2.5 times the earlier chip’s bandwidth, consistent with Microsoft’s figures.
The system grows with the chip. A Maia 300 rack contains 96 accelerators, providing 27.6TB of aggregate HBM capacity and about 1.7 PB/s of aggregate memory bandwidth within a 167-kilowatt power envelope, according to Microsoft. The company says a fully configured rack can deliver up to seven times Maia 200’s token throughput, another company projection.
Its Ethernet-based scale-up network combines electrical connections within a rack with an optical second tier across racks. Microsoft says the design can connect as many as 12,288 accelerators in a superpod. Whether those system-level claims translate into better economics for particular services will depend on production deployment and workload results that were not included in Microsoft’s disclosures.
This story draws on original reporting from SiliconANGLE.
