Recently, Microsoft officially launched Maia 200, its second-generation in-house AI accelerator focused specifically on inference workloads. This announcement marks a significant leap in Microsoft’s independent silicon strategy, while simultaneously posing a direct challenge to main competitors like Google and Amazon in the field of AI cloud computing.
Based on the announcement, Maia 200 is built upon the foundation of Maia 100, with the primary difference lying in the process technology and the resulting efficiency. While Maia 100 utilized TSMC’s 5nm process, Maia 200 has transitioned to the newer TSMC 3nm process, which allows for increased transistor density and better power efficiency.
From a specification standpoint, the Maia 200 is quite impressive, supporting 216GB of HBM3e with bandwidth reaching 7 TB/s. It also features 272MB of on-chip SRAM to accelerate data access, native Tensor core support for FP8 and FP4, and is equipped with an integrated Network-on-Chip (NoC) with a bidirectional bandwidth of 2.8 TB/s. This enables ultra-fast communication within large clusters of up to 6,144 accelerators.
Microsoft also publicly released a comparison table placing Maia 200 directly against competing chips. Based on the released data, Maia 200 produces FP8 performance nearly double that of Amazon’s third-generation Trainium accelerator and approximately 10% higher than Google’s seventh-generation TPU. These performance claims demonstrate Microsoft’s confidence that they are not only catching up but have also surpassed the pioneers in the field of custom AI silicon.
In addition to much better performance, Microsoft emphasized the business efficiency aspect of Maia 200, claiming that this new accelerator offers 30% better performance-per-dollar compared to the latest generation hardware currently utilized in Azure. Furthermore, unlike the Maia 100 which was announced long before its actual implementation, Maia 200 has already been deployed and is currently operational.
Reportedly, this chip has been deployed in Microsoft data center regions in US Central (near Des Moines, Iowa) and US West 3 (near Phoenix, Arizona), showcasing design maturity and production readiness. To support this infrastructure, Microsoft recently collaborated with SK Hynix for an exclusive RAM supply. Overall, the launch of Maia 200 is not merely a product update but a strategic statement from Microsoft that they are serious about developing independent silicon solutions that are not only cost-effective but also capable of competing directly with the best in the industry today.