Microsoft has introduced Maia 200, its second-generation in-house AI accelerator, positioning the chip as a major step forward in inference-focused computing. Rather than chasing raw token volume alone, the company is emphasizing efficiency, throughput, and optimized execution across large and complex AI models.

Built to support diverse AI workloads across cloud environments, Maia 200 is engineered specifically for inference on advanced reasoning models. According to Microsoft, it represents the fastest and most efficient custom accelerator the company has deployed to date, and the strongest first-party silicon offering among hyperscale cloud providers.

Industry analysts note that Microsoft’s strategy diverges from competitors that prioritize training-centric platforms tightly coupled to proprietary stacks. Instead, Microsoft is treating inference as the long-term battleground for AI infrastructure, designing Maia 200 for agent-driven and multimodal AI systems that must operate efficiently in production environments.

Performance and architecture highlights

Microsoft reports substantial gains in low-precision compute, where inference workloads increasingly operate. Maia 200 is said to deliver roughly three times the FP4 performance of Amazon’s third-generation Trainium chips, while its FP8 throughput exceeds Google’s seventh-generation TPU.

At peak capacity, the chip provides:

• Over 10,000 teraflops of four-bit floating-point performance
• More than 5,000 teraflops at eight-bit precision
• 216GB of high-bandwidth memory, surpassing comparable offerings from AWS and Google
• 7 TB/s of memory bandwidth, enabling faster data access for large models

Microsoft also claims a 30% improvement in performance per dollar compared with the most recent hardware currently deployed in its own data centers. The expanded memory capacity allows models to remain closer to compute resources, reducing bottlenecks during inference.

Rethinking data movement for AI models

Beyond raw compute, Maia 200 introduces a redesigned memory architecture aimed at improving token throughput. The chip incorporates a custom direct memory access engine, on-die SRAM, and a specialized network-on-chip fabric. Together, these components are intended to support faster, more efficient data flow between memory and compute units, which is critical for large-scale inference.

Microsoft says this architecture enables the chip to handle today’s largest models comfortably, while leaving room for future growth as model sizes and complexity continue to increase.

Built for multimodal and heterogeneous AI systems

Maia 200 was designed with modern large language models in mind, particularly those that go beyond text-only interactions. Microsoft points to growing demand for AI systems capable of processing images, audio, and video, supporting multi-step reasoning, autonomous agents, and more advanced decision-making workflows.

As part of Microsoft’s broader heterogeneous AI infrastructure, Maia 200 will support a range of models, including the latest GPT-5.2 family from OpenAI. The chip is tightly integrated with Azure, and products such as Microsoft Foundry and Microsoft 365 Copilot are expected to benefit directly from its deployment. Internally, Microsoft’s advanced AI research teams also plan to use Maia 200 for reinforcement learning and synthetic data generation to further refine proprietary models.