Maia 200 is said to be Microsoft's "most efficient inference system yet," and all of their press releases have been split between praising the high numbers the system offers and touting Microsoft's lip service to environmentalism. The Maia 200 offers 30% more performance per dollar than the first gen Maia 100, an impressive achievement considering the new chip also technically advertizes 50% higher TDP than its predecessor.
Maia 200 uses TSMC's 3nm node and contains 140 billion transistors. The chip itself can achieve 10 petaflops of FP4 compute, three times higher than the competition offered by Amazon's Trainium3. The Maia 200 also has 216 GB of HBM3e memory onboard with 7 TB/s of HBM bandwidth, as well as 272MB of SRAM.
As can be seen above, the Maia 200 boasts a significant lead in terms of raw computing power against the Amazon in house competition, and it makes for an interesting comparison against the top dog GPU from Nvidia. Of course, one can immediately see the foolishness of comparing the two; the Maia 200 is unavailable for purchase by any outside customers, the Blackwell B300 Ultra is designed with much more powerful applications in mind than the Microsoft chip, and the software stack is designed with the Nvidia GPU launching with a massive lead against anything the current market has to offer.
The Maia 200 does have one advantage over the B300 Ultra: it is more efficient, which is important in this day and age as public opinion against AI's environmental impact is growing. The Maia 200 operates at nearly half the TDP of the B300 Ultra (750W vs 1400W), and if the Maia 200 is anything like the Maia 100, it will be running at below its theoretical maximum TDP; the Maia 100 was designed to be a 700W part, but ran at 500W in operation.
Maia 200 is designed with the focus being on serving the inferencing of AI models, which are in need of the performance that FP4 has to offer, as opposed to complex operations that are required by FP8. Much of the R&D money that Microsoft is pouring into their chip design has been put into the memory hierarchy that is found within their 272MB of high efficiency SRAM bank, which is divided into "multi tier Cluster level SRAM (CSRAM) and Tile level SRAM (TSRAM)," in order to accommodate the increased operating efficiency and the philosophy of spreading workloads evenly across all of the HBM and SRAM dies.
It is difficult to quantify the advancements that have been made by the Maia 200 compared to its predecessor, the Maia 100, as Microsoft's official stat sheets for both of their chips have almost zero overlap in terms of measurements. What we can say about the new chip is that it is going to run hotter than the Maia 100 did, and it is 30% better in terms of performance per dollar.
The Maia 200 has already been deployed in Microsoft's US Central Azure data center, with future plans announced for the US West 3 in Phoenix, AZ, and more to come as Microsoft receives more chips. It will be part of the heterogeneous deployment, working in concert with other different AI accelerators as well.
The Maia 200, codenamed Braga, caused a stir with its long overdue development and release cycle. It was supposed to be released in 2025 and deployed, perhaps even ahead of the B300, but that was not to be. Microsoft's next hardware release is far from being set in stone, but it will likely be built on Intel Foundry's 18A node, according to October announcements.
Microsoft's focus on efficiency with their Maia 200 campaign is just another example of their latest trend of emphasizing their corporation's concern for communities surrounding their data centers, going to great lengths to deafen the backlash against the AI boom. The CEO of Microsoft, Satya Nadella, recently gave a speech at the World Economic Forum about how, if corporations are unable to open the eyes of the public to the benefits of AI development and data center construction, they risk losing "social permission" and creating the feared AI bubble.
👉🏻 Found this article interesting? Follow us on Facebook, Twitter and whatsapp to read more exclusive content we post.
