Microsoft Chief Technology Officer and Executive Vice President of Artificial Intelligence Kevin Scott speaks at the Microsoft Briefing event at the Seattle Convention Center Summit Building in Seattle, Washington, on May 21, 2024.
Jason Redmond | AFP | Getty Images
Microsoft would like to mainly use its own chips in its data centers in the future, the tech giant’s chief technology officer said on Wednesday, in a move which could reduce its reliance on major players like Nvidia and AMD.
Semiconductors and the servers that sit inside data centers have underpinned the development of artificial intelligence models and applications.
Tech giant Nvidia has dominated the space so far with its graphics processing unit (GPUs), while rival AMD has a smaller slice of the pie.
But major cloud computing players, including Microsoft, have also designed their own custom chips for specifically for data centers.
Kevin Scott, chief technology officer at Microsoft, laid out the company’s strategy around chips for AI during a fireside chat at Italian Tech Week that was moderated by CNBC.
Microsoft primarily uses chips from Nvidia and AMD in its own data centers. The focus has been on picking the right silicon — another shorthand term for semiconductor — that offers “the best price performance” per chip.
“We’re not religious about what the chips are. And … that has meant the best price performance solution has been Nvidia for years and years now,” Scott said. “We will literally entertain anything in order to ensure that we’ve got enough capacity to meet this demand.”
At the same time, Microsoft has been using some of its own chips.
In 2023, Microsoft launched the Azure Maia AI Accelerator which is designed for AI workloads, as well as the Cobalt CPU. In addition, the firm is reportedly working on its next-generation of semiconductor products. Last week, the U.S. technology giant unveiled new cooling technology using “microfluids” to solve the issue of overheating chips.
When asked if the longer term plan is to have mainly Microsoft chips in the firm’s own data centers, Scott said: “Absolutely,” adding that the company is using “lots of Microsoft” silicon right now.
The focus on chips is part of a strategy to eventually design an entire system that goes into the data center, Scott said.
“It’s about the entire system design. It’s the networks and the cooling and you want to be able to have the freedom to make the decisions that you need to make in order to really optimize your compute to the workload,” Scott said.
Microsoft and its rivals Google and Amazon are designing their own chips to not only reduce reliance on Nvidia and AMD, but also to make their products more efficient for their specific requirements.
Compute capacity shortage
Tech giants including Meta, Amazon, and Alphabet and Microsoft have committed to more than $300 billion of capital expenditures this year, with much of that focused on AI investments as they look to satisfy booming demand for AI.
Scott flagged that there is still a shortage of computing capacity.
“[A] massive crunch [in compute] is probably an understatement,” Scott said. “I think we have been in a mode where it’s been almost impossible to build capacity fast enough since ChatGPT … launched.”
Microsoft has been building capacity through data centers but it’s still not enough to meet demand, the CTO warned.
“Even our most ambitious forecasts are just turning out to be insufficient on a regular basis. And so … we deployed an incredible amount of capacity over the past year and it will be even more over the coming handful of years,” Scott said.
