It makes sense that businesses like Microsoft are looking into in-house solutions as an alternative to off-the-shelf hardware from the leading player in the dedicated GPU area because the scale of generative AI models is expanding faster than the compute capabilities needed to train and run them.
Technology behemoths are already vying for AI supremacy, and the competition goes beyond simply feeding massive quantities of data to models like the well-known (or infamous, depending on your point of view) ChatGPT that dominates nearly every news cycle.
Microsoft Working on Athena AI Chip
The Information alleges that Microsoft is creating a unique AI processor to train its models. “Athena” is the codename for the enigmatic silicon chip, and the firm is said to have begun developing it in 2019.
It is currently only accessible to a small number of Microsoft and OpenAI employees so that they may test how well it works with big language models like GPT-4.
Large and small businesses alike are snatching up every enterprise-grade Nvidia GPU they can get their hands on in order to construct the robust infrastructure necessary for training artificial intelligence models on chosen data sets.
The same hardware is used to run the models and perform “inference,” which is the act of using input from the real world to produce material that is helpful for a particular application.
The issue is that Nvidia can only produce a finite number of A100 and H100 GPUs, and they are all extremely expensive.
The more recent H100 GPUs sell on eBay for over $40,000, while A100 GPUs cost over $10,000 each.
Team Green is transferring some of the production capacity for GeForce RTX 4090 GPUs to produce more H100 GPUs since it is so ecstatic about the unmet demand.
So it shouldn’t be surprising that organizations like Microsoft are attempting to lessen their reliance on a single vendor for their machine-learning initiatives.
Because of the potential future cost savings, Google, Meta, and Amazon are following the same path and producing their own chips.
It will be intriguing to observe what happens to Microsoft’s current cooperation with Nvidia given that 300 people are working on the Athena processor and that numerous generations are planned for the upcoming years.
In any case, the Redmond behemoth has been hard at work integrating AI technology into each and every product or service it provides, including the Edge browser, its Bing search engine, the Microsoft 365 office suite, GitHub, and more.
It’s interesting to note that a different rumour from Thurrott claims Microsoft is also creating a neural processing unit (NPU) for its Surface line of products.
It presently uses CPUs and SoCs from Intel, AMD, and Qualcomm; to date, the Surface Pro 9’s NPU is only found in the Qualcomm version.
To read our blog on “Microsoft reportedly working on its new crypto wallet,” click here.
