In the early hours of Wednesday, Amazon Web Services (AWS) announced at the re:Invent conference that Wall Street’s long-awaited new-generation in-house AI chip, Trainium3, is now officially available.
According to the company, Trainium3 is Amazon’s first chip built on a 3nm process. Compared with the previous-generation Trainium chip, Trainium3 delivers up to 4.4 times the compute performance, 4 times the energy efficiency, and almost 4 times the memory bandwidth. It is clearly positioned in the AI compute race as a high cost-performance (value-for-money) solution.
Servers built with Trainium3 can also be interconnected. Each UltraServer can house 144 chips, and a single application can scale up to 1 million Trainium3 chips, ten times the capacity of the previous generation.
The company claims that, compared with systems that also use GPUs (graphics processing units), Trainium3 can reduce the cost of training and running AI models by up to 50%.
That said, investors hoping Amazon’s Trainium3 could go head-to-head with Google’s TPU or even seriously challenge Nvidia may be somewhat disappointed, as there is currently no data to suggest the chip has advantages beyond cost efficiency.
Amazon has declined to disclose benchmark performance comparisons between the new chip and Google’s or Nvidia’s latest products, and it has not released detailed power consumption figures either.
The chip’s biggest selling point is price-to-performance. Its main goal is to give customers more choices across different compute workloads. For Amazon’s AI chips, the biggest weakness is not the hardware itself, but rather the lack of sufficiently deep and easy-to-use software libraries.
Apart from Amazon itself and Anthropic — the AI startup in which Amazon has invested heavily — there are currently very few well-known companies using Trainium chips. In October, Anthropic said it would purchase up to 1 million of Google’s TPUs, and in November it announced an equity investment deal with Nvidia and plans to further purchase Nvidia’s compute capacity.
Market Commentary:
Amazon’s share price has been moving steadily higher in a choppy uptrend, supported by the ongoing iteration of its AI chips. The company also appears well aware that customers will inevitably compare it with Nvidia.
In the Trainium3 announcement, Amazon made a point of highlighting that Trainium4 is already under development, with its key selling point being the ability to work in tandem with Nvidia chips.
