How Big Is the Datacenter AI Chip Market?

This could be the Year of the AI Chip. Despite claims that the artificial intelligence hype cycle is slowing a bit, AI component news keeps coming as 2024 gets underway. And there are predictions that strong, even meteoric, growth could build this year. The question is, how high can the upward line go?

Late in 2023, Advanced Micro Devices (Nasdaq: AMD) boldly declared that the total addressable market (TAM) for datacenter AI chips will reach $400 billion by 2027, representing a compound annual growth rate (CAGR) of more than 70%. AMD’s prediction got mixed reactions (more on that momentarily), but it spoke to the momentum behind the components designed to process the large language models (LLMs) at the heart of generative AI (GenAI) applications.

The size of the potential market for these chips is demonstrated by the success of NVIDIA (Nasdaq: NVDA), the market leader in graphics processing units (GPUs) used to train LLMs. NVIDIA’s stock price has grown 217% over the past year. Over the past three years, it's grown 140%.

In its latest earnings report in November, NVIDIA posted revenues of $18.12 billion, of which $14.51 billion was datacenter revenue. Overall sales were up 206% year-over-year, and datacenter sales were up 279% in the same period. Added to this is the well-known shortage of H100 GPUs NVIDIA has experienced. All of this makes it conceivable that datacenter chip sales are on a steep upward trajectory. The question is whether that line tracks as high as AMD hopes.

Given that NVIDIA by nearly all accounts owns at least an 80% share of the datacenter AI GPU market, it's possible to envision sizable growth over the next three years. But a whopping $400 billion TAM would call for NVIDIA to perform even better than it has and for other players to exceed expectations as well.

How High Can the Market Go?

One well-known Wall Street analyst wrote in detail about what AMD’s projected TAM would mean for the market. This kind of growth would have lots of huge implications for how cloud providers allocate capital spending (capex) in the future.

“Nvidia and all other silicon suppliers would massively beat expectations, even if this scenario only partially materializes,” stated Pierre Ferragu, managing partner at New Street Research, in a post on X on January 10, 2024. In a chart on the same post, he indicated that the top four public cloud providers – presumably AWS, Microsoft Azure, Google Cloud, and Oracle – would have to increase their datacenter capex by 150% over what his firm currently projects for 2027. Tier 2 providers would have to grow their datacenter capex by 253% over New Street’s projection, and other spenders would have to exceed their New Street number by 208%.

First, here is what $400bn spent on AI chips in 2027 would look like: GPU and TPU shipments quadrupling, Other in-house accelerators ramping to represent 2x the TPU footprint.

First, here is what $400bn spent on AI chips in 2027 would look like: GPU and TPU shipments quadrupling, Other in-house accelerators ramping to represent 2x the TPU footprint. pic.twitter.com/DoFtm7LAmw

— Pierre Ferragu (@p_ferragu) January 10, 2024

It's notable that Ferragu stresses the role of the cloud hyperscalers’ contributions to the datacenter AI chip market. In a Raymond James industry update last October, Simon Leopold and colleagues stated that AI datacenter spending in general isn’t yet an enterprise issue. “Partners have not observed AI affecting enterprise spending plans, and some see the topic as over-hyped, [at] least in the near-term. We have the impression that it is a small portion of sales but growing quickly,” stated the analysts.

AMD and Others Aim High

Despite the skeptics who insist that AMD is far behind NVIDIA in the AI chip race, AMD has set its hopes on dramatic growth over the next three years. In December 2023, it announced its MI300 series of chips, which are designed to outperform graphics processing units (GPUs) from NVIDIA in inferencing, or querying a trained model with live data. Another product unveiled at the same time, the AMD Instinct MI300A accelerated processing unit (APU), combines CPU and GPU cores with memory in one platform.

“And as we go through 2024, we would expect revenue to continue to ramp quarterly, and again, it will be mostly AI,” stated Lisa Su, president and CEO of AMD, on the company’s third-quarter conference call last October. “Within the AI space, we've had very good customer engagement across the board from hyperscalers to OEMs, enterprise customers, and some of the new AI start-ups that are out there. From a workload standpoint, we would expect MI300 to be on both training and inference workloads.”

Other major suppliers aren't standing still. Intel (Nasdaq: INTC), which runs a distant third behind NVIDIA and AMD in datacenter GPUs, last month unveiled the fifth generation of its Xeon processors as part of a push further into the datacenter AI market. Qualcomm (Nasdaq: QCOM) offers its own chips for AI inference.

The cloud providers may be among AMD and NVIDIA’s best customers, but that isn’t stopping them from pushing their own AI silicon. Amazon has its Trainium and Inferencia chips, Google its TPUs, and Microsoft and Meta are working on their own projects. Still, it’s likely the hyperscalers will continue to rely on NVIDIA and AMD for AI processing power in addition to their own components.

Ultimately, at least three conclusions can be safely drawn from AMD’s outsized prediction: First, the datacenter is the focal point for AI chips in the near term; second, the datacenter AI chip space is on a steep upward trajectory, though how steep remains a question; and third, NVIDIA will continue to dominate the space, though AMD and other vendors will try and erode its position, possibly widening the TAM.

Futuriom Take: NVIDIA remains the dominant player in a fast-growing datacenter AI chip space for the foreseeable future. But its success will spur AMD and others to challenge its position.