AMD's GPU Ecosystem Keeps Rolling

AMD is the primary rival to NVIDIA when it comes to GPUs. It takes more than just the hardware to compete, though, as a couple of small announcements this week remind us.

AMD acquired MK1, an inferencing software startup, for an undisclosed sum. Separately, a company called Spectral raised $6 million to continue developing a framework that makes code more portable between GPU architectures.

Neither deal is huge, but each is significant to AMD’s progress, as we’ll discuss. They’re also little signs of continued momentum, coming on the heels of AMD’s Q3 news about big deployments (coinciding with AWS’s self-proclaimed success with Trainium, a different type of alt-NVIDIA momentum).

It’s Not Just the Chips

AMD is an underdog against NVIDIA, but many ecosystem players, incluidng investors, are committed to cultivating NVIDIA alternatives. Quite a few clouds offer AMD chips, specifically. TensorWave is betting its existence on the architecture.

But as we detailed in the recent report, AI, GPU Clouds, and Neoclouds in the Age of Inference, AMD must compete not only with NVIDIA’s chip technology, but also with the surrounding ecosystem. That includes software, models, developer tools, and the installed base of applications already groomed for CUDA.

Running NVIDIA Code on AMD

Spectral is addressing the CUDA point. Founded in 2018, the company is developing a toolchain called SCALE, which launched in beta in July 2024.

SCALE is a superset of CUDA and AMD’s framework (called ROCm); it compiles CUDA code, untouched, to produce runtimes that suit AMD chips. It works independently of any NVIDIA runtimes, so it’s not a “translation layer.” Spectrum’s software is also not an emulator—a term that comes with implications of bumpy performance.

(It seems like SCALE should work in the other direction too—porting AMD ROCm code onto NVIDIA chips—but let’s face it: The bigger installed base out there, and the one most likely to be seeking alternatives, is on CUDA.)

CEO Michael Søndergaard presented the idea at Beyond CUDA, a guerrilla event organized by TensorWave and held outside NVIDIA’s GTC conference in March. At that time, he said Spectral had build support for about 90 percent of CUDA’s APIs. For the developer audience, he also explained some of the tiny details that made porting difficult between CUDA and AMD’s ROCm.

Spectral announced on Nov. 10 that it’s raised a $6 million seed round led by venture firm Costanoa, with participation from Crucible and angel investors. The company is interested in adapting SCALE to lots of different hardware environments. That implies working not only with Intel, but also with hyperscalers’ chips such as AWS’s Trainium and Google’s TPUs. That’s a long road, but AMD is the logical place to start.

Inferencing on AMD

MK1 has already gained some attention because co-founder and CEO Paul Merolla was part of the founding team at Neuralink, the company working on a brain-to-computer interface.

MK1, launched in 2023 in Menlo Park, California, is less dramatic. Its mission is to make AI inferencing more efficient. Earlier this year, Merolla posted some benchmark results on LinkedIn, touting the possibility of “DeepSeek-style optimizations” on AMD chips.

It sounds like MK1 was initially more generalized; there’s a reference on Merolla’s website about applying it to NVIDIA. By May 2024, MK1 had partnered with TensorWave, and now it’s part of the AMD family.

MK1 would have good reason to focus on AMD. In addition to providing a milieu where a startup can get more attention, AMD can claim some technical advantages over NVIDIA, particularly when it comes to memory. The MI355X sports 288 GB on-chip versus 180 GB for NVIDIA's GB200, for example, allowing models to run on fewer GPUs. It sounds like those kinds of architectural differences caught MK1’s eye.

The M&A Trail

AMD has been tooling up for the fight with NVIDIA, with acquisitions being one of its levers. Some of these deals have been small enough that the terms went undisclosed. Those include MK1, Enosemi (silicon photonics), and Brium (compiler technology).

In a recent SEC filing, AMD noted that it’s made small acquisitions totaling $36 million during the previous nine months. It’s possible this is the sum of MK1, Enosemi, and Brium.

Some deals were larger. AMD bought European AI lab Silo AI for $665 million in 2024. The big-deal acquisition, though, came right after that: AMD’s $4.9 billion grab of ZT Systems.

ZT gave AMD the expertise to build rack-scale systems. That’s part of the new rules of neoclouds and GPU clouds; the speed of deployment calls for products at rack-scale, row-scale, or even database-scale—as opposed to buying individual servers, which is how hyperscalers scaled up. AMD’s MI400 series, due next year, will bring the company into the rack-scale game—one of many pieces required for being a full-blown rival to NVIDIA.