SmartNICs and DPUs: A Market Update
Smart network interface cards (SmartNICs) have become essential building blocks in networking for AI. Alongside their more advanced versions, known as data processing units (DPUs), these devices streamline AI networking, ensuring that GPUs continue to run efficiently without interruption.
SmartNICs and DPUs combine network interconnections with integral processors that offload tasks that would otherwise consume CPU cycles. Examples include packet processing, encryption/decryption, KV caching, collective operations, and multi-tenant isolation control. These devices take over these tasks to keep GPUs humming when otherwise bottlenecks would form as CPUs become preoccupied.
There's a distinction between SmartNICs and DPUs, though both have CPU offloading capabilities. Like other NICs, SmartNICs connect the server to the network, but they also offload specific networking functions, such as traffic filtering, along with some storage and security features. DPUs are more powerful systems that operate on their own chips and act as “third leg” sources of compute power in AI networks, alongside CPUs and GPUs.
A variety of networking vendors provide SmartNICs and DPUs, including NVIDIA, whose ConnectX series SmartNICs and BlueField DPUs lead the market. But there are many other products that feature integral capabilities to alleviate CPU stress. All are complex devices featuring a variety of different functions.
Key Differentiators
To access the rest of this article, you need a Futuriom CLOUD TRACKER PRO subscription — see below.
Access CLOUD TRACKER PRO
|
CLOUD TRACKER PRO Subscribers — Sign In |
