How the Datacenter Power Issue Is Being Addressed

Power: That’s the headline behind the news this week. As OpenAI unveils a raft of new Stargate datacenter sites, and NVIDIA pledges $100 billion to help OpenAI build 10 gigawatts worth of AI infrastructure, the question persists as to how these buildouts will be fueled. The current electrical grid won’t be sufficient. What’s the solution?

That question has multiple answers. But let’s look first at the context. According to CNBC, 10 gigawatts of power represents what’s needed to serve 7.5 million homes, or a geographic area the size of New York City. It would require the power of 2.5 state-of-the-art nuclear reactors, 15 large natural gas plants, and about 3,000 wind turbines covering an area the size of Rhode Island. “These are massive numbers and the capacity does not exist right now,” said CNBC anchor Brian Sullivan.

Many issues arise: Given the current U.S. administration’s aversion to renewable energy (and claims that it’s not reliable), how fast can nuclear power plants be licensed for datacenter use? Currently, it can take up to 10 years to get a nuclear plant up and running.

Apart from nuclear fuel, natural gas has taken the top spot in fueling datacenters in the U.S. Here, the trend is to go “off grid,” an approach in which gas providers siphon off portions of their main fuel pipelines and dedicate those to specific datacenter customers. This has the advantage of keeping the datacenters from competing with other businesses and residences. On the downside, natural gas remains a carbon-emitting substance and is generally considered to be a “bridge” solution between traditional electrical and future renewable energy sources.

Speaking of which, talk of renewable energy for AI datacenters has stalled in the U.S., where the current administration has damned it as flawed and unreliable. Even SB Energy, a SoftBank company whose solar-based power grid will be part of the Stargate project in Milam County, Texas, seems to have removed all mention of renewable energy from its website.

Given these developments, it’s interesting to hear that Meta has, through a subsidiary company named Atem Energy LLC, petitioned the Federal Energy Regulatory Commission to allow it to deal in wholesale energy at market-based rates. This could help Meta satisfy its own power needs while profiting from selling off-grid datacenter energy of one kind or another to other companies.

Energy Savings Inside the Datacenter

As noted, though, utility-level resources are only part of the answer to the datacenter power question. Another major part of the solution is happening within datacenter racks, where liquid cooling is touted as a solution to the datacenter heat problem. This approach uses water or alternative liquids to cool down machinery and reduce the heat they produce. These solutions are gaining ground because they are cheaper and more efficient than computer room air conditioning (CRAC).

Liquid cooling has become a key option in AI infrastructure. There are different approaches: The most popular type of datacenter liquid cooling runs water or other liquids through pipes or plates located at the back of an equipment rack, cooling the gear and/or exchanging heat for cool air in radiator fashion. There’s also direct-to-chip cooling, in which a plate containing cool liquid is located next to heat-producing CPUs to draw heat off the chip. The liquid is then cooled by water or turned into a gas. A third option is immersive cooling, in which entire servers are placed in tanks holding dielectric fluid (which doesn’t conduct electricity).

This week, Microsoft unveiled a fresh take on direct-to-chip cooling that uses microfluidics, or the process of putting tiny coolant channels directly onto chips to remove heat and reduce the need for other types of power-guzzling cooling methods. “An important breakthrough from our teams,” wrote Microsoft CEO Satya Nadella on LinkedIn. “[A] new approach to liquid cooling that … [opens] the door to more efficient, sustainable, and power-dense datacenters than conventional methods.”

Focus on Models

Yet another effort to reduce AI datacenter power consumption is aimed at the AI models themselves. By shrinking them down, proponents of this approach say, resources devoted to training can be significantly reduced, corresponding with power savings.

In a paper this summer, experts at University College London (UCL) claimed that tweaking large language models could result in more efficient use of processing resources:

“Researchers from UCL Computer Science conducted a series of experiments on Meta’s LLaMA 3.1 8B model to assess the impact of changes to the way AI models are configured and used on how much energy they need, and how this affects performance…. They found that by rounding down numbers used in the models’ internal calculations, shortening user instructions and AI responses, and using smaller AI models specialised to perform certain tasks, a combined energy reduction of 90% could be achieved compared to using a large all-purpose AI model.”

On the downside, model adjustments risk making results less accurate. Still, IBM, Microsoft, Google, Apple, Mistral, Meta, and other major AI suppliers are homing in on model compression and quantization, the latter term referring to fiddling with the internal processes of models to reduce their precision while increasing their speed during inference. The trick is to compress the model while sacrificing as little accuracy as possible.

Clearly, solving the AI datacenter power problem is a complex, many-sided matter that involves a range of approaches on multiple levels, with innovations required throughout. It will be a progression that hopefully keeps pace with AI requirements.

Futuriom Take: The power required by AI infrastructure has become a central issue. As massive datacenter buildouts hit the headlines, work is underway to find cheaper sources of power, to design datacenters more efficiently, and to address the fundamental requirements of AI applications.