Key Takeaways:

• Liquid cooling and higher temperatures are revolutionizing data center efficiency.
• Liquid cooling makes AI operations cooler, faster, and greener.
• Waterless cooling and higher temperatures cut energy use and boost sustainability.

Introduction

Currently, data centers consume about 4% of the total electricity generated in the United States.¹ At least one-fourth of this power is solely used to cool the data centers and not for any useful computational gain. Traditionally, decreasing the electrical consumption of the cooling system has meant increasing water consumption, and in an increasingly water-constrained world, this leaves data center designers with two undesirable options.

Current and future trends in cooling design systems, such as the rise of liquid-cooled hardware and higher operating temperatures, have the potential to break this paradigm by enabling a transition to both low-electrical and low-water consumption cooling systems. Both current and future design trends will be explored, detailing their potential impact and innovations.

Air Cooled Hardware

Effective cooling is crucial for maintaining data center efficiency and reliability. Without adequate cooling, temperatures can rise rapidly, leading to computer shutdowns and increased operational costs due to emergency repairs or replacements.²

Currently, most operating data centers utilize air-cooled hardware; in these systems, a fan moves cold air across the computers, and the resulting hot reject air is cooled by the data center's auxiliary cooling systems.

ASHRAE (the American Society of Heating, Refrigerating and Air-Conditioning Engineers) has guidelines for the maximum allowable operating temperatures of the air-cooled systems. The best current practice is to provide inlet air to the servers at or below 27°C/80°F for most of the year.

Air Cooling Technologies

There are many different ways to cool the hot reject air in a data center. For the purposes of clarity, they have been divided into two categories: evaporative cooling and waterless options.

One of the most electrically efficient cooling technologies is direct evaporative cooling. In these systems, outdoor air is cooled by evaporating air directly into the airstream, which then moves through the servers and is rejected into the atmosphere.

Other cooling technologies which evaporate water include indirect adiabatic coolers and chilled water plants with cooling towers. Both options consume more water and electricity than a direct evaporative system, but they have the advantage of insulating the data center interior from the outdoor environment.

On the waterless cooling technology side, we have options such as Computer Room Air Conditioning (CRAC) units, air cooled chillers, and chillers with fluid coolers. These technologies all use the vapor-compression cycle (the same one in your refrigerator at home) to move heat away from the computers and out into the atmosphere. What these options all have in common is a higher electrical consumption than the evaporatively cooled systems, but they consume no water at all.³

Due to many data centers' 27°C/80°F control temperature criteria, maintaining hardware operation requires significant amounts of water, electricity, or a combination of the two resources. Significantly raising the control temperature is one of the best ways to save energy on the cooling system; however, meaningful attempts to increase the control temperature criteria run into real-world limitations of computer hardware performance.

Liquid cooling technologies can offer an alternative, potentially reducing reliance on these resource-intensive methods.

Liquid Cooling and AI Hardware

Liquid cooling technologies have gained traction in recent years, particularly for high-performance computing (HPC) and artificial intelligence (AI) workloads.⁴ Unlike traditional air-based systems, liquid cooling uses liquids to absorb and transfer large amounts of heat more efficiently. This includes direct-to-chip cooling, where liquid channels circulate directly across heat-generating components, and immersion cooling, where entire servers are submerged in non-conductive coolant liquids. These methods excel at managing the intense thermal loads produced by AI accelerators, Tensor Processing Units (TPUs), and Graphics Processing Units (GPUs), which often operate at much higher temperatures compared to conventional servers.

These advanced liquid cooled hardware systems have significantly higher inlet temperatures than the conventional air cooled systems. For example, many liquid cooled systems today can use water temperatures up to 35°C/95°F, with current trends allowing for even higher temperatures in the future.

Higher Working Temperatures and Waterless Cooling Expansion

The ability of modern AI hardware and high-performance computing systems to operate at higher temperatures has expanded the feasibility of waterless cooling technologies in more environments with minimal increases in energy consumption. Energy-efficient waterless cooling systems, such as fluid coolers, can be deployed in regions with warm climates without the risk of performance degradation. By raising working temperatures and eliminating the vapor-compression cycle (and attendant electrical consumption), future data centers can significantly decrease the percentage of electricity they consume for non-computational activities and reduce the overall burden on the power grid.

By maintaining higher working temperatures, data centers can reduce the need for intensive air conditioning and fan systems, thereby minimizing the electrical consumption of cooling infrastructure. This reduces operational costs and supports sustainability initiatives by decreasing overall power usage and carbon footprints. Data centers can achieve greater energy efficiency and operate more resiliently under a wider range of environmental conditions.

Much has been written about how the current AI boom has the potential to strain the power grid. By adopting new cooling solutions and raising acceptable operating temperatures, data centers have an opportunity to move toward more sustainable and efficient operations while supporting the cutting-edge technologies that they are designed to serve.

References:

1. DOE Releases New Report Evaluating Increase in Electricity Demand from Data Centers. (December 20, 2024.) US Department of Energy. Available at: https://www.energy.gov/articles/doe-releases-new-report-evaluating-incr…?

2. Datacenter outages are on the decline, but when they hit, they hit hard. (April 2, 2024.) The Register. Available at: https://www.theregister.com/2024/04/02/datacenter_outages_declining/#:~….

3. Investing in the rising data center economy. (January 17, 2023.) McKinsey & Company. Available at: https://www.mckinsey.com/industries/technology-media-and-telecommunicat…

4. AWS bets on liquid cooling for its AI servers. (December 2, 2024.) TechCrunch. Available at: https://techcrunch.com/2024/12/02/aws-bets-on-liquid-cooling-for-its-ai…