OVHcloud details the “Hybrid Immersion Cooling” technology

Data center cooling infrastructure has not changed much in recent decades. While the power and efficiency of computers have improved remarkably over the years, many data centers still rely on racks whose cooling depends on a variety of fan systems. Heat, particularly waste heat, remains a major concern for keeping servers running. While some data centers have opted for power-hungry solutions such as air conditioning (direct expansion), others have a free-cooling approach.

With the evolution of the power of computer equipment, OVHcloud R&D (research and development) has worked on a new generation of computer cooling solutions. data centers. One of the most promising technologies today is immersion cooling, through direct fluid contact with hardware components. It allows to build data centers worldwide by maintaining low PUE/WUE indices even in difficult climatic zones.

OVHcloud Hybrid Immersion Liquid Cooling

Immersion cooling is the practice of immersing electronic devices in a thermally, not electrically conductive, liquid.

The typical immersion cooling solution usually employs pumps, heat sink structures, heat exchangers, condensers, sealed evaporative equipment, etc. which either consume large amounts of power to operate, or require sealed enclosures, or occupy relatively large areas that limit the number of servers that can be implemented.

OVHcloud has developed its own hybrid immersion liquid cooling technique. It is composed of a water cooling system directly on the chip and a single-phase passive natural immersion cooling system containing two fluids:

  • Water: cooling of a heat sink by water blocks placed on the CPUs and GPUs with the same solution as that used in all OVHcloud servers and a proprietary convection coil connected to a pumping substation (PSS) and a dry cooler to evacuate the heat outside the DC.
  • Immersion cooling (IC): the fluid is contained in a reservoir and cools all the computer equipment of the server rather than only the CPU and GPU; this fluid essentially replaces the air circulating in OVHcloud servers, thus improving the efficiency of any component not cooled by OVHcloud water cooling systems.

It comes with a new design of rack 3-tier servers, in a library format, which we can populate with up to 48 servers (1U) or 24 servers (2U), in a book format. Each server is immersed in its own reservoir for independent cooling, enabling large-scale deployments. In addition, each server benefits from a specific monitoring system of all environmental factors, guaranteeing safe and secure operation at the server level.

Advantages in terms of power consumption and efficiency

The OVHcloud Hybrid liquid immersion cooling system has several advantages that make it an effective solution:

  • The new passive design of racks means there is no pump or fan, resulting in zero cooling power consumption at the rack.
  • High power racks can operate with data center inlet temperatures up to 45°C, allowing for different cooling loads depending on climatic conditions.
  • Power consumption and CAPEX are further reduced by eliminating the evaporative cooling system (without pump) used on dry coolers for data centers located in areas with ambient climate temperatures below 43°C
  • The difference between the supplied temperature and the recovered DC water temperature (DT) of 20K can be achieved through highly efficient CPU and GPU cold plates and a large thermal contact surface between the coil and the dielectric fluid. The patent-pending coil design occupies a very small area in the server chassis, maximizing space for components.
  • The energy consumption of the overall data center cooling infrastructure is reduced by at least 20.7% compared to the OVHcloud water cooling system.
  • The energy consumption of one server per year is reduced by at least 20% compared to air-cooled servers and by 7% compared to water-cooled servers.
  • Lower energy consumption means lower operating expenses (OPEX)
  • Finally, in a context where the TDP of CPUs/GPUs is constantly increasing, the solution allows for a higher density of computing power.

A new generation of data center

This new hybrid approach to immersion cooling has several key benefits for data center operators:

  • A real footprint of up to 37U/m2 can be guaranteed without additional pumping systems and condensers. The design is expandable to two or three times its footprint if the racks Immersion chillers are installed in stacked shipping containers.
  • Capital expenditure (CAPEX) remains low because there is no need for sealing, nor for heat exchangers or sophisticated pumping circuits.
  • With a data center exit temperature of 65°C, waste heat is better captured and affordable heat recovery systems can be considered.
  • More flexibility and freedom in design at OVHcloud, where the new design of rack about requires access from the front panel, unlike the racks conventional requiring access from the front and rear sides.
  • Redefining the indexes of data centers.

With such a game-changing new technique, typical data center performance metrics change dramatically.

PPUE 1.004 improvement: The PPUE or Partial Power Utilization Efficiency defines a certain portion of the overall PUE of a data center within a clearly defined boundary. The PPUE for infrastructure cooling is as follows:

Reduce PPUE 0 for data centers located in areas with ambient air climatic temperatures below 43°C: WUE or water use efficiency is a measure of sustainability to assess the amount of water used by data centers to cool computer equipment. Annual site water consumption includes water used for humidification and water vaporized on site for power generation or cooling of the data center and its supporting systems. It is defined as follows:

Environmental Considerations

The hybrid immersion liquid cooling technique is based on a non-volatile hydrocarbon dielectric fluid. OVHcloud R&D has tested and qualified the system with a variety of fluids on the market. Although each fluid has different properties, we paid particular attention to their compliance with the following considerations:

  • Non-corrosive
  • Ultra-low vaporization
  • Nontoxic
  • No allergy
  • Biodegradability in 30 days
  • Very high flash point
  • Dielectric strength up to 42 kV
  • PRP[2]=0
  • ODP[3]=0
  • Operational changes in the data center

The new rack design in 3-storey library format is optimized in terms of space for a high density of servers and, incidentally, computing power per square meter (U/m2). It can be equipped with manual/automatic guidance to extract servers, which reduces the impact of maintaining one server on the entire rack.

The new hybrid liquid immersion cooling technique also improves reliability by reducing the failure rate by up to 60% thanks to the elimination of the risk of dust circulation. It contributes to a quiet data center with the elimination of any fan at the level of the racks and servers. Finally, several components are no longer necessary:

  • No fixed heat exchangers on the racks (3 heat exchangers are removed by rack),
  • The 36 fans usually installed per rack are eliminated
  • All small fans inside the servers are eliminated
  • Cooling modules consisting of pumps and plate heat exchangers are eliminated.

And after?

The new OVHcloud Hybrid Immersion Liquid Cooling system paves the way for increased computing power in modern data centers. Its unique, patent-pending design is durable and preserves the opportunity to recover waste heat through higher operating temperatures. In addition, it reduces energy and water consumption, as well as improving the footprint, while adapting to regions with difficult climatic conditions. We’re incredibly excited about the very first use cases for hybrid immersion liquid cooling in banking, medicine, and scientific research, not to mention gaming workloads. We look forward to sharing more with you in the future.

Leave a Comment