(Image credit: Tom's Hardware)
It is not a secret that proper cooling ensures longevity and enables hardware to demonstrate its full potential. But when it comes to data center AI hardware, proper cooling also means higher sustained performance, which directly translates into money earned by the owner. Frore Systems, a maker of cooling solutions that are made using semiconductor-grade tools, seems to have a perfect idea of how to reduce the temperature of next-generation AI accelerators and increase their performance by 15%.Tom's Hardware Premium RoadmapsFrore Systems last week published a white paper which suggests that improvements to the entire cooling stack — from the GPU packaging and thermal interface materials (TIMs) to coldplates and coolant temperatures — can increase token generation per watt by more than 30%. Meanwhile, one of the company's boldest projections based on an analytical thermal model* is that its LiquidJet coldplate technology alone can lower Nvidia Rubin GPU junction temperatures by up to 12°C, which translates into a 10% to 25% improvement in tokens/Watt, while a 10°C reduction could increase token generation by around 15%.Indeed, modern AI accelerators, such as the upcoming Nvidia Rubin, can dissipate up to 2,400 W, and their die temperatures can easily hit 95°C or more. But while 95°C is not necessarily a problem for silicon longevity, leakage current certainly is. Leakage current rises exponentially with temperature, approximately doubling for every 10°C increase in maximum junction temperature, which is when transistor switching itself also becomes less efficient. As a consequence, hotter GPUs require higher voltages to sustain clocks, which eventually forces Dynamic Voltage and Frequency Scaling (DVFS) to reduce clocks to remain within thermal limits, which in turn will reduce performance and token generation.
