> Data center liquid cooling is going to happen<p>I mean yeah, but I doubt its going to be mainstream anytime soon.<p>At the moment its cheaper to just use forced air, and yolo it. Running at half density is expensive, but not as expensive as backfilling everything with liquid cooling.<p>Also given that we've been able to run 2 socket blades at full bore without liquid cooling kinda suggests its not actually needed.<p>Having radiators on the front/back of the rack directly works really well. We had it on our render farm. combine that with enforced hot isle/cold isle and you can reduce the need for aircon dramatically, without multiplying your leak risk by at least 96 times.<p>The massive problem here is that its really difficult to hotswap anything. Those cooling pipes need to be removed before you can pull out the server. Unless they are self sealing (like hydrolic lines) then you need to drain the loop first. That costs a shitload of money at scale.
I cannot help but chuckle at how what is old is new again here. In the 70's and early 80's all of IBM's mainframes supported liquid cooling. Basically when a computer "uses" X kW of power it really means that it generates that many kW of heat while it is operating. Removing heat at scale has been a thing for a long time.<p>And what was alluded to in the video is the thermal mass of 'air' kind of sucks. So the old design of chilling air down to 67 degrees and filling a room with that air so that it can circulate around electronics putting out prodigious amounts of heat, and then collected and re-cooled, is not nearly as efficient as one would like.<p>Cooling water, piping it to the heat exchanger in the back door of the rack and then (unlike the video's idea) sucking air through it first, and then pushing the cooled air over the electronics to 're-heat' it, works better. Then you don't really care what temperature the air in the data center itself is as long as the heat exchanger can remove 'x' watts of heat from it before it gets blown over the computers. Suck air in from the floor (the coolest air) and blow it out the top (where it continues on to the ceiling).<p>Still, that only doubles the power capacity of the racks (maybe 2.5x if the heat exchanger is filled with actively chilled water)<p>Prior to heat exchanger doors people would have "cold" aisles and "hot" aisles. The cold air from the CRAC units would come up from the floor behind the servers, get sucked through them and exhausted forward into the "hot" aisle. There is a whole little mini industry of "cold air containment" which has stuff to build doors/covers for the cold aisle so that all of that air is sucked through servers.
I'm curious why liquid cooling for computers still uses compression fittings and other odd methods.
Compression fittings are widely out of favor due to their tendency to leak, and nowadays everything is copper, pex, or metal+flared fittings.
I wonder if there's a PC running PEX-A for tubing.<p>Further thoughts:<p>Brake lines use flared fittings, with either metal tubing or plastic tubing with metal ends. Uses a special bolt that allows liquid through the pipe to enter the caliper, and is removable.<p>I would imagine something like that could work to make the lines to the individual servers serviceable without relying on flaky plastic "quick connects".
A lot of high power plasma processing equipment is water cooled. RF and DC generators, matches, etc. Stuff that requires a coax cable as thick as your arm. The switch from air to water cooling happens around .5-1Kw, this is the same range they seem to be targeting for datacenter stuff.<p>Aside from leaks, the building maintenance staff really needs to stay on top of the cooling water chemistry. Cooling loops tend to make slime and if you let it get out of control it clogs up everything and becomes a real problem to clean.
Liquid cooling also enables heat recovery and free cooling all year long.
Here's a project using liquid cooling to recover energy from Data centers to heat greenhouses.
<a href="https://www.qscale.com/" rel="nofollow">https://www.qscale.com/</a>
It seems odd to discuss liquid cooling but spend the first part of the article talking about AC PDUs and their issues. If you are able to use liquid cooling in a data center, aren’t you also sophisticated enough for modern DC to point-of-load?<p>I can see why supermicro needs to pursue this direction, but it seems inevitable that their business will get eaten up by OCP and integration at the rack, row, and building level.
One place i worked used flourinert to cool the circuit cards of a integrated circuit tester.<p>Each card has a 'cold plate' where the outside of the cold plate has a brass tube brazed to the plate in a serpentine pattern leading to two quick disconnect connectors that automatically engage/disengage when the card in inserted or removed from the card cage.<p>The other side of the cold plate (the 'inside') has spring loaded fingers to pull the heat away from chips. We had some chips that dissipated over 100W.<p>This worked great - you'd lose maybe 1 drop of fluid per card insertion, etc. We used flourinert, but DI water, mineral oil or water with ethylene glycol could be used as well.
Technically, the best solution is distribute. Specifically, compute stuff on user's own devices, as opposed to moving everything to these centralized clouds controlled by just a few large corporations.<p>Too bad that's unlikely to happen.
What makes this time different?<p>I've literally heard this is coming to datacenter for 20 YEARS and it hasn't yet.<p>Liquids like mineral oil, etc to be the cooling agent yet it never gains measured adoption.<p>And with so much of servers now being centrally managed by just a few major cloud providers (AWS, Azure, etc) - unless you can break into those few accounts - how will this time be any different than the past?<p>From 2003: <a href="https://www.hitachi.com/New/cnews/E/2003/0217/index.html" rel="nofollow">https://www.hitachi.com/New/cnews/E/2003/0217/index.html</a>
Cool stuff. I remember seeing liquid cooling solutions that immersed the entire system in some non-conductive fluid, like mineral oil. Of course, that's a bit messy, but I wonder if all components were designed to be immersed, whether it'd be feasible to consider the entire server enclosure as the watertight unit, e.g. in a rack, rather than hoses to individual servers.
Direct liquid cooling makes a shitload of sense from a purely academic standpoint.<p>At what point in power density would it not even matter if the datacenter was at meat locker temperatures? I feel like we are getting pretty damn close.<p>The efficiency gains at scale must be really good. I can see why this is not super popular though.
Data centers have been using chilled water to cool air for a long time. It would be interesting to see cooling delivered directly to the CPU/GPU from a/the chiller.
I think datacenters will opt for more ARM architectures before introducing liquid at scale. Things like Gravitron and Apple Silicon are changing the landscape there.
Why is that guy wearing a mask? Either servers are infectious or wearing a mask is now just "expected".<p>Do any of you do a doubletake when you see people on tv without a mask?<p>Have you noticed characters in cartoons wearing masks? Check out the weather frog, sitting alone in the middle of a forest, wearing a mask.<p>This how we become Morlocks.