UT08x10: Cooling at the Edge with DUG

Modern AI servers generate a lot of heat, but the industry is ready with revolutionary technologies like Immersion Cooling. This episode of utilizing tech features Micah Jordan from Doug, discussing specialized server solutions with Janice Roski of Solid and myself. Join us in learn how these servers are dissipating heat and becoming more reliable in the process.

Welcome to Utilizing Tech, the podcast about emerging technology from Tech Field Day, part of the Futurum Group. This season is presented by soy and focuses on AI at the edge and related technologies. And today's episode is especially talking about AI at the edge.

As you'll soon hear, I'm your host Steven FoST, organizer of the Tech Field Day event series, and joining me today from SOY as my co-host is the fabulous. Janice Roski, welcome to the show. Thank you, Steven.

It's great to be back. Appreciate that intro. It's great to have you.

Um, you know, this has been a lot of fun talking to various companies and of course, uh, it's always fun to nerd out with the latest technology. And that's just what we're gonna do today. We're talking some pretty cool cooling technology, aren't we?

Very cool. This is gonna be literally and maybe figuratively the coolest episode we've done. We'll see.

Well, I guess with that introduction, uh, you know, let's, I guess let's dive in. Um, do you wanna introduce our guest today? I would love to.

I am, I am so excited about Micah Jordan, who is joining us from down under Geo. Micah, welcome to this program today. Thank you.

I appreciate the, uh, invite and welcome here. Yeah, as you said, my, my name is Micah. I'm the account executive here at Doug Technologies, and, uh, really excited to be here and chat a little bit about cooling.

So you guys are doing something very, very cool. Now, you're not just blowing a bunch of air at these things, you're cooling these servers in a special way. Tell us a little bit more about it.

That's right. I feel like I need to push my glasses up for this bit. Um, so I'm gonna geek out a little with you and tell you a little bit how immersion works.

Um, we're using immersion cooling to, to cool all of your high performance compute GPUs, ai, right? So your standard data center, the way it works, they have them in racks. Um, they're air cooled via crack crawl units, hot and cold aisles.

We take those racks, lay 'em down on their back, um, and then fill them up kind of like a bathtub with a dielectric, dielectric hydrocarbon fluid. Um, that fluid pulls the heat away from the servers and then, um, expresses it out via kind of like a car radiator. They're called fluid control modules, um, which then either pulls the heat out and expresses it with a chiller or, uh, idio or dry cooler.

So it's a much more efficient method of cooling your IT hardware and something we've been using for, uh, decades. Decades. So we're talking a lot about, we just jumped right in and started talking about the cooling.

Right. Um, but tell us Micah, a little bit more about the overall solution. Obviously down under Geo creates some very fascinating technologies, um, but this one is, is very, very different and unique.

So as you mentioned, lots of cooling capability, which is terrific. But tell us a little bit more about the overall container and what's inside. Well, so a little background information.

Um, Doug was built on a, a simple idea, um, find smarter ways to solve really tough geoscience problems, right? We're called down under geo because that's where we started, was in, uh, uh, geosciences. So when we started back in 2003, we didn't really have a lot of money.

Um, just a lot of really big ideas and, um, the drive to get things done. Um, so that meant we had to stay very agile, um, invent our own technologies, uh, from the ground up, including, um, very energy efficient and immersion cooling systems. Uh, well, those technologies led us to create one of the largest and most efficient immersion cool data centers in the world at Skybox just outside our US office in, in Houston, Texas.

Um, so we've been running skybox now as an immersion cooling data center at scale for well over a decade. Um, yeah, and let me tell you, the benefits have been pretty amazing from reduced water usage, um, to increased IT hardware lifespan. Um, it's just been very surprising.

So when we would show that location off to customers, they started telling us you should sell this. And that's what we did. Uh, Doug Nomad is built on that idea of immersion cooling efficiency at the edge.

Um, so you can put HPC AI anywhere you need it. Uh, it's where the data is born and processed, um, using our patented dug cool technology. So we're talking, I mean, the harshest environments when you could put a NOMAD 10 anywhere from Abu Dhabi to Anchorage.

You know, The, the benefit here, as you said, is that with immersion cooling, it's not just cooling like one spot or one component, it's cooling the entire system, right? And, and so essentially, rather than, than having this sort of, you know, hot and cool and lots of fans and lots of air guides and all that kind of stuff, you're just pulling the heat out of the entire server. And that, I, I think that would lend the server to, to, to being more reliable and, and, and being able to be maybe turned up a little, a little notch, right?

That, that's exactly right. Uh, a lot of the things you see nowadays is directed flow, uh, especially direct liquid cooling, um, to the hottest pieces of the IT hardware, right? Your CPUs, your GPUs, and absolutely those, those places need help.

They need, uh, directed flow. But the benefit of immersion here is that it covers everything, including the things that are starting to generate a lot more heat now and used to not. Um, plus you said it yourself, you remove your fans, uh, there's no more vibration, there's no more dust settling in the, the, the IT hardware components.

And that's what I meant when I said we were surprised by some of the benefits. We're recognizing a lifespan increase of some of our IT hardware by two, three, even four times. Um, and that's, mind you overclocked, that's running them hotter and stronger than they normally are run in an air cooled environment.

Um, so pretty surprising and, and remarkable benefits. So, you know, a a lot of of edge computing today is driven by mission critical workloads. Um, can you comment a little bit on how this, this device or this box is really being used for mission critical AI workloads at the edge?

Yeah, yeah. Uh, so we, like I said, we have been using our immersion cooling for our own products, you know, since 2003, right? Um, we, uh, geoscience data processing in our data hall in, in Katy, Texas.

Um, and we essentially take everything that we've learned, all the experience we've, we've gained from those, uh, compute processes in-house and take that experience and put it inside our Nomad 10 and help our customers run their AI at the edge. Um, now mind you, we are building the infrastructure. We let the customers or clients, uh, run it how they want, right?

Because it's gonna be different than, than a, a geo solution company is gonna run it. Um, but we can kind of handhold a lot of them through what that looks like in the infrastructure side because we've been doing it And, uh, you know, I, I imagine, yeah, geoscience, for what it's worth, I used to work in Houston for a geoscience company as well, and so I know the demands that they've been placing on hardware for many, many years. But of course, AI frankly has taken this whole thing to another level.

You know, I mean, it's just like with HPC technology, um, you know, what's happening in, in AI workloads especially at the edge, is, is pretty remarkable. And this is one of those areas, I think, where people are starting to wake up and realize that they're gonna need not just a little compute, but a lot of compute all over the place because we're collecting more and more data everywhere. I think that, um, the, the container that you sort of, the, the containerized liquid cooled solution that you're, that you're suggesting, what is the scale of that thing?

What kind of, of processing can be performed in one of these containers? The, the space in the Nomad 10, we're, we're talking about 26 r ru of space in one of our tanks, right? Um, essentially 56 kilowatts of it heat rejection.

So we have a outstanding partnership with hypertech, um, via their Trident line where you can essentially put six of the, um, H 100, uh, Nvidia GPUs in a two RU space, right? Um, you put 10 of those in one of our tanks, and what is that 60 GPUs in the, the space of a 10 foot container? That's quite a bit of, of workloads, right?

When you compare that to a standard data center, uh, a rack running at 5, 10, 15 kilowatts per rack, it's night and day. I mean, getting that level of power at the edge is unheard of. So that's, that's an interesting, gives me an interesting thought here.

So who really is, uh, utilizing these types of systems, like what, um, what types of end customers do you see really gravitating toward this technology? It's a great question. Um, without using any names, um, just talking kind of verticals, right?

Uh, so obviously AI use cases, so, um, I, I think I mentioned it earlier just in conversations. Um, uh, uh, an LLM company could build out their, uh, data centers somewhere remote because it's less expensive, and learn the LLM and then put some of our products at the edge where their end users are, say San Francisco, they could put five of our nomad tens so that the latency is not nearly as, as, uh, intense as it would be if it was in the middle of South Dakota, right? So, LLM companies are really interesting, uh, uh, fits for this, but some of the, the less, um, obvious ones that kind of surprise some of us would be things like cloud gaming, um, where latency is absolutely imperative, um, putting that at the edge, uh, precision agriculture.

So gathering a bunch of data around your crops and your livestock, processing it at the edge and making decisions same day, um, without having to, you know, run all your fields. That's a pretty exciting, uh, option, right? Um, defense, uh, contracts and being able to deploy something like this, which can deploy in three months in the middle of Abu Dhabi, or well, let's say a more remote location, that's something that not a lot of people can offer, right?

Yeah. The Desert somewhere. Yeah.

There you go. Yeah. Thank you.

Yes. Um, and I mean, there's others like, mining is a big interesting fit. Oil and gas and energy.

Uh, you know, I, I usually don't do this, but I have a question for you, Janice. Um, yes. So theoretically, you know, maybe not specifically soy, but you know, component suppliers, when somebody like Doug comes to them and says, Hey, we're gonna be taking this server and immersing it in this special liquid for cooling, um, how does a component supplier react to that and what's their, um, well, I mean, I imagine that you react by saying, cool, a fun technology challenge, let's make this happen, but what does that mean for component suppliers and, and, and, and and how do you build and modify your products to be used in a, in an environment like this?

Yeah, yeah. Um, it's a great question. And there's a lot of, uh, you know, SSD suppliers out there kind of looking at okay, the overall solution, right?

The GPUs are running really hot a lot of times, um, the overall system. Um, you've got fans to cool those things, but you don't always look at how do you cool the SSD, right? And so our company is, um, really big and bullish on this.

They're going out and figuring out how do we liquid cool our SSDs in a number of ways. Uh, the diabolic cooling that, uh, Mike is talking about is a little bit unique for us as well. When we started working with Hypertech and now working with, um, down under Geo, it was an opportunity for us to say, Hey, how can we dip these into the, the cooling solution?

And the worst thing that happened at first was I think the label fell off. Um, but so we had to get, uh, get that put back on. But at the end of the day, I think our team's really always on the cusp of innovation, and we take this very seriously as an opportunity.

We know energy and Edge is a, a growing market, and we see this as an opportunity tot to take a different look at how our SSDs work. So our team is continuous, uh, continuously innovating with, um, companies like down under Geo n Hypertech to make these solutions real. I honestly don't know of any others really leaning into it, but we're kind of fearless.

That's all time when it comes to these things. Yeah. Micah, I mean, kind of the same question back to you too.

So we just heard, you know, what do you, what has been your experience in working with customers? 'cause there are so many varied components in these systems. Um, you know, how, how do your suppliers react and, and what have they needed to do to make their products ready for a a, an immersion cooling environment?

Well, the easy answer to this question is, uh, talk to hypertech, uh, or, uh, an IT hardware provider who has already done this work for you, right? Um, because they've built servers from the ground up for immersion. Um, now to be clear, we are totally IT hardware agnostic.

So it doesn't matter whether you're buying Dell, Lenovo, HPE, hypertech, cs, what have, right, what Hypertech takes, um, and, uh, say Dell through unicom takes is they remove the fans first off. Um, they remove the, um, thermal paste on the processors and replace it with indium foil. And then some of the cables that connect, different things need to be replaced if they have PVC in the casing, uh, because that PVC does break down in, uh, some of the dielectric fluids.

Um, and it'll just get really brittle. So yeah, those are kind of some of the steps. And then I think, uh, bios, you, you need to go into the bios and say like, Hey, don't freak out.

You don't have a fan running. Right? I imagine You would.

'cause I think most of these servers would be like, ah, no, Man. Yeah, exactly. I know what's happening.

Um, so it's definitely, we're, we're in the stage, like literally the cutting edge of, of moving, um, shifting this, this entire industry towards a more efficient means of heat rejection because air just can't keep up with what's being put out by the likes of Nvidia a MD and Intel. So, so Micah, where, I mean, I'm gonna take it back for a second because I'm just really interested in the use cases. I kind of keep coming back to this, but what's the most interesting place that you guys have deployed this?

Can you share that? So, one that has been really exciting, uh, for me is an, an area that's close to a coast, um, in a very hot, arid environment, right? That has been historically one of the hardest places, uh, to cool your data centers, right?

Um, and so think about like Iceland would be perfect if you could have all your data centers in Iceland and you just open the doors and everything is cooled with the environment right? Now, imagine, uh, putting it in, you know, the middle of a, a desert that's right on the coast, so you've got really humid air, you have very hot temperatures that don't really get below well, 70 degrees, right? Uh, those two things, I mean, you're just kind of stuck.

What are you gonna do? How are you gonna build the data center that if you want low latency? That's an example of one of our, I mean, our Nomad 10 is running in that environment today and performing spectacularly.

It's really exciting to talk about some of that stuff. I wish I had a little bit more, uh, meat to say like names and, and locations. But, uh, that's always been really fun to, to point to, um, and, and stay with confidence too, you know, because we're doing it.

So How do you power all this if you're dropping it into a remote location like that though? Yeah, yeah. Uh, so there's a lot of ways, um, if you wanna be entirely green, there's some companies who have, uh, very remote deployable, um, solar, uh, setups that kind of fold up into like a 40 foot container shaped thing.

You also have have like a portable wind. Uh, those are really green solutions. Um, you also have, I mean, generators, uh, you can put a generator out and run that, uh, anywhere.

And then some of them, like the example of running five in, uh, uh, San Francisco just run power and networking directly to the nomad, right? Um, then it's powered. All we really need is level ground.

And Of course, um, you know, it's not enough just to pull the heat away from the servers. You have to dissipate the heat as well. So you got me thinking when you're talking about that hot humid coastline, uh, how are you dissipating the heat from the container itself?

Depends on the, the, the mechanical method by which we're heat rejecting, right? If we were in a, a nomad 40, which uses either idiomatic or dry cooling, it expresses the heat based on your, um, dry and wet bulb temp. We kind of tailor the, the, um, chiller or the cooler based on where it's going.

But the nomad 10 uses a, a chiller, which is refrigerant, right? Um, not as efficient but deployable anywhere no matter the, the environment. And that's what we're using in that, uh, example I gave.

Yeah. So, so be, because that's a, a pretty proven technology. The challenge is getting that cool, that cool down to the component level, uh, not being able to dissipate the, the, the, that amount of, of heat.

Well, I mean the fluid, just basic physics fluid is a much better method of heat rejection than air is, right? Uh, you think about jumping into a, uh, 70 degree swimming pool, right? And standing out in a 70 degree temperature air, one of those is much cooler, and that's because it's pulling your body heat away a lot more efficiently.

It's the exact same method that we're using here, right? Just a little bit hotter, uh, and um, more consistently. And of course it's not water, it's a specialized, Right?

Yeah. Right. That's valid.

Actually, I'm glad you said that. Uh, I do kind of wanna talk about the fluid a little bit. That's one thing that we are also agnostic on.

Uh, doesn't matter what dielectric hydrocarbon fluid you use, we, um, it all works in our system. Um, but some of the similar properties they all have would be, um, so dielectric, so they tend to have flashpoint, um, upwards of 250 C. Um, I've seen some as low as 150 C.

Um, they're all biodegradable. They break down their, um, base carbon, their, their, uh, carbon based, um, components within 30 days in environment. So if there is some sort of a spill, it's not something you have to, it's not like an oil spill.

Uh, this is a, a completely different type of fluid. In fact, when we're transporting, it's considered a, um, so it's non-toxic. I've actually tasted it doesn't taste good, but it's edible.

Um, and, uh, I think actually I've heard that there's talk about a, a peanut butter and jelly flavored one. So we, We don't want that. We do not, do not want to be drinking our peanut butter and Jelly Electric Marvins out of our data center.

Um, I Dunno, Steven, I think I would try it. You wouldn't try it. No, I would try it.

It's funny though, because, you know, you go to these h HPC shows and you see companies like yours on display and right next to them are some of the big names in industrial chemicals. And, um, you know, you know, and, and, and, and I think people are scratching their heads like, why is, why is the company that I buy gasoline from at the, you know, supercomputing show and this is the answer, right? Yep, That's right.

That's exactly right. These fluids that are being produced by them are pretty remarkable. And mind you, they have been around for decades.

This, some of this fluid, similar fluids are used in your transformers, and they're right around you right now. Um, they're the same types of dielectric fluid used there. Um, they're tailored a bit more for the, uh, data center market, um, and made specific, but they're not anything new and cutting edge in that respect.

They're tried and true and, uh, it's perfect for this use case. Yeah. 'cause we've been seeing, uh, immersion cooling in computing and high performance computing and specialized approaches for a long time.

I guess just sort of to sum up, I mean this, the interesting thing here is that you're taking this tried and true technology, but that you're deploying it specifically for challenging applications and specifically at the edge, which again, I think a lot of people might say, whoa, whoa, whoa, you're taking this stuff out into the, into these challenging environments. But in many ways, these, these things are better at the edge than conventional solutions, right? That's exactly right.

I mean, think about your, your entire infrastructure is insulated by this fluid, right? There's no more dust concerns. Uh, there's no vibrations.

It's lasting longer. Um, and this is expensive hardware. This is the kind of stuff you wanna protect, and that's what this does.

Exactly. So even though it might be a little scary to people to think about, um, you know, it's not like you're dipping your server in water. It's, it's, uh, proven technology.

And, uh, yeah, I I think that the fact that it's insulated from shocks, uh, thermal shocks, physical shocks and so on, I think is, is another thing that people might not consider, but that could be pretty valuable. Well, it, it's incredible stuff. Um, will we see you at some of these, uh, future shows?

Where can we get our, I i, I don't wanna say get our hands on it because frankly I don't wanna my hands in there, but well, You can softest hands in the data center world. So where are we gonna see you? Yeah, where, where can we run into this, uh, solution?

So you can actually come and see the Nomad 10, uh, fully kitted out, um, at the YADA conference in a few months in Vegas. I think it's September 8th. Um, we'll have our Nomad 10 there showing it off in a booth.

Um, and our CIO will be speaking at that, uh, conference as well. So, um, really interesting conference created by, uh, some of the creators of, uh, data Center Dynamics. Think George Rocket is his name.

So come check us out there in Vegas in September. And of course, Vegas in September is a hot environment that you could deploy one of these things. It is, absolutely.

Yeah, it most certainly is. How about you, Janice? I mean, we'll be all over.

Where are we gonna see us next? Yes. Uh, well you might see us at the YADA conference as well, you know, but certainly, uh, Dell Technologies world.

Um, we'll be at, uh, flash Memory Summit. We'll be at the Open compute, uh, summit as well. So we'll be in a lot of places.

We go everywhere, but we might, we might show up with, uh, Micah and his and his team in a couple more occasions, so stay tuned. Maybe not quite ready to announce anything, but it would be very fun to be able to get, uh, I'd love to get the Tech Field Day delegates hands on some of this stuff too. So Micah, we will make sure that we do that at some point in the next, uh, little bit here because it's just neat.

It's just very cool stuff and I really appreciate having you on the show. Yeah, I appreciate you guys, uh, uh, letting me join and talk a little bit about this really cool tech. Thanks for listening to this episode of Utilizing Tech.

Uh, if you can, uh, if you enjoyed this, uh, you can find this, uh, podcast in your favorite, I'm gonna just redo that 'cause I stumbled twice. Here we go. Well, thank you very much for joining us and thank you everyone for listening to this episode of Utilizing Tech.

You can find this podcast in your favorite podcast application as well as on YouTube. If you enjoyed the discussion, please leave us a rating. Please leave us a review and, uh, maybe provide a comment too.

We'd love to hear from you. This podcast was brought to you by Soy as well as Tech Field Day, part of the Futureum Group. com, or find us on X Twitter, blue sky, and Mastodon as utilizing tech.

Thanks for listening, and we will catch you next week.