Daren Shumate – Advanced Cooling Technologies Tackle the Energy Demands of AI Data Centers

Hello, along to the latest edition of the Textron AI Leadership Insight series. I'm your host, Mike Bazaar. Today we're with Darren Shumate, who is CEO of Shumate Engineering, and we're talking about AI infrastructure and how to cool it, because if you haven't noticed, this stuff consumes an inordinate amount of energy and well, we don't have enough of that to go around.

Darren, welcome to show. Glad to be here. Describe if you would, and kind of bring everybody up to speed, but what is the fundamental challenge here?

'cause we, I think everybody knows GPUs consume a lot energy, but I don't think we kind of fully understand the implications of what that means in a data center, how it all hooks to the grid. So kind of, you know, lay it out for us. Well, so, uh, this is, this is has always, there's always been an issue with how do you cool data centers.

Uh, and you talk about an inordinate amount of, of, uh, of energy being used by these GPUs. Well, basically, data centers now are being deployed in a hundred to 200 megawatt buildings in campuses that have, you know, a, a gigawatt, uh, of it load. And to date, uh, most data centers are cooled with air cooled chillers, which is a, you know, a self-contained unit on, on, basically on, usually on the roof.

That's then providing chilled water for the cooling of the data center. But an air cooled chiller is not real high tech, and it's very energy inefficient. So basically, for every, for every 10 watts of compute, you need five watts of power to cool it with, with the air Cool chiller.

5 gigawatt on the grid. So that, that, that's one of the big challenges just from a macro standpoint, from a micro standpoint, the, the issue is these racks with, with the, uh, Nvidia and their competition type chips, uh, uh, run very, very hot. And, and, and how best to cool them because air cooled chillers are providing chilled water to coils with air blowing over them.

And the air is the medium that takes the heat away from the racks. And we've reached a point where the air just can't, can't do it. So we now are deploying, uh, systems whereby the racks are cooled by some sort of a cooling water loop and connected then to the building through some sort of a, a heat exchanger called a, a cooling distribution unit, CDU.

And, and that's the issue. And the big question is how do you provide that water, which would typically be in the 85 to 90 degree, uh, range when to your air handling side, you need around 68 degrees. So that's the challenge.

So there's really two challenges. How do you get, where do you find that much power because you need that much power to cool in addition to your, your computer load, but how do you actually segregate power that's going at, uh, in, in water that's going at, in that 85 to 90 degree range versus a 68 on the air side. Mm-hmm.

So do we need to, for lack of a better phrase, refurbish all our existing data centers to make in a combination for this AI workloads? Or are we essentially building new data centers to kind of handle this particular class of workloads that will all ultimately supersede the previous generations? Well, so, um, the, the retrofitting is, is definitely possible and, and feasible.

We, we have talked to a handful of the developers about that. Uh, right now, most of the ai, uh, requirements are going to brand new data centers, uh, that are, you know, built at very large scale. When, you know, when I talk about gigawatt, that might be part of one of those campuses.

And we know of several, you know, I, I think it's pretty public knowledge about several of those very large ones in Texas. And there are several around, uh, the country that are being deployed at one or two or three gigawatts of critical power. Um, so the, the, the, anything that you're looking at retrofitting is a smaller data center.

I mean, holy smoke, four years ago we thought a hundred megawatt was, was a big data center, hyperscale. And, and schuit engineering is involved with designing buildings now that are 200 megawatts of it in one building, you know, three megawatts tol, or 300 megawatts totaled with the, with the, with the, uh, the cooling. So the retrofits are gonna be on buildings like, uh, you know, many that were built 15, 20 years ago where a building might be six megawatts or 10 megawatts or 12 megawatts.

Uh, definitely candidates for refurbishment. But I, I think if, I think it's less critical for, you know, for, for those data centers. So are there things to be done with the cooling side of the equation that would put less stress on the amount of power being consumed in the data center and the energy that's being kicked off?

And I mean, what is the current state of the art? So I'm, I'm glad you asked. Um, I'm glad you asked.

I mean, basically, uh, uh, humid engineering has come up with a, uh, system that we call hybrid dry, a datic cooling. Uh, that's a lot of words to describe a, a process that we have actually applied for and won a patent for. And, uh, why, why is it state of the art?

We, we have basically paired, uh, a an air cooler that's got a, uh, um, uh, an amatic pad with it, with a chiller and a, a pump with valves and controls. And we're able to, in one loop, um, in, in one closed loop, produce two temperatures of water, we can produce the, the, the 90 degree that goes direct to chip through the CDUs, and we can produce 68 degree water that goes to the air side. And, uh, what that does is it solves one of those two problems that we talked about before, which is how do you get, how do you get in an, in a data center that, that still has to have air being cooled?

How do you also cool the direct to chip racks through the CDUs? So our design, uh, actually in one loop produces those two temperatures of water. And oh, by the way, uh, we get free cooling up to 60 degrees.

So up to 62 degrees, all we're, all we're doing in the system is running, uh, water through the system, blowing air over the coils and producing 68 degree water. So we're getting an eight degree lift off of the 60 degree air outside. And then from 60 to 82 degrees, we actually divert some of the water that's coming off the, uh, air cool, the, the air cooler and diverting it through the chiller and lifting it to the 68 degrees and above 82 degrees.

We start, uh, trickling water over the a DA datic pad. 5, uh, I hadn't called it out as A-P-U-E-A standard data center. 5 PUE connected, which is your worst, your worst case on your hottest day.

3, uh, connected annualized. 06, depending on if it's mainly direct to chip or, or air airside. Uh, what does that mean, and why is that important?

Uh, we like to call it a collateral benefit. The, the, the original design intent was to solve the problem of how do you get the direct to chip and also to the air side, realizing that we've run out of the capacity of how much we can cool with air. The collateral benefit is on a gigawatt size build.

So, and, and like I said, we're involved with a couple of them right now. 5 gigawatt and that's 1,500 megawatts from the utility. Or if you don't have a utility, you're building natural gas generation on site.

2 is equal to 200 megawatts of infrastructure, meaning cooling infrastructure, generators, power distribution, et cetera, in the building, you don't have to build. And it's also that 200 megawatts that you don't have to buy from the utility. So your substation is smaller.

And if you're building onsite generation, that's 200 megawatts. 5 billion in just first, first cost. 5 gigawatts available, 1,500 megawatts available from the utility, and we're using air cool chillers, we can get a thousand megawatts of revenue.

But with the shumate patented HDAC, you can actually get about another 150 megawatts of revenue. And at, and I'm gonna totally mess this up, at $140 per megawatt per per month, uh, that adds up. That's, that's many, many millions, if not billions of dollars of additional revenue.

And, you know, for developers that are, you know, trying to attract, you know, you big investors, uh, that matters. And environmentally, uh, I'm, I'm driving over here today and my, I'm talking to my mom out in Kansas and she says, Hey, did you hear what Bernie Sanders said yesterday? I go, no, what Bernie, he's calling for a moratorium on all data centers.

And I said, well, I thought I liked the guy, but now I don't because why is Bernie Sanders saying there should be a moratorium on data centers? And I, I haven't looked it up yet, but it's gonna be something like, we haven't studied it, we haven't studied what the environment's going to be, Eli, you know, what's the impact of the environment? And of course, we in the data center industry know that there's a lot of the concerns about pollution, et cetera.

3 gigawatt total. So we're saving energy. We're saving energy, you know, and, and all of the, the, the consequent pollution and supply chain, you know, you know, polluting events, et cetera.

So we think that, that this really is the solution going forward. So, and who is actually building these data centers? 'cause you'll see, you know, the open AI of the old world and Oracle all talking about they're constructing these data centers.

And yet when I go look at the industry itself, there's a lot of companies that nobody ever heard of, and they seem to be actually doing most of the actual work. So how does, how do these data centers come about? Who funds them and, and, and who kind of carries the, the economic weight of them?

Uh, well, uh, obviously my, my purview is right out of the gate is mechanical electrical engineering. But, and, and, uh, we, we look at these deals and ask kind of the same question and, and you're looking at say, and, and to be real honest, the one that we're working on, I'm not sure who is underwriting it, but you know, the Blackstones and the Black Rocks are, are major players. Starwood Capital is a major player.

And, you know, similar, similar types of private equity are, are deep into the data center space. Uh, I, I have a, I have a, I have a, I know of a firm that design builds data centers, and they are, uh, they are underwritten by, I believe it's Blackstone, and, and they're, they're building, you know, at, at the gigawatt scale. So I think a lot of it's is the, is the private equity.

Yeah. And I'm asking the question 'cause there's a lot of speculation about the so-called AI bubble, and a lot of it is tied back to are we building too many data centers for these AI workloads? And you're close to the space, so, you know, what's your assessment here?

Are we over provisioning at this point? Because some folks would say, well, the LMS will get smarter and smaller and we won't have to have as much compute. And other folks will say, these things are just gonna keep growing.

Uh, well, I am not a soothsayer, but, um, I, I tend to be an optimist. com bust, the, the bubble in 99, 2000. And that truly was a speculative bubble.

There were a lot of real estate companies that got into building data centers and got way over their skis and were building spaces before they had signed leases. And, uh, you know, the spec market, you know, finally the, the, the, the lack of, the lack of the, the tenant actually finally caught up spectacularly. Uh, now, I mean, you can count 'em on seven on, on your, on two hands.

The, the, the what's driving the market are the big guys. It's meta Google, Oracle, uh, Microsoft, um, apple, Amazon, and there's another one. I, I always, I always forget who they are.

Uh, they're, they're driving it and, and what's driving it. I mean, every time you go onto your little, your little trusted device, which, which we all have, uh, now when you do a Google search, you, you get their AI answer, whether you, you're not even asking to use their AI and they're giving you their ai. So it's, it's, it's, it's, they're, they're building it in.

So, you know, most of this data center demand, we drive ourselves ev every citizen. I mean, I challenge to find, I challenge you to find anybody whose life is not completely tied up in this thing. Everything has to be, and everything that's on this device that I love is on a, on a computer in a building that's taking a lot of power and needs to be cooled, and it just grows exponentially.

So I'm not a soothsayer, but I, I don't see this as being a bubble maybe in a couple years, but I, I'm, I'm very optimistic and, um, uh, I I think it's gonna keep going because it, it's gonna get better and smarter. I mean, frankly, it's kind of frightening. I mean, I always ask anybody, when we're talking about ai, if you've seen the movie, you know, 2001 Space Odyssey, um, and we, we hear of those kinds of, you know, odd events, it's like, uh, Hal, I I need you to open up that space door and you know, Hal is Dave, I, I can't do that.

So, you know, where, where is this thing going? Who knows? Um, will the market kind of sort this all out for us?

Or do we need something maybe that feels like some sort of national, uh, policy or approach to energy and the AI age and we need to think this through, or, you know, what's your assessment of how this should be managed? You know, that, that, that's a great question and I'm, I'm glad you asked because, um, uh, like I said, my mom called, I talked to my mom today and she was talking about, did you talk about, did you hear Bernie Sanders? And on my drive to lunch today, I'm listening to, you know, the, the NPR and the Washington DC area, and the, the lunch program is, is one of the reporters talking to, uh, uh, one of the executives in Prince George's County, Maryland, where they have a six month moratorium on, on data center development approvals.

And she cited the same thing about it's not smart growth, it really doesn't bring the economics and, and, you know, we're worried about the environmental concerns and we're worried about the neighbors, you know, et cetera, et cetera, et cetera. And then, then they kind of glossed over the data centers. Then they get to the end of this thing where it's like, well, you know, prince George's County has a $90 million shortfall.

You know, where are you planning on stopping spending? And it's like, this is an economic engine. Loudoun County, Virginia brings in almost a billion dollars a year from, from their data centers.

So, so, and the reason I said that was whenever you hear op, op, any opposition, and I, I literally have a stack of, it was, it was the New York Times business last weekend. It's Wall Street Journal, it's the Washington Post. Uh, one of my colleagues at work brings in the New Yorker article, and there was one in the Har Harper's, uh, magazine, and they're all about data centers.

And they all, they all say that data centers are driving up electric costs. Well, here, here's, again, I'm not an expert in this, but the re but the reality is electrical costs, electrical rates have been flat for 20 years. And, and why and why is that?

Well, we've not been adding a lot of big, uh, big demand. And, and why is that? Well, we have replaced incandescent and fluorescent lights with LED lights, and that has gone, you know, a building like this one that I'm in, we would've counted on this being two watts a square foot in this building for lighting, and now it's less than a 10th of a watt per square foot.

So our lighting power to our lighting in this country has, has, has, is, is cut like two 5% of what it was before. And the sitting with air conditioning, we've gotten way more efficient with air conditioning. We're way smarter with air conditioning.

We weigh better, way better in slate buildings. We're way more energy conscious. So we've done a great job in preserving our resources.

Uh, now the demand has caught up. Uh, at the same time you hear about, you know, these energy prices going up, you hear about, you, people complain about the grid's bad, we can't count on the grid, and the grids are unreliable, and the grid's at least 50 years old. Well, guess what, folks?

Uh, the grid's gonna be expensive. We, we need to replace it. So at a, so at a certain point, uh, it, it's, it's kind of built in that we need to find ways to improve the grid, the regulating thing, or regulating data centers.

I think they're focusing on, on the wrong things. I think we should be looking at how do we improve the grid overall, the, the, the notion that data centers are not paying for their fair share of the upgrade is ridiculous. When, when you have a, a, a client who has 1000 megawatts and they're paying, maybe it's a discounted rate, but what they're paying, they're paying for their infrastructure, they're paying for the substation, they're paying for the power to it.

And oh, by the way, if that, it means it's improving the grid between Amarillo and Armstrong County. Well, guess what? The citizens are gonna get the benefit of that upgrade.

So, um, I, I think it needs to be not a data center centric, it needs to be a grid centric, it needs to be energy as well. Uh, unfortunately, we have a, an administration now that's like anti wind, anti-solar, which, which is I think is folly. Uh, the, the reality is the quickest way for us to get power quickly is the most, re most abundant re resource, which is natural gas.

The biggest challenge there is there are not enough gas turbines available today to meet the load. So part of, of the problem now is that we're, we're in a supply chain choke point in, in getting, uh, equipment to build the power generators. All right.

Hey folks, you heard it here. There's no easy answers and there's a lot of pros and cons, but I think when you put it all together, probably the right answer's gonna be all of the above and everything we need to do to get there. Darren, thanks for being on the show.

No, it's great. Thanks. All right.

Thank you all for watching the latest episode of the Techstrong AI Leadership Insight series. You can find this episode and others on our website. We invite you to check all those out.

Until then, we'll see you next time.