48. AI Innovation Inevitably Drives Massive Energy Consumption – Tech Field Day Podcast

Innovations, uh, inevitably, uh, have consequences. And one of the consequences of our increasing reliance on technology has been an increasing use of energy. How are we gonna deal with this?

How are we going to provide new energy? Is it nuclear? Is it solar?

What are we gonna do with the energy required to cons, you know, to transmit data, to store data, to, uh, compute? And where does the heat go after? That's the question for the panel on this episode of the Tech Field Day podcast.

Welcome to the Tech Field Day podcast, where each time we meet, we bring together a group of technical experts from our tech Field day community to discuss a single concept. This podcast features a variety of perspectives from members of our Tech Field Day community. And today we are actually recording it live at AI Field Day in San Jose, California.

It is currently January, uh, 2025. We have talked a lot about ai, uh, for the last week, but one of the things that didn't really come up as much as I think many of us would have liked is the question of power consumption, pollution. Uh, basically the fact that, uh, or the the off repeated thing that AI is, is burning down the rainforests and consuming as, as much energy as, I don't know, Ireland, I dunno, pick a country, right?

Um, but it's not just ai, it's all of modern society. We're, we're electric now. Uh, we're, you know, building dams.

We're electrifying the whole darn state. And what does that go to? That's the topic of this episode of the Tech Field Day podcast.

Before we begin the conversation, let's meet who's on the show. Hi, I am Jim Rinky. I'm a technical consultant, and, uh, I'm from Chicago, Illinois.

Hi, Jack Paul, our principal analyst with Paradigm Technica, focusing on cybersecurity and ai. Hi, I'm Andy Banta. I'm an independent consultant, uh, covering all sorts of technology issues.

And I'm Steven Foskett, uh, organizer of the tech Field day events and frequent pundit on things like this. So this is an area that I talk about a lot on the Textron Gang. For some reason, they have me pigeonholed as the guy who will come on and talk about renewable energy and green power with Bonnie Schneider.

'cause every week she brings up another positive and hopeful green story, and everybody shoots it down. I try not to, uh, I think that we are actually looking toward a positive, uh, green future despite the fact that power consumption just keeps rising. But I wonder, do you guys agree?

Yes and no. We're, I think that we are coming up with new green energy sources to cover an awful lot of the, uh, new demands that are coming up. But there's also the, the new power density of devices is incredible.

This is a topic that came up, uh, last year at one of the field day events where one of the vendors talked about having 120 kilowatts in a rack. And since then I've heard discussions about going up to like 400 kilowatts in a rack. That is a huge amount of power.

And they're even talking about having 400 volt buses in the racks to cut down the amount of copper that's needed in them. So we're, we're talking about incredible amounts of power, uh, usage. Uh, there's another topic that was discussed in one of the conferences last year was that roughly a fifth of the power in an electric car goes into the data systems rather than motivating the car.

And this is another example of how much power is getting consumed by data systems for a variety of different purposes. Uh, I, I've done a bunch of research on a lot of these different topics and have quite a few things to talk about, but I don't want to be, I don't want this to be a monologue. Well, I'll say that the power density is good, right?

It is really good to get things as, as dense as possible from a power, um, consumption. The challenge we now have is getting the power to the computing equipment, right? So we can get, we can build data centers.

We have, it's very easy to find cheap land out in the middle of nowhere. It's very easy to get data to those data centers laying co uh, laying fiber optic co. Uh, to them to get the data there is easy.

But getting power to them is a big challenge. And what I've heard is that data center reservations out to 20 27, 20 28, are okay in terms that we've figured out how to get power to those data centers. As people are looking and building data centers out past the beginning or middle of 2027, they can't find enough power to power those data centers.

So have we really run out of power and what are we gonna do because we, we need more and more and more compute equipment. Yeah, and that's an interesting point too, Jack. Um, that, uh, I'm an investor full disclosure on a company called New Scale, right.

Which I was really excited to invest in because it's small modular reactors, right? And Bill Gates, for example, for some of the, uh, Microsoft data centers has been talking about using these small modular reactors. They have a different version that the, the one new scale is actually liquid cooled, but they, uh, have a newer version of this, uh, that's actually liquid sodium cooled.

So super safe, uh, you know, none of the three Mile Island concerns that people had though recently, uh, to power a data center, they were discussing about, since it's still licensed, reactivating part of Three Mile Island, uh, to actually power, uh, the infrastructure for, uh, a data center, right? So the power utilization curve that sounds very Star Trek mm-hmm. Is not changing, you know, it's, it's certainly accelerating.

And back to your point about getting the power to where you need to get it, right. Uh, many people don't know this about the US power grid, but there are disjoint parts of the US power grid, and it's extremely difficult, in fact, almost impossible to shift power from outside of Texas into Texas. And this is something that we were kind of on a trail, uh, to re electrify our country.

Uh, who knows what's gonna happen as a result of that, but imagine if you could ship power much more effectively. Well, you know, Steven started out the conversation talking about green power, and I believe the greenest power is nuclear. I'm a hundred percent belief it that the challenge we've always had is there's a huge fear factor around it, which he brought up.

Mm-hmm. And there's a huge regulatory factor around it. So yes, the right, yes, Microsoft invested and said they're gonna invest billions of dollars in restarting three mile island.

That's still many years away. Right? And the, the belief is because of the regulatory hurdles and the, the, the, the fear factor around nuclear that as small modular reactors are not years, but decades away.

Yeah. Well, I also think that that solar presents as a greener option nuclear. Not that I, not that I have any problem with nuclear power, but I, I wanna get back to the IT side of this problem where yes, there, there are issues with getting the power to the data centers, but, um, the, the data centers themselves are producing a huge amount of heat.

Yeah. And it's, it is, this is a matter of you need to dissipate the heat somewhere. People talk about liquid cooling servers and that's, that's cool, but the liquid has to go somewhere to dump the heat.

Yeah. They're talking about geothermal cooling for liquid cooled Servers. Right.

And there's, there's a couple of different topics here. There's, uh, a company that I won't name that is in the Bay area, probably not too far from here, that has essentially a two story cooling tower out behind their development plant, because that is what they use to cool their data center. And this is essentially dumping heat into the center of a, a heat island already that, right.

So what you're talking about is dumping heat into the atmosphere, Right. Dumping heat into the atmosphere. Right?

Absolutely. And another thing that was brought up by, uh, somebody I talked to from Switzerland is that in the u in the eu now new data centers are not allowed to dissipate their heat into the atmosphere. Yeah.

Existing data centers can, but new data centers have to figure out some other way to dissipate their heat. And there's a place in France where they're using it to heat a swimming pool. There's places in the Nordic countries where they're using the heat to, you know, basically pump into the, the heating system for the town.

But at the end of the day, day, even if we have, even if we're carbon neutral on how we generate the power that goes into these data centers, we are still producing heat. And that heat is going to, it is going to be causing issues, especially if it's dumped into heat islands already. Yeah.

And, and that's a really interesting point because there's basically the, the energy has to go somewhere. So there's, there's a couple of things here. So first there's generation of electrical energy and how we're doing that, whether it's, you know, coal or, you know, uh, nuclear or one of the green renewable sources.

I will point out that, uh, photovoltaic solar panels, the learning curve for them has held steady at about 44% per year for the last 20 or 30 years. Which means, what that means in practical terms is that you are getting a little less than half as much energy per dollar from solar as you did last year. And that's caused real, really strange economic impacts where it's almost like a deflationary curve where it makes sense to wait another year before you buy your p PV panels because they'll be so much more effective and or so much cheaper next year.

It's a really weird situation, but essentially those panels are being produced at such massive scale too. Um, you can't put that genie in the bottle. It's already cheaper to buy solar panels than it is to just continue to feed a, a gas powered turbine.

Yeah. Um, so, so I feel like solar is, is just going to happen, but what you say is, is absolutely true then, then what Any power you put into a data center produces No. In the physics terms of things work there, it's every, every watt you put into a data center turns into heat.

Mm-hmm. And it, and then you gotta do something with it. But See, see here's where it gets interesting from a IT technology perspective is these are civil engineering problems.

Unfortunately, when you look at the actual compute boxes themselves, right? The we can stuff more and more transistors on a chip and make that chip lower power consumption and do more inside the chips. That's, it's moving the data off the chip, it's the iOS where the power is.

And absolute, I mean, that's, that's actually, that's, that's part of what's driving some of the density. That's where we're end ending up with these densities. Exactly.

120 or four. And there was a presentation at, uh, open Compute Project Summit last year where, uh, one of the people from one of the networking companies essentially said that they fear that over the next five years, fiber optics are going to consume $7 billion worth of electricity. Well, absolutely.

And that's, that's that it's a, it's a, it's a huge amount. But then when you look at, when you look at, for instance, we're here talking about ai, when you look at a, a rack of equipment, right, for ai, right, the comm com communicating between components in the rack box to box in the rack, you can use copper because it runs efficiently short distance, right? Right.

You can run it at, at very low power compared to fiber, The fiber optic fiber driving the laser, which Is why they're, they're trying to cram things into a tighter and tighter right. Uh, density. And you're right, they can run it at a lower voltage.

And the same, um, the same discussion just drew a distinction between active copper and passive copper. And part of the, part of what you need to think about here is that on serial links, uh, you need, even if there's nothing being carried over a serial link, you need to continually keep an idle stream going to keep the parity proper on that screen. So even if you're not carrying any data that that serial link has fiber Optic is still being used energy, Well, fiber optic and copper And copper still using Energy.

So it's, it's like an elevator, you know, consumes power to go up and down empty as well as it does full, because It's, well, it, except in this case, the elevator has to, the, so essentially it has keep moving no matter what you do. And you, you have to keep the electronics, the, the, the copper bits moving even if it's not carrying valid Data. Right, exactly.

And they, they do good distinction between active copper, which is what I just described in passive copper, which you can essentially turn off if you're not transmitting any data. Interesting. And it was, uh, it, I'll, I'll have it included in article soon, but it was a very nice layout of essentially how much power is being consumed by passive copper, active copper, and three different fiber optic technologies along the line.

And as I said, their, their estimate was $7 billion worth of electricity in five years. And well, so there's a flip side of this discussion as well, and that's, that advances in semiconductor manufacturing don't have to be used to improve performance. Exactly.

They could also be used to improve efficiency. And so we were talking earlier about the incredible things that are happening in the IOT space with, um, I was bringing up the ESP 32, which is just an incredible little microprocessor that can run on battery power for months at a time. It's got a full TCP stack, it's got an arm processor, and it basically does everything we've talked about except instead of optimizing for performance, it's optimizing for efficiency and your phone and your laptop and your iPad and your whatever.

All of these things are incredible examples of basically turning this whole question on its head. I wanna put it to you. Is there a chance, or am I nuts that data centers could similarly be optimized for efficiency instead of optimized for power density?

Well, Absolutely, and I, part of it is if they, power density is optimizing for efficiency, where if you cut down on the amount of, uh, on the amount of power needed to transmit data across wires, if, if you cram everything into as tight a space as you can, exactly, the amount of energy you use to transmit data between those various different things, uh, that comes down. And that is why they are trying so hard to, to get things as dense as possible. And yes, I do think it's important to start optimizing for energy.

And, uh, soy gave a great talk at, uh, the AI data infrastructure field day last year where they, they addressed a problem by taking a look at the total cost of ownership of a data center. And that included, uh, replacement costs, energy costs, cooling costs, maintenance costs, and they attempted to, to demonstrate that these are all factors that you should take into account, not just the performance and the, if you look at like, the lifetime of, of a data center or disc or any one of these, typically you spend more maintaining that thing just powering it and cooling it than you do initially the initial cost. But, but to also, to your point Steven, about how designing things, right?

Um, you know, we're, I, I'm doing a project where I've had storage, I've had spinning diss running in a, in a NA box for 13 years, right? And it's like, that's As, so you are selling not Years, 11 years, here's 13 years, right? It's, it's scary to have spinning discs running for 13 years.

It's time to replace it. And, and it'd be great if I could replace the spinning discs with low power SSDs because that I'd love to, you know, cut my power consumption down by, you know, 50%, a hundred, you know, 90%. But SSDs today are not suitable for the use case because of the properties of, uh, of, of non-volatile memory versus the properties of spinning discs.

Well, it depends on how you go about it. And I mean, cer certainly you can do SSDs at scale. And this is actually something that Jack and I were discussing, uh, just a few days ago.

Uh, we did, we took a quick look at, um, Seagate drives and between a Seagate 5,400 RPM drive and a Seagate 7,200 RPM drive, the power consumption was double. Hmm. And, and then you have to ask yourself, uh, we had a, another tech field day presentation I wanna call, call your attention to, uh, Fujifilm.

The tape manufacturers presented the same, uh, picture at, at Tech Field day a few years ago. And they were demonstrating just how, uh, less power consumption a tape, you know, store a bit on tape is than a disc or an SSD because it's disconnected. And also they went further into the manufacturing and, um, of those of those things, they were showing how, you know, how much less intensive it is to manufacture a tape cartridge as opposed to a disc drive.

And it was really interesting to kind of consider all of these aspects because I think that we're all just so in love with the state of the art and we're in love with faster, faster, faster. Maybe it doesn't need to be faster, faster, faster. Yeah.

One other aspect too that, you know, I'm gonna go a little bit out in the future here. We were talking about small modular reactors, but I am consistently seeing news stories, again, news stories about how companies are not, are, are, are actually building the, uh, equipment to enable fusion power transmission and other things from a fusion site. And there are now, again, is this vaporware, I'm not a hundred percent sure, but people are already anticipating that nuclear fusion, not fision is going to be a thing much more quickly than we would've thought even 25 or 30 years ago.

Um, and the implications of what would happen if fusion were available, a clean power source, right? That literally you can take in deuterium from sea water and power, immense amounts of anything, what that's gonna do. And I think we're much closer to it than, than we think, right?

What is that going to do to this entire factor, right? Uh, or this entire equation? Because what, you know, the, Steven, I know you and I both own EVs, right?

Uh, the cool thing about an EV is I can put output from my solar panel or my, um, my wind generator into, uh, an a non ice vehicle. I can't do that to an F-150 that has an internal combustion engine in it. Right?

But you can do that for an F-150 that has, that's an Electric if it's an lightning, yes. Yeah. But so I mean, this is gonna be, if this happens, and the interesting part, 'cause we were at AI field day this week, uh, was that, uh, there are companies that are using that, uh, technology to design the Toka Mac or whatever model to contain the plasma without having to actually build that.

And IF you know, if we're gonna spend money or we're gonna burn fossil fuel, let's burn it to power a data center that can get us fusion. It's, I'm kind, I'm looking for, uh, an end game, you know, but I think some sort of high ground anyway. I Think, I think, I think energy and energy consumption is just like storage.

Whether it's, whether it's, whether it's it's spinning discs, SSDs or horizontal surface in your house. You will expand to consume all available storage and we will expand our need for energy to consume all the available energy we can generate. And if we can generate oodles and oodles of energy, we will build oodles and oodles of data center and compute centers and whatever to use that because we see that that gives us more and more benefits and cool things that we can do.

And there's a whole lot of cool things we can do. Definitely Very warm things, very Warm, very really hot things that we can do. And It is, it is a really, it is a really interesting thought.

Uh, and I, and I think that to your point, I mean, I don't know how likely we are to get fusion power. I don't know. I'm not really a believer in small modular reactors either.

But that being said, I'm also not gonna write 'em off because the truth is, you know, this stuff tends to surprise us. Technology tends to improve to the point. I mean, Andy, uh, you and I watched and kind of chuckled at the prospects of solid state storage until we were not laughing anymore, right?

I mean, it was pretty pathetic at first and then suddenly it was not. And it's the same with a lot of this other technology. I mean, you know, the, the, you know, artificial, the first AI was pretty pathetic, and then suddenly we hit a, an inflection point and, and it does things pretty well.

Yeah. It, it's the same with solar. It's the same with a lot of these things.

And also another aspect that happens is there's a lot of cross pollination of technologies. So like electric vehicles gave us improvement, you know, a need for improving battery technology. Yep.

And that battery technology is now improving the reliability of data center power and the usefulness of solar panels because you can, you know, we have much, much better batteries than we ever did, right? And so it is really interesting to think about how this, you know, that kind of cross contamination of ideas can, can, can actually improve everything. Yeah.

Amazing. So I'd be interested when we actually get to the point where, where data centers are, are capable of being run off the DC that you would take directly off a battery, and I mean, I know telephone switching equipment is already that way. Yeah.

But it would make sense that if we actually are going to be able to supply things through, uh, through like a battery powered, solar powered battery, that we should be able to power our electronic equipment directly rather than having to do the AC to DC to It's DC to ac to dc. And we do that because we can, we can power the equipment on DC but we can't transmit the power optic. Absolutely.

I I, no, I under I understand the issue, but it's, it's a matter of doing the transmission. You, you, you have to do power transmission and it's, I mean, that goes to the, the whole idea of running 400 watt buses or 400 volt buses in, in a rack. Like, are you insane or you're gonna kill somebody doing this?

But it, it, it, it, it is, it is interesting. I I do hope that what, that this is a positive thing, uh, you know, kind of as we're getting to the end of the discussion here, um, you know, what's your take? Is this a positive prospect or is this a negative prospect?

What, where are we going? I, I think it's a amazing opportunity for cross pollination of all kinds of things happening. And I think the example of the electric battery technology leaping further and further ahead as each year goes by and finding out that, for example, electric vehicle batteries actually last a heck of a lot Yeah.

Longer than anyone ever expected. And the ability to take them, even if they're at end of life for a car and plug them into an array that you could save that green energy or less dirty energy into and have it, you know, I mean, that's something that people really didn't think about when this all started. Uh, I think it's a very much net positive.

It's, uh, I'm a full believer in, um, the future of technology and what it can do for us overall. And if that means, you know, we learned a hundred, 150 years ago that we are an energy, you know, technology uses energy and we're an energy consumption environment and energy economy, and it's, we are gonna continue that way, whether it's coming from dead dinosaurs or splitting atoms. Andy, and I think this, this hearkens back to another discussion that we had previously about, uh, did we ask for this?

And the answer, the answer is no, we didn't really ask for this. But it does present interesting problems that do need to be solved. And in, in terms of, I, I see that as a, you know, cross between negative and positive in that yes, it's going to create, create problems and at the same time it's going to create opportunities to solve these problems.

Uh, from a technology perspective, I, I kind of, I see where Jack's coming from, but it's, it's like we are going to have technological problems that we need to address while doing this. And I, I'm going to call it fairly neutral, but there's gonna be both positives and negatives and it's not something we ask for, but we're gonna get it anyway. And I will put one more thing in there, and that's that if we've seen anything, it's that constraints breed innovation.

Whether it is in AI model training, like we've been discussing this week, or electric vehicle batteries or the development of solar, the development of nuclear power, the development of better, you know, microprocessor, you know, fabrication technologies. If it wasn't for the demand from cell phone industry, uh, which provided all the money in the world, we might not have proceeded so quickly in developing the ever better microprocessors. And all of this, like we've talked about, kind of tends to come around and cross pollinate and, and improve the situation.

And so hopefully, uh, the power situation in the data center and the power situation will be improved. Uh, at the very least we'll probably come up with new and hopefully better solutions to solve those problems. So thank you for the discussion here.

Uh, before we go, uh, where can we connect with you and continue the conversation? So you can find me on LinkedIn, uh, Jim, I'm gonna spell it. C-Z-U-P-R-Y-N-S-K.

I don't ask how to pronounce it. Uh, or you can find me at, uh, blue Sky at Jim, the Y guy Blue Sky Social, And, uh, I'm Jack Poller. com or on social media as at Poller, P-O-L-L-E-R.

And I'm Andy Banter. com. And I'm Steven FoST.

Uh, you'll find me at s Foskett on most social media networks, including the Blue Sky and the Mastodon and, and others. Uh, you'll also see me on, uh, the Tech Field Day podcast on the Gal It Rundown, uh, Textron Gang. And, and, and actually now, uh, we're returning with another season of Utilizing Tech our Monday podcast.

So check that out. We're gonna be talking about AI at the edge, and that's gonna be a lot of fun. So check that out on your favorite podcast application as well.

Thank you for listening to the, this episode of the Tech Field Day podcast. If you enjoyed it, uh, please do subscribe. You'll find us on YouTube or in your favorite podcast applications, and we would love to hear from you if you want to give us some feedback.

The podcast is brought to you by Tech Field Day, home of it experts from across the, uh, industry, and is also brought to you by The RUM Group, which is our parent company. com/podcast and you'll also find us on Techstrong tv. Look for a new episode every Tuesday.

Thanks for listening, and we will catch you soon.