98. Rising Memory & Storage Costs Make On-Prem Hardware Uneconomical – Tech Field Day Podcast

Is it a bubble? Is it coming? Is it going?

When are we going to be able to afford to buy RAM and storage for our on-premises infrastructure? Should we just move everything to the cloud, or do we have to keep waiting? All this and more on today's Tech Field Day podcast.

Welcome to the Tech Field Day podcast, where we bring together a group of IT technical experts to discuss a single idea about key concepts in the industry. This podcast features a variety of perspectives from members of the Tech Field Day delegate community and is often recorded in association with one of our events. Tech Field Day is a part of the Futurum Group, and this podcast is also published on our sister company site, Techstrong TV.

On this episode, as we head into Cloud Field Day, we'll be discussing how the rising cost of memory and storage make on-premises local hardware uneconomical. Before the discussion, let's meet who's on the panel today. com, and I'll be talking about memory and, and how expensive it is.

Good Lord. com. Um, I will be talking about very similar things, including how that impacts truly local things like new laptops.

And of course, I'm Alistair Cook, an event lead here at Tech Field Day, and also now working with the Futurum Research Group, looking at data center and cloud, and one of my top predictions for this year, 2026, is that that rising cost of RAM and SSDs, and now we're finding that even hard drives are sold out at all of the usual manufacturers. Uh, this drive for, sort of contracts to buy all of this hardware to build out, the AI data centers of the future, um- Yeah ... all these long-term contracts these, cloud providers have and, and these, AI providers have seem to be squeezing out anybody who wants to build infrastructure on premises, and maybe this is gonna drive the death of upgrades to infrastructure on premises.

It's certainly going to constrain it. So if you were planning on a hardware refresh for your data center, you're gonna take a good hard look at what your memory needs actually are, and if you can get by for a few more years on your current build, because I don't think this is a permanent state of affairs, right? We've, we've been through this before.

There have been supply side shortages for all kinds of different components, especially when there was a major leap forward or demand for something. We saw this with GPUs, first with crypto, and then with AI when they were trying to build more the training than the inference, and there was, you know, not enough GPUs, and you saw gaming cards just spike ridiculously, and then NVIDIA dedicated resources to building AI-specific chipsets that weren't commodity GPUs, and that pressure decreased. Yeah, I find that a fascinating, point, Ned.

You know, I was just comparing prices because I went to go buy what I consider would be my absolute last laptop. It's made by a company called Framework, and the thing I like about it is that, you know, it, it, if I need a new motherboard, because I've broken a motherboard once, I don't have to throw away the whole laptop. Um, but between the time I priced the small amount of memory, only thirty-two gigabytes of memory, from two months ago, the prices had skyrocketed and almost doubled.

And then also looking at a solid-state drive, right, the prices almost doubled in the past two or three months since I did my last estimate. So, you know, again, that, now again, this is consumer-grade hardware. It's not commodity-grade hardware, but those pressures are really there.

But it still kinda feels like, you know, Dan Aykroyd and Eddie Murphy and, well, what, what, what was that movie, I forget, where they corner the frozen for, frozen orange juice or something on the, on the open market. It's like, you know, what's going on here? 'Cause it do- does feel a little bit like market manipulation, but who knows, right?

Uh, so much of it seems to be going, like you're saying, towards more now the inference side of things. Right. Well, it's the same fabricators that are making the consumer components.

Maybe not the drive, but the components that go in the drive. So if that's all being bought up by the cloud hyperscalers, and now more and more the AI companies that are building their own data centers, or at least have plans to, they're buying up that capacity now. I'm sure, like, TSMC is probably booked out for several years at this point with what they need to fabricate.

That will change over time, so it's like, can you weather the storm for the next two or three years while things catch up? I also think it's gonna necessitate folks finding new ways to innovate around the supply issue, finding new ways to approach the crunch on memory or the crunch on SSDs. So what can you do to replace those components or make your application or software not need to take advantage of those components as much?

That could actually be beneficial in the long run to make our processes more efficient. And I think this is one of the things that's, again, as, as you say, things are cyclic. And so we've seen phases where there's been huge leaps forward in the capability of the hardware that's, that's in our computers.

And then, all of a sudden we see innovations in the software side that consume that capacity, and this has, this has sort of been the leapfrog that we saw as Moore's law was carrying on was, m- as we got more compute capacity, we could worry less about being constrained by compute and memory, and we could use more of those resources to, to deliver innovation and new applications. I think Ned, you may well have hit one of the things that-Is an option at the moment is be more efficient in how you use resources. So maybe we'll start to see things like higher consolidation ratios on computers.

We may, on virtualization platforms, we may also see more aggressive optimization of resource usage on things like setting much more restrictive constraints on the amount of resources delivered to Kubernetes pods, for example, as they're being scheduled. I know there is, in the, the cloud governance kind of space, there is a big space for cloud FinOps, so being more efficient about how you use the cloud resources because you're paying for them by the, the hour. Well, if you've got a fixed amount, finite amount of resource on premises, and it's uneconomical to upgrade and increase that amount of resources, y-you've got to apply the same sort of discipline to the limited resources you have on premises.

So I think this may... Y-you know, there is this whole cyclic thing of being more efficient use of what's now a constrained resource versus we've now got an oversupply. But we will also, as you're saying, you'd see that this shortage is not going to be forever.

Now, it may be that this, this shortage of resource, this high price of resource runs for another twelve to eighteen months, but I would anticipate that the market will change again sometime within the next two years, and this increase in cost, this very high demand due to the expectation of what people are going to build with AI, this is also going to pass. And so this period of time in between is the challenge, is how do we get business innovation while we're constrained for resources. Mm-hmm.

And if you want to think about a company that would really look towards gains from efficiency, the cloud hyperscalers are the ones that are possibly going to pioneer this process. " And that may cause them to look at how they're doing inferencing right now, 'cause that's probably the most expensive part of their AI offerings, is how can we make inferencing more efficient and less memory intensive? I don't necessarily have an answer for that, but I suspect that some people who are way smarter than me are already thinking about that challenge and what they can do to shrink the model size, 'cause that's part of it is that model has to sit in memory somewhere, and also just the processing time to, process tokens and, and generate responses.

That's definitely something that is going to be on their minds, and we will benefit from that as they find more efficient ways to run those models. And it's interesting, I was preparing for, an event that I'm working at right now, in about a month, and, I was looking through some of the stuff. Uh, I've been working with Oracle technology for twenty-five years, and they are now touting...

I was really surprised that this bubble up to the top. " So you don't have to move anything. You can leave it where it's at, leave your data exactly where it's at, because as we know, right, AI and the other demands of...

just not AI, but anything predictive or, you know, analytic needs data. The, the, the thrust towards, making it simpler to categorize, or even just get the training data or be able to apply a model and inference, you know, an inference against a much larger set of data, right? Leave the data where it's at is becoming more and more the trend, but now the thing I've seen over the last year is, "Leave your data where it's at in its own data center.

" Um, so that's an interesting aspect. Another aspect is, I... You know, we're all hearing about vibe coding and everything else like that.

" You know? So it's... And, really, I mean, th-those of us who still have that dirt underneath our fingernails know there's a way to make things run faster and take better advantage of, memory, even things like a, a proper index for a vector, type column.

We're not hearing those kinds of discussions. It's, ship the code, right? And I'm wondering what that's gonna do long term to not just technical debt, but also in terms of inefficient use of memory, or especially inefficient use of GPUs or TPUs.

There's every possibility that AI could actually solve its own problem in that case. If you can take the knowledge that you've gathered and the efficiencies, the efficiency techniques that you figured out and enshrine those in a skill, then AI can make use of that skill and solve for efficiency much faster than a human being can, because I can say, "All right. I, I wanna explore four different potential optimization techniques for this query.

So I would like you to farm out... You know, come up with four different optimizations, farm them out to sub-agents, test your, you know, performance improvements from each one, select the one that has the greatest optimization, you know, enhancement, and use... " That's something I can ask it to do, and then I can ask it to do that across all of my queries that I have, you know, stored across all the different applications that I've written.

And I can just have an agent do that. So you may actually see, like, a weird spike in agent usage that is specifically around optimization to make things better in the long run But it's, it's that optimization that AI's, coding agents are least able to do, it appears. So yes, the, the idea of having an agent go and find all of the optimizations, what you need is the, the, the corpus of here is how to recognize the situations that require optimization, and then here's the optimizations that, that are applicable to test with it.

And the difficulty we have at the moment is most of the agents are trained on here's, here's a pile of code, not the, the sort of what we would've historically called an expert system- Yeah ... of here's how to understand the consequences of the things that are in the code. 'Cause of course, current AIs don't understand consequence.

Uh, this is why you would need that agent to go through that sequence of just randomly test possible optimization until you find one that works. And, and again, that's not an efficient way of optimizing. Uh, you need to optimize your optimization methodology a little more than that.

A- and back to our original thing right there, Al, was wh- wh- what does this mean? Does this mean that really on-premises computing is in its death throes except for where on-premises must be on-premises because of, government regulations or financial regulations? I-I...

Is it really just my, my cloud or somebody else's cloud at this point? Uh, wh- where are we headed in that regard? Yeah, and so if you can't afford to upgrade your on-premises environment because cloud providers have long-term supply agreements that are making them more cost effective than you can currently get hardware on premises, does that change again the, the calculus, right, the cost benefit?

Because usually it comes down to is there enough benefit to moving to the cloud, that it's worth us taking that effort. It's not, it's not usually there is no way this can ever be put in the cloud. We've, we've seen that pretty much anything can be put in the cloud, and, sometimes it costs an awful lot to decide how to put it in the cloud.

Usually those same workloads cost an awful lot to decide how to run on premises, and I've certainly seen things like financial environments where there is huge resource under commit on premises, that leads to excessive cost on premises, that then means that being able to have a maybe a less resource over commit on, on cloud was more cost effective for that company. I wonder what the impact of this will be on the secondary market for hardware sales. I'm sure it's already happening, because if the price goes sufficiently high that it doesn't make financial sense to buy stuff in the primary market, buy directly from vendors, then perhaps it makes more sense to buy slightly older but more of the same hardware and memory, and just run that older, less potentially powerful or efficient, but actually having the memory you need, at a lower price point.

So I, I would be very curious to see what the secondary market looks like. I don't know if that happens primarily on eBay or if there are some vendors that really specialize in it. There's one called, ServerMonkey that resells, you know, servers.

I actually have one in my lab when I needed a- an oversized, HP box. So, hmm. It's...

I'd be curious to see if their prices are also spiking for similar reasons. Yeah. I definitely saw some coverage of using this, not throwing out the storage that you're replacing, retaining maybe your older storage.

It was in the context of, of flash storage SSDs that were, retain your old SSDs because, you maybe want to use them as a lower performance tier. And the more expensive SSDs that you maybe have to buy because you need the performance, you don't wanna hold your cold data on those. So some discussions, as I say, Ned, about either retaining longer than you'd always have retained or potentially buying, used, secondary market, components and putting them into your data center.

And maybe architecting for the fact that these older components are going to have a, a shorter life or a higher failure rate and designing systems around that. Um, given that we maybe feel that this is only a one-and-a-half to two-year-long issue, then you gotta balance. You know, is it worth optimizing for this transitory situation that we're going to be in for the next year and a half?

Is there a business benefit to putting all of that effort in, or do we just accept that, maybe we're going to be constrained for a while and we need to find other places to make innovations in our business? Curious what the two of you think regarding whether or not that we're in a bubble. 'Cause if the AI bubble is truly a bubble and it pops in the next six to 12 months, then those data centers that are supposedly gonna get built aren't going to get built, and suddenly the amount of available capacity spikes tremendously and prices crash.

So I'm curious, like, Jim, do, do you have thought... Do you think we're in a bubble and it's gonna pop in the next six to 12 months? I'm going long on tulip futures these days.

Uh I'm joking, of course. I, I seriously believe, we're in a bubble. Absolutely.

I just don't... From the deeper studies I've read about the, special purchase vehicles and all kinds of stuff that are going on from a financial perspective, you know, the circular purchase agreements, if you will. Uh, certainly from a financial perspective, Ned, I think we are.

Uh, now, are w- are we really... You know, what will we do if the capacity doesn't get built, right? And it turns out we really need it, and then you've got the flip side of, well, what happens if we end up with all of this-Capacity, I think.

I do not remember exactly which company it is, but before they became, a hyperscaler or supplier of hyperscaling, they were actually a cryptocurrency company. And so you know, so maybe what happens if we are in a bubble and this pops? Will we suddenly see, "Hey, you know what, let's...

Cryptocurrency is the next thing again," you know? And will they switch back over to that? I mean, because this, if, if the capacity's built, right?

Um, really interesting. 0, if you will, after that. What's gonna happen here, you know?

Be- so it, it could go both ways, you know? Not from a financial perspective, from a technology perspective. 0 really happens, whatever that's gonna be.

Uh, we are in, in uncharted waters and circled by sharks at this point, in my opinion, in my personal opinion. The next th- big thing is, is quantum entanglement simulation with GPUs, I think. That's- Yes ...

that's my guess. It's, it's quantum- I- ... quantum, quantum ...

I would agree to you. Yes. Yeah.

And let's... Oh, what that would do to cryptography and, and possibly alternate energy sources and all kinds of different ways to analyze things, it, it, it could be fascinating. Qu- quantum has some, some really interesting applications around material science, around designing how to build, how to construct materials, and so the quantum piece.

So in terms of, of net quantum simulation is not quite so interesting, but using the GPUs to do the error correction so that we don't need to use more qubits and more quantum, compute to do the error correction for the errors that are inherent in the quantum system. So if we can achieve that, and there's already been some work around using CPUs to do the quantum error correction, because one of the challenges with any quantum system is, as you grow to more qubits, you need hugely more qubits in order to handle the error correction. Well, if you don't need the huge more for error correction, it's much easier to, to scale the systems to get real benefit out of quantum.

So maybe we'll see that. But, that, that... It does bring us a- an interesting thought, that if these data centers that are contracted, that are what's driving this shortage, this high price for memory and CPU, if these AI data centers get built and then the people don't come, then yeah, what kind of innovation can we build out of this, what will be incredibly low cost, compute resources?

Uh, I mean, at the moment, we're mortgaging our, the industry to build these s- resources, but if, companies who are building these huge resources end up defaulting, on that mortgage, well, there'll be a, a lot of very cheap resource to be had and a lot of very unhappy people who, are not gonna get repaid. True. And we also have to consider that there's, like a concerted pushback on the building of these new data centers, at least all across the US.

Tons of states and local communities have pushed back and said, "You are not putting a massive new, you know, gigawatt data center right next to our town. Just no. " You know, maybe some construction jobs while it's getting built, but then the actual staff needed for that data center is, like, 15 people.

Right. So- Right ... one of the things to be aware of is that it's not even the, the, the power that's the issue so much as the water that's gonna be consumed.

Yeah. So of the modern designs, the, the, the... This co-location of power generation seems to be a very popular thing, but co-location of water generation, how are you actually cooling this thing that is generating gigawatts worth of, of heat?

Uh, it's the water consumption that's gonna be really terrible, particularly in some of the places where there's, there's ample power are places where there's a distinct shortage of water and where older techniques like free air cooling are, are just not practical because it's either too damp or it's too hot. So it does make me wonder if the delay of building these data centers and the bubble popping may lead to these contracts just never getting fulfilled, and suddenly we are just flooded with memory and SSDs in, like, a year. And if that's the case, then, I would feel very silly buying an overpriced machine today unless I absolutely had to.

So I'm wondering if you're also gonna see consumers and enterprises holding off for 12 to six, to, you know, 16 months on purchasing until the bubble does pop, and then you'll see a ton of purchasing happening right after that. Fascinating idea. And one other thing, another point.

Let's say these danter- data centers don't get built or don't get used, but the energy supplies that are being built to supply them, like small modular reactors, which just recently in the US got approved, or I think it's either Google or Microsoft, I believe Microsoft is actually licensing, I'm not making this up, a fusion plant, which hasn't even happened yet. Right? But they're planning for that.

So imagine, okay, well, nobody wants to use AI, but we've got, you know, extremely low cost and reliable nuclear energy as a result, and is that the death of the fossil fuel era because people finally see that, oh-Energy's, you know, not that expensive. So it could... There's a lot of disruption that could happen here.

It would be wild if the best thing we get out of AI is the next, like, revolution in sustainable energy. That would be wild- Yeah ... if that was the actual good thing that came out of all this.

Yeah, yeah. And for the other scenarios, I suggest we all take a break and go look at a couple of episodes of Black Mirror, because I'm sure we can come up with some really interesting ones, in terms of what that all could mean. You know, do we...

I remember one episode where they were storing human consciousness basically inside what looked a heck of a lot like a data center. What if you could do that, you know? I mean, who knows what...

Y- y- you know, we quantize people. I don't know. Yeah, I think we get into all kinds of fun stuff with, the, Netflix series Altered Carbon as well, where it was stored on a chip that you could insert in the back of your head.

Um, but let's circle back to this idea of the really high cost that's, of memory and storage that's being driven by the AI, is it a bubble or isn't a bu- a bubble? Let's get some, some numbers on, on some guesstimates, because, you know, predictions are always tricky, particularly about the future. Um, if we look at, look at your crystal ball, both of you, and work out how far out are we from this current squeeze turning into an oversupply, because we know this is the sort of the way the pendulum swings.

So I want you to, to come up with a number. Um, Ned, have you got a number for how many months out we are from this changing, this supply squeeze changing? Oh, geez.

Yeah. Well, I think the biggest thing is it takes a certain amount of time for chip fabs to ramp up production to meet demand, and that lag is gonna be anywhere from, you know, half a year, 18 months, to two years. So if I'm assuming they started really ramping up about a year ago, then we're probably about a year out from this squeeze ending.

If the bubble pops sooner than that, then we, we might get out of it sooner, but I think we've got at least another year. So I'm gonna say March 2027, come back and ask me and, and we'll check and see what memory prices look like then. Okay, Jim, when's gonna be the best time for you to actually buy that fully, full upgrade kit for the Framework laptop then?

I, you know, I'm gonna go sooner. I, Ned, I, I, r- riffing off your idea of the pushback that I'm seeing everywhere on data centers being built, that w- I'm gonna go October 2026, especially with tariffs running out. Yeah, tariffs running out in around October or September.

So we'll see. All right. Well, I'm, I'm gonna go late 2027.

I think things are going to take a lot longer to come to a, a big change than we expect. I think, these cycles are slower, and it just takes longer. So I, I, my bet is that it's gonna be late 2027.

I'd go with October 2027. So Jim's October 2026. Ned is March 2027.

I'm October 2027 as being the best time to go out and buy some new hardware for yourself. We could continue to discuss all things around hardware and the times to buy and the causes of bubbles and also some of the great fantasy stories that we see out on, on the, in the movies. Um, do check those things out if we have, if you can.

But before we spend all day, I'd like to thank you all for joining us today on the Tech Field Day podcast. And now, we've hardly started this conversation. I know Ned and Jim and I will be spending much of next week at Cloud Field Day 25 having equally interesting and strange conversations, but where can the people who are watching connect with you to continue this conversation, Ned?

com and, and reach out through the contact form. com or, as Ned said, on LinkedIn. I guarantee you there's only one with that last name and first name out there.

And of course, you can find me, Alistair Cook, Alistair Cook with an E, to make sure you don't find the English cricketer guy, and I'm on LinkedIn as well as across a bunch of different social media, and you'll find me at Tech Field Day events, coming to a city nowhere near you unless you're close to where we have our Tech Field Day events. Uh, thanks for listening to this episode of the Tech Field Day podcast. If you enjoyed the discussion, please subscribe on YouTube or your favorite podcast application so you don't miss an episode.

Do consider giving us a rating and a very nice review of how much you've enjoyed this conversation. This podcast was brought to you by Tech Field Day, the home of IT experts from across the enterprise and a part of the Futurum Group. com/podcast or view us on Techstrong TV.

Thanks for listening, and we will see you next week.