27. Data Infrastructure Is A Lot More Than Storage – Tech Field Day Podcast

The rise of AI and the importance of data to modern business has driven us to recognize that data matters, not just storage. This episode of the Tech Field Day podcast focuses on AI data infrastructure and features, Kimberly Bates, Andy Banta, David Clee, and myself, Stephen Foskett, before our AI data Infrastructure Field Day event this week. Welcome to the Tech Field Day podcast, where we bring together a group of IT technical experts to discuss a single idea about key concepts in our industry.

This podcast features a wide variety of perspectives from members of the Tech Field Day delegate community, and is recorded today in association with our event this week, AI Data Infrastructure Field Day. Tech Field Day is part of the future and group, and this podcast is also published on our sister company site techron tv. On this episode, we're digging into the concept of data infrastructure and suggesting that data infrastructure isn't just storage.

It's something a lot more than that. But before we begin, let's meet who's on the podcast today. Hi, my name is David Klee.

I am the founder of Hair Flux Technologies, and I spend my days focusing on SQL Server performance tuning and everything underneath it. Hey, I'm Andy Banta. I am the storage gender from all over the place, and I'm here because I've been working on data infrastructure for a large part of my life.

Hi, and I'm Kimberly Bates. I am a analyst with the FU group, and I have been focusing on data infrastructure and data storage and data protection and data security. Now for too long, we won't go into that.

Excellent. And I am Steven Foskett organizer of the Tech Field Day events, and yes, yes. I also consider myself to be a storage nerd, but I reflect on the fact that really storage doesn't matter.

Data matters, right, Kimberly? Mm-hmm. Absolutely.

So a couple years ago we, I started talking about this as data infrastructure as opposed to data storage. And part of that is because half the storage vendors didn't wanna be associated with storage. Something was like icky or dirty about it.

I'm not sure what the problem was. So we just started calling it data infrastructure, and they seem to be happier about it. Um, and then we've got the AI sy AI stuff popping up, and all of a sudden we're having this odd merging piece of it is like, you know, we have these multi, these data manager kind of folks that we knew for data warehousing and the data storage people and the data pipeline and all that kind of stuff.

So it's gotten kind of pretty blurry, Steven, and that's kind of why we're talking about this and the poor guys that are having to make sure that things keep on moving and the performance is there. Well, that's who we're trying to talk to. Yeah.

And I, it's, uh, I understand why people aren't interested in talking about storage because nobody really actually wants to think about storage anymore. Storage is just something that's there where you have resources and you can use it if you need it, and not use it if you don't need it. Uh, and I, I do think that data infrastructure might be a, a better catchall than storage for what this, uh, idea is.

But I think at the same time, it's, uh, the, the data infrastructure or storage has always, uh, vary depending on what uses you're use, what things you're using it for, uh, what the workload is, um, you know, whether you're, uh, doing research or whether you're a consumer business or that type of thing. So I think that the whole idea of data infrastructure all of a sudden being different for AI is that's the way it always has been. I have a bit of a different take on it.

Uh, I I, I agree with you all completely. I would love to see it expanded a little more to understand more of the enabler storage. You don't buy storage just to have storage.

You buy storage to put data of particular types on. Uh, it could be mixed general purpose. It could be, you know, machine learning models.

It could be large databases, could be anything you want. Um, I would love to see the AI portion of all of this expand more into how it's being used so that you understand why it's being used and if you understand why it's being used, you can manage the whole thing smarter more efficiently. I feel like in many ways, AI is driving us to the conclusion that we've been making for a long time.

In other words, we've always known that storage didn't matter. We've always known that it was, you know, writing data to be able to retrieve it later is never, it, it, it's, it's a, it's a, a, an element of a stack, but it's never been an application. That's, that's really not a thing.

What you're doing with that data is the important thing. And we've, even in the storage industry always known that there was sort of horses for courses when it came to storage system architecture when it came to interfaces and access methods and, and things like that. But AI has really driven this point home in a way that other applications didn't.

Um, you know, if you look at cloud, uh, most cloud systems used just regular old file system storage. They kind of don't care. AI really does care just like databases and, and data lakes and data warehouses really did care.

And I think that it's really forcing us to kind of come face-to-face with this truth that storage doesn't matter, data matters. Uh, at least that's how I look at it. Well, I'm kind of looking at as a layer cake and, and, and I think a lot of the, um, people that are developing and bringing systems that are looking at layer cake, you know, the layer, the, the bottom layer is the, the drives, the networking pieces of within the, you know, how you're networking your systems, how, what you're using in terms of the asics or custom hardware that's being down there, the DPU kind of pieces of it.

Um, but even then, I mean, it is like what's in that hardware matters, you know, it was just with a, um, one of the vendors this la this last week and talking about, you know, they're not using DPU because they're using Intel's an Intel processor that has got a data reduction system there. So they don't need to do use A DPU. So even in the hardware piece of it, there's some highly customized stuff items that are doing this just not off the shelf for a lot of the vendors.

The next piece of it has to do with the protocols piece that you just talked about, whether your file block object or whatever, and that does matter BA based upon the ai The piece on top of that is the data services with the traditional data services that we're looking at, which is all the replication and, and those items that are going on. And then adding more into has to do with all the cybersecurity things that they're bringing in. And then the next thing on top of that that seems to be for AI is more of the data classification that we've been trying to get people do for years and years and years.

But when it comes to ai, all of a sudden this thing become super important because I have to be able to do the items about identifying all that metadata to be able to figure that information out, what I need, hide, hide the privacy information to be able to drop it into my vector database. And we're even now starting to see the data infrastructure vendors bringing vector databases into that stack, um, which completely changes in what they're delivering on. Um, it's no longer just, you know, those other layers that are below, but it's, it's gotten bigger and taller.

Well, those, those would be largely the data services, as you mentioned, that fit on top of the data infrastructure. And I think the demands that's make of data infrastructure would be potentially some of the things we see out of ai, which would be the, um, much higher amount of storage or much higher amount of, uh, storage that's needed and much faster bandwidth to the storage that's needed. And these can allow some of the data services that you just mentioned to work much faster.

Uh, you know, at the far end of the spectrum though, there's still, the AI is generating huge amounts of data that you need to end up somewhere as well. And a couple weeks ago at SDCI went to a talk by IBM about talk, talking about tape systems for ai. It's, uh, so it, it's like the data infrastructure spectrum really hasn't changed that much other than the, just the raw amount that's needed and the speed, the access to some of the more, some of the data that's in more demand.

And, uh, I think you'll, I think you're correct that the data infrastructure companies are providing more of these data services to make these things more available and probably working on getting their data infrastructure to handle this, uh, fairly well. I, one, one other aspect that I wanted to throw into this mix that I am gonna be very interested in hearing from our vendors in the next few days on is, uh, power consumption is a huge issue, especially not necessarily in the US as much as it is all over the world. It should be more an issue here, but unfortunately it's not.

Well, that's another issue I think that AI has given rise to, and that's, as we've talked about it, AI Field Day and a lot of other events. Um, because the AI hardware is so ridiculously expensive and capital consuming, it, it, it is important to make sure that you're optimizing these systems more than previous systems. So essentially it's, it's, it's, it's incredibly important for storage to be very, very fast, very reliable, low power, low heat dissipation, et cetera, because every ounce of all of those things is gonna get slurped up by these GPUs and these advanced, uh, you know, offload processors.

And, and those things are so much more expensive and so much more valuable, uh, and mission critical than the storage that companies, you know, we talked about solid, IM, we've talked about this with them, that it's driving sales of more advanced storage. I mean, I remember talking about this with some of the other kind of scale out storage companies as well. Uh, one of them in particularly privately confided to me that although they're a leader in AI data infrastructure, they're not even an afterthought when it comes to specking out these systems because their systems are so cheap relative to the cost of an AI supercomputer that essentially it's all about specs and it's not about price anymore, which is wild situation to be in because the enterprise storage, uh, world has always been so incredibly driven by cost and it's so incredibly profitable.

They're really sidelined because of the cost of the rest of these solutions. And that's shocking. I'm surprised to hear that because we're also, the flip side of that, Steven, is that couple good friends that are doing the, are doing quite a bit of work in helping these clients, you know, actually consume their GPUs that they spend a lot of money on, are also now going through the process of looking at what data needs, what they need to store all this data on, um, to support the GPUs.

And the few of them are getting sticker shock. So, okay, so wherever that sticker shock is, so, I mean, maybe it's not the millions, you know, hundreds of millions of dollars that GPU is. So, so maybe that's the issue here is where the balance of that first transaction is.

But as, as this other side has happened, they, they're going through and saying, wait a second. Well, how are we gonna do this? How are you gonna really, and I think that's why IBM was talking about take, because it's cheap, right?

It's very cheap to, to keep and is also, um, power efficient to, to do that. So it's kind of an interesting space to be in. Yeah.

This was specifically in relation to one of those mega projects where they were buying, you know, literally billions of dollars of hardware. And in that context, the storage wasn't that big of a line item. And that could be very, very, yeah, I, I can see that definitely can see that.

And I think that's probably true if I look at a supercomputer that the storage is often not that big thing. The, the supercomputer is always, usually it's the network and the servers that are costing the, is paying is really the costly piece of it. That'd be a fascinating breakdown to look at scale of these deployments and then watch the graphs shift as the scale of it grows.

And in terms of, of cooling in power as well. Um, you know, it's, it's interesting to think about the impact of networking and storage on power consumption in the data center. We heard about that at AI Field Day from in Fabrica, and I've certainly heard that from storage companies as well, to the point that essentially they cannot use disc drives, not from a performance, well, certainly from performance perspective, but mainly from a power and cooling and and perspective.

Essentially they need that space for, for GPUs And, and we're, and they're so, so they're moving away from the hard drives, which consumes, supposedly consumes more, more power. Um, and there's arguments on that out there as well. So I'm, I'm, I'm not gonna get into the, to, to the arguments and everything then the solid state, which reasonably doesn't consume as much a power because just kind of the design of it and you're not having to keep them spinning, um, in order to free up power for the GPUs and the servers and that kind of stuff.

So yes, we're definitely seeing some of that, that process happen. It's, I mean, it's kind of like throwing, throwing a lot of things up in the air trying to figure out how I'm going to architect for it. And a lot of it, a lot of it, a lot of it goes back to metadata management.

You know, it goes back to your point earlier, how much of this data can we de stage onto something like tape and get it off to the side, stop paying for the expensive storage, and then how quickly do you need to retrieve it based on what you're doing with it? I think that a lot of that's gonna depend upon how, what the use case in the application is, because there is this discussion of certain use cases that you're gonna be keep retrieving that data and teeing it up to retrain and retrain and retrain. Um, there is also the use case of which I think they're calling rag, which, uh, or, or they are calling rag in, in which you're taking, um, new data, you know, and constantly bringing in new data and training it.

Um, so maybe that is completely training on top of the current, current one that's been engine that has been trained, but there is a constant flow of data that's going through this pipeline in order to keep things moving and current and accurate in what it's producing on the other side of the house. And as this happens, I mean, there are not awful lot of intermediate results that you probably need to keep track of in, in some way piece bits and pieces that you, you don't necessarily need to make reference to right away, but, uh, that you might wanna make reference to at some point. And that can certainly be some of the deep, deep archival type things that could be pushed off the tape or could be put on spinning media.

And this is a, another hot topic with me in, in AI is that, uh, that there needs to be some auditability in the way AI processors are done. And keeping these, uh, records of how we got to where we are today is a necessary step. It's interesting too, one of the things I heard from a company that was implementing, um, model training was you, what you just mentioned, uh, that there needs to be traceability in terms of the exact components that were used for the model, but also that, um, preparing data for training was a huge, uh, user of, uh, storage capacity and storage performance and storage features.

Essentially, they needed to collect and move and organize data to prepare it to be used in training. And that was something that was taking them a lot of time and resources as well. It's metadata management.

Well, and I'm wondering if it's even more than that because you're getting into, well, metadata management is classification. You're getting into the privacy of the information, being able to, to protect any information that's, you know, black it out. So I, when I'm looking at a record, I'm looking at just the information that I'm allowed to have access to and some of that access management.

But I absolutely agree with you, Andy, just like the traceability becomes really critical because, you know, we will see those rules go through probably in Europe first and then over California and then trickle across the United States the way it seems to, seems to happen. But for, for being able to say, okay, so this is how I came up. And I, I think about it, if we use, start using AI for risk management for any kind of decision that impacts somebody's financial environment, their health environment, any decisions that are made in that person there is opening, you know, there's always opening up for liability.

8, et cetera. And this is, yeah, straying off data infrastructure at this point, but that, um, that is sort of the point that I look at here where it's, if you, uh, if you're counting on the answers you're getting, you need to demonstrate that those answers are based on knowledge. Uh, and this goes into, you need somewhere to store this traceability.

And this is not necessarily going to be the, uh, wide fast storage that the GPUs and the CPUs and the, the engines that are actually doing the AI processing are going to need. But the, you will need these along the way as well. So I think the, the, um, you talked about the multiple layers stacking up of the data services, I think the, the multiple breadth of layers as well, where you have every, everywhere from tape up to, uh, you know, the, the newest vast high bandwidth memory, uh, is, is important as well.

And if we're gonna be talking about data infrastructure this way, I also think that we need to sort of keep in mind the concept has blurred between what is considered storage data infrastructure and what has been considered memory infra data infrastructure. These layers are just continuing to pile up. And the distinction between which is memory and which is storage are getting fuzzy.

Isn't that based upon really persistence? Or why would we, why would we? And and your, your, and I'm not, you know, arguing here at all against that, but how, why would you merge something that's not persistent with something that is persistent?

Well, and lots of times things are worked on in non-persistent ways that are, that are immediately flushed to a persistent way, simply because you don't, you can't get the bandwidth to a persistent way mm-hmm. Immediately. But the, the layers, the layers still pretty much stack up that way.

Mm-hmm. Yeah. And with the advent of CXL and the advent of network CXL, you're going to end up with the idea that you have an awful lot of the storage networking problems you had over the past 20 years.

30 years are going to start showing up with memory. Definitely true. And, um, you know, another thing I'd like to bring up too, that we're gonna hear about at AI Data Infrastructure Field Day is the various protocols that can be used to interface these things with, um, AI systems.

So we heard from Elastic at AI Field Day about, uh, vector database interfaces for rag applications. I know that we're gonna hear a lot of talk about object storage, uh, from MIN io. We're gonna hear a lot about other storage methods, uh, you know, in terms of supporting Nvidia, uh, super clusters and so on.

Um, what, what's your take on database versus storage versus object and, and so on? Well, this, this is a question that, again, came up at the storage developer conference a couple weeks ago where there were about 40 variables that, that were thrown up on the screen talking about automotive data systems. And they, they brought this to a point of saying that power consumption is one of the biggest concerns with automotive data systems.

And if, if you, if you approach it this way, some of this becomes rather, you know, some of it, if you're basing it entirely on power consumption, object storage becomes a very obvious choice where you're not delivering blocks if all you need are bites and you can deliver large objects in as a single object, rather than having to do multiple operations to get those objects. So, uh, but that's, that's just one factor, and there's a variety of different factors depending on what your use is. That's, that's why we have so many different storage protocols today.

Yeah. Short answer is yes, Yes. It, it's a dessert topping and a floor wax.

So we've hit a lot of topics here. Everything from literally the, uh, physical media being used to the importance of metadata and storage services, uh, protocols, databases. What's your summary here?

What makes data infrastructure more than storage? And Kimberly, let's start with you since you know this was really, uh, your area of expertise. Well, it's because we need to understand what's inside of the data, more so than we did ever before.

We need to be able to do that in order to track how we're training something to do something, you know, whereas before, you know, we were making the decision because we were presented with the data. Maybe we dug in to say, how did we get to that decision? But now we're turning this thing potentially over to something that we'll be making some, not necessarily decisions, but hard recommendations and making action items.

So that data and how that data is trained and how that data goes forward becomes really critical. And the other piece of it is that we've spread our data across the world and needing to manage to that, where that data resides, et cetera, is part of also that piece of that geopolitical, you know, who owns what information, et cetera. And, and I, I see data infrastructure being different from storage and in largely the ways you talk about that there are data services, there are, are, there are the concepts of metadata that you need to pay attention to.

Uh, but I'm, I'm going throw in the cynical comment that we talk about data infrastructure rather than storage, because storage companies don't wanna talk about storage. I completely agree with all of that. The, the way I look at it, data infrastructure versus traditional storage.

Traditional storage is the what and where data infrastructure adds the why and potentially even win. Interesting. Uh, yeah.

That's, that's, that's very much true. And that really is the difference between storage and data. I mean, it's the, it's the, it's the what and the why more than it is anything else.

And, and I think that we in the storage industry have been keenly aware of this question for a long time, and now we are finally, uh, really facing it with the advent of ai. So thank you all so much for joining us, not just here on the tech field, a podcast, but as well at, uh, AI Data Infrastructure Field Day, uh, which is happening. com.

Before we go, uh, tell us a little bit more, where can we connect with you and continue this conversation? com. Uh, and I'm definitely looking forward to the next couple of days.

Yes, and I'll certainly be available all of AI data infrastructure field day. You can find me on Twitter on Blue Sky, uh, both as Andy Banta. com.

Uh, Steven, thank you. I am going looking forward to AI Field Field Day, or ID Field Day, which is chunk of words. Um, you can definitely find me at the group website and Infrastructure Matters is my podcast.

Um, and we'll see you there. Great. And as for me, you'll see me here on the Tech Field Day podcast as well as, uh, most Tuesdays on the Textron Gang and of course, at our various field day events.

So thank you for listening to this episode of the Tech Field Day podcast. If you enjoyed this discussion, please do subscribe. You'll find us on YouTube as well as your favorite podcast application, so you don't miss an episode.

And also, please do give us a rating or review. We'd love to hear from you. This podcast is brought to you by Tech Field Day Home for IT experts from across the enterprise, part of the Future Group.

com/podcast or view us on Techstrong tv. Thanks for listening, and we will catch you at AI Data Infrastructure Field Day and another episode of the podcast next week.