VAST Data look at future innovations for AI and the AI OS with Solidigm

Phil Menez here. Go to Market Execution lead at Vast Data and today we're gonna spend a little bit of time talking about some of the things we're doing beyond storage. Um, so again, vast really launched our product to the world here in 2019.

We were founded in 2016 on the basis of this new shared Everything architecture that really breaks the scale and efficiency challenges that customers are facing as they bring legacy challenges or architectures into ai. First thing that we did was build a storage product. Then we added a database product that rides right on top of that architecture.

Now I have the capability to have structured data next to unstructured data. Think about the world of having vector database local to where the actual source data is. And beyond that, we've now started to bring more compute in to figure out how customers can execute, um, models and build agents.

And we're gonna talk about some of these things today in only a little bit of time. So one of the things that we have seen, common theme that's coming up and I mentioned shared, nothing shows up everywhere. Not just storage, but eventing infrastructure.

Eventing is becoming really critical to inference infrastructure. And that is really in two ways. One, it's like events are happening.

That's how my AI knows things are changing. So someone's purchasing something, a stock price is changing. Uh, an IOT uh, event has tripped, right?

Something's broken. But what we're also seeing is all these players that are actually deploying inferences scale are using eventing for feedback on how the user experience is, right? How long did it take to give me that answer?

Uh, if I gave the customer, do you like this answer or that answer? What data that is? So all the data around the user experience, how these models are working is being sent home being a venting infrastructure, right?

And because of venting infrastructure is based on shared nothing. We see these scaling challenges, right? I'm a legacy storage guy.

I had to learn about eventing infrastructure at vast. It is very complicated, right? When I scale eventing infrastructure, again, I'm breaking up my data across different nodes.

Every node's a broker. That node owns a partition, which is like a piece of what we're capturing in eventing and partitions, uh, brokers own partitions. Um, there's ordering of venting.

Infrastructure is only maintained within a partition, right? So I can guarantee data in a partition came in order. This one came first.

That one came second. I can't guarantee which one came first in a separate partition. So it's really complicated to build this out.

My main example for that is you need something called a zookeeper to keep all this stuff working, right? Because this stuff is like wild animals, very difficult to scale. And ultimately what you generally see is customers having to break up their eventing infrastructure into smaller clusters, right?

We see a lot of right amplification here. We use something called replication to keep that data. What if a server fails, right?

Comes in. I've got right that to generally two different other servers exploding capacity, um, you know, really killing utilization. And then the other problem here is that the analytics on eventing infrastructure super weak.

It's really hard to query this stuff because it's not in like a table form. It's basically like a long run on sentence where every event is a new word. So I have to, if I'm doing analytics, I've gotta go out to all the different nodes and really like replay the events that happen looking for the data I'm looking for.

So customers don't really do analytics on this. And that means depending on how quickly that data moves from my eventing infrastructure into a data lake, I might have a 15 minute gap or hours long gap between, I have real insights into what's happening in real time as we're moving to more real time ai, I wanna be able to act on things that are happening now. Someone's stealing something, something's on fire, whatever it is, I wanna be able to act on it now.

So the shared nothing architecture not only solves the scaling challenges of storage, also solves the scaling challenges of eventing infrastructure. Because I now have this parallel architecture. I have the compute layer, right?

Everything that lands in the system is acid. So we will tell you what order everything came in across partitions. We don't have to worry about any of that.

We have NVME parallel access to every single device in the system. That means I don't have to worry about any of that ownership ordering. What we actually do is take data that's events and every topic that's like what we're talking about in terms of eventing infrastructure.

A topic's usually broken up into partitions. And every topic is a table in the vast database living on that day's architecture. And every new event is a row.

And, uh, something I learned about databases, row based databases are really good for capturing data. Column-based databases are really good for analyzing data. So what we actually do is capture the data coming into the system in storage class memory in row format, and migrate it down to column format into the QLC.

So that means that I have really fast capture on storage, class memory, and really strong analytics on the QLC. So now this is really important as I want to be able to have agents interacting with data that's still on my eventing infrastructure so I can make decisions on what's happening in real time. But we're also seeing customers love this for more traditional data analytics workloads that have eventing infrastructure.

I can't get any insights from it. I've gotta move that to a data lake, move that to a data warehouse bill. Yep.

The challenge with, you know, events and brokerage and things, it, it's quite high throughput activities and you're building a a row relational database out of it and then moving it to a column and database. Um, did you have the compute structure to be able to support this at, I don't know, million transactions a second? I mean, what would that look like from a cluster perspective?

Absolutely. So fund lead-in, uh, we've tested this per server. Not only is it simpler, more scalable, we are dramatically faster, right?

5 million transactions per server. You know, one of the first ever customers that I knew that was looking at vast, they needed 11 servers in their eventing infrastructure, three on ours, right? So it's faster, it's easier to scale and it's dramatically simpler, right?

So when I look at trying to consolidate this and give my agents a view into what's happening in my real time environment, this is a game changer for our customers. The whole white paper on the details of the performance. Yeah, love that T in question.

Okay, so couple things that we're gonna hit on. Shared nothing, it's everywhere. It's causing problems everywhere.

We're also seeing huge problems with shared nothing in the vector database world because it's the same thing. Another distributed architecture vector databases, right? Some of the things that are challenging with vector databases, I wanna scale beyond a server.

Same thing like we're doing with data reduction. I've gotta create shards right now every one of my servers only has a piece of that vector index, right? These are very memory bound.

Generally speaking, when you're building vector databases, you're trying to get all the activity in the database to happen in memory. So I either need to scale more servers, right? Which adds more complexity or beef up the server's memory footprint.

We've seen customers with like 10 terabytes of memory in a server trying to squeeze as much as many vectors as they can into one server. And then another real problem here is that inserts and updates very, very slow, right? The data structures that make it easy to search vectors make it really difficult to add new ones.

'cause I've gotta like understand how do you relate to what's already in the system? I wanna make sure that you actually do some clustering. So data that looks the same, you're trying to keep it close together so I don't have to go really far apart to find things that are close together.

So that's some big scale challenges. We see more of the scale challenges as customers do more data or get into more rich types of media. I wanna do AI and inference on video instead of documents, right?

So we do see customers and like our vector databases are working fine, but something that we've realized if all you're doing is like PDF search, you're not gonna create that many vectors. So here's a a a a real example. Customer 600 million documents, 600 terabytes of storage.

It's like 4 billion vectors, right? It's not that meaningful. If I looked at, you know, a blue chip enterprise with a hundred petabytes of data, I wanna vectorize the whole thing.

43 trillion vectors, right? That's a lot of different vector databases that I'm gonna have to sard up and move around. And my data is very fragmented.

I have to build a lot of intelligence on top of that to make sure that my AI can get to the right index that's gonna have the data that I'm looking for. So we now have a vector database built on the day's architecture that solves a bunch of these scaling problems. So one, we're able to get to trillions of vectors, right?

So I can store a ton of data and consolidate without worrying about sharding and breaking up my databases. One giant vector database, we did a really cool smart cities project. That customer had 52 different shards of their vector database one on vast, right?

They really couldn't get it to work. Um, vector search scalability. Again, a lot of times these struggle with search at scale.

As I add more, I'm constantly trying to move data around, keeping things that I expect people will ask about right next to each other. That becomes harder and harder and harder as the system scales insert scalability. The fact that we can do real time performance at scale into that storage class memory means that we can do rapid inserts.

Generally speaking, that's one of the biggest areas where we see vector databases tip over. I want an agent to watch a video feed and while that video feed's coming in, I need to be embedding it and chunking it up and storing new vectors. That in-memory model does not work for that.

It just tips over. Pretty much everyone gets to video, everything falls over PDF's, fine video bad on legacy vector databases. And another problem is around security.

And this is something really cool, like why are we not just all these different products? It's one product because we can actually take the same permission structure and apply it from the document image video that you've given us to the vector that's created based on that video, right? So it's really easy to say like, I can access a document and Scott can't, right?

That's kind Enterprises have figured that out. But what they didn't figure out was how do I make it so my large language model won't answer a question about that document for me if I don't have access to it? That's hard.

The first thing we saw happen when enterprises started dropping everything into large language models. Everyone started asking questions about stuff they weren't supposed to know. Mm-hmm.

Right? Like, how much does my CEO make? Where does he live?

It's like, alright man, here we gotta shut that down. So how do we put that in place and make sure if the permission of a data, ch piece of data changes, that the permission of the vector changes? And we can do that with Vast because I apply a common data set of permissions across tables, objects, files, right?

And I know we're really running on time. Something that we can also do. We haven't talked about our data engine.

And I can also create data-driven workflows on Vest. So as a video comes in, I recognize it's a video. I launch the embedding process for a video and I store those vectors back into the vast database.

So I know I'm at time. Um, there's other things I wanted to cover but we won't hit on. But we do have our first user conference coming up at the end of next month, February 24th in Salt Lake City.

We're gonna have a bunch of announcements, a bunch of great customers there. Um, so if you wanna learn more, it's definitely a great place to tune in on what's happening with Vast and what's going forward. Thank you again, Scott and the team and all the delegates for having us today.