MinIO AIStor, S3 Express API, NVIDIA GDS,m and BF3 Overview with MinIO

Okay, I'm ab anan. Babu per, if you ddu my name, you get ab, uh, Storage joke. So the, uh, I, I'm one of the co-founders and, uh, CEO and, uh, I also, uh, deeply involved in the product tech, architecture, everything, right?

So the, uh, so getting back to the tech side, uh, these, uh, three interesting things that we are working on and, uh, this, actually, these are all already out. I'll explain. So, uh, S3 express, uh, Amazon actually calls it S3 Express single Zone.

Actually, it means two things, that it is S3 express is not S3, it's actually a subset of S3. It's a slightly different, then single zone means that by default, when you, they give you an object store, it's actually kind of stretched across multi zone. They couldn't meet the AI demands, the kind of modern applications pound on the storage, and they have limitations in terms of number of requests per prefix.

You can hit the AI applications in, in a gi you will eat that up, right? They needed to come up with something that's like per as performance as Mio AI store. They actually came up with this S3 express, API single zone.

For, for us, we support a single zone, multi-zone from the beginning because it's software. How you want to deploy is customer choice. Single zone is not a big deal.

S3 express was something interesting. We actually, uh, very early on, if you looked at Min S3 API, it looked like single, single zone, right? And S3 express.

API, we had some grudges, uh, where the way Amazon designed S3 API, that the a, the some of the things that while the API is beautifully designed, there were corner cases that they inherited post like baggage, like listing still has to look like files and folders, sub directories and sub sub directories. And how you navigate the directory namespace, they still gave four six C behavior. That's nice from a legacy application, but for modern AI applications, it comes in the way.

And there were bunch of other details like that where the, the S3 express, S3 API was a general purpose. API, the AI workload is so critical that if a general purpose API can meet everyone's needs, even as even AI workloads. If you're talking about the master AI data set, your AI repository S3 is the right API, but we are talking about, uh, pushing the AI workload for all the performance and scale a general purpose.

API ha kind of holds you back. And that's where the S3 x plus API comes and Amazon basically stripped down the API set and for the, for the a PA that came in. It's not just a stripped down API, the APIs that, that are in, like for example, get object every time you're doing, get object, put object every time you're doing MD five sum computation.

MD five sum is harder, harder to sim decelerate. It comes in the way of performance. And CPU, it hits your latency.

There are other ways to do get the same reliability, uh, uh, and durability without relying on MD five sum. And then the directory, like legacy names, like a directory, uh, fake directory, things like that. Directory tree walk, all that.

Even if you list the objects, objects in a folder, Amazon S3 API would actually start and give it to you. If you're talking about billions of objects, trillions of objects starting is enormous base stage of computer source for the AI workload. So they got rid of all that.

So some, some of the subset of APIs like get object, put object, even multi-part object and list object. While they look like S3 API, they're, they're actually slightly modified. The list looks like list, but it's not sorted.

The get object, put object, there is no MD five sum requirement. There are other ways you can still do the checks on verification. There are more details.

I don't want to bore you, but it's just enough to understand that S3 express is a subset and a refined set of S3 API, who is it really useful for high performance applications. Who are those high performance applications in the modern world? AI workloads.

Then the database data processing workloads. That is what S3 X process is. We started this way and then over time, because the community grew, we have like 2 million dock pools at day.

The community so large, we couldn't convince the community that this is the right way to do. So over time, we actually relaxed and became full end-to-end. It just, if it works on S3, it should work on work on mini store.

It just that we maintained that compatibility. As soon as a SS three was in S3 express was introduced, we are like, thank you. Now this is, this is the right set of APIs.

So we actually implemented exactly like Amazon S3 express, API. When you start the Minio server, you can actually say that you want S3 express, API or you want regular S3. API by default is a regular S3 API.

You can actually restart the server and go back to the S3 API. Uh, you don't need any data change format, existing deployments. You can just move over to S3 express by just restarting it.

And that's S3 express. API right. Now Google introduced something recently similar.

Test three express only faster. I mean, does it? No, they don't actually, the thing is right with Google and Azure, it's not even S3 compatible Azure.

Completely not. Blob is completely different. Incompatible with AWS Google gave a partial S3 compatibility, but basic things like you upload like say, uh, uh, 20 megabyte object, the application would break it up and send multi-part upload that's broken.

Some of the basic things are broken. They, uh, they, otherwise Google wants you to use G-C-S-A-P-I, which is Google Cloud storage specific, API that's incompatible with S3 API even you move from one cloud to another, not just, not just the API, right? Even the bucket policies, right?

The security mechanism to access control to everything is different. They're all object store, but they're incompatible with each other. That's where for us, right?

You deploy AI store in Google Cloud, on-Prem Edge, your laptop, it's the same experience everywhere, right? And this, I think we touched upon this. I I, I didn't follow the slides, right?

So I I thought like you're controlling it remotely. I'm not. So the benchmarks you can see, right?

Here's where you can see the, so the latency, for example, if you're doing list, if I have a, a a hundred million objects in that folder, I have to start and then I start streaming fast. But before the SART completes, I, I cannot stream anything. So the time to first bite matters, right?

The latency shows up in multiple ways. Like even if you see gith objects, right? The hashing function, because it has to compute the hash.

There's something called eag. EAG is like in file system world. You call I node.

It's a unique identifier for every object. Amazon, for a long time, they made object tag, the EAG unique identifier based on MD five some. Now they relaxed it.

You don't have to be MD five, some is just some unique identifier. When you strip all these things, the result is better performance, better, uh, response time, lower latency. So o overall, it actually just, this, this is the kind of performance AI needs from AI scale.

The performance is actually about scale, and everyone can choose performance in a small scale. But when you have the these thousand nodes, trillions of objects, when you're pounding on it, can you show this? That kind of performance, even tiny calls, like every single API call, we cut down.

It immediately shows a benefit, uh, for the, from the application point of view. These, uh, I'm not going to walk into the performance details in specific, right? Just too much technical details, but, uh, Avi will share the slide deck with you Now, before I go into GPU direct, any questions on S3 Express?

Just go back to that one slide. Sure. Yeah, that one.

Thanks. Yes. Just in case there's one more performance deck as well.

This funny, I I you got it. So GPU direct. So for those o of you who are not familiar with what GPU direct is, the modern infrastructure, GPUs are becoming the main processor, and CPUs are like co-processor.

You load the operating system, you load run the Python runtime, all that. But the real workload, the work hours of the modern application is actually all these matrix, matrix multiplication. All of that is actually running on the GPU.

The CP, the the CPU. And the CPU memory starts becoming a bottleneck. How fast can you feed the GPU memory while the GPUs are fast, they're only as fast as how the data can reach the GPU memory, and this is where Nvidia came up with the, with the a, with the interface where you can bypass the CPU, you can bypass the host memory and then directly, uh, read and write from storage into the GPU memory.

And that's GP direct. And G-N-V-D-S started with GPU direct file and block, because the enterprise was mostly file and block. The cloud was all object.

Soon, Nvidia came to realization that the modern AA workloads, the born in the cloud, they all need object. And, uh, if you look at all from open AI to aro to everybody, they're actually built on from even Databricks. All of them are built on object.

So they realize that object you need GP direct an object unlike low level like a SC RDMA, like SRP, there are a bunch of protocols, like NVMU or fabric object is H tt PH TT PS. How do you do RDMA or H TT P? That H HTTP fabric doesn't support RDMA, but H TT P is today the TCP IP of the modern internet and all of the look at the data traffic, it's not fiber channel, right?

It is not like regular UDP or anything. It's H TT P carrying bulk of the internet data. And, uh, when it comes to the storage, if you see inside every cloud, you will find that object is the traffic, is the data traffic, but it, it's all at HTT PS layer.

How do you mix RDMA here? This is where that, uh, so the, uh, this is where the, the GPO direct object interface comes. So NVIDIA has not announced GA on GPO direct object yet.

We work, uh, very closely with them. We have joint, uh, joint pilot customers. Uh, uh, the development is complete.

The benchmark is complete, and it's showing promise. So why do we need it? Increasingly, networking is becoming the bottleneck.

If you put 800 gigabit, still, it is the bottleneck. Can I do 800 gigabit today? No, you can't because PCA Gen six is not yet available on these servers.

Gen five is there, maybe you can do dual 400. It's as good as it can get. And still compared to NVME drive, compared to the GPU, uh, the network is the bottleneck.

But in the, in the network, when we push this kind of data, that data, if you look at the host, the g the these are like the DGX boxes, right? If, if the data is flowing through the arm chip, grace chip into the gpu, into the CPU memory, and then going into the GP memory, you wanna cut that. You don't want any kind of bottleneck.

Increasingly, GPU are the more important, most part important part of your fabric. So the GPU direct or uh, GPU direct object of age works is from application point of view, it's the same S3 API, you do still get object, put object and all the regular operations that the, the, the data that is sitting on your object store that's accessible through regular S3 API is not like trans. You have to migrate that over to different kind of storage.

The data is still the same thing, but your GPU direct enabled application, when they do GPU, when they do GPU direct enabled client SDK, that is when they do get object. They don't actually, the way they, the GET object works is they give the object name, bucket name, just like any other, any, any other, uh, uh, S3 API, any other application. But they will tell here is my GP memory pointer and the data, the get object, the all the access credentials, object name, bucket name goes over https.

This is the control channel. It goes to the server, and then server Recogni recognizes you are actually doing a GPU direct call. In fact, the GPU direct memory is just a HTT P header embedded inside the request call.

Otherwise, the request looks like any other get object. When the Air store server sees, this is a GPU direct get object, it gets the gpu, the remote RDMA memory pointer, that's the GPU memory and it, it's not necessarily one GPU, right? Even single system has multiple GPUs and you have multiple racks of those servers.

The GPU direct memory pointer is unique. It knows exactly where across the, uh, the network, which node, which pointer we are talking about, and the server recognizes, okay, now you're asking for this, say, uh, a hundred, a hundred gb, maybe even 10 10 gb, 10 GB object to be sent to that memory. The server actually then makes a GPU direct response back.

That's a data channel. That's A-R-D-M-A channel. It pushes it back.

It's also smart enough that if the data is large enough, we can actually break it up into multiple pointers. Then a group of a cluster of servers will push the blocks simultaneously. That way you're kind of like a not bottleneck by single nick, you're bot, you can now push from multiple nicks and then the GPU machines as far as they can, as fast as they can take, we can push it in.

Yeah. But I have a question. Uh, Diana, uh, network, uh, card needs to support that.

So, uh, today, NVDS says that, uh, it has to be melanox, uh, linox nex, melanox n and MELANOX spectrum switches. Mm-hmm. That that's what they have validated.

They Have RDMA over ethernet. Correct. Underneath it is all RDMA work converts ethernet.

Right? In theory, it should work for Arista found networks or anybody, right? Edge core switches.

But, uh, N NVIDIA has to validate their software stack. The driver, the RD the, the driver is today certified only for this V already. It's more like other vendors have to work with NVIDIA to get, I have a question here because it's kind of interesting.

Uh, you like disconnect the control plane from data plane. Yeah. Htt PS Yeah.

So via https you have just a control plane, right? The API calls, blah, blah, blah. Just communication, right?

Yeah. So it's don't need much bandwidth. Correct.

And then you take kind of block data. Mm-hmm. Right?

And you stream that block data mm-hmm. Over the ethernet, uh, with what in our DMA protocol. Yes.

Right? Yes. So it's a pure unencrypt unencrypted data.

Yes. So good question. Yeah.

And, and that means that we have to have one communication layer Yeah. That it's, let's say, going through the traditional network. Yeah.

And then direct galvanized separate channel for air DMA traffic. Yes. So, uh, a bunch of details here.

First, the RDMA traffic is, uh, underneath is, is actually built over RDMA over converge ethernet. Uh, right? So that rocky interface, but then, um, if you, uh, what about Infinity Band, right?

The ethernet versus Infinity Band enterprises are largely ethernet based. And, uh, if you look at the Linox Nvidia, Nicks the same. Nick is basically the same fabric.

It's just a software layer is different. Otherwise you get basically the same capability. Converged ethernet essentially borrowed the, borrowed the INFIN band ideas into the ethernet.

Now, from a customer point of view, they like ethernet because they already, like, they took a bet. Right? Now, the, the interface, the software interface to talk to the rocky layer, if you see what, what, what's the library you would use?

You'll use IB verbs library. In fact, the driver to talk to the RDMA layer, it's the same code, uh, whether it's Infinity Band or ethernet. So if you, if you have the R dm, a fab, if you have the RDMA uh, uh, driver, it'll actually work over Infinity band as well.

Now, coming to the, uh, the question, uh, the encryption, right? It, it, uh, at, at this, at this level, yeah. Nobody's asking these questions.

Don't ask, don't tell, uh, here is a problem, right? Even CXL, same problem, right? Okay, now how would we make it?

How would we make it, uh, uh, encrypted? Then you don't get the DRAM performance. DRAM is not encrypted.

You then if then otherwise it starts looking like any other Redis cluster or something, right? But GPU direct in particular that we, we can act, the O object is encrypted. We can actually, we can send the encrypted data, but here is the problem.

Who is going to decrypt it the moment someone is intercepting to decrypt it at the client side, the RDMA, uh, path is broken. Then you need CPU can GPU do the decrypt. If NVIDIA enables RDMA encryption, decryption, uh, they actually are now working on it.

A a, uh, even compression for example, they actually have compression function that's GPU accelerated compression. Mm-hmm. And we already have it.

Uh, we already have it working in our, uh, in our labs. It looks promising. It is possible to do GPU accelerated decryption as well.

Uh, but you will take some hit on the performance. Sure, sure. Yeah.

Um, Good. So, yes, so Mex, uh, has bluefin chips, you know, as high smart nicks and stuff. Does it require smart nicks, uh, GPO direct, or You don't need smart nicks for this as long as the e even even the older, older, as long as at melanox today, what N Media says is they melanox chips connect X five, connect X six, like as long, they're all, they're all already DMA capable to a, but typically nowadays the newer deployments are 400 gigabit.

So Connect X seven is a good bet, Right? Right. Yeah.

Why do you need this? These are basically, if you are GPU applications, right? You're on the CPU, you, you can still, uh, the, the way ox, uh, the Nvidia guys designed the GPU direct object is it'll work even for CPU traffic, even if there is a Python application, some application are running on the gray CPU even they can take RDMA, that GPU direct object directly sends to.

It makes RDMA call to the host memory. What if, what if it's just a, a end-to-end melanox fabric, but it's just CPU machines. Even that box, the same driver should work, right?

So anywhere you need R-D-M-A-R-D-M-A class, it's really about CPU bypass on the client side, eliminating the CPU bottleneck. So you can directly talk to the memory is, is the benefit of it. When you enable GPU direct object, you will find, if you go to the client machine, you look at the CPU utilization, it'll be one 10th of the normal workload.

The otherwise it's not a throughput itself. Even HTTP can saturate a 400 gigabit network. It's really about reducing the CPU reliance on the com, on the compute nodes, right?

So the Bluefield, so we have been working with Bluefield since Bluefield Gen one. This is pre melan, uh, Nvidia acquisition of, of Melanox, right? The Bluefield Gen one.

It's basically a smart nick. It's an arm ship, right? Like smart nick, smart drives.

Actually, if you look at the Theder vendors all the time came to us and they said, we have a processor sitting on the nick, we, it's just sitting free. Can you run on the syst on the controller itself? We said, sure, maybe it's no different from running on a raspberry pie, right?

It's just a arm chip. But then, uh, our guys just a, a, we love performance. It's kind of a OCD problem, right?

We went and implemented, uh, the neon instructions. Neon is like the CODA instruction for the, for the CPUs. Uh, the equivalent in the Intel world are a MD world intel, typically AAV X five 12.

And a MD is, uh, is AAV X two. Now they actually have AAV X five 12 as well. What's the sym instruction in arm chip?

It's basically neon instruction. So we add the neon instructions and we actually, we graviton particularly on AWS was like so good. It was cheap and it gave the best performance.

Now ARM is making its way into the data centers. So this smart nick made a lot of sense. So all the op optimizations we did just worked here, but, uh, then we found, hey, there is even a better wider instruction set.

Like that's SV arm supports both SVE and our neon. We even did that optimization. All that benefits the smart nick and, and Bluefield.

The, it's a key requirement. Like, here's the thing. If the Nick is running, uh, if, if the smart, smart nick is the computer, so that means we have to run the software entirely within the Smart Nick AI store is the only software around in this market that is small enough to fit inside this system.

16 core, right? And just GPU, right? Uh, uh, the GPU, uh, uh, um, uh, the, the, the c the CPU memory, the arm memory, and the arm course we can run in a lower source environment, but it's also optimized for the CMD instructions in the smart nick.

These are 400 gigabit smart nick. We can fit it entirely in This is, there was a basic problem with this system, right? That's basically leads to this, this one, if I bought this smart nick, and if you told me to, to how do I put this smart nick in the rack, then if you have to buy a server, like a MD or Dell based server, like super micro Dell server, then your host processor is more powerful sitting their idle, then why do you, why do you have a proper server and then a smart nick just to power through a PC slot?

It didn't make any sense for sure. The software, a store software ran within the smart nick, but it didn't make sense to buy a server. So AED gave the controller to some of these industrial s server grade, uh, system manufacturers to build servers around it.

Finally, this is hitting the market through like super. The first one to launch a server. That server has no intel or a MD motherboard.

This is just as smartnick and bunch of drives and a PCI switch directly attached to the smartnick. It's as light, as light as it can get. We actually deployed this at, uh, PayPal and few of our customers already, that it looks very promising.

What is it is simply a jab, o and a smartnick. Nothing more, nothing less. And that is low power.

We all know in the AI world, what's the biggest problem power, right? So this is the path to that, and we are the only one who can fit inside this system.