Aligning Object Storage with MinIO’s Ugur Tigli at KubeCon Paris 2024

This is Textron tv. Hey everybody, welcome back. We are at KubeCon Paris, KubeCon Cloud Native Con 2024 Europe.

Uh, you know, the buzz, just the energy. A number of people here is kind of insane. It's just, I was just talking with one of our friends here and it seems like we're at a next level of this conference.

It's really kind of exploded in a, in a good way. So it's great to be, uh, talking with great friends and, uh, colleagues from the industry. Uh, speaking of which I'm joined by ER, who is with our, um, friends at MIN io object storage.

Yes. Object storage. We've talked to you and Jonathan and folks at different times, kind of keeping up on what's happening.

Well, first tell us, tell us about your role at the company at Miao. Yeah, ly, I'm the CTO at Miao. And Miao is a, this high performance distributed object storage, all about enterprise object storage in the modern world.

And today's, today's workloads, AI and CubeCon here in Paris is amazing for us because it's our best conference. And from this, this is the first day, so the first day I've been at the booth for about two hours and we got more than 300 people stopping by at our booth, just by the two hours that's been, and it's nonstop since Morning. Yeah.

It's not even, well, it's barely lunchtime already. I just walk away to meet with you here. Like it's been crazy busy at the booth.

There's a huge interest for Mineo. We've been quite popular. Mineo has been around for about 10 years now.

We are going, we are going to be at the end of this year, it's gonna be our 10th year. It's been an open source company doing S3 based object storage. We use S3 APIs in the front end full stack enterprise grade object storage.

And we just announced last week our mini enterprise product. It's six products put together from our learnings from our largest enterprise customers to using AI workloads. Mainly we've been always a popular choice for data workloads.

And with that, we have built six of these products to enable enterprise customers to do KMS on their own, to do load balancing on their own in front of Minio software to do their own catalog database, which all the namespace of minaya can be reflected on this database. And we give them a graphic QL fast search of different objects that they have in there when, when they grow from hundreds of objects, hundred thousands of objects to millions of objects, this becomes critical for them to be able to understand what they have on their data infrastructure. So this is our inflection point in our growth of the company because we are going from different deployments in enterprise to providing them different software products to manage it at large scale.

We have huge customers now going into hundreds of petabytes and they're planning exabytes of deployment with us. So it's critical for us to provide them the tools to manage that. And that's why we are excited to be here because of this announcement last week that with Minio object storage, enterprise object storage for enterprise customers and coupon is our best conference ever.

Like for many years it's been our best conference and the crowd here loves Minio and they like to use mineo in different project from smallest projects to large deployments with large enterprises. Now being a fellow CCTO, you know, you, you really do appreciate what it takes to go from a startup 10 years ago. Absolutely.

Introducing a product and trying, trying to support very large scale, very large enterprise service providers. Yep. Whatever it might be.

It is, it isn't just more of the same. It's new problems get introduced. Totally.

How do I manage all those objects? How do I manage the security of 'em, the key management of it? Exactly.

How do I, maybe they aren't all provisioned the same way or Exactly. Configured the same way. Where are they?

Right, Exactly. Where are they? So they are everywhere.

We have enterprise customers doing deployments on Kubernetes. We have enterprise customers doing the same customer can be doing the deployment on VMs or bare metal. So our goal is to bring all of these deployments under one single pane of glass.

I always talk about Minios, uh, inflection point in their, in its history. 20 years ago when VMware started with their hypervisor with ESXI, vSphere as the core of the product, it was just hypervisors. They were just com making sure that the hypervisor is chosen and the best product.

We have been in that phase up till last week now. And VMware did what they, what they did with enterprise. They built their vCenter.

Yeah. To build and manage everything under their umbrella under networking and security and security. Everything.

AYA is under the same phase, uh, during is going through the same phase. We are just in our vCenter inflection point, just like what VMware has grown into after they came up with vCenter and Management. We are giving all of our enterprise customer a single pane of glass to manage all their deployments regardless of where they're running it.

They can be running in a public cloud on a a KS kind of a service, a Kubernetes service. They can be running on a edge on a single instance, single node. We can still manage it under this umbrella with mini enterprise object storage.

That's why mini Enterprise is bringing everything together but also giving these five other different components like a KMS. People don't want to deal with an external KMS In the enterprise world. Yes, there is a CSO organization that has a KMS, but the object storage people, they don't want to do the integration.

They want a simple solution for their key management. And we built our KMS with basic key rotation. We have no desire to compete with any KMS product or different, uh, companies.

But it's different for data. It's different, different than devices and users. They want a simple, we always implemented the Amazon S3 framework for key management for encryption, whether it is A-A-W-S-S-E S3 or S-S-E-K-M-S.

We used that same framework. What we did with this enterprise products, we u build our own KMS and we are providing the KMS to our customers. Or we built a catalog where they can search their, in this, uh, database approach.

They can search all of their objects or we build something called cache Minaya Cache. The cache product is built because to your point, we learned from our biggest customers, one of the biggest, the world's biggest streaming company uses BIN io. And that company needed a lot of caching to feed into the CDs.

The same problem that to this morning, I'm talking with the AI folks. The same problem is there as well. And caching will help them to feed into the CDNs for a streaming company or the AI foundational models needs to be loaded fast.

They don't, they, they can do training fast as well, but training is not their main concern. Main concern, after they train the model, they wanna upload their models into the, the DGX boxes or into the H 100 or whatever, uh, product or a cloud-based product that they're using wrapped around. They want to upload those model fast and they need AYA and they're coming to AYA for that.

And whether, whether you using some specialized hardware, you don't need any of that to fund foundational models to be uploaded fast, you need a very fast network and a high performance object store. And we provide the high performance push part 'cause of the technology we have and the networking they need to design. For one customer, I was talking this morning, he has a 400 gigabit network, four times a hundred probably or 400 natively.

I didn't understand if they're doing both. But if they have a 400 gigabit network, they can upload very fast using MVME technology, commodity hardware, mine software, you can feed any GPU any way you wanna do the work. Whether for foundational model uploads or you're doing training, doesn't matter.

You can get very fast AI data infrastructure with Mion. Very good. Um, so just to step back a little bit, I think I asked Jonathan each time I talk to him about this too.

Um, you know, streaming's a really clear example. 'cause you can see, you know, very large pieces of data that are getting, uh, stream distributed out to end users, viewers, whatever it might be. Exactly.

What are some other like perfect uses of object storage versus structured and structured traditional kind of databases? Exactly. So streaming, you think that streaming is easy from a theoretical object storage use case.

But honestly learning with this large customer doing streaming, it's very random data. And when you think about streaming as a use case on its own, you think that, oh, it's just streaming of data is big blobs of data or origin data. So next, next block on the disc, next block on this, send it on.

But when wish it's that easy. But when you're feeling in a large streaming company or any of the media companies that are doing DVR or Live World or like video on demand or live, uh, video tracking or those kind of applications that they have, whether external or in-house that they make when they come to storage, they're very random. So it's very critical to explain to, and I try to explain to everybody, oh, why do we need for a streaming use case?

Why do we need high end storage like MBME or SST Flash type of storage Q-L-C-T-L-C type of storage. You do need it when the numbers are large and all of them are coming to you as a storage origin, that becomes very random IO profile. So that's the biggest misconception actually with the streaming use case.

Yes, streaming of one, uh, origin media file is linear or serial. And that could be a really good use case for a big blob of object, but it's not, it's random when you're feeding a lot of them. So that's number one way.

Number two point, or to answer your question, are there use cases which you think of really streaming may not be today. We have customers in large enterprises at financial services, um, a transactional, um, a large, uh, service company that is doing a lot of transactions, let's put it this way, a west coast company. And they're doing a lot of all of their merchant transaction reporting using our data.

So it sounds like a very different use case, but primary storage, they use object storage us all of their content management. They use us. They started using us.

They went from an appliance storage to Minio as a software and a commodity hardware. They liked us a lot. Now they're doing that.

Uh, one of the other credit card companies using us for their, um, cybersecurity operations or cybersecurity analytics. That's 20 plus petabytes of data that they're collecting and they're running their analytics on top of us. So that's another use case.

Um, we have data infrastructure, large use cases of Hadoop. I've been saying this for five, six years, but Hadoop is, and one of the supporters of Hadoop is kind of getting not liked by the industry a lot. So Hadoop, especially HTFS is an a downtrend and people are using, taking out HDFS, putting Minio with its erasure coding.

They're saving from the get go of 50% off because Hadoop uses three copies, minimum replication mode or sometimes four if they need that extra. 5 depending on how, which erasure. It's a big difference.

Yeah, it's a huge difference coming. And they use, You're talking about petabytes. Terabytes, that's a lot of data.

Exactly. And I, before joining min io I was at Bank of America and I was running their infrastructure for storage server and data centers. And biggest customers of mine seven, eight years ago was a Hadoop cluster or the Hadoop team.

We would get a lot large amount of, uh, customers, like a hundred plus nodes in a Hadoop cluster. But the CPU utilization on those clusters were 2%. Mm-Hmm.

And talking to customer, everybody has the same experience. They, a lot of customers buy Hadoop clusters just for their capacity, and they don't use those CPUs. It's, in my opinion, it's waste of CPUs.

That's why Hadoop use case is an excellent use case for Minio because of the work that Hadoop community did with S3 a. S3 A is a connector. You change the HT FS connector to SVA and Miao runs automatically.

No, no need to change anything else. So that's another use case. I think 1, 2, 3 different use cases.

Very popular ones. And that's just the start. There are many, That's just the start for miao.

But I think a common thread are all that is you're doing it a scale. It's a much different problem than doing it at serving half a dozen a minute or something. You know, data lakes, there are lots of names for it.

Data lake, data warehouse, data lake house Fabric, data, fabric, all That open table formats is very popular. And we are all playing a role in that as well. And that's, Hadoop is part of that with ml, like machine learning before machine learning, this has data analytics has been done in many companies with machine learning.

It was ML ops machine learning. Now with ai, it's AI ops and AI data infrastructure. But the core of it is a, is a data layer, is a some large data store where you keep it.

And S3 APIs is the de facto standard with all AI workloads. The new generation, whether it's foundational model tools, uh, to build them, to train them or to vector databases underneath them is all object storage. And if you look at them all is at io, like their vector databases, they all document min io in their documentation from the start.

Really? That's interesting. And AIOps ML ops is changing in the AIOps.

They're all using, so there's maybe 10, 20% of the ML ops AI ops world that needs, uh, transactional DB for one reason or another to track some of the, the rest of it can go into an object store easily. So we play a large role in the ML ops AIOps world as well as underneath vector databases. And because of that, the tools all use S3 and they come to us.

So that's the story for us. Large amount of data, data lakes, we, we are, we are Switzerland when it comes to data lakes. You can use Trino, Presto, drio type of modern query engines.

They all need to do some S3 queries against us. Or you can do external table from Snowflake to, uh, Teradata. They all use offloading with NOS in Teradata old school appliance world to Snowflake doing external tables on AYA.

And you can run some same snowflake query against us extension of snowflake tables. Well, I, it's, I'm sorry we couldn't get you going to kind of talk about this. You honestly No, that's a great topic.

Who knew data and storage are still sexy. It is. I mean, nothing happens without it.

Right. For sure. And especially at the, at the volumes we're talking about.

Yep. Well congratulations on the enterprise announcement. Thank you.

And, and the compliments that that brings to the customers that have been using the product. Yep. Who maybe don't even, don't need all of that yet.

But for the folks that do, you know, those kind of things are vital. One or the other, but like some of them needs all, but they're very useful. They're all solving one specific problem.

Yep. Absolutely. Well definitely stop by the min io booth if you haven't already.

Or check them out online. Definitely Online And, uh, Min Io, min io. Thank you very much.

Very Simple. We go with simplistic across all of our marketing websites, cards, everything is Very simple. Ben io I think there's a hidden meeting there.

I kind of think I get it now. Okay. No, it's great talking to you.

And, uh, thank you so much. Good luck with the rest of the show and congrats on the announcement. Thank you So much.

Great, great to be here. Fantastic guest again, thanks for being with us for this segment. We have more, we have a full day, a full lineup.

We're just getting started. So stay tuned and if we're, we're replaying a, a, a, a previous c**n interview, great. Hang tight.

We'll be back with you with another live streamed interview in just a minute. So we'll see you in just a, just a sec.