Jonathan Bryce on AI, Digital Sovereignty, and Open Source Innovation in the Cloud

Hey, everyone. We're back here. Live on our, what you could say, it's day three.

Or you could say it's day four. I've been here four days, but this is the third day that the expo floor is open. So take your pick.

But we're live here at Cube Con Cloud Native Con in Atlanta, and I'm really happy to have my next guest. We've been kinda waiting for him all week, whether you call it day three or day four. Jonathan Bryce.

Jonathan is the, uh, executive Director of the CNCF, but he has a dual role. He is also the executive director of the, uh, open Infra Foundation, which is also now part of the Linux Foundation, or under the umbrella or whatever auspices of Del F. Uh, Jonathan, welcome to Text Drunk tv.

Thanks For having me. Pleasure. So, Jonathan, I I've wanted to ask you this 'cause I, I admit I don't know the answer at all.

What, what, what have you been doing that got you to this position? Uh, well, it, it, it all started, uh, in the nineties with the web. I, uh, I spent a lot of time, um, doing software development, uh, for the web.

And back then that also meant you had to build your servers and put 'em in a data center. And, uh, and then I, I joined a company called Rackspace. I was one of the Oh wow.

One of the early, early employees there. And, uh, we, um, you know, built out more data centers, and then I started a cloud company. Rackspace bought it back in, and, um, that was where we, we, uh, launched OpenStack from.

Sure. Um, OpenStack started to really catch on, and so we wanted to give it a neutral home. And this was kind of before the Linux Foundation, as it is now, is really a foundation of foundations.

Um, and we, we actually talked with Jim Ziland a lot in 20 10, 20 11. He was super helpful and we created the OpenStack Foundation, I remember. Um, which, uh, we, we, uh, as we added other projects to it, we, we rebranded to the Open Infra Foundation in 2020.

Yeah. And, and finally, you know, kind of full circle earlier this year, um, merged it into the, uh, the Linux Foundation. Um, so, you know, it's been a lot of all, all infrastructure, uh, some software development, some hardware engineering and network engineering, and a lot of open source.

So let me ask you, in the nineties when you were building out servers for the websites, what was your platform of choice? Uh, well, one of the things that, um, that got me into open sources, I was a teenager and I had no money, and I started doing this that's that cheap because, uh, I, I realized I could, I could, uh, make more money than mowing lawns by, you know, building people websites, but I needed everything to be as cheap as possible. So it was, uh, it, it was X 86 and Lennox and, you know, the, the very first versions of, of PHP and my SQL and, um, yeah.

You know. Yeah, no, it similar. I mean, I was older than you, but I started as a hobby building websites, and then we had a store, 'em, and same kind of thing.

It was an, I I, I still remember my, the first X 86 server. We bought our own server to store 'em on in a little ISP and Long Island called L inet. They gave me free, they let me put a server in, uh, it was a, a four-way X 86 machine, and I thought I was a digital landlord.

As we added more sites, I just plugged another hard drive in there, a vigor hard drive. Then, then, then the web really started taking off and we, we wound up moving the Sun Ultra Sparks, and we were running Netscape Yeah. Server over Solaris.

Those, we got acquired shortly after that. But that, that was, that was the web then. That was, that was the internet.

And it was fun as all hell. I know. Um, good times.

An interesting, you know, your fact pattern with, with, uh, Rackspace is a, is a common one. A lot of people started a Rackspace, went out, did well, fairly well, and got brought back in. Mm-hmm.

Right. Yeah. Cisco does that a lot too.

Yeah. Right. So it was an interesting thing.

Um, while we're on the topic though of Open Stack and Open Infra, you still are the ED for Open Infra Foundation. Have it mentioned it once this week here on Techstrong? Yeah.

Give us a a, if you wouldn't mind, a a Yeah. Well, we just had, we just had our open Infra Summit, uh, in Paris about two weeks ago, and, uh, it was awesome. Sold out crowd there.

Um, the, you know, OpenStack is still the, the largest project in the open Infra Foundation. Sure. And right now, there's some real tailwinds for OpenStack driven by all of these massive infrastructure investments.

Um, they're, there, there are are kind of two SubT trends in that. One is digital sovereignty, which was a big theme for, for the event in Europe. Um, you know, there, there's, uh, there's a desire to really know where your data is, who has access to it, what laws apply to it, and, uh, and make sure that you, as you're building systems, they're resilient to changes in the geopolitical environment.

So digital sovereignty is leading a lot of folks to, um, you know, to kind of think about where the servers live and, and who's running them. And, and that's led to a lot of investment in, in, uh, in Europe that has mostly been built out on OpenStack. Um, AI is another piece of it.

And one of our other, uh, really popular projects is called Kata Containers. Um, kata containers is a secure execution environment. You can plug it into, uh, into a Kubernetes pod, and it gives you a, uh, a, a very lightweight virtualization wrapper that protects against, um, container breakouts and those kinds of things.

But it also has some other really interesting features that make it nice for ai. Mm-hmm. Uh, which is that you get to have a kernel in there.

Right. And that kernel can have customizations for, uh, for special workloads. So we, we have a number of AI companies who are using COTA containers to, um, to, to create GPU as a service businesses, some of them at quite large scale.

And, uh, and, and so, you know, the, the Open Infra foundation is, is often one level down from where we are here at KubeCon. You know, KubeCon, cloud Native Con, uh, a, a lot of the tools here expect that you have a cloud, that you have infrastructure with an API on it in the Open Infra Land. We're building infrastructure APIs on top of hardware.

So it's kind of one level lower, but still, I love data centers and I love that level. So I love being able to, to kind of span both groups. Has there ever been a more interesting time to love data centers?

Oh God. I know, but do me a favor. How do you spell Kata?

KATA. Yeah. KATA.

Just wanted to make sure people got that. Thanks. Um, you know, it, it's interesting, there are, you talk about tailwinds that are moving it there.

Let's talk about, if you don't mind, we'll spend a little time. Yeah. Look, you know, the whole VMware Broadcom licensing thing has caused, I am not here to debate whether it's worth the money, but it's caused people at least to say, Hey, this is a good inflection time.

Mm-hmm. Should we look at something else? Yeah.

Should we look at going to public cloud? Should we look at a different, uh, cloud solution, hypervisor solution in general? Right.

Should we go hybrid multi stay just in the private mm-hmm. Data center? Um, it's certainly, it's an agent of change.

Yeah. Or, or at least an agent of ref a time to reflect and, and make some choices going forward. We, We, we did a survey of our Open Infra Foundation members earlier this year, and, uh, over 80% of them had gotten inquiries about migrating from VMware.

Over 60% of them had already done a migration. Really? Yeah.

Off of VMware. Off of VMware, yeah. Over the course of this year.

And so it, I think, you know, what it, what it did is it injected enough uncertainty that, as you said, you're willing to consider a change. And from a business point of view, you know, I, I don't, uh, I don't necessarily think VMware made a bad business decision like they are. They're, they're focusing on, on profitability and yeah, He's done pretty well for stuff that guy.

Uh, but you know what, it does change the dynamic of where their customers have been historically and, and where they would be in the future. So yeah. It, you know, it, it, it made people consider, should I move to something else?

And, uh, you know, what would that be? I, I agree with you. And, and it may very well be that their decision was they're better off with 50 or 60% of their existing customer base paying three X the time.

Yeah. They make more money. And, and, and, and it's a very, uh, focused customer base.

Yeah. Be that as it may, it makes opportunities for a lot of people in different things. Yeah.

The other thing, driving it, of course, as you mentioned, ai Yep. Right? And, and what's going to, what is the AI stack of the future look like mm-hmm.

And what platform is it running and what cloud or, or what have you. Yeah. I, you know, I think there, we, the jury may still be out, but it certainly, anytime you could get people to say, Hey, wait a second, change is coming and I gotta think about what I want to do.

It's a good thing, I think for like Yeah. The infra, open infra foundation and the tools and projects in there. Yeah.

We have several, um, GPU cloud providers that are, are running OpenStack to power that. Some of them use cota, as I mentioned. Um, one of them is a top 10 buyer of Nvidia GPUs.

So it, it's, uh, it, it's great because if, if you're talking about a handful of GPUs and a couple of systems, then you know, you, you may just go with a simpler set of tools to manage that and deploy the workloads and, and, and go with a, with a simpler option. But if you're talking about putting tens of thousands or a hundred thousand GPUs in a data center with all of the associated infrastructure around that, then you really have to have something that's very focused on the compute, storage, networking management. And I think that's, that's where, um, you know, OpenStack has, has done really well this year.

I agree with you. I, I, I, you know what? Look over the years OpenStax had its ups and its downs.

I really thought, I guess when you changed to open in, was, was that 2022? It Was 2020. Yeah.

It, it was a bit of a, it was a good shot of adrenaline in the arm. Right. That reinvigorated it.

Yeah. Um, and look, we, I think it's better days, may it's best days may be still ahead of it. We, we just crossed 55 million cores of really compute and in our user survey, uh, that we just wrapped up around the summit.

So that's a Lot. It's crazy. It's a lot more, more open stack than ever before.

Absolutely. And it's a good thing. Look, choice is good out there.

Freedom is good out there. Yeah. Um, let's pivot over to, to CubeCon.

Yeah. NCF. So this is your first CubeCon as as Ed here.

Impressions, thoughts? Uh, yeah. I've been coming to CubeCon mostly since the beginning.

I've been to, to most of them. And, uh, it's always a, a really interesting event because this is where the, where the industry comes, you know, and, and, uh, it's, it's, it's a good way to sort of test the waters and see how people are feeling. And, um, you know, we we're here in the sponsor hall, we sold out our sponsorships this year.

Um, the it, and when, when you look around, you know, you'll see, you'll see a lot of backdrops with AI on them. And, uh, I that's, that's different than even six months ago. And I, and I think this is, you know, what's on everybody's mind, and, and to me, there's a, um, uh, what, what I've been trying to, to have a conversation around is, you know, which part of AI is the part that fits here.

Mm-hmm. Because AI is so big, you know, it's everything from, from really, really deep AI science and machine learning, Right. The neural net part Up to, you know, chatbots and, and agents and this kind of thing.

Um, which part in that, in that entire spectrum is the right part for our community to work on. And, um, you know, my feeling, I I, I'm obsessed with inference right now because I feel like that is a, uh, an an area of AI that's really getting overlooked. You know, we, we kind of have skipped from, from being interested in, in these deep learning and machine learning and LLM creation techniques all the way over to agents.

And, you know, agents require models, models require inference. Agents are gonna operate at a much, much higher transaction rate than humans do when we interact with models. So now we have to expand that inference by even more multiples.

And, uh, and this is, you know, going to just increase the already extreme demand for, um, for, for, for access and data centers. Yeah. We can't, that we probably can't meet in the, in the in, yeah.

You know, the timeframe they're talking about, well, This, this is where the software is really important because you, you know, you're right. Like you, you were talking about power earlier and, and data centers, and we can't build nuclear power plants more rapidly and we can't, you know, install servers more rapidly past a certain point. But software can change very quickly.

And, and if you look at, at the core pieces within, um, AI inference stacks, we've, we've seen incredible efficiency improvements this year. Six X in VLLM eight X with really between, uh, caching techniques on top of, um, Kubernetes routing primitives. And so, you know, you, you, you get a few of those, those advances and you are 50 to a hundred times more efficient just through software.

Right. And that means that, you know, that nuclear power plant, you know, you, you, it's 50 x more, more, uh, efficient there as well. Absolutely.

If we get it approved, sorry. Right? Yes.

But, but you know, what you hit on here is, is again, this is a, this is a very normal fact pattern, right? First these things come out and, you know, and it's, wow, they're great. But now you start thinking, well, how much power does he use?

How much water do I need to cool it? What, you know, all of these things and, and efficiencies become the, the, the rule of the day. Yeah.

And software is always about making it more efficient, right. That's how we've always proceeded all through my time in tech. Yeah.

Right. Software is where we pick up those efficiencies. Um, I don't, you know, you talk about inference and, and you, I don't think inference has had its day in the sun yet, is the problem.

I don't, yeah, I don't think so. I, I think we've been so focused on training, training these models, and yes, training the models is intensive energy, intensive resources, but eventually, like we don't have enough data to train much more of these models, right. We gotta have synthetic data and all those things.

But now I, especially with agen ai, I think inference is where the actions are gonna be in the next, I I'm ashamed to say, I don't even wanna say three years, 18 months. 18 months. The fight to do 18 months is plenty.

That's actually where I, that's the timeframe. I think we have to, um, you know, to capture the opportunity right now. Yeah.

Here, here's the thing. If you look at all of the largest infant systems out there, which most people don't know that these even exist, but we had OpenAI give a keynote yesterday and, uh, and they were talking about, um, their, their fluent bit usage and Yeah. Uh, how they were hitting performance limits.

They fixed, uh, you know, they made some small patches to a few lines of this. It Made a huge difference. 50% reduction.

We, We spoke about it on text and getting, I think yesterday. Yeah. And, and you know, so, so this is a, an example of where that the pattern that they have is different because of the amount of data and the amount of track transactions.

It's a little bit different than what a, a happy path Kubernetes application was two or three years ago. And so this is what we need is we need, we need these systems to, to, you know, be, um, testing the new limits that, that we're gonna find. But if you look at all of the largest inference systems, they're all running on Kubernetes right now.

I think the, the, the, the area that's a little weak that we need to focus on over the next 18 months is that most of them are doing it in slightly different ways. So we're not truly benefiting from the full power of an open source community here. No.

And, and the thing that I love about it is, when I talk to, to a lot of these companies who are, who are doing inference systems right now, they all go, what's everybody else doing? Yeah. And they say, this is not differentiating.

Like, I don't believe that my inference system is differentiating for my business. My data is differentiating the Agents that I create, but that's always what the gator is, what's important, The data, the agents, like that's gonna be differentiating. This is holding me back.

How do we get people working on it? So I think this is really the 18 month opportunity is to get some, um, you know, some, some real patterns and, and real reference architectures for this. But I, I think this fits so perfectly in the open source model, right?

Yeah. Because where we are now is a bit of a Cambrian explosion. We have all kinds of weird animals out here, six eyes, 12 legs, but eventually life finds a way, right?

And that efficiency kind of sets in. Yeah. And the best model will, will, you know, the cream rises to the CRO to the top, and the other stuff stinks to the bottom.

Yeah. I think we're gonna have that. I hope it happens in 18 months.

Sometimes these Things Have a way of stretching out because anti open source money becomes involved, and a particular company with a lot of money pushes a particular model, which, you know, may not be the, the most efficient or best. Yes. We don't use OS two today.

Right, right. And I left OS two, but in any event, I, I do think that's where we are with inference, and I think it's gonna reignite, you think there's a lot of AI now, wait, right, right. When this inference stuff is real, and we've got that working right.

Um, I, I, I, you know, bother, what is the term? Katy bar Bo doors or whatever. Yeah.

Katie bar the door. I didn't think how much Of New York growing up, but you know what I mean? Um, I I I do think it's there.

I, you know, from the CNCF point of view, right? Mm-hmm. So you've got all these projects and they're all, they're all being touched by this one way or the other.

Right? Right. How do you, how do you, um, orchestrate to bed Edward, to use?

How do you orchestrate Yeah. All of these AI needs and Yeah. You know, push and pulls going on here.

Yeah. Well, we, we announced an AI conformance for Kubernetes this week. And I think that's kind of step one is to, um, to give people like a very baseline target to, to start aiming at.

Um, it's, it's definitely not, not the end, you know, there's, there's a lot more to do, but what it does is it gives people a baseline target to, to start aiming at. And the, um, the other thing that, that, you know, I, I spoke about this in my keynote. We need to highlight it because when I do that, I walk, you know, through the conference the rest of the week and everybody's coming up to me and they're saying, okay, well I'm using Ray on Kubernetes.

Uh, you know, I'm using ser, I'm using Cube flow, I'm using, you know, but this, I run into this problem, or I, you know, I did this thing and I was able to do this Prefill caching, and it like totally changed the, our gen AI efficiency. Mm-hmm. And so, you know, I write them all down and I start to connect them and, you know, this is how, how it works in open source.

And when we get to Amsterdam, I think we'll, we'll see progress, uh, within Three, four months. Yeah. I think we'll, we'll see some progress and, um, and, and you know, that it, it can move very rapidly once you can I think once this consensus Yeah.

Once that, that's, yeah. Once you can Get people together, it can move other. And I think that's what you're really saying.

Look, hopefully by Amsterdam we're gonna have a consensus here, and then it's full speed ahead. Yeah. Right?

We get rid of the 12 eyes eight Legg that make no sense. Yeah. And, and go forward.

I, I do think that's it, you know, but you gotta remember, to me, open source is like democracy, right? It's the most least in, it's the most least inefficient form of government. Yeah.

But the best that there is. Yeah. And, and so sometimes you gotta let this play out.

Yeah. Let the, let the community and the market decide what is the best thing. Right.

You can't, yeah. We can't dictate. And the, the thing happened.

Yeah. The thing that I see as a change in AI right now is I, I say that, uh, chat g PT was a proof of concept, you know, that, that it was not, it was not actually the end goal for AI chat. GPT was a proof of concept, but we, it, we put a, a human voice on top of AI and it made us all go nuts, you know?

Yeah. No, there, there is there. It's sexy.

Like Yeah. 'cause it had that a magical that even, look, you and I, we've been in this game, you know, we, but you show it to like my wife's family. Yeah, I know.

I think it's magic. I Know the computer is is Has a brain. Yeah.

All of the sudden it became real. Right. It became real.

And so I think it's like, it's actually distracted from, from some of the, the fundamental progress or in other areas. Mm-hmm. And, uh, and we're starting to see a resurgence of small models Yes.

Of specialized intelligence. And those are the things that are gonna drive, you know, inference on the edge, inference inside of enterprises, all kinds, uh, Optimize. Just do you think see those as open source projects?

Or do you think the, the, the amount of money is just so you can't even, it's like talking about how light years in space, right? Yeah. You can't wrap your head around it.

Yeah. I mean, does that take away from the open source? Um, so Akamai just announced, uh, an inference edge platform, um, I think maybe last week.

And, uh, and you know, it's built on Kubernetes. Is it? It's in our AI conformance program.

Beautiful. So, you know, I think that, that we will see services for sure that are, are monetizing the open source As they should. But I, I think we'll see a lot of, you know, we'll see a lot of this in, in open source.

I love it. Thank you for coming on here. We're over time.

I apologize. Yeah. Hey, that, that might have been our highlight for, uh, Q Con this year in Atlanta.

I hope you've enjoyed it.