AI in DevOps: John Willis & Oliver Eikenberry on Innovation, Platforms, and Future Tech

Hey everyone. We're back here live at Q Con. It's Tuesday afternoon, getting a little punchy.

We've been doing a lot of interviews today. They got popcorn and cupcakes out here. It was hard getting me back in this seat, I'll be honest with you.

But, um, nevertheless, if there was one person at this event who could get me back in the seat, it's my brother from another mother here. John Willis. John, welcome.

And you owe me a dollar, right? And I owe you a dollar. I owe you a dollar.

I, I tried to throw the bag in the garbage. Can I shot a, like I was from Indiana or from Miss. Anyway, John.

Hey, pleasure to have on. Sounds great to be here. It's been a little while since I've been on the, it has been.

It has been. You've been busy and running and running around. Let me introduce you to the man in the middle though, playing the five spot.

Just checking how much you know about basketball. Yeah, I know. Alright, so I, Oliver Eikenberry.

Yeah. So Glad to be here. Glad to be able to come and talk to you guys.

Um, Oliver Eikenberry, I'm a principal architect with Rio. Um, we're a small boutique, uh, consulting firm that specializes in DevOps transformations, um, at largest large scale enterprise organizations like We. And we really focus in on, uh, building with not for.

So we come into an organization, we, we do value stream mapping sessions. We under really get to understand the client and where their problems lie and what they're doing. And so, and part of what I do and what I drive the most is I'm, I lead our platform engineering, uh, initiative.

So I'm really focused on how to build great platforms that support delivery for these organizations and really leaving them in a better spot so they can continue forward faster and on their journey to provide value for their customers. I love it. Let's do a little digging here though, Oliver.

Sure. That was a great, a great, uh, segment. But we got to, you know, put it into chunks that people are gonna grab.

Let's start with the name of the company. So Rio. Um, so there's this flower called the ous Flower.

And Chris Blackburn, our CEO, when he was founding the company, he saw that flower, he kinda liked that name. It grows in the desert, it grows in a lot of different places. Very beautiful flower.

io domain? io. And that's kind of where the convergence of the name came from when he formed the company.

Very. And it, it also Was TRO io or li io. io.

So Rio on a plan name and the, uh, the common name for the ous flower is Blazing Star. And we are blazing stars and really drive and shine a light on a lot of things that we do with the organizations that we work with. So, So you're telling me the company's founded on flower power?

Yeah. You might say that, but Yeah, I love it. You know, it's a lucky thing you got both domains.

'cause the DO io is supposed to be going away, right? Yeah. Thank God you you did, we've that was, we've had that for probably 10 Plus years at this point.

Yeah. So Very cool. Um, so we mentioned the domain name.

We mentioned what you do. I just wanna make sure we get our housekeeping outta the way. Yeah.

John, what do you just, you haven't been on in a while. Give people a quick update on you. Yeah, so I, you know, I finished, um, a book that I worked on for two years.

Uh, it's called Rebels A Reason. It's the history of ai. You know, I, if most of you, I, I wrote a book about Dr.

DAMing. I've, I've found this rhythm of telling stories like historical it, but taking complex subjects so that everybody can sort of understand 'em. And, uh, yeah, I finished that.

It's been out almost a year now. Um, and right now I'm, I'm starting, I'm fascinated with quantum computing and how that's gonna like invade probably not until at least five years, but like, I, I think now's the time for start preparing For few days is coming. But, but speaking of Theatro is, I've known Chris Blackburn from the earliest dev stays.

He, him and Damon Edwards work together. Yep. Okay.

So when Damon was moving away from moving more into product and moving his very successful consulting business, he partnered with Chris. And a lot of that stuff that Damon had built on like some of the best value stream mapping stuff that's ever been out there. Chris took that over and I've just watched Chris over the years build a culture, you know, and, and you know, in all transparency, I am working as a consultant helping some of their sales team right now.

But I've been a fan of Chris's what he's built. 'cause the truth is, regardless of whether I'm getting paid or not, they've been always doing DevOps the right way from the beginning. And I think what's fascinating now is as we're seeing everybody wanna do AI native and how I do that, those principles are more important than ever.

And, and that's why like sort of decided I wanna do some work with them. 'cause I think it's fascinating to use those principles that we know work we've all been sort of talking about for 15 years and, and, and stay away from the people who are just going right into AI and using none of this, the things we've learned over the last 15 years. So yeah, that's, that's sort of my story around why we are here or I'm here.

Excellent. Alright. So guys, let's dive into it though, right?

Just when we thought we were getting our arms around DevOps, right? It's real. We, we know how to turn the dials.

We know what dials to turn or we think we do. You know, we, we we we got to the top of the mountain and hopefully it's a little bit more of an easy sale. Now this AI thing comes along The mountain.

Yeah, Right? It's, it's the theory of constraints all in real life for us. And everything you thought, you know, let's throw the cards back up in the air and see where they land this time.

Oh, and let's mix in, mix in a little platform engineering, some cloud native. Hey, maybe even we'll talk about a little quantum down the road. You ain't seen nothing yet, right?

Basically, we haven't solved nothing Oliver. Well, in one way I guess it gets, it's what gets you up in the morning saying, Hey, I got a fresh mountain to climb. Right?

On the other hand, there gotta be days where you say, geez, does it ever get easier? Yeah. There, it's, it's both of those because I mean, I've been, so I've been pretty focused on platform engineering in that space for the last two to three years.

Owning and driving that, figuring out how to really drive how that connects to DevOps. Because a lot of people say DevOps is dead platform engineering is the new DevOps. And that's where the principle of and really drives this DevOps is a culture, it's a philosophy, it's not a thing.

And it always was DevOps Let's Clear Right? At its purest point. And so I get up and I, I look at what's new in the platform space and all of a sudden there's this new thing with AI over here that's gonna solve all my problems.

But I, I look at that and I go, well, that's just actually the foundations of a platform still exist. I still need a place to run all those agents, run CPS and interact with all the existing systems that exist today and manage those. So now my, I get up and I go, oh, how does AI inflect and change in this space?

And we have a whole team that's run by Robert Kelly, our VP of innovation at Rio now that is an innovation team. They are focused on what the next thing looks like in this space around ag agentic development and how to get that and transform organizations and that flow. And the struggle I have is, it is too much.

There is, so AI moves so fast at this point that you can't just go buy a tool or subscribe to any one LLM or other area and, and say, that's it. I can stay there. It moves too quickly now.

So the struggle I have in the pain is, is like you have to keep up. It's a constant like hamster wheel effect now. It is in that space.

The problem is, is you can burn out on a hamster wheel. Yes. Hey, I wanted to make a point that, you know, like the why the fundamentals matter, right?

And, and to today's, uh, keynote, uh, the open guy from open ai, great story. So this is open ai and he talks about how they're logging. He did some performance review.

So you think about all the stuff that has to happen to get us chat. GPT Yep. And GPT five and all that stuff.

They have to log, they have metrics all over the place. Of course they have that, right? And he talks about how they evolved into that.

But then he shows this, and I think the presentation lies how one line of coach is saying 50% CP utilization across the board. And basically what he found, going back to the fundamentals, the fundamentals are always gonna be here no matter what we build with ai. He, they, he found through basically some perf tools that there's a library in one of the new, um, you know, fluent bit, bit fluent or whatever.

Um, that was doing, um, an FS stat, just a basic, you know, c library FS stat to get metrics from a file to log that was happening in an iNOTiFY routine as part of the colonel. And he literally made an open source change that he could toggle that FS stat. And he saved 50% CP utilization across all open ai.

This is why. So as we're talking about AI native CPU or GPU or both, No C-U-C-P-U-P-U, but as we're talking about open, you know, Claude code and all this stuff, it's all fascinating, but the fundamentals are never going away. And that's a case in point where like 50% utilization at open II is solved by a true understanding infrastructure io at all levels.

Right? So let me talk fundamentals. Fundamentally, if it's costing you a dollar for every 20 cents I give you, you gotta fundamental problem here.

And, and I think that that's to a certain extent where we are with this ai, you know, there Yeah, there's a lot of platform engineers and a lot of DevOps folks and a lot of developers who are saying, ain't this grant, ain't this grant until they get that first bill for tokens. Yeah, yeah, yeah. Right.

Token usage and so forth. Right. I I think I, I'm not anti ai.

I use AI more than most people I bet. But I, I don't know if we have industrialized, I guess is a good word, industrialize. Its usage at scale in a, in a sustainable, and I'm not talking about energy.

I'm Right. Sustainable from a business point of view manner. And, and you know what, Oliver, you're fighting the fight on the front lines of this, right?

You're like a firefighter in California here, but is it, is it a wildfire? Is it something we can get a, hold on. I Think we're, we're seeing emerging trends and patterns with like Claude Flow and some of these things that are really enabling an individual engineer to have multiple agents go do things and help them do things faster, produce things that they can review faster.

So we're seeing that acceleration at the local engineer level. All the tooling is very focused to that. But we're starting to see emerging parts, pockets, and parts in the market now.

Like at GitHub Universe, uh, just a few weeks ago where they announced, sure, oh, I can run my agents in GitHub space and they'll manage those agent agents for me. Where in order to really unlock and go from 10 Xing an engineer to 10 Xing an entire organization to really get to that value, you're talking about from a business perspective, we have to figure out how to give the engineers the ability to use these new tools in a shared way. Because AI is context.

Context, windows, figuring out the context, parsing what you need to do to get the right results out of it because it's deterministic in a lot of cases. So as we figure out how do we scale that and say, oh, this agent is really good at producing these things in this organization and it's referencing what I've already built as a fundamental in my organization. And how do engineers, are they able to share using that, spin it up in a collective way versus having to only do it at the level of their machine and what's going on, right?

So we're seeing emerging things in that market's moving that direction, but it's still so open as to what you can do and how you can figure out how to do it. Well, it's On, it's wild west and it's very, Yeah, it's very wild and open. We we're still creating the maps, if you will, before sailing the ocean.

There's a DevOps stay is Dallas. There was a gentleman from, um, I think, um, one of the car, like EDS or whatever sort of, um, One of the third party Bargains. Yeah, yeah, that's right.

And, and he was talking about economics of token to not just sort of token costs and tracking out, but there we're starting to see the, what's happening is finops Yes. Is getting involved. Yeah.

And finops is sort of mandating like, I don't, don't explain it to me. You just need to explain these costs. And so I think like everything we do, I mean we, I mean I, you're right, it's very dangerous.

The token cost can get like scary really fast. But, but we, we source some of this in cloud. Like everybody raced out to the cloud and then we, we figured out some cloud Economics efficiencies.

Right. Efficiencies. And we sort of, and That's not, that's by the way, that's not a technology thing.

That's right. You could any innovation. First you do the innovation, then you figure out the efficiencies of it, right?

That's right. Well, you look at a big farmer when they do, uh, experiments on new drugs, first they figure out does it work? Then they figure out what, how, how much is it gonna exactly cost and how do we optimize its efficiency.

And I think that's where we are in this. Let me, let me, let me bring up another fundamental for you. And it's really important to every single person and booth here at this event.

At the end of the day, is it about using AI to make our platforms better, to make Kubernetes better, to make observability better, to make DevOps better? Or, or is it, in order to use ai, we've gotta reinvent all of those things. I just told you platforms, we need platforms that are optimized for AI inference and or learn or training.

We need, we need, Kubernetes isn't going to be the orchestrator for AI applications. We need a different, we need something, call it native AI that is going to orchestrate my AI stack. 'cause is AI gonna create a new stack that renders all, everything you see here obsolete?

Or do we make this stuff the stack for ai? What do you think Oliver? I think it's both.

I think too, the, is is there a whole new stack for ai? We don't know yet. That is an undiscovered country that I think all of these engineers that are like really playing with this and pushing the limits and boundaries of what AI can do, they will figure that out that will emerge on its own as it happens in the space.

And likely we won't see it till it happens. Um, in terms of like runtime for ai, when you talk about the, like there's a huge push into using Kubernetes to run AI and l workload. That's Right.

But are we pushing rope up the hill on that? Maybe I, there's other organizations that are, that are spinning up entire data centers just for GPUs. So they're data centers that are just rank Trillion worth GPUs to then enable people to run that.

And they're building declarative Kubernetes stacks on top of that. I honestly still go back to even the lower fundamental of that, of how compute works. Kubernetes is great for scale.

If you don't need that scale, what do you, what do you use it for? And how do we look at, so it's a kind of a non-answer, but I think we have to look at both, both perspectives and be prepared to shift one way or the other and what's right. Because really what AI is doing is it's un it's giving us the ability to build larger shared context, put more information to the system, and build bigger, better things and solve bigger problems.

And so it's, that platform will emerge, but in order to do some of the stuff in the experiments, we have to provide a space for people to even try those experiments. So that's where I, we, when we look at what people, organizations that are getting into ai, they're like, well, which tools should we buy? And we're like, give everybody access to all of them.

Don't sign a three year agreement. Yeah, yeah. With chat GPT sign a six month agreement.

Yeah. Yeah. Give give them access to the different tools because everybody's workflow is different.

And that's the unique part of this change is that you can't really put it in a box. It's too dynamic at this point. Alright.

I think that, I love that question, Alan, because, uh, we were just on a tech Field day podcast and we had a good debate about this. And you know, there's sort of the, the, there's two ways to think about it. One is we can look at the proof that OpenAI and and Anthropic are all running Kubernetes mm-hmm.

To run large, the largest infrastructure. And I'm certain Google does too, right? Um, for Gemini.

So that kind of tells you that maybe the hammer and nail is never gonna change. But the beauty now is companies can step back and do things they've never been able to think about doing. Like, and, and because the, the cost of writing your own has always been the maintenance of it.

But now that the, the like the same things that are solving that I can do things in a week that normally took teams six months to do. I, it's the same acceleration for maintenance and update. So I think there's an opportunity organizations, one other story I've heard a few times is some of these larger organizations, instead of sort of getting rid of some of their classic developers Are looking at some of these big ticket ELAs.

Like a, not saying Workday specifically, but like, could we rewrite Workday? Could we rewrite Workday for us and build it specifically so it works for our organization? And, and some of those are like, let's try, right?

So I think there is a, there is a world where somebody might look at Kubernetes and say, you know what? We've been forcing ourselves down this thing. Let's prototype an alternative.

And by the way, there have been some good alternatives that have died not because of technology, but because marketing Nomad from, from HashiCorp was, was Too Syndrome. Well, but, but even in the, in the, in the platform space, yeah. No form for Docker and, and Nomad.

Those were perfectly legit solve 80% of the problems that went away for non-technology reasons. Yeah. Well The other thing is, is that a lot of these AI tools make using Kubernetes easier.

Yeah. That's because Kubernetes is, yes, everybody can say Kubernetes is easy, but then you get an SRE stand up next to 'em, they Kubernetes is not easy. Yeah, yeah, yeah.

Um, type of scenario. So that's the other thing is, that's a good point. These cha these tools give you better shared context of how to work with that thing.

So I get to your point, John, they don't have to worry about managing Kubernetes. Yeah. They're more worried about building their app and their workload.

Yeah. And then they start to question, oh, do I need this SaaS tool or can I build it to the specification of what I need? So it kind kind of comes back to softwares eating the world.

It's gonna eat the world even more because we're gonna have more people building software with ai. And the Double down on that is that, you know, as I think about listening to all the keynotes to Kku Con today, right? Like, like I get, I get tainted, you know, like, oh, Kubernetes come to KubeCon, watch people, thousands of people march around learning how to build helm charts.

And but to your point, and the point that, that they were making the keynote is like, all the contributors are now using the accelerated ai. So maybe at the end of the day, we don't have to build Kubernetes. 'cause it gets built through an evolutionary role that, right?

Like we're getting better at delivering a better platform based on CNCF or Right. Cloud Native or Kubernetes. Ideally A platform is just removing a, a layer a context load from all the engineers so they have a place to build their apps that they're creating.

So you wanna remove that context load. So AI just continues to layer in the ability to, to bring that context out and further out for them to work with. So You know what, it's gonna be interesting times that's for sure.

Place out. Definitely. You're Right in the middle of it.

Absolutely. Every day new articles, my feed gets filled every day I have an entire brow separate profile on my browser. Now that's just AI that all those things go to and it just stacks up for me to go read.

You're Not using an AI browser that does this for you? I've tried, I've the ai I've tried using that, but part of the consumption or information for me is I want the details to a degree. Yeah.

Yeah. And so I do AI summaries, I've used AI to do deep research on large, complex projects that would've taken me teams of people months to do down to few, a few weeks and, and things like that. So that's about a balance of figuring out the right place to use the tool to build that Content.

It's funny, on the AI thing, there's a couple of newsletters I subscribed to and I was using AI completely to do summary. And one day I went back and I looked and I was like, I, I looked back on what I'd been doing and it was missing stuff all over the place. And so I've reverted now to reading the art, reading the sort of newsletter summary to his point.

Like, yes, the summarization is awesome. Like take a research paper. Mm-hmm.

For me, I'll take a research paper and I'll put it in there and say, let me know if I need to read this. Right. But, but on things I actually wanna know the details.

I'm, I find myself falling back to less summary mode. I love it. Yeah.

Guys, we're about outta time. Yeah. Oliver's a pleasure meeting you.

Yeah, absolutely. Indiana's a big basketball. I was just, there you go.

New York's okay too. It was New Yorker in me that just, you know, you say that, I'm gonna tell you look up to see how big that building is before I snatch your wallet. Um, John, always a Pleasure.

That good? Yeah. It's fun stuff.

We're live at Cucu. We're not done yet today. Stay tuned.

We got more.