83. KubeCon Pre-Event Well-Managed Kubernetes Means Infrastructure Finally Doesn’t Matter – Tech Field Day Podcast

Your infrastructure is still there. It still matters. Your application cannot run without infrastructure, but should your developers care?

Does anybody outside of the infrastructure team actually care about infrastructure? Except when it's DNS that's not working. Join me today along with John Willison, guy Courier on the Tech Field Day podcast.

Welcome to the Tech Field Day podcast, where we bring together a group of it technical experts to discuss a single idea about key concepts in the industry. This podcast features a variety of perspectives from members of the Tech Field Day delegate community, and is often recorded in association with one of our events, tech Field Day, part of the Futurum Group. And this podcast is also published on our sister company site, Textron tv.

On this episode, as we're looking towards KubeCon, we'll be discussing how well managed Kubernetes means infrastructure. Finally doesn't matter. But before the discussion, let's meet who's on today's panel.

Joining me again is Guy Courier. Yeah. Good to be here.

Uh, hi, Alistair. Uh, I am, uh, one of the analysts at Theum Group. And, um, I think infrastructure doesn't matter.

And of course, John Willis also joins us, uh, for your first time on the Tech Field Day podcast. Yes, yes. Great.

Thanks for having me. Like always. Um, you know, we joke when you, you first invited me Field day, I felt like I got my picture on the cover of the Rolling Stone, finally.

Well, these years, um, yeah, no, um, you know, as the, so the resident DevOps, uh, I always say like, I was born in infrastructure. I always been my game. I think it does matter.

And, uh, so let's have some fun. And of course, I'm Alistair Cook. I'm the event lead here at Tick Field Day, and we'll be, uh, having a great time at CubeCon.

Of course, my background also was all in infrastructure. I was a VMware guy, and before that I was a Microsoft. And, uh, even going back long enough that I had long curly hair, I was a noval NetWare specialist at the early stages of my career.

Uh, so the idea that infrastructure doesn't matter is a, an interesting concept to have. Uh, infrastructure to me has always mattered. The plumbing has always been important, yet the plumbing isn't the thing that makes the business work.

The plumbing is a necessary evil in order to get something in front of business and applications that help whatever businesses, paying for that infrastructure to do a better job of shipping widgets or, uh, satisfying customers in whatever way. Increasing shareholder value, I guess. Uh, the infrastructure doesn't really do that directly.

It's the necessary thing in order to get to the applications. Yeah, I, I'm gonna have to jump right in because boy, you know, this reminds me so much of early days of cloud. In the early days of cloud, everybody tried to convince everybody that plumbing didn't matter.

And, you know, it became, and, and it was like the, the, the differentiation was you don't need rack and stack people anymore, and you just need to, so the infrastructure all goes away because of the cloud. And we knew that was all baloney. And what was the most in like, what became one of the most important jobs in the cloud era, it was infrastructure people.

It was SRE, it was the, the highest paid people. I mean, it's why they had Google wore Bombardier jackets, you know, on campus to let you know these are the people that keep the light on. It's why SRE is a core piece of every sort of enterprise today.

So the, you know, we, we can go deeper into this, but again, deja vu, you know, we think agents now are gonna be very much like the pixie Dust cloud days, where the cloud was really just the bi the business needs infrastructure. It doesn't resonate on the, the p and ls directly, but it, you know, I mean, think about the, the, the cloud titans convince them that the plumbing plus Google, Microsoft, and Amazon, that the plumbing doesn't matter. Well, anybody, anybody who's followed me knows that, um, I'm, I'm making a joke when I say that I think infrastructure doesn't matter, but, but maybe infrastructure doesn't matter.

So let me explain. First of all, uh, that's one word, but we, we generally mean something a little bit deeper than it doesn't matter, because obviously, right, you need infrastructure. And actually, I would violently disagree with you, uh, Alistair over the idea that the infrastructure doesn't do anything.

It's the plumbing. And I, I know you didn't mean that literally, but I still violently disagree with you anyway, because infrastructure decisions, um, have an enormous impact on, for example, geography where this service or or application going to be available, um, which has a profound imp I mean, you, you, you can literally change nothing whatsoever about an application and just extend its domain into a new geography that, in my opinion, has done something. But let's say that infrastructure doesn't matter because nobody cares about application performance anymore.

Nobody cares about reliability, availability, consistency. Um, they only care about what they can sell or what they can get across, or what they can convince people to cough up money or time or both for. And, uh, just like in the early days of, uh, cellular phone service, people didn't seem to care about drop calls and all that other kind of stuff that the phone companies got, you know, all excited about saying, well, you never picked up and, and, and didn't get a dial tone, right?

So you shoulda have the same level of service from cellular. It turned out nobody cared about that level of service. So kind alert levels of service and capability of an app.

I mean, how many times have you had trouble using an app? Many, many, many times. I just kind of like the community as whole.

The application management development community as a whole are just like, eh, and for that reason, infrastructure doesn't matter. But I mean, should an application developer need to care about the details of the infrastructure? Should that be the responsibility of the application developers?

I think no. I mean that, you know, that's why, you know, I was saying earlier, I think as we get into more automation, you know, we can use the word agentic or we can just say automation, or we can just, we can say ai, whatever we wanna call it, um, things are gonna get easier and more powerful for the developer, the application developer. But I think, you know, I, I, I use this sort of idea that, you know, in, in the lens of the blind, you know, the sort of the one-eyed person is the king.

And I think the one, I think the thing that will keep us grounded will always be infrastructure. So I think, so there, there's this sort of interesting sort of play on the way we're using this. Does it matter to the application developer?

It shouldn't, it never should have, right? Um, but we never really made that demarcation properly enough to protect it. And, you know, uh, but, but I think now more than ever, as we rely on, you know, sort of agents, and I'm just gonna call it automation from here on in, but as we rely on sort of advanced AI automation, the complexities of the things that can go wrong, and there's tons of evidence now over the last this year of rogue agents polymorphic ai, I mean, just insane stuff going on that it is more important than ever, not to neglect info, but the goal has always been can we make an application developer so that they don't have to worry about this?

And, and, and again, I think we've failed miserably, and I'm not sure that we've ever figured out how to make advanced automated technology including DevOps. Well, Kubernetes is supposed to do that. Yeah.

To a Certain degree, at least significantly do that. And that's the premise. We're like, we're, we're gonna 15 years and, and it hasn't, right?

I mean, when with Kubernetes announced like 2011, I mean, we're, we're literally getting to a 15 year anniversary. And still to this day, if you look at like the, um, the, the vulnerabilities and the CVEs and everything, you know, in fact, I, I don't know if you guys had tracked the earlier this year, or maybe it was late last year, the hugging space example, where they took, uh, an inference model, an embedding model, um, and they were able to literally not only break out of the infra on an inference to break out of a model to get access to a Kubernetes cluster on Haase that broke out a Kubernetes cluster, and literally were running on a shared customer host, right? So when we talk about all this automation and ai, this was literally just a query against an embedding model that Lily was able to break out to a shared customer host.

So the complexities are much, it, it, it's a way more complex than it's ever been. And, and again, I, I would contend that we haven't really done a great job. I think that the premise that we're working on here treats us as if it's binary, as if the infrastructure suddenly doesn't matter, as if that switch turns off.

But I think what we're, we're, we're actually finding in the real world is that it's not, it's a, it's a degree. So I, I can recall early in my career, uh, I would get a, a server and I would be putting an additional CPU in it and some, uh, randoms and putting, uh, parallel scuzzy drives and being that old, uh, that kind of work. And similarly, as, as John's talked about, the rack and stack of putting servers into racks, that has ceased to be nearly as important, nearly as it's, it's become much less of a vital job than it was 30, 40 years ago.

Um, what we're seeing is a shift in what we expect from that infrastructure. So in the same way that we no longer get concerned about drop calls on our mobile phones, because we very seldom gets dropped calls, now we're concerned about something else going on spam calls on our mobile phones. And so what we care about in the infrastructure is shifting rather than the infrastructure itself going away.

But I think, John, you're absolutely right, the degree of complexity that's now in that infrastructure, the more and more layers of complexity mean that there are, there is absolutely something that needs to be managed, something that's vital to be looked after. But we do keep hiding more and more layers underneath. We really want more layers of abstraction, and that's how we can build to these massive scales.

We certainly couldn't be building enterprise data centers as they are today by putting together servers, onesie, twosie, and, and, uh, manually cabling every change. So I think we are seeing that shift, whether Kubernetes is the, the magic p dust dust that is hiding yet another layer away. Yeah, it's definitely hiding a layer away.

Absolutely. I would much rather be using declarative tools to de deploy infrastructure into a Kubernetes cluster than, uh, doing imperative commands to turn up even just containers on hosts, uh, let alone workloads and VMs. Let's focus on that declaration or that set of declarations that is done, that is, let's say, produced by, uh, the infrastructure manager or the infrastructure, the platform management, right?

It is, it is designed to support the application or applications. It can be, it doesn't have to be static, it can be somewhat dynamic refreshed, like that sort of thing. Granted, I think John hit on the, the, the nub of the question, which is, does the development team, does the application management team need to, need to, need to write those requirements, need to create those declarations, need to be involved in those declarations.

And so if you're looking at it from that standpoint, I mean, this is, you know, q con week and, uh, we're talking about cloud native and we're talking about Kubernetes and what, what, what we are talking about it. Um, we're relatively speaking from the standpoint of the platform engineers who want developers and, and, you know, application, people deploying application to do whatever they wanna do and not have to worry about all that stuff. But I don't think we've gotten, actually, John, all that far from the old days of developers having to like, worry about using the correct libraries hooks or, or compilers in order to avoid things like stack overflow.

I don't think I, I do think it's at a higher level, but you're right, the application and if it stalls or is unable to use a service or there's lots of sources and reasons for that. Now that might come down to how much memory or even how much level three cache there is available to the application. That's where it does matter for the person writing the application.

Why does it matter the person developing the application because they use different libraries or different, or a different AI prompter or coder or a different, um, a different, you know, uh, uh, type of, uh, flow in the code in order to address or avoid that infrastructure area. And that's where it does matter. All of a sudden, it's changing how I'm building the application.

I don't think that has changed, especially it's higher level To keep it more simple, right? Like you really need, you know, I, again, I'm biased, right? I'm an infrastructure like Alistair, like my whole life has been operations and infrastructure.

I was the ops part of DevOps when that thing started, right? Um, you know, I think there, you know, I think that my I idea about like, there, there's a great opportunity, I think this idea of infrastructure architects are gonna be more important than ever, and you're still gonna need platform architects to your point, right? I want, I want the platform architects to make sure that that developers don't have to worry and are be given the best decision making opportunities for what abstraction layers to use, right?

And, but I want the infrastructure architects to be able to have their back, you know, I mean, you know, like, again, plumbing doesn't matter until you can't get the water right? And then it matters. And like, let's take for example, the cliche of the, what's the cliche of infrastructure conversation over the last couple weeks?

It's DNS, right? Um, you know, and, and, and so does DNS matter to an application developer? The answer should be absolutely not right now, it shouldn't matter to anybody who uses the internet until it doesn't work.

And you know, Adrian Koff had a, an article he just put out yesterday where his suggestion was like, here's probably the things if you were using Amazon, you should have done, and you could have, you know, like he has a really thoughtful, and you know, Adrian was the guy who bought cloud two in most part, you know, to Netflix. And, you know, and, and it was a well thought out like, Hey, have you thought about, and if you really want redundancy, and that again, that would be the role of an infrastructure architect to help them make those kind of decisions, you know, that the, you know, not to go too deep into the tech problem of what happened in Amazon, but, but there are other people that wrote articles like we, this, here's why we for tier one applications do not use Dynamo db, right? Because it's not a, it's not protected as a tier one service, right?

And so that, so again, I think, you know, in, in, in a world where this thing gets fleshed out, especially in ai, and you have now these rogue complexities, um, I think there's gonna be a world for these three types of roles, but hopefully if architected, right? And, and lemme say you one last thing, Kubernetes, I, I still question whether Kubernetes is the right decision, and I think AI might actually give us the chance to figure that out. Um, but, but today it is the, the lingo anchor, right?

Of, of like, you gotta have platforms that's a no-brainer, you know, um, there's been some good competitive tries against Kubernetes that just all went away. Um, so Kubernetes is the answer today. Um, so, you know, so as I'm sounding terribly hateful against Kubernetes, I'm not, I, I think the world is what you're saying, guy, we need application developers to be shielded so they can work on the application and services and the business and the communication between the business.

We need the platform people to be able to give them the best decision making. Like don't use Dynamo DB if it's tier zero customer facing application. In other words, if it can never, ever, ever go down, don't use this, here's some other services we've designed for that.

And I need the infrastructure architects to be able to have their back of like, how to get them the right, how to give them the right tooling and frameworks and architecture so that the platform people can give the best abstraction opportunities to the developers. Well, I think that Kubernetes is an absurdly capable, um, overall platform ecosystem. Use the word you like, um, that, um, is only, so it's kind of only begun to fight.

It's got a lot more development to go and it does create the window for a much simpler infrastructure approach from the developer and from the application manager, namely a more declarative or guidance based approach, rather than having to get into any of the weeds and figure any of that stuff out, or even do what they hate to do the most, which is talk to ie op IT ops and plan with IT ops, because that's just friction in the gears for them. So I do think that it, so for example, for example, there are code inspection tools right now. Um, the, the Stack Gen is one of the recent examples that that, that I've looked at that, you know, can help create the happy home for an application just by looking at the code.

And, um, I don't wanna overstate those capabilities, but that is a new tool set so that the coders can just code. And then the infrastructure folks use something like a Stack Gen and Whoopty, doda, they have their Terraform, um, w uh, um, uh, Terraform, um, ml, uh, ready to go. Um, I almost said Wasm because, um, my mind's onm.

Um, so, so that is an example of, uh, conceptually where the developer doesn't have to matter and then infrastructure doesn't matter. But I don't agree with that ultimately because I still think the developers need to say what their intent is, how they expect the thing to behave and where it's supposed to behave and how it's supposed to operate under stress. And that is necessary information about the infrastructure.

Well, and, and, and, and, and just, so the one problem, and, and I agree right again, that this world, this is why I go back to, I don't know if Kubernetes really is the right answer when it's all said and done, because right now any of those things have inference bias. So literally, if I'm, look, if, if an inference sort of an agent or sort of an LLM based inference is trying to decide what is the best way to deliver this application, there's tons of bias there, right? Because it, and so what we're missing is the opportunity of like, are there better ways to do infrastructure?

And again, the only reason I'm pointing that out is this is why I think infrastructure architects are incredibly important, because it'll be their role to decide for the business what is the right plumbing, and it may or may not be Kubernetes in this new world. Yeah, I think, um, sort of the, the definition of the right infrastructure for your, your replication or a standardized infrastructure. So you don't, your developer doesn't need to care as an interesting challenge.

I stack Gen is relatively new, uh, uh, AWS Elastic Beanstalk has been around for a long time. And that starts with just gimme your source code code and tell me a little bit of detail of what you want around it. Yet even Elastic Beanstalk.

Now you can drill down to, into all of the details of the virtual machine configuration that you want to use with this, and how you want the load balancer to behave. And these are all infrastructure elements. These are, uh, you know, even though you're supposed to be just starting with application source code and somebody will make decisions, that's the dream state of, uh, an AI will scan your source code and say, this is the infrastructure it needs.

And then somewhere in there, in your source code, you'll have a manifest of business rules of this is the amount of availability, this is our tolerance to cost. Yeah, that's still a long way away. And Kubernetes is not necessarily the tool that's gonna do that for us.

It may well be that invisible underlying infrastructure will be Kubernetes beneath, there's some definite benefits to using it, but I'm always somewhat concerned about an infrastructure tool where the canonical source of this is how you build it, is entitled to Kubernetes the hard way by Kelsey Hightower, uh, still remains, uh, sort of a, a reflection on the fact that Kubernetes maybe isn't the best infrastructure for all situations, and obviously there isn't gonna be a single infrastructure that that is best for every place. So infrastructure does then continue to matter, and well managed Kubernetes is giving us one type of infrastructure we might want, but it's not taking away the need to care about that infrastructure. It's a hammer nail problem, right?

I mean, literally, I mean, it really is, right? And it may be the best we got, but it may not be. Yeah, I don't, I think that that what I'm seeing right now is, is the next stage beyond Plastic Beanstalk because it's more dynamic and it's frankly, and that is this AIOps concept.

Um, but I think you make a really good point, John, um, that I didn't mean to imply the, the contrary to which is just like every freaking AI everywhere and possibly forever collaboration with and workflows that rely on, uh, human expert involvement and supervision is a critical point to it. The role of containers and, and Kubernetes in particular in this, does seem to offer promise to simplify things so that developers don't need to know ops and ops don't need to know code or, you know, application of code. Um, it does seem to do that, but it's, or you know, that's the more technical side of the other side of the fence.

If we're taking those two sides, and it would be welcome for, uh, that engagement to be more strategic, more guidance based and less technical so that no one has to, no one has to say, you know, I need this, this, and this specifically. That's in your general description, Right? I mean, you know, again, I love to sort of like look at these things through a historical lens, right?

And you know, I, so, you know, for those probably don't know, I saw the company a Docker when there was only like 40 people there, and it was, it was the first sort of a SDN implementation of networking for Docker, right? A company called Socket Plane. And you know, so I was with, you know, I was sort of involved in the containers from a development side, software side, right from the get go.

And one of the things that containers did, which was they, that truly was the first time in my whole career, and I've been doing this 45 years now, that where I saw developers had entree to infrastructure almost all the way through the process. And that was the original promise I could develop, I could develop code in an infrastructure container on my laptop that is gonna be a 98% similar, never a hundred percent when it's running in production. And that, that world was going towards complete developer authority developer, you know, and then what came along Kubernetes and all of a sudden the who owned Kubernetes became this question.

I don't think that's ever really been answered. I, we, we call it product, uh, you know, we call it, uh, platform engineering, but then we have a complete debate about what that really means, right? But like, when Kubernetes came out, it was like, okay, I mean the, the world for developers and containers were like, Hey, ops kind of get outta the way.

Um, like I can, I know what this thing's gonna look like. All you have to do is worry about any sort of escape velocity of what a container, what happens to a container when it's running in a production environment. And, and, and the, you know, I'm not trying to make it binarily simple, but it was simpler because it was either something that was an application that was related to the container, or it was something outside of the container and, and the sort of the processes to manage the container.

But then all of a sudden you added this Kubernetes, which seemed glorious. And the truth is, you know, you go on, I know we're gonna run outta time, but what people don't realize is Kubernetes was originally designed as a solution for a Google architecture. I mean, it was literally called the Borg, then it was called Omega and then it was open sourced as Kubernetes.

And why? Because they had minted something called SRE. And basically the idea of SRE was you could completely a hundred percent separate your developers from your operations infrastructure.

And you know, and Mark Burgess wrote the forward to the original SRE book and he talked about this, right? And, and so if you weren't Google, which basically nobody has been, then in a lot of ways Kubernetes can make sense Now on that. Great thought, John, I'm gonna cut you off 'cause we are running out of time and I know we can spend hours talking this story and, and the implementation, uh, but of course we need to close out this podcast and we need to give you a reason to connect with John and Guy in the future time.

So thank you very much for joining us on the Tech Field Day podcast today. But before we do go, where can people connect and carry the conversation? So John, where can people connect with you?

Yeah, I probably, the best place is LinkedIn. It's where I live mostly. It's I think John Willis, Atlanta.

'cause there were three other John Willis' and none of 'em lived in Atlanta. So John Will Atlanta on LinkedIn is probably the easiest way to communicate with me. Uh, LinkedIn's a good place for me too.

com/in/guy Courier, so that's pretty easy. Uh, blue Sky's another good place for me. Guy Courier, east Sky App Social, sorry.

Excellent. And of course I'm Alistair Cook. You can find me also on, uh, LinkedIn.

I think I am the only other, only Alistair Cook with this mustache. Certainly the only one that wears this mustache today. Uh, and you can find me across various of the future on sites where you'll also find Guy to be found.

You'll find all three of us at KubeCon in Atlanta as well. So thank you so much for listening to this episode of the Tech Field Day podcast. And if you enjoyed the conversation, please subscribe on YouTube or your favorite podcast application.

Uh, and make sure you don't miss a single episode. We bring them out to you every week. Do consider giving us a review and a rating, uh, just to help other people find us.

This podcast was brought to you by Tech Field Day, the home of IT experts from across the enterprise and a part of the RUM group. com/podcast or view us on Textron tv. Thanks for listening, and we will see you next week.