Kubernetes Cost Control – Cloud Native Now Podcast EP24

Hello, and welcome to the latest edition of the Cloud Native Now podcast. I'm your host, Mike, er once again with Paul Nade, practice lead for application development for the FU group. Paul, good to see you, Mike.

Great to be here. Well, we had a little surprise. IBM turned around and bought a company called it Cost It.

Um, is there something going on with the expenses these days and Kubernetes and trying to bring this under control that a company the size of IBM would feel the need to go buy a tiny little outfit like Q costs? Yeah, I, you know, it's an, it's a, it's a great move in my opinion for IBM because, you know, when we look at the adoption of, uh, cloud modernization and cloud native, uh, adoption in general, um, container cost management is a factor, right? And without having a pure kind of understanding of, of the, uh, uh, activities that are going on with your refactoring of a, of applications utilizing containers is a, is a big part of that, right?

So we often have this conversation whether or not, you know, heritage applications need to be refactored at all, and then once they are decided to be refactored, there's usually a, uh, challenge of getting it refactored. But there's also a, an after effect that says, oh, well, now that we've refactored it, it's costing X amount of dollars to run it. So having Cobe costs as part of the mix allows to do IBM's finops, uh, uh, approach prior to the refactoring conversation.

So adding in finops into Cloudability and Omics allows for that real kind of approach on, um, you know, determining what the costs are going to be for containerization as well as your heritage applications. So I think it's a great move. I mean, I think it's a, it's a good move to kind of expand, especially when we see in our own research that 43% of organizations have 30 to 60% of their production applications in containers.

Um, you know, I think it's an important understanding of how the, what those containers are cost, uh, what's it cost to run those containers? Mm-Hmm. Yeah, you and I were both on, uh, the text drum gang this earlier this week, and, um, we were talking about the fact that developers, there's a tension in the system.

Developers want to have the most amount of resources available and well owned naturally overprovision, even though theoretically Kubernetes is supposed to scale things up and down for them dynamically, and we shouldn't have this issue, but here we are. So, um, do we need to kind of figure out a way to automate this whole finops motion as it applies to Kubernetes, especially, and its complexity. So, um, are we on the cusp of some fundamental change where maybe the, the controls for finance are embedded in the DevOps workflows for managing the cloud native environment?

Well, I mean, if, hey, look, if you look at, if you look at Cobe cost's customer base, they have 12,000 people plus that are running, you know, cobe cost, right? So those 12,000 organizations feel a need to understand what's going on with their cost spending for container container management. Um, automation of it absolutely would be, uh, the next step.

I think having a, a simple, uh, approach to say, uh, almost like a, a, a calculator that says, here's what the factors would be to do their, to do refactoring entering the formula, and it produces a result. That is, uh, that is what I'll, uh, several of these, um, kind of finops solutions are doing today. Um, but I think a full automation of, of finops and a full automation of understanding what's going on with the environment, it's still a ways away.

I would say probably, you know, just roughly looking at, uh, just some data that I, that I've pulled from, I would say roughly 10% or less of organizations that are doing modernization efforts, um, are ready for an automated finops solution. Well, cost has been a recurring theme as of late, and we just saw Docker Inc has, uh, changed its subscription model, and if you look at it on paper, that's like a 40% height, but we're talking about the difference between, uh, five bucks to nine bucks and nine bucks to 14 bucks. And some people are saying, well, that's the cost of a latte.

What's your take on it? What is the impact of this price increase? And are we getting more value for money because, well, there's a more bundle in that subscription.

Yeah, Mike, you know, I mean, this is a good, um, good kind of conversation. I, I, I met with the, the Docker team, and we went into, into kind of a lot of detail around this. You know, I think with the acquisition of Atomic Jar, uh, last year really was a, um, kind of a good, a great move, especially with test containers, right?

Bringing the, the whole testing into the CICD pipeline and putting out Docker build Cloud as well as docker scout to kind of complement all that. This price increase really is an, a simpler way to roll out the suite of products, right? It's, it, so it's incorporating all these different, um, tech stack that goes within the, the Docker Pro subscription.

Yeah. They move from that $5 to $9, which is a, a significant increase, but the, the, the bigger factor here to consider is you don't have to have, um, all these different components individually in configured. It's, it's part of the suite now, um, which is a, a nice way to kind of simplify the deployment across the Docker hub.

Um, you know, which I, I think is really attractive to developers because they can turn on and turn off services as they need them, uh, without having to worry about, you know, bringing on another part of provisioning and getting the cost to download it and, and such, this is all part of the same platform. So I think it's a good move. Um, I think that, I think that the, you know, the, the developers will look at it and look at it as a, as a easy, a faster time to value, so to speak, for their applications, because they don't have to worry about, um, you know, a, a a different change, uh, in, in, in their environment.

But let's, let's kind of really dissect it. When you look at what organizations are actually going to be hit with, it's roughly on the average about a $400 per month increase of $4,800 per year increase. Um, but in turn, you're getting all this value of frictionless access to scout, to build cloud, to test containers.

It's all part of the CICD pipeline. It's all part, you know, part of the, the platform. So if your organization is, is all in with Docker, this is a seamless way and easier way to deploy.

I'm working a pet theory that says that AI is gonna force this issue anyway. And what we're seeing is the silos between all these different tools are starting to collapse, and I'm gonna create these automated workflows, and I don't really wanna have an automated workflow that spans multiple processes being hung up because there are different skews for each of these tools and processes. So I think, um, we're kind of looking at this as a, as a much larger wholesale trend that's gonna occur across all these software development life cycles.

Yeah, I mean, I think that, that you're spot on. I mean, that's part of the reason why Docker is in instituting or pushing out, uh, for a GA announcement in November of, uh, Docker Insight dashboard, right? And that's gonna give the ability to see what's happening across the workflows and, and it's, you know, it's currently in preview right now, but it's, um, but it's definitely something to, to consider for cloud native developers.

And again, to your point, it's going to streamline the process. Dr. com that takes an issue with the whole notion of inter loops versus outer loops and says, it's kind of an antiquated way of thinking about things, but how should we be thinking about software development life cycles and the age of containers and microservices and Kubernetes?

Does that loop metaphor still work? Well, I mean, I think if you look at it from the context of the SDLC as an infinity loop, right? That that's continuous and the agile development processes of, of, of continuously improving your product in your, in your application as you're pushing it out the door.

Yeah, I think that that's still a valid way took the SDLC. Um, I, I think that if you, if you don't have some type of, uh, call it some type of loop or some type of way to kind of do this across the, the, the, the pipeline, um, where, where do you, where do you stop the, in the continuous improvement piece, you know? So I think that loop helps with that continuous improvement.

I think that that's an, a value, a valuable way to, uh, to drive results and very quickly to put your application out the door. Yeah. I'm just not sure the loop itself, if you take it apart, you could just run it straight in the linear line and it would show you the same thing.

So I'm a little sometimes scratching my head going, uh, you know, is do we just make it a loop to make it fit on a chart or, um, or is it just kinda like, look, the reality of the situation is we do different things at different times based on the task at hand. Well, I think that goes back to whether you running an agile software development methodology or you, or you're running Waterfall, right? If you're running Waterfall, then you have a continuous line and you have milestones and you meet those points and then you deliver and then you're done.

Uh, with the, with the Agile software methodology, it is meant to be continuous. So that's why it is meant to be a loop. Speaking of the software and development lifecycle, there's been some news outta Jfr that affects the cloud native community, and, um, let's go first with the big names, but jfr, uh, aligned with Nvidia that connected their Artifactory to the repositories that NVIDIA makes available for microservices that I, they call nim.

I think that stands for something, but nobody seems to remember what, but, um, there's also these pre-trained artifacts that NVIDIA makes available as containers as well. How are these cloud native workflows and AI models coming together from your perspective? Well, you know, I think jfr, OOGs align with, with Nvidia is, is powerful.

I think it makes sense in the AI world. I think that, uh, the, the registry, the Artifactory or registry that kind of goes with jfr really is part of that, uh, development, lifecycle development process. So having, having, um, JFR kind of have an alignment with Nvidia only helps build the, uh, the, or solidifies the AI application.

So if you, if you're looking at, uh, the, the running AI applications within, uh, production, you know, I think, uh, if you've listened to any one of my, my sessions pre previously, I've made the point of saying that 18% of production applications nine months ago we're running ai. We ran that study nine months later, and now we see that 54% are running AI and production applications. So I think Jfr is smart to kind of get involved with this because it's a, it's part of that, um, that process, but it's also allows for, to add the APIs into the whole process.

So you have the ability to manage, um, manage the applications using, uh, the J Jfr, uh, model registry. Jfr talks a lot about the melding of DevOps and ML ops. Is that happening from your perspective?

And what does it look like? Yeah, I mean, I think if you look at, um, ML ops and, and DevOps, I think ML ops is part of one of the, one of the deliverables or one of the parts of DevOps. I don't think it's, uh, I don't think it's separate from, I think it's, uh, no different than understanding how it can most effectively, um, in incorporate ML lops and DevOps together to meet those business KPIs.

If you, uh, if you are driving towards results and you have to have a specific, um, you know, uh, actionable insights or, or, you know, specific uptime or, uh, service level objective that your business is driving towards, um, you know, having the ability to do so through DevOps and using ML ops to kind of get you there makes a lot of sense to me. Um, it's a tool, it's an enabler to, for success. It's not a, uh, call it a means to an end, but it's also, um, it's also part of the whole, the, the workflow that, that you would put together as part of the delivery process.

Mm-Hmm. It also feels like Kubernetes is kind of become the default platform for building and deploying a lot of these AI applications. Is there at least two things becoming joined at the hip because, um, if I don't put them in containers, they're too unwilling to build and if, uh, but then they're in multiple containers and I need to orchestrate that.

Yeah. So I think I'll take what you said, kind of break it apart into two sections. One, um, microservices and containerization absolutely is the preferred meth Methodist method for creating any net new application.

I mean, just for a, just the, uh, cloud elasticity, taking advantage of cloud native applications, uh, absolutely makes a lot of sense. Kubernetes then having the, adding the orchestration layer to manage the pods of these applications in these containers, you need some way to manage it, right? So having the ability to have, uh, Kubernetes or, or do the orchestration of the containers makes a lot of sense.

Building net new applications in containers and microservices makes a lot of sense. And combining those two things together, uh, obviously for AI applications, uh, allows you to scale and grow, uh, dynamically and fluidly much faster than you would be able to do so on a monolithic application. So I think that that makes, uh, and from my perspective, it makes sense to utilize, um, you know, containerization and of course utilize Kubernetes to do the orchestration of, of those containers, um, for those AI apps.

All right. Janet Fry, of course, has been up to some other things. They have a runtime platform now, and they're making a case for, uh, pushing responsibility for application security both further left and right.

And the idea here is that at least on Kubernetes clusters where it works, that I'll be able to scan and identify security issues in the room town environment that will then feed back to the, uh, jfr tools for remediating and scanning for vulnerabilities, um, at the far left in terms of, uh, at the point, at least where developers are writing the code. Um, is this the right approach to DevSecOps? I feel like, um, so much of what we've talked about for the last few years has been ship left, and this feels a little more like ship right and left.

Yeah. You know, I think you're right. I mean, I think that when I look at where, um, organizations are, what this announcement really does is allows organizations to continuously analyze the workflows and workloads, as well as analyze the Kubernetes clusters in real time as, as cloud native applications, environments are put together and they can be done dynamically, right?

And this is something that's not just a shift left. This is definitely a, a shift left shift, right? Like you're saying.

Um, but it's also management of those, uh, of those environments, uh, holistically across the, the entire ecosystem. So I think that, uh, that makes a lot of sense. I think that if you, uh, if you look at what, uh, a lot of organizations are spending, um, they're roughly spending about, you know, five, $600 per week, uh, per application developer on security related or DevSecOps tasks, right?

This is a way to kind of overcome some of those challenges, right? And I think that that's a, uh, that's a good move, uh, to move it closer to an automated approach. Um, but then also, um, really what's it made for is to, to block those, uh, vulnerabilities that, that, um, that, that are caught by DevSecOps.

So I think it's a, it's a, it's a much better way to incorporate your CVE into your, into, into your environment And seeding of security. There's a story on cloud native now talking about how, um, distributed denial of service attacks are kind of seeking out cloud native applications. I mean, they've been after cloud applications for a long time, but, um, a lot of the cloud native applications have different characteristics, and in theory, at least, you should be able to reroute API calls if something becomes unavailable.

But, um, do we think that through and are the bad guys about to disrupt our little happy microservices universe? Well, I mean, I think that it makes, uh, it makes sense to have a, a redundant architecture. So in the event that something does happen like that, there's an, an alternate path to get through to their, to the right results there.

Um, you know, look, I mean, I, I think that there's always going to be a challenge with security, and there's always gonna be bad actors out there that are trying to break in and do the, do what they do. Um, but at the, at the end of the day, uh, the, the things that we can do to kind of put, uh, measurements in front of it to prevent it, uh, is, is probably more important in my mind than capture it after it happens. Um, so I think that if we can put those measurements in place to, to understand how these things occur, uh, and block that from occurring in the first place, we won't have to deal with the resolution at the end in the on the back end.

All. Paul, thanks for joining me from the latest edition of, uh, shall we call it, as the Cloud Native World turns. Thanks for having me.

It's been great. Alright. Thank you for all watching the latest episode of the Cloud Native Now Podcast.

And we enjoyed presenting it to you. You can find this episode on Spotify and all your favorite channels, as well as our website cloud native now. We invite you to check that out.

Until then, we'll see you next time.