The Sticker Shock of AI Infrastructure Costs

Hey guys, thanks for the intro. We're here with Alexander Thompson, newly appointed CEO for CloudBolt, and we're going to have a little chat about cloud optimization, Kubernetes rightsizing, and all the things that go with that. Alexander, welcome to the show.

Thanks, Mike. Great to be here. I guess, first question anybody asks anytime there's a new CEO is, is the fundamental mission of the company changing?

For you, what's the priorities? That's a great question. That is always, I think, the focus when there's a change.

Look, I think the reality is that CloudBolt has built a great business over a number of years and evolved through many iterations. And obviously, last year, they made the acquisition of StormForge, which brought that Kubernetes rightsizing into the business. But the long-term vision is similar, but making that next step on that journey.

So a lot of the questions that we hear from our partners and customers is, "You've helped me govern and manage and automate my resources and workloads," whether that was in private cloud, then public cloud, and then that next generation is those agentic workloads, those agentic resources, MCP agents, which obviously run on Kubernetes. So the mission itself, I think, is the same overarching value of how you handle that governance. What's changing is how we accelerate into helping enterprises and service providers who deliver those services make that next step in this AI world.

Yeah. And to your point about that, how automated is all this going to get soon? Because I think what people want to be able to do is have some sort of policies that are automatically enforced, and somehow or other, that has to get tied back into some sort of automation system that I'm assuming is going to be AI-driven.

But how do you think all this plays out? Yeah, look, I think it's a great question. It's the questions that we're asking ourselves internally and challenging ourselves as a company for our partners.

The reality is that businesses as a whole, most of them are going to go on a journey. They're going to start off with, how do we evolve with that ecosystem? How do we evolve with those changes so that we can not be the blocker to those advancements?

I do envision a world not too distant where-- Today, I think in the infrastructure world, people are still maybe a little bit concerned about handing over the reins to make the changes fully to agents. They're very keen to take the recommendations and action upon them, and maybe even have almost like a human-in-the-loop type approach. I saw this a lot in the cyber world, which is where I was before I came back into infrastructure recently.

And in the cyber world, it's a similar notion. They want the human in the loop. And I think that's very much where infrastructure world is as well as they tie into this governance story.

But at some point, that will change. The reality is you want more automation around that and the old methodology of putting the tickets in in ServiceNow, in Jira, and so on. That will still happen, but more from an audit perspective versus actually that's the workflow of then they get prioritized, then they get action.

Because in the AI world, Kubernetes world, container world, et cetera, you can't afford to have a two-week lag on those resources being deployed. You can't afford to have that rightsizing to take X period of time. Cost plays a part in that, but also the type of workloads and that continuous uptime approach, et cetera, plays another part.

So I think that the governance is going to be interesting because the reality is while everyone dreams of that state where everything is free-form and free deployed and engineers being able to do what they want and those things. And I think that obviously accelerates businesses, but you still need to have governance and control. And the fallacy I see is, well, we've created a policy, so the policy means people aren't going to go do stuff outside of these rules.

But the reality is you're still going to have that shadow AI, those shadow pieces take place. So the reality is what you need to do is enable the users and the teams to be able to go consume in that self-governance way, but still have that visibility and control on the back end as the IT or infrastructure team. I don't know if that answers your question, Mike, but that's how we're thinking about it from our lens.

I've often felt that a big part of the problem is just simple visibility, and I think most engineers, if made aware of how much something might actually cost and what they're consuming, will make a better decision. But for whatever reason, we haven't been able to surface that insight into their workflow. So, can we find a way where as somebody is configuring something or deploying something, that they get the insights to what this thing might truly cost?

I believe we can. I think that's where the world on how you consume those data and insights has changed. Most people don't want to have to log into another tool, another dashboard, another kind of database.

They want the data where they live. And I think that's where those MCP connectors play a large part to deliver those insights as resources are being deployed, so you can almost create the just-in-time approach in how that is delivered. So there's an understanding of here is the forecasted cost or resource or whatever it may be for what you're deploying.

And I think that's where you'll see that evolve into. So you joined CloudBolt from where, and what's your background? Yeah.

Well, I've been in software and SaaS, and been around the houses in the sense of working with security and cyber infrastructure, channel MSP businesses, the likes of what eventually became N-able, part of SolarWinds back in the day, into Veeam, some companies. And then also, being actually in a digital analytics space company called Quantum Metric that does a lot of stuff around changes and that ecosystem on the mobile apps and websites. So very similar to what we're talking about, but just more on that customer-facing side.

And then most recently in the cyber space with a company called PlexTrac, which recently was acquired by another cyber company called Brinker. So, a lot of experience across that kind of ecosystem, whether it's on the infrastructure side or security or cyber side, but then also tying into service providers, MSPs, and channel partners. And the interesting thing is it doesn't matter where you see, the problem's always the same where people are kind of always in this mode of, I have this problem that I'm trying to solve.

I'm trying to make the right decisions with the datasets I have, but it's the outcomes that they're focused on. And often what happens is how do you build trust across teams and across those business units? Because at the end of the day, if the trust on the data, you talked about the data insight of what it's going to cost to create that resource, deliver it, et cetera, run it, that trust on the data has to be there so that you can make the right decisions.

When that trust is broken, that's when there is an issue between those two teams. Do you think, and circling back to your earlier point, but as we deploy AI agents, the amount of IT infrastructure that it's about to be consumed is several orders of magnitude greater. So I guess my question to you is, do people really understand what it is we're looking at here and what the cost implications are?

I think people believe they do, and then they realize it when they see the bill as they start deploying it. I think that's the reality. I know in my prior roles in life as we deployed agentic workflows across customer success teams and support networks and so on, it is a bit of a sticker shock when you get that first bill.

When I think a lot of time, there's that kind of, well, we're going to have that consumption model and token base, then it's going to drive. But the reality is it has to be paid from somewhere. There's no magic money tree in my experience in most companies.

I wish there was, and that would make life a lot easier, I'm sure, on certain things. But I think people- Don't fully appreciate the fact of the, at the end of the day, it's the energy that's the cost as well. It's not just the resources itself that's kind of driving that cost that you have when you spin up these resources.

We saw this ourselves. Obviously, we have development units and engineering teams doing stuff on GPUs and CPUs and so on, as we develop new functionality. And we've seen our costs increase exponentially, because obviously those types of workloads and resources require different infrastructure that's more expensive.

So there's definitely, I think, a bit of a dirty secret right now that maybe is not fully rearing its head. Because the adoption in a lot of the enterprises, while there, has not been full scale. And then obviously, these frontier model companies, they're also in a state where they want market share and volume.

So there's a lot of incentives around tokens and thresholds and usage credits. And as those things change and evolve and these companies end up going public and different goals change when it comes to their margins, their profitability, and what they're trying to do, those costs will only go up. Is the reality.

Whether that's increased usage or obviously the existing usage kind of improving gross profit margin for those companies. So ultimately, what's your best advice to IT leaders out there as you look at all this and these challenges that they face, and is there something you wish more of them were thinking about? Yeah.

I think my advice would be, 20, 30 years ago, there was an ecosystem where, maybe even longer, where we're going to lock everything down. We're not going to allow people to bring their own devices, do things like this, et cetera, from that side. And I think, as an industry, we've evolved past that, where the reality is you can't just block things and sort of rule with an iron fist.

You need to almost be part of that solution on the other side. So, when I speak to IT leaders, the ones who are at the cutting edge, what they're doing is they're embracing it. They're not putting up walls that stop this advancement and adoption, but creating guardrails within their ecosystem to deliver those outcomes.

And what I wish we did more of, I think, is be a bit more looking around the corner around what's going to happen to those cost pieces. And that's why I think you get into the ecosystem of if we build the right efficiency plan up front, then you can avoid the pitfall of that kind of sticker shock at the end, where you've had all these resources spin up and then that kind of big bill that potentially could hit on the other side. So I think it's sort of looking around the corner to see what's coming next is on the ground.

All right. So this time next year, what are people going to be thinking about Cloudable? Look, I think the reality is we have a great partner and ecosystem.

For me, I hope they still see us as that trusted partner in their enterprise workloads and resources to help kind of govern and manage and automate those goals. But that we're helping them on the journey towards how they govern those AI use cases and MCP connectors. So delivering governance in the AI age.

It's a difficult journey. It's a change. Change is harder than people realize, in my experience.

And we believe that we can be that foundation to give people that visibility, reassurance, and trust that they're not going to have this exponential rise on their cost, but also they're going to be able to still show the governance and controls for security and visibility into those new resources. All right, folks. You heard it here.

Managing cost is going to be a little more challenging in the AI era. There's just no two ways about it. So buckle up, because we're about to have some new adventures.

Hey, Alexander, thanks for being on the show. Thanks again, Mike. All right.

Back to you guys in the studio.