AI Agent Governance Moves to the Center of Enterprise IT

AI Leadership Insight Series. I'm your host, Mike Bizberg. Today, we're with Steve Lucas, who's the CEO of Boomi, and we're having a little chat about these phases of AI that we're going through, and well, where are we now, and where do we think we're going to be in a few short months?

Steve, welcome to the show. Thank you, Mike. It's good to see you again.

Good to see you. We have rapidly moved through a couple of AI cycles here, and it's not clear to me that the average enterprise is able to keep up. But from your assessment, where are we at the moment?

Well, whatever I say, Mike, may be invalidated a week from now, with how quickly things are moving. I think we're at a phase where-- Stating the obvious, the innovation's incredible on the one hand, and I'm not even sure that that innovation is even consumable by organizations today. We're moving so quickly.

But what I do think is that while we have these frontier labs that are creating amazing models, the open-weight models are catching up rapidly. So while we're seeing this state of change, while things are changing quickly, I think that organizations need to take a hard look at the data, the data retention policies of these large frontier labs, and whether or not they should be adopting using, or they should be looking at these open-weight models as the repository for their intellectual property and their private data. To your point about that, it seems like every one of those providers of an AI model says that they're not going to train their next model based on my data as their customer, but there's more to it than that.

Basically, as I understand it, they still see your code, and they still see your metadata, so how easily could they move into an adjacent category, as it were? Well, I think they could very easily do that. I think we're at a place where these frontier labs providing organizations, companies like yours, mine, whomever, with their best model, those days are over.

They're retaining the best models for themselves. So we know that, number one. But let's let history be the teacher here.

Once upon a time, Amazon sold books. Mm-hmm. True that.

If we take that and we apply that to these labs, look, these AI companies that provide cutting-edge models will provide healthcare services, financial services, insurance, because their models will be incredibly good at delivering an end-to-end solution for that. Gone are the days of frontier AI companies providing only frontier AI models. We are maximum a few years, if not months, away from that.

If that is the case, when I look at a lot of those frontier AI models, they seem like, at this point, a lot of overkill for a lot of enterprise workflows. So to your point about open-weight and open-source models, not just for my IP, but are people going to dynamically route prompts to different models based on cost and capability? And that needs to be baked into some sort of process somewhere.

Yeah. Well, that's absolutely going to happen. The answer is which model is right for me?

The answer is all of them. The question is not necessarily which model, it's how do I route the volume of prompts, whether it's from a user or a human or an agent, or it's code-based. Whatever I'm expressing as an organization, those prompts just need to get routed to the right model at the right time for the right cost.

I wrote a white paper on this. The title of it is "Economics Always Wins," and this will absolutely prove out. Organizations need an enterprise routing layer where they can just interface with the different consumers, the humans, the agents, the code requests, and then route that to the right prompts.

This is not a question of if, it's a question of when. We're working on an enterprise prompt routing layer, but there's other companies that do this as well. Perplexity is a great example of a company that routes prompts.

We just had another prompt routing company that just got acquired, and so here in the enterprise, so we're seeing this over and over and over again. It's not if, it's when, and the when is now. Do I need a separate platform for all that routing, or is the routing among the AI agents and applications just going to become a function of my existing infrastructure?

I think that question has been asked and answered a thousand times over in the history of software. Some organizations go full suite. Some organizations go best of breed and build their own stack.

I don't think we're going to see a one-size-fits-all approach. Mm. As you look at what IT teams are up to today and the rate at which agents are being deployed, many of which they probably don't even know exist because some line of business unit or some individual rolled them out, what are the governance issues going to look like going forward?

Because as I look at it, there could be millions, billions of AI agents, and we don't have the cognitive capability to keep track of what they're all doing asymmetrically. Well, I think you're asking potentially the question is what work will be done by what entities? Easy to track humans, a little bit more difficult to track humans and AI, very difficult to track purely thousands potentially tens or hundreds of thousands of agents working within your own enterprise on the behalf of humans or interacting just with AI.

So I was thinking about this this morning and trying to come up with a visualization for this. But the visualization, it's difficult because today it's humans and AI and agents working in minutes, sometimes hours, but it's minutes to seconds to milliseconds. That's the construct that we're used to, and that's going to change fully in the enterprise.

So what is considered outcome latent today will be outcome instant tomorrow, and it will be driven by, again, tens of thousands to hundreds of thousands of agents within your own enterprise. That is the new construct that we're building and will be built. And as that gets built, governing these things will be an extraordinary challenge.

How do I understand when an agent decided to extend an offer to someone? That would be a really simple example. But how do I understand when thousands of agents collaborated together to offer a discount to a particular customer or an entire base of customers?

How did they achieve that conclusion? Two years ago, when we started using the term agent, and we started talking about governance. We started talking about control towers to govern those agents.

Most people said, "Ah, we're not going to need that. " But here we are. It makes all the sense in the world.

I was at a big tech conference last week in Europe, and all we talked about was governing agents, governing agents, and governing more agents. It seems to make all the sense in the world today, and what will make all the sense in the world 24 months from now is doing that at sub-millisecond speed. Mm-hmm.

So given that scale, how will the AI agents not just get orchestrated, but negotiate amongst themselves and with us to perform or complete a task? Because I don't think that everything's going to be fully autonomous. There'll be degrees of autonomy, no?

There will be degrees of autonomy. It's much like self-driving cars. It'll be level one, level three, level four.

Rarely will we have level five autonomy in the next 24 months. Some of that will happen, but we'll have a whole lot of level four. You'll hop in a car, it'll drive you where you want.

You'll get a little nervous if it's taking a left turn across traffic or a right if you're in the UK, so you'll get a little nervous, but you'll get comfortable quickly. So I think we'll have Waymo-esque type activity, cyber cab type activity happening across our enterprise. But I think it will be largely level four, mainly because of trust.

And look, we said this a couple years ago, but it's reality today. The way I think about it is how Uber changed the world. Pre-Uber, we were told not to get in strangers' cars.

Mm-hmm. But it's the most common of things that we do today. We just walk up to a complete stranger and get in their car because the app told us it's okay.

We assume there's some vetting that's happened. There's a five-star rating. This is going to happen with AI.

8 star rating, and we'll be fine with it traversing our network, much like Uber or self-driving cars traverse our highways and byways today. This is going to get normalized because the governance app says it's okay. 9 star rating, and we're perfectly fine with it running promotions for us.

But more than that, today at my own company, we have AI agents that make payments to our vendors on our behalf. Do humans look through our governance platform, or what we call our control tower? Absolutely.

This is happening now. There is, of course, a lot of concern about AI agents going rogue and a lot of discussions about regulation. Do we need more regulation here, or are the existing regulation frameworks going to be sufficient once we understand how to well manage these things?

We need technology to govern the technology. So there's going to be entirely new layers of tech that will be and are being invented right now to govern this. I think regulation is a tough word.

That's a tough one, because it means very different things in Asia, in the US, in Europe. So we need companies to self-regulate and pace themselves, because I do think pacing is a big part of the conversation. We need other organizations that come up with governance platforms, like what we've built at Boomi with our agent control tower.

And then we need other organizations that create secure constructs like containerizing agents, which we must do. That's good practice. We have the technology to do that.

So if an agent behaves poorly, we can turn it off, we can turn it on, we can do whatever we want. So we're seeing entire industries get invented and reinvented right now to govern these agents. But make no mistake, humans won't interact with software.

Humans will interact with agents, and agents will interact with software or traditional software. That's how this is going to work. So there's this global abstraction layer that will be put in place, but governance, governance, governance is the critical key piece here.

This is not Sarbanes-Oxley type or Sarbox tools from the past. This is AI in real time looking at agents that are containerized doing real-time things. Here's a great example.

So Boomi, what we do, our first principle is we move data. This is what we do. We move data for 30,000 companies in real time, from AppSec, from databases, from APIs, but we move data.

That's what we do. And in the future, it's not going to be, again, seconds, minutes, hours, sometimes months. It's going to be seconds, milliseconds, and even sub-millisecond movement of data, by and for AI.

Now, we created this technology that we haven't launched yet. We call it Autopilot. And what Autopilot does is it moves all of your data from your AppSec, your databases, your APIs, and even your agents autonomously.

" Well, that's right. But humans need to govern the agentic activity of moving data throughout your network. So that's something that we're going to be launching here, coming soon to a theater near you.

But the point is, you need the governance tools to trust the agents to move your data. In that context then, do all our existing applications and software essentially become back-end headless services because the only thing that they're interacting with is the AI agents, and we in turn are either typing something or maybe speaking to those AI agents? Yes, Mike.

Yes. Everything moves to the middle. Everything becomes headless.

Everything. We've seen organizations announce those things, "Hey, we're headless. " Our perspective is welcome to the party.

Our job has been to provide a headless interface for businesses so that they can move their data freely and fluidly amongst all their AppSec and their databases and their APIs. We've been doing that for decades. This is mission critical tech when it comes to AI.

Your business has to have a single read/write interface. It has to be queryable and programmable entirely, not thinking about a single app. But how does my business have an API?

That's a heady thought, but a different way of thinking in the world of AI. Do you think that the way we work will ultimately change? Because today we have all these silos around marketing and sales and manufacturing or whatever it is, but the reality is we are trying to execute some sort of process that probably spans them all.

So is this notion of, I'm going to move between these silos one application at a time, going to give way to something that's a little more integrated and cohesive? 100%. It has to.

Think about, again, just the act of how we move data today. So we have, there's a house that you could think of as an enterprise application, and we put that house at the surface level on ground. So if I have to move data from that app, that app sits on top of a database and code, and that database and code sit on top of an OS, and that OS sits on top of a network, that sits on top of routers, that sits on...

and on and on. And if we go old school, it's like the seven-layer OSI model. That was a question on my Microsoft Certified Systems Engineer test 30 years ago.

And so that layer, we have to go through all those layers just to move a single bit of data. Agents aren't going to want to, let alone speak in English, but go through all those complex layers just to move some data. Agents will find new ways to move data.

They will demand it. We certainly hope that they will tell us what that language is, and we need to be able to securely but freely allow them to move data more efficiently. Think of it as like a hyper-optimized trading network.

When you're trading stocks or equities, you're not going through all those layers because that little bit of a millisecond advantage gives you arbitrage, right? That's going to happen with agents when it comes to enterprise data. But we're going to get into heady topics like, well, how do we demand that those agents have quantum key encryption on both ends of that?

So in the future, this is not just going to be about moving data. It's going to demand things like quantum key. It's going to demand that we allow agents to help us more efficiently move data, because whatever was human designed is going to give way to whatever is agentic or agent designed.

How will we know what the AI agents are up to if they have their own language and their own way of talking amongst themselves that we may not understand, per se? So how would we kind of layer something in and around that? I don't think it's human in the middle, but it's got to be human hanging around somewhere.

Yeah. Humans are going to have to hang around the hoop for sure. But this is going to be governance, governance, governance.

What we think of as governance today is what are these agents doing? What AppSec did they access? What actions did they take?

What APIs did they interface with? That's how we think of governance today. And then in the future, we will have things like optimization, like what we talked about, prompt routing.

That's less governance. That's optimization. But governance and optimization will work hand in hand.

Going forward into the future, though, we will need interpreter layers. We'll need to understand that an agent is speaking this language, kind of lower level, closer to ones and zeros. But how do we interpret and understand that?

There's a lot of research today that goes into looking at neural nets and understanding chain of thought when it's not expressed in English. That will transfer over to AI as well. So we can demand things like agents use A to A protocol, or they use MCP servers or MCP interfaces.

But agents will find that, in the future, incredibly inefficient. So as they do that and they evolve, and AI proposes new protocols, new lower-level ways of communicating that don't have to go through those six or seven layers of inefficiency that we humans invented so we can track it all, that's going to change, but it will evolve. What doesn't change?

Governance, governance, governance. So what is your best advice, therefore, to all those senior IT leaders out there that are kind of struggling with this in their minds about how to get started? Because in a lot of ways, when you look at this, it's kind of overwhelming.

And you and I both know when people are overwhelmed, they do nothing. Well, I think that there's a precedent that we are used to and we're acclimated to that can be applied here. If we think about just the world of data, every company has someone like a chief data officer.

They know what their data stack is. They know what their data standards are. We have a data lake.

We have some databases. This is transactional. This is warehouse.

And they define that data stack. That's a conversation that we fluidly have today, right? So when we speak that language in IT.

Similar thing happened with applications. What's our application stack? What's our best-of-breed approach?

Do we go full suite? How do we think about that? The same thing happened with APIs.

Do we have API catalogs? How do we wrangle these APIs? We invented words like that.

And what is API management? We built a stack for that. We're seeing that emerge with AI.

It's not just about what model do I use? Do I use Frontier? What we've already talked about, and Mike, you and I talked about the last time we met.

Do I use Frontier models? Do I use OpenWeight? But what is my agentic framework?

What's the governance layer underneath that? We have to have that conversation, and I think, because you shared this with me as well, so you're trying to get this message out, is you got to have the conversation. Now, two years ago, the word agents were just barely being whispered, right?

And we were kind of thinking, you mean like The Matrix? Is this weird? Is this even going to take off?

Agents are here to stay. This is what's going to happen because software vendors, including Boomi, just decided this works, so this is how we're going to do it. So you got to, as an organization, have this conversation.

Models, agents, governance, what is real-time, what's latent? And even having conversations like, well, what are the instant outcome results that I want with agents and how will I govern them, versus outcome latent, where an agent may think on a long horizon. How do I understand that?

How do I interpret model and agentic languages? And I know I'm off on a bender here, but there's precedent for this from data, AppSec, APIs, just using analogs, to now agents. And I just wonder how many organizations have stepped back and just had that fundamental conversation about the AI stack that their organization will have, or are they just trusting that the AI gods will figure this out?

All right, folks. Well, you heard it here, the AI agent conversation, early, but arguably overdue. So no time like the present.

Hey, Steve, thanks for being on the show. Thank you, Mike. AI Leadership Insight series.

You can find this episode and others on our website. We invite you to check all those out. Until then, we'll see you next time.