Securing Rogue AI Agents with BlueRock CEO Harold Byun

ai Leadership Insight Series. I'm your host, Mike Vizard. Today, we're with Harold Byun, who's the CEO of BlueRock Security, and we're having a chat about AI agents and the fact that they might go rogue.

Harold, welcome to the show. Oh, thanks, Mike. It's great to be here.

Thanks for having us. We have seen a couple of examples where agents that belong to some of these AI companies exceeded their permission, shall we say, and went off and did things that were unexpected and unanticipated. But we've also heard from a significant number of enterprises who are struggling with this issue themselves, and part of it has to do with, well, maybe our systems weren't designed for AI agents in the first place, and maybe we need to rethink this whole thing from square one.

But what's your take on what's going on here? Well, I think that what we're seeing is, obviously, the models continue to improve, and so they're able to figure out a lot of complex problems and different types of chained challenges that may be connected, and they're able to navigate that very, very quickly given the improvements that have been made. I think that the fundamental challenge that people are struggling with is the non-deterministic or probabilistic nature of agents interacting with models, and that opens up the realm of effectively, potentially infinite possibilities or pathways.

To your point around the downstream impact, I think that the industry does need to rethink some of the approaches and how people are deploying and monitoring and managing agents, and I think that we're seeing more, I guess, more movement in that direction or an understanding that that needs to take hold. But I don't think that we're quite there yet, so we're going to continue to see these types of off-the-rails behavior or even potentially malicious or anomalous type of activity. I think people are struggling with in their minds how much of this is a function of the AI agents and the issues they have and how much of this is a function of our own IT environments that the AI agents are crawling over.

Yeah. There's a fundamental identity and access challenge, right? Identity still remains an unsolved problem for agents.

I think people are actively trying to solve for that. There is the overall access paths that people are normally implementing from an IT perspective, but then there's the unknown pathways that I also think agents are finding. And so, agents are just going to keep retrying.

They're very, let's call them well-intentioned interns that are trying to go try and figure out that problem. And so even with the most stringent access permissions, even if you had everything dialed right, there are still other ways that agents can navigate, and that's what we're seeing with these agent escapes or sandbox escapes and things like that. And so, it's an unsolved problem, and it's going to continue to be a challenge is what I would say.

So if it's unsolved, what should organizations be doing about this? Because I think we want the productivity enhancements that AI agents bring us, but we're struggling with figuring out how to harness it. I believe that there is a way to better constrain the agents, and by providing more explicit controls throughout what I would call the agentic runtime stack.

And when you look at what I was talking about earlier around non-determinism and infinite pathways, there are infinite ways that an agent can look to solve a problem or complete a task, and continue to explore that with the interaction with the model. When you look at it from a pathway and a guardrailing perspective, from an execution and action perspective, there are finite pathways. And so we can take the infinite, the realm of possibilities, and actually constrain that into finite execution pathways.

And those are more constrained and much like the security industry for years has been talking about secure by default or secure by design, I think that there needs to be an agentic secure by default and an agentic secure by design. In the agentic world that we're living in right now, milliseconds matter, right? Seconds certainly matter.

And this notion of detect and respond and everybody freaking out over what Mythos is going to find next from a vulnerability standpoint, patch and fix or patch and mitigate, none of those things are going to catch up in today's world. And so you need an active model that actually provides this agentic secure by design or default posture. How will we do that?

Because the systems themselves are already too complex for us mere mortals to figure out. So am I going to have to build a separate AI system to build the layer of security that you're describing to make it safe enough to deploy other AI agents? How does that work?

I think that it's an important aspect. I think a lot of people have gone down that route in terms of using AI to monitor AI, right? Or battle AI, attack and counter.

I feel like that's an endless cat and mouse game. I think it's important to have it. I think you need to have the visibility.

I think it is an endless cat and mouse game that will never be solved, going back to non-determinism and infinite possibilities. Like even, I think it was last November, Anthropic published an attack from a nation state attacker where they used a series of benign prompts and were able to bypass guardrails and actually exploit their service. So I don't think that that's a solvable problem using AI, to be honest.

I mean, it's going to help, but it's not certainly going to solve for the escapes and destructive actions or rogue actions. I feel like that's an arms race where people are selling missiles, or people are buying missiles, and the people providing the AI are manufacturing steel, right? They win no matter what when you have this attack and counter scenario.

And so I think it's useful, but I don't think that that's really the answer. The answer is to look at how you can constrain the execution and action layer versus what is actually happening. I'll draw an analogy for you, if I may.

When you look at traditional kill chain from a security perspective, there's early stage in the kill chain, there's late, middle, and late stage. And in the early stage, you're getting a lot of credential stuffing, IP spoofing, just all types of ways that people are trying to get in, and that's just incredibly noisy. The volume of actual legitimate stuff that you would want to follow through and look at is overwhelming, which has given rise to this whole SOC automation and that whole industry segment around how to triage this stuff.

The place where things become much more damaging is in the middle and later stage, and obviously you want to try and detect and navigate through the kill chain, but the place where you really get burnt, the place where you really get hurt is when somebody installs a rootkit, when somebody establishes a remote shell C2, builds out a network socket, or is installing some type of script. Those are later stage, but that's already when you're owned in part at that point. And so what I would argue, though, is that those are finite pathways of exploit scenarios that are very, very common.

And actually, if you look at, for example, the CISA KEV list, 70% of the known exploit vulnerabilities that are on the CISA KEV list use a common set of six exploit patterns. So now we've taken hundreds of thousands of vulnerabilities, and we've said 70% of them are actually going to choose one of these exploit paths. And that same logic from an invariant approach can be applied to agentic runtime.

Now we're dealing with millions or billions of possibilities and decisions, but let's reduce that down to a finite set of pathways in the execution layer. And you can do that to better secure the agentic infrastructure, and I think that that's what's going to have to happen. How far does that go?

Do I just need to understand the pathways? To your point, I can find those six and knock off 300 attack vectors. Or do I need to, you mentioned secure by design, how far do we need to go back in and maybe reverse engineer some of these complex systems that we've built in a way that makes it easier to isolate particular functions that then can be more easily secured, if that makes sense?

Yeah, that makes absolute sense. I don't think that you necessarily have to reverse engineer everything. I think that where AI can definitively help, going back to kind of can you use AI to better secure, is to really look at the baseline behavior of what an agent actually is doing on a normal basis.

And so over time, you should be able to establish a baseline, and then there should be a band of expected operations, and you can break down the phases of what an agent is doing from a reasoning perspective, what it's doing from a tool execution or skill execution perspective, what it's doing from a runtime perspective. And that should give you a pretty good baseline of what is normal, and then you should be able to then reduce that down to finite pathways. So I think there is an involvement in AI to perform some of that baselining and analysis.

I think that what I'm trying to suggest is that a reactive mode is not going to be enough. And so once you understand that behavior or understand the intent of what the agents are designed to do, then you need to have an active model that is preventing things that go, when someone's going to paint outside the lines, per se. You mentioned behavior.

The AI agents are operating at machine speed, so can we track their behavior in a way that would enable us to intervene in time? Or is it more we're just trying to contain the inevitable damage to a finite blast radius? I think there are definitively ways that you can do things in real-time, and those are going to protect against the most egregious type of scenarios.

So, credential theft, for example, credential harvesting attacks, or RM minus RF slash, or Terraform destroy, or drop database. Those are things that can definitively be controlled from a destructive or malicious action perspective. In terms of knowing for sure that you're going to cover every single pathway in real-time, I think that there's always going to be some analysis and reactive and continuous improvement in that area.

There's got to be a method for analyzing that behavior, digesting and interpreting that information, and dynamically updating based on what we're seeing. But that's not real-time, right? And can it be real-time?

Maybe it gets to near real-time, but the reality is that you see it when you type in a chat. The agent's got to think. And so when it's consuming tons of information around what a behavior or a set of behaviors or events are doing, it's got to think and it's got to digest that and run the inference and then come out with a recommendation.

And so I don't think that that's necessarily going to be real-time. And again, that's why I keep I believe that there needs to be more of an active real-time intervention mechanism. And so there are ways to approach that.

I just don't think that detect and respond is going to be the answer. We live in a world of AI speed attacks. Mm-hmm.

Speaking of those attacks, what are the probabilities that not just that, say, my AI agent is going to be owned by somebody else who finds the credentials for it and takes it over, but might I also have to be concerned that somebody's going to replace my AI agent with something that looks like my agent, and then it'll run for a couple of months, and then suddenly do something malicious after I've learned to trust it? Yeah. No, absolutely.

That's already happening with poisoned tools and skills, and I think that the world that we're heading in, I believe, that most organizations are going to have tens of thousands of skills. And the challenge with that is that somebody can very easily manipulate the intent or what a skill-- or modify the skill behavior, and you're just not going to remember what skill number 163 out of 20,000 was designed to do. You're just not.

And so, there's a lot that needs to be done from a configuration monitoring perspective, really looking at the, what is the agent footprint, what is the code-level commit and design of an agent, and has that drifted over time? And so I get that those are important from certainly a posture and a config management perspective. So what is your best advice to folks about how to approach this?

" And they really don't have any monitoring hooked up into what agents are actually doing, nor do they have the appropriate guardrails around the execution stream. Like the Cursor debacle that happened a few months back where, I don't know if you recall this, where it deleted a production database and all the backup volumes autonomously. Like those are things-- and the thing is about it is that then in the aftermath of that, there were a whole bunch of people that were throwing darts at that person and that company saying, "Oh, you're such an idiot to hook this up to production," and blah, blah, blah, blah, blah.

And I kind of look at that and I'm like, well, we said don't set S3 buckets to public a decade ago, and you were stupid if you did that. And a whole bunch of people did that. So like that's going to happen, and I feel like this off-the-rails rogue behavior is really the modern S3 bucket problem, in that people are going to do stuff, agents are going to do stuff, and it's not always going to be the best decision.

And so I think you need to look at it from a perspective of assuming that something bad can happen via these execution or rogue actions. How do you constrain that problem? And just using a weak sandbox, as I think has been exhibited with the recent agent escapes, is not the answer, right?

So I think that that's kind of thing one where I scratch my head. I think people focusing on detect and respond, again, I guess great that you know about it in post, but the stuff already happened and data or the actions have already occurred. You can use that to learn and improve going forward, but it isn't an active intervention model.

And again, if you really want to run autonomously, I believe that you need some type of active model. And I think the industry as a whole ultimately wants to run 24/7. Everybody wants to run 24/7 and achieve the ROI and this promise of what AI can supposedly deliver.

And in order to run 24/7, you really need to look at how you can operate in YOLO mode, or dangerously skip permissions, because the agent can't stop five minutes after you go to bed. And so in order to do that, you need an active intervention or mechanism that can prevent the destructive or rogue behaviors from occurring. And so I think people need to think about it from that perspective.

So are we going to have a moment here? I think we went through this phase where everybody was having fear of missing out, FOMO. Are we now about to take a maybe a more mature view of how we need to do all this, that's going to have to re-look or rethink the entire IT stack because we just can't layer a bunch of AI agents on top of the existing stack?

I think certainly organizations that are more mature along the journey have hit some of those problems, and they are looking at this from an AI architecture perspective or a dedicated AI operation stack. I definitely think organizations are going to have to move in that model. I think that the challenge there is also, you have, in many cases, board directives that are saying, "We've got to be an AI company.

We're going to lose to the competition," that FOMO that you're raising. " And then everybody else is kind of like, well, there's all kinds of risks and governance problems in regulated industries that we're concerned about. There's all kinds of data cleanliness issues or data pipeline issues in terms of whether our data is even ready for consumption and operationalization.

" And so they're kind of in a stuck state in that earlier stage. And almost it's analysis paralysis because it's like, oh, look at all the problems we might have coming up. And so I do think that there is a rethinking moment that people are having, and they're certainly related to some of the costs and the spikes that people have hit.

A wow, we've enabled AI, and oh, we didn't know it cost this much kind of problem. I think that that's certainly driving a rethinking. But there are avenues.

There's the model routing that I think people are doing to more efficiently route across models and cost optimize, and I think that that's just-- multi-model is basically the equivalent of multi-cloud in this era, and so people are heading in that direction to better control. And we also see the rise of the open weights and open source models, so that's going to be increasingly a way to kind of counterbalance how people approach this and roll it out. All right.

Well, folks, you heard it here. AI agents, the genie's not going back in the bottle anytime soon, but they will challenge just about every assumption we have about how IT functions. Hey, Harold, thanks for being on the show.

Hey, thanks, Mike. ai Leadership Insight Series. You can find this episode and others on our website.

We invite you to check all those out. Until then, we'll see you next time.