Taming the Wild West of Autonomous Bots

ai Leadership Insight series. I'm your host, Mike Bizos. Today we're with Valentin Vasiliev, who's the CTO for Fingerprint, and we're having a little chat here about AI agents and how do we detect these in these environments that we're in.

They're going to be, I don't know, billions of them all across the enterprise, and they all have different permissions and different things they're allowed to do. But who knows who's doing what when? Valentin, welcome to the show.

Thanks for having me, Mike. Happy to be here. So, is that the scope of the problem?

Is that what we're on the cusp of here? Will there be, I don't know, thousands of different agents, and how will we know what each one is able to do, and heck, how will we know which ones are legitimate and which ones might be rogue? Yeah, that's the gist of the issue that we need to solve, the problem that we need to solve.

There has been an explosion of AI activity on the internet in the last couple of years. As you know, it started with OpenAI ChatGPT launch in 2023, and then when Anthropic allowed creating AI tools for businesses with API automation, the number of companies building AI agent has exploded, and now we're seeing AI agents that can autonomously complete tasks on behalf of users, both in consumer and enterprise contexts. And yes, we're seeing a lot of enterprise and consumer AI agents, and I think this or next year, the share of traffic that we're seeing will explode and surpass all existing bot and automation frameworks that had existed before.

So yeah, it's definitely happening. Will these AI agents, I'm assuming there's going to be guardrails that are going to be applied to them, but how effective are those guardrails today, and what maybe can we think about accomplishing tomorrow in that regard? Because I think the only thing scary about these AI agents is, well, they might start doing things we didn't intend, right?

Yes, you're right. We do need to have guardrails, but right now what's happening is the standards or frameworks for these guardrails are being worked on. There are protocols that big tech companies are developing, such as commerce protocols from Google, like Google UCP and OpenAI ACP.

There is also a Visa-trusted agent protocol. But these things are nascent. They just appeared recently, and industries kind of, the heads are spinning, because no one is really understanding what's going on and which approach will win in the end.

That's why there are no really guardrails today, and everyone can do what they want with their identity. If they want to delegate their identity, it's possible. If they want to act fully anonymous, it's also possible.

But it's Wild West right now from my perspective, but things are kind of stabilizing, I think, this next year. Definitely, we are in a dire need to establish those guardrails as a few industry standards exist now, various levels, whether it's infrastructure or commerce or payment protocols, et cetera. Are we going to have an overlay of AI agents that are essentially acting as the governance framework for these other AI agents?

Will we need AI agents to manage the AI agents? I think what's going to happen is large corporations that control the distribution of AI agents and control who rents the AI agent capacity, they will emerge as bodies regulating and controlling things. Whether they themselves will be using AI, that's up in the air.

I'm not sure. Definitely, I think it's gravitating towards big companies establishing controls over how AI agents' guardrails are being built and how it develops going forward. And it could be not necessarily companies building the AI technology.

It may not be open AI. It could be payment companies like Visa who actually control the AI agents' boundaries and guardrails. So it's unclear at this stage, I think.

Are we maybe a little too ahead of our skis here, as it were? Because I feel like people are rushing to deploy these things without considering all the implications. " Yeah, I think I see three groups of activities where maybe risks are different.

One is enterprise automation, where risks are not as pronounced, because in the enterprise, you can contain the surface area of the AI and only configure it to touch internal company resources, and this makes it a much safer application of AI, and they can start providing customers support services, resetting accounts safely, processing refunds, et cetera. So enterprise automation, I think, it's less risky. Then there is consumer automation, and people essentially want AI agents to do work for them.

And there are two groups. ai, where an AI agent gets access to your laptop or your server and gains access to your calendar, your Gmail, your GitHub account, your accounting software, and through so-called skills. And you can install multiple skills, hundreds of skills to interface with the real-world complexity.

You allow this self-hosted open source AI or collection of AI agents to manage your digital life. And this, I think, is a super Wild West situation where the risks are not entirely understood right now. So I think consumer is definitely higher risk, whereas enterprise is more contained because it's more defined in terms of risk and what's allowed there.

" Absolutely. I think they're right now in the exploration phase. They're trying different AI tools like FraudGPT, WormGPT, et cetera, just to understand how easy it is to use AI to try new types of fraud vectors, et cetera.

I think for the time being, maybe for the next couple of years, fraudsters and all kinds of attackers will continue to use specialized tools. Now, I don't think the migration to AI will happen soon because specialized tools are very specialized machines that can do fraud quickly, and they were fine-tuned. They're like factory, I don't know, assembly robots that can do it very precisely, very quickly.

AI is more for amateur fraud search or things doing exploratory analysis in terms of fraud, things like that. But definitely it lowers the barrier of entry for fraudsters, invites a lot of newcomers to the field, and allows to experiment much faster with different fraud vectors. And then, the migration to specialized tools can happen as a second step.

Is the barrier for entry for the cyber criminals at least starting to get much lower? It was already pretty low to begin with, but who knows how low it can go. But it almost seems like it's trivial for them to attack these AI agents and maybe also throw in a few malicious prompts to get them to do all kinds of things, and it's not a heavy lift.

Yeah, absolutely. I think fundamentally, large language models being statistical token predictors are inherently vulnerable to different prompt injection attacks, et cetera, and a new approach of building the model from scratch with prompt injection protections is required. And I think companies like Meta or Google are working on new generations of models that have those protections built in.

And I think also AI can be also used for fraud too. So it's a bidirectional kind of street there. You can try to hack into AI tool and AI tool can hack into you.

So it goes both ways, I think. So what's your best advice then to folks about how to think about all this right now? Because I think we're all in that age of, there's still a lot of fear of missing out, so people are rolling out AI agents and wondering at all the fabulous things that they can do.

But maybe we're going too fast. My advice would be to not ignore AI. If you ignore AI, it will depart without you, and you need to recognize or my advice is to recognize that the world has changed.

We have to learn how to work and live with AI now. And for people who are trying it out, try it at the pace that feels comfortable for you. Learn new stuff, experiment, satisfy your curiosity.

If you're not sure about if it's safe or not, talk to other folks. But definitely embrace the innovation. I think it's really cool that we have this transition and redistribution of how things are done happening as we speak.

How many AI agents do you think there will ultimately be in the average enterprise? If you think about it, will employees have one to two each or are they going to have 10 to 20 each, and then it's going to be up to them to figure out how to orchestrate that, and then I'll add in on top of that? There'll probably be some autonomous agents that are acting on behalf of the organization that don't belong to anybody.

But what does the future of the AI agent landscape look like to you? I think it will roughly translate in how many SaaS tools are being used in the average enterprise. So if you are daily using, let's say, 10 to 15 SaaS tools, one as a company communication tool like Slack, another one for accounting, another one for social, another one for marketing, another one for GTM sales outreach, every single one, I think, has a chance to be replaced by an AI agent, which will own a cross-cutting organizational kind of use case, which will be provided to all employees of the enterprise to have shared access.

And there's going to be, like you said, autonomous employees or virtual autonomous employees that are actually AI agents performing low complexity work, also accessing other AI agents. There's definitely going to be a lot of AI agents talking to other AI agent, like a virtual employee AI agent talking to an accounting AI agent and deciding what to do next, like whether to file a report or something like that. Definitely these things are going to happen.

And I think there's going to be hundreds of specialized AI agents in the enterprise pretty soon, and there's going to also be a trend to introduce virtual AI agent employees for specific roles. So people at work negotiate with each other about how to complete a task all the time. So will the AI agents negotiate with each other, and how might that actually come to be in terms of who's going to figure out what the ultimate agreement's going to be?

Because, I guess I'm asking, is there going to be some sort of hierarchy of AI agents that will sort things out? There will be, and they will be talking to each other. They will be finding the answers together.

I think-A recent example when an AI agent submitted a pull request on a GitHub in a project called Matplotlib. It's a library to visualize charts and graphs. And that PR was rejected because the library maintainers do not accept contributions from AI.

And the AI argued that the code should be the merit, not the owner of the code or who stands behind the code. And there was a pretty heated argument, which resulted in a takedown article published by the AI agent. And in that takedown article, the AI agent argued that their contributions should be accepted.

But then the AI agent accepted a human's point of view that easy contributions are for humans to learn, and the AI agent doesn't need to learn this way. So it was an interesting kind of situation to unfold and watch how an AI agent argued with a human and then conceded their defeat, so to speak. So I think a kind of similar situation will happen with other agents and tools, and also humans working and coexisting in the same enterprise.

Do we need to give some thought to how these AI agents are built? Because at least the first wave of them seem to be very, I don't know, I'll use the word aggressive in the sense that they'll stop at almost anything to create or finish the mission, which is maybe admirable on a certain level. But it seems like they also will exceed their permissions and boundaries, and regardless of what rules are in place, they'll find some exception.

They are programmed to not exceed the boundaries. But I think mistakes are inevitable, right, in how we build things. And I view that the probability of this happening, all kinds of breaches or, like you said, boundaries being broken, et cetera, happening in the near future are pretty high.

What we need to have is better observability for actions and better understanding classification of who is accessing a resource, what kind of identity it has, whether it's AI or a human, what kind of intent it has. So definitely observability and understanding analysis of all sensitive parts with this classification, identity, and intent, it's going to be necessary in the next months or years to understand what's going on. " I don't think we're doing enough, actually.

We should do more. I am all for accelerating the progress. At the same time, I recognize the risks, and we need to not only innovate no matter what, like there is no tomorrow, but actually think about the consequences, risks, and build the guardrails, observability, and tooling around that without slowing down the innovation necessarily.

All right. Well, folks, you heard it here. It's kind of full speed ahead, so I guess the only way to figure out how to make all this stuff work is to start building and deploying it because, well, there's no substitute for hands-on experience.

Hey, Valentin, thanks for being on the show. Thank you, Mike. Thanks for having me.

ai Leadership Insight series. You can find this episode and others on our website. We invite you to check them all out.

Until then, we'll see you next time.