Stateless MCP Moves AI Agents Toward Enterprise Scale

ai Leadership Insight Series. I'm your host, Mike Bizzerra. Today we're talking with Clare Liguori, who's a senior principal software engineer for Amazon Web Services, and we're having a little chat about the latest version of mCP, otherwise known as the Model Context Protocol, which is now stateless.

Claire, welcome to the show. Thanks. Great to be here.

So what does this mean exactly for the end customer? It's great that we have stateless, but what is that benefit and what should they be paying attention to here? This is one of the largest and most significant change to the mCP protocol since it was launched a couple years ago.

And since then, we've seen a huge explosion of mCP servers be out in the wild. But I would say the vast majority of them are still local servers that you have to run yourself. What we heard very clearly from folks trying to build a remote mCP endpoint, trying to host a server on an HTTP endpoint for their customers and their users, is that it was really hard to scale, really hard to build.

And part of that was because it was the stateful protocol. A lot of the features of mCP required that the client and the server have a persistent stream connected together, and that ends up being really hard to scale, and we heard that from many of the hyperscalers as well as other folks who were trying to build their own mCP servers. And so with a stateless, what that means is that you can much more easily scale your mCP server and host it remotely for your users, and that makes it a lot easier for anybody to connect to it, whether it be from their coding agent like Hero or from their custom-built agents.

Right. Have you seen any changes in terms of who it is that's constructing or building out these mCP servers? And I ask the question because initially, it seemed to be very much driven by developers, but as I talk to other folks, it almost feels like IT organizations are now dropping mCP servers in front of everything and making that their kind of lingua franca for accessing data.

Absolutely. One of the things that we see with Bedrock Agent Core Gateway, which is our managed mCP server service, we see that a lot of enterprises are taking their existing API definitions and wrapping them in an mCP server, making them available right away to people's agents. Especially inside of Amazon, what I see is that more and more people interact with internal services via their agents.

And developers are interacting with tickets via their agents and via mCP. People are interacting with information about vacation time. All the HR policies are being pulled into and accessed by agents.

And so it really opens up this new way of working that is becoming much more common, where you're doing your work via agents as opposed to going to a website by yourself. Will mCP eventually supersede all those APIs, or what will be their relationship? Those APIs are still useful for, again, powering websites for people for human use, but also for applications.

So there's still value in a deterministic API and deterministic applications that call those APIs. But we are often seeing that-- So I don't know about supersede, but certainly it will become more common, and I hope more traffic shifts to those mCP endpoints as people are able to use them through their agents. And that's part of what this specification does, is make it a lot easier for both the clients and the servers to connect to each other.

There has been a lot of conversation about security in mCP, and a lot of folks are, well, shall we say, some are critical. How do I go about securing mCP? What is the best way to think about that?

And what should people kind of make sure they do so something bad doesn't happen? I think there are two things there. One is that this specification does bring in some stronger auth requirements for mCP servers.

0 and OpenID Connect. " and say yes. And that's really becoming the standard, and is specified in the mCP standard for that kind of enterprise-grade security for authentication to the mCP server, making sure that both the user is kind of eyes open about what they're allowing the server to do and having a strong identity for the request tied to the user.

But we're also seeing, just generally in the industry, a lot more interest in sandboxing, both agents and mCP servers. As I said, many mCP servers run locally, and so sandboxing them helps to protect your environment, your laptop, or whatever computer you're running these on, from getting access to anything that might be on that computer. One of the things that you do hear about AI agents and workloads in general is that they're spiky, and it's very difficult to predict exactly what they're going to need when, and unlike, say, some of our other application workloads that were very predictable.

So I'm assuming that the fact that mCP is stateless kind of has a role in all that, but how is the nature of these workloads different than what we have seen in the past, and what should people be getting prepared for? I will say they still do follow what we typically see for applications that are interactive in terms of it's mostly during business hours, things like that. We are starting to see much more people using their agents, especially coding agents, and sort of these always-on agents that people have now to schedule work.

And so we are starting to see a few more bursts at midnight, maybe. People have started to schedule nightly jobs that their agents go and do, and so sometimes the mCP servers see that burst. But generally, it is still very interactive workflows, and so you'll see the daytime highs and nighttime lows.

But I will say that agents often will move faster than humans, and so we do see a lot more burstiness, where you'll get many more requests in a single minute by an agent than you would a human who's reading through a website and kind of slowly building context in their head. The agents are able to build that context much more quickly. So given that, does that not kind of force the conversation around the cloud?

Because I don't think a lot of organizations can have a lot of idle IT infrastructure just sitting around waiting for some bursty workload to come along. That just seems prohibitively expensive. So how should we be thinking about where we deploy these AI agents?

I think it points to cloud, and it points to serverless, and that's really the value that the mCP spec becoming stateless enables, is serverless enables you to have no idle time compute cost. And so if you don't have a persistent long-running connection, if it's normal request response, stateless APIs, those are such a great fit for serverless compute options. And so you can dramatically reduce potentially your costs by hosting them on something serverless and similar to Bedrock Agent Core Gateway.

Mm-hmm. Gotcha. " One of the things that I've seen be interesting is the shift in some of the MCP use cases as agent skills has taken off.

And that's probably over the past year. I used to see a lot more MCP servers that were effectively documentation search. They'd have a documentation search tool and a retrieve documentation tool.

And as we've seen agent skills take over a lot of that static context, we're starting to see a lot more skills about just how to work with the existing documentation that's out there. So I would say I'm starting to see a trend towards MCP servers becoming much more dynamic in that sense because we have skills to provide that static content. Mm.

Where do you think MCP will evolve from here? What are you looking for? If they gave you the magic wand, what features would you be thinking about next?

One of the exciting things about this spec release is that we formalize the idea of extensions. So extensions are a way for the community to experiment with some new ideas, and experiment in a way that is not going to have any breaking changes in the core specification, but enables extensions to make some breaking changes, go through multiple iterations before they're really stable. One that Amazon has been contributing to and leading is Tasks.

So we're looking at ways to model long-running jobs that can be done by MCP servers. This could be any kind of long-running job or even another agent. Maybe you're calling an agent, and it's going to go off and gather some information for you or do a task and come back.

And Tasks allows us to model that kind of start job, wait for job to finish, and then get the results. The other one that we are working really closely on is Triggers and Events. We're seeing that agents are becoming increasingly event-driven, especially these always-on agents that people are running on Mac Minis and things.

And so we're working on an extension for Triggers and Events where you'll be able to get events from an MCP server, let's say telegram messages or messages off a RabbitMQ queue, and then also to receive webhooks. So things like get a webhook when there's a Slack message, or have your MCP server send a webhook to your Slack channel. So we're starting to see some really exciting use cases there and continuing to work on those into the fall of this year.

All right. To your point about that, the concept of event-driven applications has been around forever, but they were never easy to build. So will that become a lot easier for mere mortals to go build event-driven applications using those hooks and MCP extensions?

What do you think? I think it's still a challenge. I think one of the agent architectures that I'm interested right now is serverless agents.

Because as we start to have these always-on agents, today, people are literally buying Mac Minis, or they're spinning up EC2 instances there all the time, and maybe the agent is only actually running a couple of times a day. And today, that's because the agent basically sits there going in a loop and then waits for new work to come in and then runs in a loop. And it seems like we should be able to do serverless agents.

We should be able to shut down when there's no work to do. And so that's one of the ideas that we've been architecting out with Triggers and Events is how do we design it for serverless agents, which is not a very common architecture today, but I think it's going to be on the rise. As you kind of look forward, are there going to be thousands, hundreds of thousands of these AI agents crawling all over the cloud?

And will these become kind of the dominant workload, or where would they fit in our pantheon of applications? I think it will become a large amount of traffic, certainly. I think that I'm really seeing two things happening at the same time.

One is a lot more companies starting to build agents internally for their own use and to automate work. And so I'm hearing about hundreds or thousands of agents within a single enterprise. But we're also seeing that for interactive agents, we're starting to see somewhat of a consolidation.

People are starting to step away from building custom agents for themselves and start to use a lot more of maybe some of their coding agents or some of the personal agents that are already out there more as their daily driver. So I'm at least starting to see internally in Amazon, people spending so much more of their day within KIRO, for example, as kind of their daily driver that they interact with all day. And so that is certainly driving a lot more of the API traffic that I personally drive myself day-to-day.

There are other protocols floating around. There's Agent2Agent, among others. And so how will the relationship between MCP and these other protocols kind of evolve, and what should people be thinking about?

Well, I think they serve different purposes and different use cases. A2A is very focused on agent-to-agent communication. And certainly, that tends to happen within, let's say, if you're building two custom agents and you need them to hand off work to each other and share context within a multi-agent architecture.

And then MCP, I see as very useful for really any kind of agent. It's the standard protocol for how agents talk to tools and gather context, and with tasks, start jobs, and with events and triggers, receive information about real-time events. So they certainly serve different purposes, and I think there's going to be probably a lot more standards that are popping up as well.

We were just involved in the agent plugins specification that just launched last week, where you can standardize how you configure an agent. And so I think we're starting to see there's a lot of different ways to configure agents, a lot of different ways to connect agents to other things. And I think my hope, at least, is that we do start to standardize this a lot so that you can reuse a lot of this configuration regardless of what particular tool you might be using.

Ultimately, everybody's talking about AI and agents, but what are we not focused enough on right now that we should be paying more attention to? One is how to ensure that your agent is doing what you want it to do. Often when we talk to customers, they want to build agents.

They want to build, let's say, business automation agents that automate some current process. But the question that always comes back is how do I make this more deterministic? How do I ensure it doesn't go off the rails?

And so there are a lot of techniques that we are investing in. One is a technology called Dogwood that we just open-sourced last week, actually. It allows you to put policies in place to specify policies that your agents have to follow, and including something that we call a temporal policy.

" Very important that it does this, real business implications if it doesn't do this. But if you just tell it to always do that, there's always going to be some percentage of time that the model doesn't do it. And so Dogwood allows you to put in place a temporal policy, meaning A must happen before B, that it's not possible for the agent to get past if it hasn't already done the income verification before it does the mortgage approval.

So we open-sourced Dogwood, and then we also incorporated it into AgentCore policy. And so you can directly add these very deterministic guardrails into the behavior of your agent to ensure that it's actually following those rules that can have real business implications, real legal ramifications sometimes for some of this work that's trying to be automated. All right.

Well, folks, you heard it here. MCP is becoming a lot more mature. There's no question about it.

But as with anything that has great powers, it comes with great responsibilities. Hey, Claire, thanks for being on the show. Thanks.

All right. ai Leadership Insight series. You can find this episode and others on our website.

We invite you to check all those out. Until then, we'll see you next time.