AI Native Platforms Define Platform Engineering with Pankaj Gupta | Platform Engineering 2.0 Ep 2

0. I'm happy to be joined in this endeavor by my good friend Pankaj Gupta from Broadcom. If you didn't catch episode one, not to worry, we'll try to bring you up to speed, but I do encourage you, go check it out.

There's some good stuff in there to learn and think on. Pankaj, welcome back here for episode two. Thank you.

For those who didn't catch episode one, can you give them a quick 30-second recap of what they missed? 0. It is an evolution.

0 is hitting the ceiling. I love it. 0, and we don't have to rehash them.

I encourage people to go watch. 0 white paper. 0, what is that platform?

And that'll be coming hopefully by the time we get done with the seven episodes here. 0 was an AI native platform. I always believe you got to define something before you can really talk about it.

How would you define an AI native platform? There are three key components of AI native platform. The number one is the AI workload should be supported native in the platform.

So that's number one. Number two is AI agents as platform citizen. This is the first time when the non-human becomes the platform citizen.

Number three is the AI-powered autonomous platform, which is more in the near future for that. So let's look at the first one about the AI workloads native to the platform. Platform should enable building the AI applications.

Not just building, that's where the current focus is, but the immediate focus is going to be also on govern and protect the AI workloads. They have unique needs like the GPU provisioning, model serving, model registries, MCP servers for that, machine learning pipelines for that, and very strong guardrails for that. And this is the field which is evolving very fast, so the platform should be able to be very agile as the new technologies and new things comes into that, or the new building blocks come for the platform for AI workloads.

Second one is the AI agents. As AI agents are the LLM powered and do a specific task in a very bounded scope, they have the unique needs for that. And we see two kinds of the agents to be supported.

The agents which IT is going to write, and second one is the agents which make platform more efficient, just like you will be able to do the triage of the incidents, or a developer asks certain thing, environment in natural language, and the agents do that for that. But the bigger need for the AI agents as platform citizen is the guardrails which is required for that. And the third one is the AI-powered autonomous platform for that.

When multiple agents, the control plane, all work together, then the platform becomes autonomous by itself. But that's very far off at this moment because having a platform which is self-healing, self-optimizing, self-scaling, self-securing is the nirvana. But I think the burning need right now is the AI workloads native on the platform, and the AI agents as platform citizen with very strong guardrails.

And this is every CIO's charter right now, or most of the CIO's charter. Agreed. " And what difference does that really make?

Recently at PlatformCon, I was up in New York, and there was a lot of talk about an agentic development platform. Yes. 0.

If I asked you specifically, just as it relates, how is that different from the IDP, the internal developer platform teams that we have today? You went through some key needs that we need for AI native. But what specifically?

I think it's a very simpler form is that current IDPs evolve to become agentic development platform. And when you look at from that lens that you have to cater not just the human developers, but also the agents who write the code, execute the code, go through the pipeline, and do a specific task for that. That's what truly it is, the agentic development platform is.

But agentic development platform will use the same APIs as the humans have been using it. But the difference is that stronger guardrails are required. Some of the agents will have a human in loop where the confidence is not there.

When the confidence build up, there will be no human in the loop for that. Another thing which is also going to happen is for agents. Agents will require very strong security for that.

And for that, you will need a lot of sandboxing. So for sandboxing, you will require the VMs or confidential computing for that, because VMs are much stronger for sandboxing. So you need a platform who supports not just the Kubernetes, also support AI, but also the VMs for that.

So having a single platform who does all those things. And this is also becoming more critical when you look at today, the resource utilizations remains very low for organizations for that. Maybe that's a topic we'll talk about in the FinOps section when we come into that.

The efficiency improvement for infrastructure is also going to be very, very critical and higher utilization. Agreed. A term I hear bandied about a lot with all of this is bounded autonomy.

Bounded autonomy, specifically around AI agents. People out here may not be familiar with that. How would you explain it to them?

The bounded autonomy means that, let's look at a little bit very simple what the agents are. Agents are powered by a lot of time with LLMs. They reason, they plan, they use tools, they execute the tools, but they have a very bounded scope for that.

Autonomy means that they can do all those things automatically without any human intervention. But the reality is that the risk for autonomous tools is that, are they secure enough? I'm sure you must have seen incidents a few weeks before where one agent deleted the customer's six months of their customer's data for that.

So that's where a lot of autonomy also has to come into that. And I think that autonomy cannot be standalone. It has to be in conjunction with the guardrails.

I think the discussion in next few months will evolve from autonomy and guardrails. They are not two principles. They are joined to the hip there.

Fair. We've got a little bit of time left in this episode, Pankaj. What you were just talking about in terms of that, I think it goes to the next question, which is around non-human identity.

And how should we think about that? And I've spoken to a lot of identity provider vendors, a lot of people in the field. With identity, we've had three waves of identity and security and identity management.

First was purely human, which as it turned out, is probably the easiest one we had. On top of that, though, all of a sudden, the age of IoT. Now you had all these machines.

Yes. Right? Including individual containers.

Every instance in a hyper had to have its own specific certificate, its own identity. Now we're in the age of agentic, where every agent needs its own identity. We're going to have 100 agents for every person.

At every step of the way, it's like an inverse pyramid. The amount of people, the amount of IoT devices, the amount of agents. It keeps getting bigger.

How should a platform team... That's the conceptualization of it, but how do you deal with it? I think we'll touch upon more on the security episode, but that's a fundamental problem we have to solve together for that.

And as you talked about the three steps, so the human identity to the IoT identity to the agentic or agents identity, that's exactly the evolution is about. The same principles like the remote RBAC and minimum privileges, they are going to be exactly the same here, but there has to be much stronger there because agents can do many bad things or things which they were not planned to do. Agreed.

Last question on this one. The world's changing so quickly, I don't even know if this is a realistic endeavor or not. And I'm not going to hold your feet to the fire 12 months from now.

But if I ask you, "Hey, Pankaj, give us a 12-month roadmap. " I think most of the platforms today have already started supporting or already supporting the AI native workloads. Many of the platforms like VCF and others already have the GPU provisioning into, they have integrated MCP server.

So vendors are moving very fast, and some platforms already support that. So this is fast-moving, but platform Vendors are supporting many functions as well as the hyperscalers today already. Yeah.

They are. But the ground is shaking a little under their feet, right? Because the models are changing.

Yeah. The weighted models, the token maxing or the token- Yeah ... consumptions, all of these things, I think.

And that's what gives us... And we're still learning what I call an air-fuel mixture. I don't know if you ever had an old lawnmower or a boat.

I did. You had to mix the oil and the gas in a certain ratio for it to run correctly. We're still playing with that ratio of humans in the loop versus agentics.

There's a lot. Yeah. It's still liquid, fluid.

And that's the reason the agility in platform is so important for that. If you and me had this discussion last year, we would not have talked about MCP servers. No.

Today- Absolutely ... you cannot have an AI conversation- Today you can't have it without it ... without MCP servers.

Agreed. Pankaj, that's going to bring us to the end of episode two. Thank you so much for joining in.

Hope we'll see you in about a week for episode three. But for now, hey, I hope you've enjoyed this. Check out episode one.

You could check out episode two. tv, our Techstrong OTT app, which runs on iOS and Android, and Apple TV and Roku and Amazon Fire, or on our YouTube channel, the Techstrong TV YouTube channel. Makes no difference.

All three of them will have this. You can check them out there. But for now, on behalf of Pankaj Gupta, Broadcom, Alan Shimel, and Techstrong, thanks for watching.