Broadcom Frames Platform Engineering 2.0 for AI | The Platform Engineering Show Ep 16

Hey everyone. Welcome to "The Platform Engineering Show" podcast. I'm your co-host, Alan Shimel, and I'm joined by my usual co-host compadre, Luka Galante- Luka Galante ...

of the Platform Engineering Guide Nord community. Hey, Luka. How are you?

I'm good. Tuning in from Sicily. I'm glad to see you.

You're setting root there. I know you've been traveling a lot. Yes.

Hey, we've got a special guest with us today on today's show. He's a mutual friend of Luka and I, Pankaj Gupta. Pankaj is with Broadcom.

Pankaj, welcome. " Thank you for having me, and good to be in company of two great friends again. All right.

Absolutely. Yeah. Thanks, Pankaj.

So thank you for coming on. 0, two dot O. 0 stuff.

0 to me, it becomes cliched. 0, so I guess we're stuck with it. Unless, I don't know, Luka, you come up with great marketing.

0? Yeah. It's tough.

Honestly, it's tough. 0, and we're going to break down how platform engineering is evolving. The way I describe it is just platform engineering hitting an inflection point, and then sort of moving beyond the initial DevX and infra focus into all these new eras, and we're going to break that down.

We announced a book two weeks ago at PlatformCon, Casper and I. " So that's just about how you apply those platform engineering principles to a broader swath of things. But yeah.

It's hard to give it a precise frame that way, I feel like. So Alan and Luka, great question here, and this was in our mind for that. The reality is that platform engineering is pretty widely deployed across multiple organizations for that.

It is no longer optional. It is a fundamental building block of any modern application delivery. It has been there since 2018.

However, maturity varies across the organization. But the reality is in that last two years or three years, a lot of market forces are asking for platform engineering to evolving, and it is not a reset. It is a evolution, moving from one step to the next phase for that.

Typical marketing guys use word next generation something. That's exactly we wanted to avoid because it is not a reset, it's not a restart. 0 or what it has been there for that.

0 rather than a next generation, not hinting in any shape and form that it is rip and replace, which you know that every IT organization hates rip and replace because it's super complex. So it is evolution. It builds on the same fundamentals of platform engineering originally has been, like platform as a product, improving the IT productivity, increasing the developer productivity, and delivering more value, and those principles have been still the same and will remain the same.

So let me give you my take on this one. " Right? And I think the paper starts off with, "What is platform engineering?

" I'll tell you, depending on the day, because we have seen this progression that Pankaj, you're talking about. It was originally if you manage Kubernetes, you were platform engineering. Then if you manage Backstage or an IDP, you were platform engineering.

Well, the cheese has moved again. Now you've got to be managing an AI stack. You got to be managing stuffs that are laid out in the paper.

But Pankaj, you used the word evolution. And a lot of what I've learned and a lot of what I've done in my career, I owe to my friend Brad Feld. Brad Feld's a very well-known VC.

He bought my first company and I worked with Brad, and he financed all the venture-backed companies I've done over the years. And Brad taught me something. " Very few things are revolutionary.

Everything we do in tech virtually is, no pun intended with my friends from VMware here, but everything we do virtually is built on what came before it. It's an evolution. And so yes, platform engineering is an evolution.

But let's not kid ourselves. The main driver here is how AI is impacting everything we're doing. Right?

If not but for AI, we would be happy with platform engineering, working with IDPs, and continuing along a rather slower evolutionary path. But every once in a while, something comes in that just, it is revolutionary, right? But it allows you to build the evolutionary And I think that's the point.

Luka, what do you think? Yeah, I totally agree. And I actually was having this conversation with a friend yesterday who works in renewables, completely different field, right?

And we basically ended up converging on, again, like, hey, essentially, how he can apply platform engineering principles to his job. " And we always had this kind of rule of thumb of, well, let's say, 50 to 100 engineers usually is when you start having those first pain points that show up where, hey, standardizing and automating all of this stuff into a baseline platform, that makes sense. Which, if you have 100 engineers, maybe you have, depending on the industry, 300, 400.

We're talking mid-size enterprise and up, right? And the reality, I think is now, to your point, Alan, with AI, you might have 20 people working, but those 20 people have crazy reach now because essentially everybody becomes a citizen developer. Everybody has a bunch of agents doing crazy stuff that they don't fully understand, they don't fully monitor.

And so the need for standardization, the need for golden paths is even stronger now. And it's felt across, I think, like we said, beyond IT and in organizations of all sizes, of all nature, really. And I think that's, to your point, Pankaj, right, we need this evolution of platform engineering that can really help as a framework in this brave new world.

And I think, Alan, you brought a great point of revolution versus evolution. I think AI is the revolution in the marketplace. It touches every part of it.

But that does not mean that everything else has to go through a revolution. Right. That's where the platform has to go through evolution for that.

Evolve. That's the intersection of- Exactly ... market force may be revolutionary, but the IT has to be evolutionary for that.

I will give point. And I think that was Brad's lesson, Pankaj, right? That it's only 10% that's the revolution.

Everything, 90% of it is evolution. Yeah. Right?

Revolution acts on it, but it's evolution. And two points you brought from the IT, is from AI for that. I think whichever customer you speak or IT organization, AI is changing the role of the developer significantly.

Every IT organization is using the AI tools already to write the code for that. The role of developer is changing from writing the code to validating the code and taking to the production. Previously, organization who could do four releases in a year, they can do 40 releases in a year.

So what it means is that the bottleneck is shifting from code writing for taking code to the production for that. That's the biggest fundamental shift that AI is driving for that. Second one is the agentic future for that.

As you tracked there, there are somewhere between 30 to 33 million developers today in the globe there. I'm sure the number of agents who will be using the platform will be multiple times of that in very near future for that. But agents- 10X to 100X.

Yeah. Yeah. And they will require platform.

Yeah. But more importantly, they will require a very bounded scope. They will require the guardrails.

That's where the platform also has to support AI workloads natively, agent support, agents to be supported. This is the first time where the non-humans will be using the platform, which is the agents, and they are going to outnumber the developers for that. And they will need the very strong guardrails.

So you bring up a point there, right? 0, its roots, whatever, right? It really comes from ops, right?

The ops people have been building platforms trying to operate people for platforms. But what was interesting about platform engineering is it was ops for devs. Yeah.

Right? DevOps, all of that. But it expanded.

SREs, security, DevSecOps, and cybersecurity folk. Now we're talking about a whole different persona, AI agents. 0, is that we've got to pitch a big tent to take in for a lot of different personas here.

There are a lot of different... And they're not all human, as you mentioned, Pankaj, right? Because even I think we're going to start seeing differentiation between the agents.

This is an agent that does developing. This is an agent that does security. This is an agent that does SRE.

I think a better word to use, persona- Agree ... is a better word to use, right? Whether that persona represents a human or a machine or some AI, I don't know.

0 has to have a place under the tent for all of these personas. No, I fully agree for that. If you look at the current platform engineering, it has one primary persona which was scattered, which was the developer, and for that, they give the standardization- Mm-hmm ...

through the golden path Make their life simpler, reduce the cognitive load for giving the IDP on self-service, click ops instead of the tickets ops, operational efficiency. And security also shifted left to the developer for that. 0 or the right platform engineering.

The persona is just a phenomenal impact is going to drive into that. Now, multiple personas emerging. First is that engineering and business leaders.

They are going to look at what is my FinOps, what is my ROI metric, what is my DORA metric, so that's one. Second one is data scientists and ML engineers are going to use the platform. They will have their unique needs of GPU provisioning, looking for the guardrails, looking for the model conformance or model delivery for that.

And then you will have the security and compliance for that. Same is true for the AI agents for that. But the reality over here that each of these persona will have their unique needs.

They will require their own abstraction of that. They will require their own dashboards for that. So if you are a product manager for your platform, you have to understand all multiple persona, their unique needs, and address those needs.

However, in any organization, they may start with a few personas first because AI is so much big impact right now. Maybe they will focus more on the data scientist or engineers or the AI agents. But FinOps and security are also the huge personas who has to use and benefit from the platform.

But Pankaj, that's going to change per organization and per, as I said, what day of the week it is. Today I'm focused on FinOps. Tomorrow, security became very important because we just got hacked, right?

Or we had a breach. That is fluid- Agree ... I think, on an organization and day-by-day basis.

But I want to move from personas a second, keep the conversation moving, and talk about something that I call who moved the cheese. You mentioned it with developers. Developers went from focusing on writing code to becoming software engineers focusing on managing the code and how to deploy it and how to get it out.

We're seeing the same thing in security. Most of my friends in security spent the last 20 years finding vulnerabilities. Now with Mythos, I could find more vulnerabilities than literally I could shake a stick at.

The real problem becomes, what do I do about all these vulnerabilities, right? How does my platform help me as I move from a limited amount of vulnerabilities to 10X, 100X those vulnerabilities? Microsoft came out this week and said you could expect a lot of big patch Tuesdays for the next couple months.

How's my platform handling that? Luka, you're at the intersection of this. What are you seeing?

Yeah, 100%. And this again is, I think, bringing back to what you were saying, Pankaj, it's like where we need this evolution from where we're just building on the stuff that we already know works, right? We have been talking on this podcast for years now about golden paths, right?

And those golden paths, to your point, Pankaj, are defined, are co-designed, right? They're co-designed by the platform product manager, the people approaching internal platforms as a product, and their respective users. And like we were saying, we can go from one user to the next and obviously we should follow minimum viable platform framework of starting small and then moving from there.

And oftentimes you don't need to have the craziest security sort of requirements built in from the get-go. But sure enough, you need them at some point, and I think, to your point, Alan, you need to have them, nowadays with all this AI stuff, you need to have them sooner rather than later. But the important thing I think is designing with security by design in mind, right?

So really making sure that your end architecture is super solid. But also understanding that your path to get there will have to iterate, and I think if you try to have your smallest viable platform really mega secure, well, you're really not going to get started, essentially, because you're going to get stuck in an endless back and forth with the security team. And again, that's where the platform product manager really comes into play and make sure that you strike a balance, find a trade-off.

And to what you were saying, Alan, that changes organization to organization, right? There is no recipe that's a solution for everybody. But I think we can all agree on one thing, which is, like I was in a roundtable conversation at PlatformCon a couple weeks ago, and there were a couple of people real insisting of on the topic of, well, look, if platform engineering is not about security, what it is about, right?

And I think that's a little bit too extreme, right? There's a lot of other things that you touched on. I got to ask you, was that a security person who said that?

Yeah, exactly. Because that's something we deal a lot, right? Because we think of the whole world through that lens.

Yeah. It's like- All of it- ... if it's not about security- ...

about security It's always about security But of course. But of course, how do you say no to that? Because of course, like you were saying, Alan, right, if you get breached tomorrow, then who cares about dev experience, right?

If your business is going down, then like. So of course, that's kind of table stakes. But- But we can't stop there.

And Pankaj, you were mentioning FinOps. There's a lot of interesting related aspects. But of course, security is super important, and in the age of AI gen, infinite AI gen, and infinite potential AI slob, and infinite potential vulnerabilities, you really need to get your game down.

But I think that also means having control. I think the interesting thing that we are seeing now in the market, and I would love to hear what you think, Pankaj. " And these models are incredible.

But if you are a VPC level something and you think you're done because you've done your job, because you got a enterprise license for Anthropic for your team, you're far from that. If you don't build the harness around these things, you can't give up control to these things, to your entire stack. You need to build harnesses, you need to build governance into these platforms, you need to build security by design.

You need to build so much stuff around it, so that potentially you can self-host, potentially you can do your own inference, all these things. We know that enterprises like control, and I think the shift that everyone is really quickly understanding now, also with all this open source stuff catching up really fast to different tier models. Well, maybe I don't have to token max on this frontier model all the time.

I can actually run 95% of my stuff on open weights things that cost me 100, and then have the expensive stuff like orchestrating, blah, blah, blah. That, however, requires platform engineering. It requires you being able to route this stuff internally.

It requires you to build security around these things. It requires you to make sure you don't have some weird Chinese backdoor in your organization. It requires a well-designed platform around it.

I will split Alan's question in two parts. One is the gap which current existing platform engineering has. Platform engineering did a phenomenal job for shift left of shifting security towards more developer.

But still most of the vulnerabilities are found post-production because a lot of SAST, DAST tools cannot find the runtime problems. Plus, bad actors are getting smarter and customers' configuration or post-deployment infrastructure constantly evolving for that. So that's the reason security has to shift down more into the platform for that.

Putting secure by default, things like least privilege, mutual TLS, micro-segmentation, putting the security at runtime. That's the table stakes right now, and for that, security has to be embedded into platform, and more importantly, it has to be immutable and invisible to developer. So that's the first one.

Second part is the revolutionary part, which you talked about, the AI for that. AI is widening the security gap extensively, and there are two key components. One is that if you look at the frontier models today or other similar projects, Glasswing and other, it's helping companies to identify what are the vulnerabilities in their code, but it is also helping bad actors to find the vulnerabilities of any code, even including for state actors over there.

So that's one wrinkle of the AI, how it's impacting the security, especially with the larger frontier model. Second one is that the whole AI workload's creating a brand new surface of security vulnerabilities which had never been thought about. There are many of them.

Few examples, the shadow IT sprawl, I think is happening in every organization today. Prompt injection, which SAST, DAST tools cannot figure it out. The model poisoning here.

The inference data leaks. So you see the variety of all different new attack surfaces which has to be taken care of. So that's the reason that platform has to evolve in the phases.

Maybe the simplest thing to currently do is the very strong model registry governance for a lot of data isolation required, where the additional security capabilities has to come in a native to platform or integrated into the platform through third party, like the prompt security, the inference audit. But everything has to be auditable, traceable, and developer enabled. So security is a huge surface, which in previous security was bolt on, a afterthought for that.

And as Luka said that when you talk to the security people, this is the top of the mind for everyone today for that. And agents are not helping- It is ... to reduce the security gap.

They are actually further expanding the Mm-hmm. And that's the issue. But guys, I promise that we keep this to a half hour, so I got to kind of consolidate here and wrap up.

org, right, Luka? Yep. com.

It's pretty long, and I think there's a link to the whitepaper there, as well as a survey that we're running on this. 0 is built on five pillars There's five pillars to this platform, the foundation, if you will. AI native platforms, multi-persona experience, embedded FinOps, security shifting down, as Pankaj laid out, and composable design.

0. Guys, next time I'd love to continue this conversation and say, why do enterprises underestimate those? Which ones?

And if we had a CIO on the panel with us, or maybe we'll look for one, what's the recommendation they give today? And I think that's what we've got to be thinking about. Pankaj, I'm going to let you go next, and then Luka, you get the last word, and we're out of here.

We got five minutes. Let's look at the CIO executives. Make no mistakes.

" They have to fast fail, fast experimentation, and also show the fast results. I think the first thing they have to look at is the set at 12 months AI native platform milestone. That is top of the mind for that.

And then look at the out of five pillars where they need biggest help, which is the priority for that. But every CIO or every engineering leader has a huge mandate from board. Set a 12-month AI native platform milestone.

Start from there. I think that's very wise and reasonable. Luka- Yeah, I think- Last word ...

it's building on top of what we were saying earlier, I think is controlling your destiny. I think that to me is really the key theme here. A well-designed platforms keeps you in control of your data, keeps you in control of how your employees essentially leverage this incredible revolutionary, like we said, technology.

Without creating a bunch of agents that are running around your organization, wreaking havoc, and really something that can drive productivity gains for your organizations, instead of a bunch of cool, impressive demos that then don't lead to anything, and you actually fall behind. And, in all of this, I think, there's going to be a lot of talk about-- I think six months ago, CIOs were like, "How do I token max? " I think now is really like, "How do I keep sovereignty?

" And I think platform engineering is, to what we were saying at the beginning of this conversation, is a clear proven path, evolutionary proven path to do so, that's been around enough, where I think you can trust is the right way to do it. I love it. Gentlemen, we got to wrap up.

Pankaj, thank you so much as always. Luka, it's great seeing you, and thank you as always. Good luck getting the house finished up.

Thank you. Because next you got to invite people over. The windows.

I'm waiting for the windows, and then all you guys should come over. Absolutely. We'll have a big dinner.

But until our next edition, this is "The Platform Engineering Show" with Alan Shimel and Luka Galante. Thank you for watching, everyone. We'll see you next time.

Thank you.