The Blueprint for Scalable AI: Alan Shimel and Luca Galante Wrap Up PlatformCon | The Platform Engineering Show Ep 15

Hey, everyone. Welcome to another episode of "The Platform Engineering Show" with myself, Alan Shimmel, and my friend, Luka Galante, who's back home. Luka, how are you?

Hey, Alan. Hey, everybody. I'm good, man.

I'm recovering. I'm still recovering a little bit. But it was good to be back home, sleep in my own bed, don't share the bed with Casper.

Yeah. That helps. But is it really your own bed?

Your house isn't ready yet, right? Fair. Yeah, it's not ready yet.

Yeah, correct. It's not my own bed. It's a rented bed, I hope until the end of the month, and then hopefully we can actually- And then you'll finally stop being the homeless man of platform engineering But I've been saying this, probably also on the podcast for three months that it's like- Yeah, I know ...

it's just 15 days away, so we'll see. Well, not only that, Luka. For anyone who's interested, if you go back and listen to a bunch of the episodes over the last year, more- Yeah ...

year and a half even, Luka's been in a different place almost every time. Whether it's Sicily or Milan or Sri Lanka or Spain or the US or somewhere else, you've been the homeless man of platform engineering- Yeah ... for two years now.

For not too much longer, though. Good. That'll be good.

So Luka, but it was a busy week of PlatformCon. I was only really involved in the New York end day of things, but give people the big picture across the whole week. Yeah.

Honestly, it was super fun. It was super fun. I thought it really represented the next level of maturation of platform engineering as a discipline.

And we'll get to that in a second, and the content and all the great talks and everything. But beside that, I think just on a human level, I thought it was a great thing for us as the community team. We actually decided to fly everyone into at least one of the two live days so that it could almost be a bit of a company offsite.

So we had our designers flying in from Nigeria and Maldives and people from all over, to really just experience, because it takes a lot of work to put this thing up every year. We had 50,000 people connecting virtually. We had 2,000 people across London, New York.

Everything in the same week. You have 300-plus virtual talks, 60-plus virtual live workshops that need to run in parallel. It's pretty crazy when you think about everything that needs to just work.

We had a virtual team that, in order to experience the event, they came to London, but they were staying in the green room together with speakers while they were hosting all the virtual workshops at the same time. Right? So it was all this behind the backstage stuff, but then also on the stage itself, Casper and I gave this keynote.

Well, we had this opening kickoff virtually, which was great. It was something like 10,000 people watched it just on day one. The thing about PlatformCon is also obviously we want to create a special experience for people that can join us in person, but we want the content to be available for everybody for free.

Right? And so that's why all the virtual workshops, all the virtual talks are available for free on the website and on the Platform Engineering YouTube channel. And we're also going to publish now the recordings of every talk, including your panel in New York, that happened on the main stage.

And so we did almost the same sort of kickoff and then a keynote presentation kicking off the two live days together with Casper, and it was just great to be there. We announced the book that you and I were just talking about- Mm-hmm ... so things like platforms.

" Whatever. " Because all these enterprises are quickly realizing, if I don't have a well-designed platform, I can't go from this impressive day zero demo to a day 100 actually functioning, value-creating AI deployment. I need that platform in the middle to scale that and make it enterprise grade.

And so who builds that platform is the platform engineering team, right? And so that's where platform engineers really, I think, are becoming one of the key roles in the enterprise. And I think the interesting thing, and another one of the announcements that we did was Agenticon, which we're going to host in SF in February 27 next year.

They're also moving beyond IT, right? So they're no longer supporting the scaffolding for AI deployments just in IT, but also everywhere else. Right?

And this is something that we're seeing firsthand on the field where platform teams are getting all these inbound requests from other teams inside the enterprise. Right? What it's like marketing teams, sales teams, legal teams, whatever, because everyone is playing with AI and agents and whatever, and everyone is basically de facto becoming a little developer even though they're not an engineer.

And so obviously that creates all sorts of problems and challenges. And so what that means is actually the platform that you need to provide to the rest of the organization needs to be even more solid and even better designed than the one that you provide to IT people. And so I think it was a very hopeful message.

I think just the human element of getting this community all in one place or two places, in this case, in person, you really feel the enthusiasm when you walk in the room. So, I think there's Kevin from the CMO at Vulture, put it really well. He was saying, you walk into Platform Con London, there's this sort of wall of sound from all the conversations and stuff.

And it's like you just immediately get into this like boom, into this bubble. And it's quite special. So yeah, we're happy.

It was really good. It was really fun. Great.

Some observations myself, but before I run into them, I wanted to just touch on a couple of things you said. Look, platform engineers are not alone in feeling anxiety over how is AI going to change my job? How is AI going to let me keep my job or not, right?

What am I going to do for my career? How am I going to take care of my family? And I don't want to sugarcoat things and tell you, "Oh, no one's ever lost their job from AI," right?

We're seeing that. I think we're seeing people make very shortsighted cuts because of AI and then realizing it doesn't work that way. Yeah.

But what I will tell you, and this is the message especially for platform engineers, is you've got to embrace AI. AI makes you a better platform engineer. Makes you a more effective platform engineer.

A platform engineer wielding AI as a tool is a lot more powerful than a platform engineer who kind of is shying away from AI. The other thing I will tell you is that in a relatively short time, AI has had a tremendous impact on platform engineers. " Because what platform engineering was even 15 years ago before we called it platform engineering, and then what platform engineering became in the age of cloud native, managing Kubernetes and that whole stack, which is still a job.

We still haven't quite cracked-- It's still not that easy. Yeah. But then a few years later, that changed too.

Platform engineering became more about IDP, managing that internal developer platform. And not just for developers, for DevOps engineers, for SREs, for security people, data engineers, right? So the main focus was managing the IDPs, that platform.

Well, now I think we're here at this next iteration, and it's about managing the AI platform. Now, the good news is the AI platform of choice, at least right now, appears to be the cloud native platform. So here we are back to Kubernetes, right?

Back to managing that stack. But that's part of it. Go ahead.

That was one of the interesting conversations that came out last week. Just because that's their setup, right? Yeah.

And that's always what happens, right? It's like, well, it's whatever I have, so I'm going to run it there. Even if a lot of people- No, I think they will.

So now you just hit on the next iteration, which is you're going to have to, right? Because I think the token maxing, the tokenomics of it is if you're going to run everything on one of these frontier models that's hosted back in- Absolutely. It's too expensive.

Yeah. Use people. You need to self-host.

Yeah, self-host for sure, right? But you're not going to do the entire training run on like because you're kind of- Well, you're not going to train a huge LLM. Yeah.

Because the open source things are already good enough, right? And so you're going to use cheap like... Honestly, actually, they're the one- But it's a 25-year-old Neo cloud.

Exactly. Okay? Exactly.

And then they are actually a cloud, right? They provide a lot more of a complete- Well, we was talking offline. I tell you we're working on this book.

This book is about using the right-- It's about where the money is as technology waves become commoditized. Right. Right?

And so the idea that you're going to use a GPU for most of what you and I and the average person are going to use AI for is swatting a fly with a sledgehammer. Right. Yeah.

You're not. The GPU is optimized for training. The inference, and you want to call them CPUs, you can, they have other names, right?

But look, Broadcom has a huge stake in that. Google, what is-- Not Tanium. I forgot what- TPUs ...

TPUs. But no, they have a name for their particular chip, though, Tanium or something. Amazon's making their own, Microsoft's making their own.

A lot of them are based on ARM architecture and stuff like this. And that's exactly why, by the way, I said you're going to get away from the LLMs, because a lot of that is using a sledgehammer for a fly, right? Right.

If I need a very specialized model for a very specialized task I'm doing, I'm probably better off using RAG and stuff like that and doing something I'm not even going to host in the cloud, frankly. I might be better off hosting it right in my own data center, in my own closet even- Yeah ... or one day we may see it on your own phone.

Exactly. But I don't care where you run it, you got to have a platform for it. Yeah.

Now, it's kind of serendipitous, that's a good word. It's kind of serendipitous that it just so happens this stack that we've built in cloud native actually lends itself well to this. Exactly.

Will there be something better down the road? I don't know, but right now that seems to be the platform of choice, the stack of choice- Yeah ... for running these AI workloads.

And more power to it, right? I think that cloud native now is AI native- Yeah ... is the upshot of it.

But it was interesting. I also spoke to the Vultr people. And I worry, so this book I'm writing on, if all you're providing is that commodity layer- Mm-hmm ...

that's not where the money is, right? Because indispensable commodities get regulated and capped. Right.

You've got to go up the vertical stack, right? Yeah. Google's not just going to run AI hosting.

Yeah. It's what they build on top of it, Microsoft, AWS, all of these guys. So it's going to be interesting, and it was great.

I wanted to talk to you a little bit, you and I spoke in New York, the difference between London and New York, and I'm not dissing New York. New York's my home. I love New York.

But there was a different vibe in London from New York, you said. Yeah, there was a different vibe. And it's interesting because I was cur- Which, I mean, you were there.

It's incredible. It's the old office- It's beautiful ... Warner Bros.

on Hudson Yards. Crazy- Mm ... views on downtown and everything.

But I think we could have- No, I mean, look, we've done a ton of events over the years here, obviously at TechCrunch. And it's funny you mention the World Cup. That unless you happen to be a football or soccer fan- Yeah ...

that may not have crossed your radar when you were thinking about where you're making it when. Because I know, let's be honest, and I mean no disrespect, but most of us have women in the marketing roles who are looking at calendars- Mm ... " Right?

" Right? " Yeah. But the World Cup's not a tech conference- You know what?

unless you're aware. I didn't think about the World Cup either. I get it.

Yeah. But, well, I'm sorry for your Italian team, but America actually plays tonight. I don't know.

It's the first do or die game. Yeah. So we'll see if they get in or out.

Yeah. But you're right. That is something.

I noticed the hotels were crazy expensive this past week. Crazy. We stayed, as you know, for the weekend, and I couldn't believe the hotel I usually stay in was over $1,000 a night.

Yeah. Where it's not cheap to begin with, but it's usually 500. Yeah.

550. It was $1,000 a night. But look, it was a great event.

And I think more importantly, and I- What did you think of the vibes? I'm a New Yorker, so even though I'm living here 23 years, I still consider myself a New Yorker. So I walked in, I felt at home, right?

And I'm familiar with Hudson Yards. I was there the first day it opened, or the first month we were there, and we often walk the, what's it called? The High Line or the Greenway, whatever it's called.

Yeah. The High Line. The High Line, and we end at Hudson Yards, and we have lunch or dinner there.

So I love that location. I think the thing about New York is New York's a big enough city where you don't need people from out of town. That's the great thing- Mm ...

about New York. Yeah. Right?

Most conferences that happen in New York, 80 to 85% of the people live within an hour and a half of the city They don't fly in. They don't even stay in a hotel. Right.

So as an event organizer, I'd say focus in what we call the tri-state area, New York, New Jersey, Connecticut. Mm-hmm. Because those are the people who take the train in the morning, and they take the- Yeah ...

train home at night. Yeah. But New York's not London.

I've done events in London, right? And you're right, there is that sense of community in London. There is that.

The Brits, they like to come early. They get their tea or coffee in the morning. They're looking for the midday snack.

And not that- Yeah ... it's all about eating, but they're engaged. They're engaged like that.

New York, you know what I mean? You got to- Yeah. At the same time- ...

it's different ... they're also more conservative. Right?

Like- Really? how to do more platform engineering. That was not the case for New York, of course.

No. It's the opposite, right? They loved it.

com, we do DevOps, all these different sites, Techstrong IT. Generally speaking, and I don't mean to diss anyone, but I think the US is probably eight to nine months ahead of the EU and Europe in general. At least.

At least. Yeah. At least.

I mean, here, if it's not AI- Well, and the problem is- ... it ain't happening ... the problem is it's getting larger.

It's not getting smaller. The gap? Yeah.

It is, but I think that gap's going to shrink. You think? Yeah, because I do think...

So let's be honest. When we talk about open source AI models right now, we're talking about Chinese models. Yeah.

Right. They're China models. I don't think anyone, not anyone, I think most people in the US and a lot of people in EU are not comfortable betting- Right ...

the house on the Chinese open source models for political and- Well, unless you get the usual hyperscalers, right? They fork it, they- Yes ... look at the code base, and then they provide it to you, then you don't care, right?

Well, no. Look, I'll tell you something. I used to sell security into the Department of Defense- Uh-huh ...

before we called it cyber. 2005, 2006, 2007. The best firewalls in the world back then were Check Point, or many people thought the best firewalls in the world were Check Point.

And they passed all the US testing. They were an approved US firewall option for the Department of Defense. They didn't sell any.

You know why? Because everyone in the Department of Defense thought the Mossad had a backdoor into the Check Point firewalls. And they- Hmm ...

may or may not have, I don't know. Right. But that was the perception.

Right. I think there's a similar perception with Chinese models. Right.

I think Europe is demanding needs and has to get a European hosted, based, developed model. It's time for AI sovereignty, sovereign AI, right? And there's a big movement, and you probably are aware of it.

My friends at SUSE are very involved in this. Yeah. My friend Andreas Prince.

Europe needs its own model so that if the US decides that Anthropic can't ship Fable- Yeah ... or whatever, no one pulled the rug out from under you. You can't- Yeah ...

plan business like that. Yeah, but I honestly don't see how we get any leverage in this as Europeans because we just can't get our stuff together. And like the-- It's so funny because we feel the same way here about us, and we think you have your stuff together.

Nah. No. Clearly the US is leading.

Mm-hmm. I mean, at this point- In sync ... the interesting thing at this point, right, is because Fable is actually not GA, like- No, it is today.

Oh, it is today? Today. They did a deal yesterday.

Fable's out. I didn't see. Okay.

Fable is out, and Sonnet 5 is out. Okay. Well, then, so the US is back ahead six months, right?

Times, right, where it's like- A feudal time, or- Yeah. Where basically, you're part of the where you have to pay your dues and you're just- Mm-hmm ... renting your stuff from others, right?

You're not really like- Anti-AC. Well, that I do have to agree with you. You got to get rid of those in-room jobs and start designing buildings that have air.

Yeah. Look, I live in Florida. No one lived here in Florida until AC was invented.

Yeah. Crazy. It's like this guy, the Singapore guy, who's like AC was the biggest leverage point in their entire history, right?

Because otherwise- Yeah ... it would still be like a super cool- Well, you've never- ... fisherman town.

I don't know if you've ever been in Singapore in July. No, I've never. That place is uninhabitable too without AC.

But hey, Singapore, so because, just as a segue, this is... there as well. You know what?

Talk to me. I'll help you with that. Yeah.

We'll talk more about it. It'd be fun. Hey, but we're 30 minutes out.

org. Yeah. com, you can...

In-person panels. The in-person, yeah. All the- Not just the in panels, the in-person sessions Yeah.

The entire main stage track, essentially. Very cool. Cool.

We didn't even talk about Weave Intelligence, but we'll hit it on the next one. Next time. Luca, I'm going to let you get back to your own bed over there.

Good luck with the house. My own rented bed. Your own rented bed, but it's coming.

" You'll have your own house soon, mate. Someone from Italy told me that. All right.

Anyway, hey, man. Great seeing you. Thanks, Alex.

Great show. PlatformCon was great. Thank you.

All right. " Bye-bye.