Techstrong TV December 17, 2025

On, don't they know it's Christmas, we're supposed to be slowing down. Um, what are we kicking off with? You're Trying to rush some stuff into the fourth Quarter.

I guess everybody wants to stick it in there. What are we starting off with? Well, let's start off with this conversation about the cost of AI agents.

And Mark Benioff weighed into this debate saying that at least customers or so he claims are telling him that the agents are gonna be priced maybe, or should be on a per seat basis. Well, you know, there's a lot of folks who say that a might make sense. And then there's others who say, well, that's not gonna make sense, because each of us might wind up having 10 agents.

And right now agents are kind of expensive. And you can start looking at a model where, I don't know, it could be as much as anywhere from 250 to $500 per agent. So, you know, what's the cost structure look like and how affordable is that?

And some folks are even saying, you know, well this whole current token based model for pricing of AI is also not working either. So Dan, how do you see this playing out? Because every software vendor and every customer is kind of scratching their head about this issue about, well, how will we afford to al's point billions of AI agents?

Yeah, I mean, I think this is really getting at the fundamental question, which is, you know, what is the ROI on ai? Right? You know, to date, there's been a, a very small few number of companies that have really made money on ai, right?

It's, the kind of joke has been, it's been Nvidia and consultants that have made all the profit here, right? And it's not really wrong in a lot of ways. Um, you know, obviously the supply chain, you know, infrastructure, you know, there's been, you know, been money to be made there as well.

But, you know, the software side has been been tougher. Um, model companies are starting to make a little money maybe where they're actually turning the model itself into an application. Um, but you know, the big enterprise software companies that are really adding ai, you know, kind of embedding ai, um, you know, there's been tension there.

You know, uh, if AI really does play out, maybe there's less people. Maybe a seat based model isn't great there. Um, I think we saw with, you know, the Doge work earlier this year that a lot of enterprise software makes a lot of money for selling software nobody's using, right?

So, you know, does, uh, does a a a fixed fee model make more sense? And, you know, this is obviously the case more around, you know, kind of consumption and usage. You know, do we go with a token model?

I think what you're seeing out playing in the market right now is really kind of that tension between the seller and the buyer kind of playing out in real time. Um, you know, I, I think the buyer's looking for some level of predictability on cost. I think the vendor's looking for, you know, some level of kind of, you know, minimum commitments along with the ability to scale up, you know, with usage, but have a little bit of protection on the, the, you know, the expense side if really these tokens get burned up in a really quick and meaningful way.

So, you know, I think we're, we're all wrestling with this fundamental question of we all see immense potential with ai. We know that it is incredibly costly to build and to put in. Um, but you know, as we're kind of getting to the value, you know, what is that business model ultimately gonna look like?

Clearly we haven't answered that question yet, and there's a lot of push and pull in the market. And, you know, I think we're getting closer to that business model, but not there yet. I'm a simple-minded guy, and I'm gonna throw this over to Tom 'cause I know he is like-minded, but, um, I don't understand.

How come we're like setting up a separate price points for AI as opposed to just saying, here's the new cost per seat for using your software and it's gonna be more expensive 'cause we are using ai, and then let's let the market determine what it will bear in terms of its actual cost versus, you know, right now it feels like, you know, the software vendors are trying to like say, all right, we wanna recoup what we're spending on AI and then add on 50% on top of that for our margins and pass that along to the customers. And I think the customers are gonna be smarter than that. Well, you would hope that, and, and if you're gonna break up a paradigm, you can't do it the old way by charging people for stuff they use, you've gotta come up with a creative and novel way to do billing so that Wall Street will reward you with an extra big stock price.

And I was just, uh, looking through my email because I feel like I've heard this conversation before 12 years ago when software defined networking was all the rage. I know that that was like back in the days of Captain Kangaroo, but really what's going on is that people who want to stick around in a company need to find a way to more closely tie usage to the dollars that they get out of it. And the reason why I bring up this email from 12 years ago is because a, an executive that used to work for a software company then went to a company that was a networking hardware company, and in the middle of a big meeting said, well, why don't we charge people to use software defined networking, in this case, OpenFlow per flow.

And everyone in the room immediately turned and looked at him like he was insane. Because you can't do that in a networking company, right? Like, think about trying to just keep track of the flows that you're gonna be billing, and now you're gonna go to a company and you're gonna say, oh, well, you know, per flow will charge you X amount of dollars or x amount of cents or whatever.

You're quickly going to just be out of pocket with the amount that you're gonna be charging. And Mike, to what you said, that's what the companies who are making this stuff want because they've spent billions of dollars investing in whatever this is gonna be. And we've gotta get our money back out of this.

So we've gotta figure out how to closely tie what you're using to what we can charge that, uh, charge for the amount. Oh, and by the way, if we can continue to make 50% profit off of it, that's even better. Because ultimately we're not beholden to Nvidia or to the analyst.

We're beholden to the shareholders who want their return. Or more specifically, they want you to take those 50% profits and buy back some more of the stock. So the value of my stock goes up.

So Mike, Tom, Dan, with all due respect, the technology industry has never been known for, for good visibility into billing and why and how. Just take a look at your cell phone bill or your cloud bill and to, to see the state of the art there, right? You, you usually, there's a whole cottage industry.

Well, it's not even cottage anymore. It's called FinTech, not FinTech. Uh, yes.

Finops, you know, to help you get ahold of these things. 'cause they, they do tend to run away with themselves and they often bear no talk. This is the craziness of it.

They bear no connection to actual cost. It's just whatever they can go for. But here's what I believe we are in a Cambrian explosion kind of, uh, era with, with, with this ai, with Agentic AI and everything else.

We may look back on it and say, why did we think eight eye creatures would be better than two eyes or six legs are better than four legs or two legs or what have you. It's a, it's a grand experiment. The market will determine this.

Uh, you know what, in 19 96, 97, I started one first company. I started in Tech Tristar Web. I was getting 49 95 to host a website, a brochure website that did nothing.

There was no SSL, there was no real commerce. They were brochures. People paid me $50 a month.

I had 5,000 people paying me $50 a month to store their websites on Sun Ultra Sparks. Two years later, that same website was hosting for $9 and 95 cents, and people were still making money at it. That nothing's changed.

It's, it's going to be the same thing. However, if 10 agents a person, you're kidding yourself, we're gonna have as many agents as we have passwords today. I don't know.

We'll see. Jennifer, I'd love to get your opinion. 'cause I'm getting ready to run for mayor of TechTown on an AI affordability campaign.

What do you say? So as, as someone who, and you know, use is a power Salesforce user and my role in Go to market, I was thinking a lot about this. And what I don't understand is, you know, Salesforce already has a very advanced pricing model.

And it's one of the reasons why a lot of companies don't start with them in their early days because they're too expensive. They start with HubSpot, what have you, try to do all their sales and marketing automation, then just move their sales into Salesforce. 'cause marketing is too expensive to do.

Marketing clouds too expensive to do both. But they have a really good structure for different groups and types of users. And then packages of how much data you can consume or how much data you can process.

And I don't know why they didn't do that with, with agents. And also it would make them more competitive for Marketing Cloud because HubSpot does not have great ai, sorry, HubSpot friends out there. Um, and would allow them to be able to combine workflows of marketing and sales much better if everybody had their marketing and sales under one umbrella.

But they could only really do that if they priced, if Salesforce priced similarly to what they do now, which is based on the user, um, understanding that the agents are very expensive, but also so is a lot of the sort of, a lot of the other things that they do. So for me personally, that would affect my buying decision. Dan.

No, there's a lesson there for us, you realize, right? Chris? You have not heard from, we have not heard from Chris yet.

Gimme a second. Yeah, that's rare enough. Um, as I thought of before on the show, this, this is this weekly thing I do with you folks is, is an interesting metric.

And Dan, you know, uh, I think about the conversation we've had, and a fascinating thing happened this week as we're all talking about this. You know, we helped a, a very, very small business in a very, uh, uh, uh, economically impoverished, war torn area, stand up an agent on an old Windows laptop. And that agent now is operating in this commercial environment, running this business on no hardware whatsoever, right?

Without the GPUs and everything else, with all the information right there, no internet connection and already changing the economics of the situation, better clarity and so forth. And it's all these things we talk about without, you know, Palo Alto and data centers with GPUs out the wazoo, right? You know, as I think about this issue, and Jennifer, everything you were saying, right?

You know, we have these complex systems and they're working just fine, right? You know, everybody's worked really hard, built these wonderful things, but sometimes they reach points. And last week, you know, last week on, on, uh, LinkedIn, both, uh, Rob Lee and Dragos and Mark Weatherford, you know, both publicly posted, uh, comments about different issues that speak to the same thing.

One was the Apple ui, uh, rud launch, right? Know the UI died and, uh, in the Apple space. And the other one was, uh, mark was talking about just the popups.

So I took American Pie and rewrote it as, uh, the day the UI died. I think we were speaking to the same thing. We're reaching levels of complexity where we had to do things differently.

So Salesforce, great friends working there, we've done great stuff. However, how do I consume that as a, in this world, we're moving into where agents are like, you know, spreading out and getting cheaper and modeling out. It's, it's falling a certain arc.

And, and am I gonna need 10 agents to your point, or will I have maybe four agents that are kind of super agents that are invoking a bunch of backend services? So I don't need all these, you know, 32 SaaS applications that I'm supposed to buy. I maybe only need five.

Eric, we're, we're gonna talk about that in the next segment on, on, uh, with ServiceNow about SaaS versus agents. But look, I I think it depends. Are you talking about persistent agents, alter egos, digital twins, or are you talking ephemeral agents that are kind of disposable, do a job and move on like containers, many containers in a, a Kubernetes environment?

Listen, I, I think we're, I think we're kind of looking a little bit of the symptoms of the bigger problem here with this whole discussion around how we price ai, right? Like, you know, if I zoom out, you know, the conclusion for me is that, you know, we got used to a world in which SaaS companies were these insanely profitable entities. And the reality is, is as you add AI to software, you massively increase the amount of compute you're gonna consume to run said software.

And so therefore, the conclusion should be, being a SaaS company is still a great business model, but it's gonna be lower margin than what you're used to. And that's probably okay. If the AI provides a value above and beyond what you used to get out of it, Is, is it still a great business model?

Although Satya, Satya says SaaS is dead, right? Uh, I, I think that's splitting hairs a little bit. I I think when he says SaaS is dead, he's talk, he's talking that agents are gonna live.

I I I'm blending all of it into this, you know, category of software, you know, whether it's or an agent still software to me. I think he said that to drive the value of the companies he wants to roll up down that, that, my opinion, Why you think he would do that Anyway, Hey, we're over our 15 minutes on this segment. Let's take a break.

We're gonna come back. You know what? There continues to be crazy news around fundraising and acquisitions and just in securities, especially cyber, we're gonna talk more about it.

You're watching Textron Gang, you've Earned it. The spotlight, the responsibility, the weight of teams, companies, and entire industries fall on your shoulders, lives depend on your decisions, your home life included that work. You are protected physically and digitally.

Nothing gets through your team without a fight. But in a globally connected world, everyone sees you, including those who mean to cause you and your organization harm. And now home your sanctuary attackers see an opportunity.

Your digital front door is wide open. And what compromises your home can breach your forklift. Because the devil's greatest trick isn't targeting your workplace firewall.

It's convincing you that your personal life isn't at risk. Black cloak, digital executive protection, defending the new attack surface your personal life. Hey folks, we're back and we're talking about some merger and acquisition activity spanning SaaS and cybersecurity.

Uh, ServiceNow allegedly is coming close to a bid for amis that's supposed to be somewhere in the neighborhood of about $7 billion. And meanwhile, we also have a report up on Security Boulevard just talking about the massive amount of money being I pour to the cybersecurity space, especially outta Israel. Alan, I know we talked about AI agents in the last section, but before we get to that, just kind of set the stage here, it seems like there's a massive amount of VC capital in the cybersecurity space.

So is there just gonna be a wave of these m and a activity? And then how might SAS play into that? So, and, and it's tied into the Israeli story too, right?

Because a bunch of the companies I'm gonna talk about actually come out of Israel, you know what, for the last couple years, and, and we've got three or four security folks on our panel today, right? The, what we've heard at RSA is where's the innovation? Where's the dynamic new stuff?

And there was a lot of dry powder on the sidelines, I think the Google whiz acquisition for whatever it was, $34 billion, what broke the dam for really bringing this money out to the forefront. Ever since that Wiz acquisition, we've seen a steady drum bait drumbeat of some really eye popping numbers, uh, on, on acquisitions. Now, Aramis, I amiss, excuse me, amiss.

I, I first met the co-founders of Amiss, had an insight, uh, in Insight Ignite event in Iceland six years ago, seven years ago. They were fairly new. They were singularly focused back then on IO security, and they still have a great IOT security, uh, uh, product service, whatever you want to call it.

But they've expanded since then. They've been hyper aggressive as many of the Israeli cyber companies are, quite frankly. Um, they just did a raise two or three months ago, I think it was a $430 million raise.

1 billion, right? So a $7 billion buy here by ServiceNow in today's market, where a hundred million here, a hundred million, there is no big deal, is not crazy. I think you also gotta look at even the broad as part of a, this broader pattern that I spoke about, right?

I, the, the, the article you mentioned, Mike, was my article. I, I did it off a report out, uh, ventures, while ventures, Jennifer, I know you know them. Our friend Andy Ellis was there for a while.

They pioneered the, what I call the underground railroad of Israeli cybersecurity companies to the us primarily to Boston. They all seem to have moved to Boston, but, um, their report this year just shows that that market is through the roof. I think 40 something deals, I think how many billions of dollars.

What's interesting, it's not just a US based VCs funding. There's, there's homegrown VCs now in Israel. There's this whole, it's not just cybersecurity companies.

It's a cybersecurity ecosystem. And it's, it's fueling that country. It's fueling our industry, right?

And, and we're just seeing, you know, it almost seems like people walk out of unit, what is it? 8100, 8200 and VCs are waiting there for them to start some company. So it, it's, it, it's definitely a real thing.

Here's the part where I think rubber meets the road for ServiceNow, and I'm interested in your comments. Is this the second acquisition ServiceNow is done in cyber in the last couple weeks, it seems like. What are they seeing, right?

What are, are they, are they just, I think, Dan, you said it in the opening, are they just, you know, a big company who's looking to buy organic revenue? But, you know, I, I know those people. They're smart as heck.

ServiceNow. They've been some of the smartest guys in our business for a long time. They see something, they see something there, and I, you know, I'm, I wouldn't where they go.

I'll follow Chris. You got your hand up. Yeah.

So ServiceNow has been very involved in the supply chain for, for a long time, for good reasons, you know, develop some of the, uh, good, good, uh, uh, protocols and, and a lot of work there. And I, I, my read is that they're making a play to be the control plane for, for the, the internet, you know, the, the, the system of record, right? You know, where did things come from and so forth.

And that makes sense. And like everything else, you know, I think it's a logical play. I think it goes right down the path.

You're, you're talking, um, I would I say to sort of what we talked about in the last segment, though. Do we need a single unitary? Is that, is this the era we're going into, or will this be a good play for ServiceNow?

So they could be a control plane that makes this easier for people in their space, but we still need to, you know, this has to be like everything else. A dare I say fully age agentic, federated process, right? Having big players like ServiceNow go down that path indicates this is where we're going.

And I would just not read too much in, in, into this as being the, it's just a, Jennifer, what's your take there? Because, you know, part of why Chris is sort of alluding to is that well, does cybersecurity, as we know it as a separate segment in the industry, just become a feature? No, I think that might be the way we're heading.

Um, not just because the moves with Surface now, but some of the moves with, you know, AI that we've been talking about in general. Because what happens, just a side thought, what happens when the AI companies start turning around and building security features, right? So there's, there's that thread as well.

But with ServiceNow, I find it fascinating. So I spent a little bit of time, a couple years of clarity, and we competed with amis, and they were all about, at the time, we all talked about was digital transformation for, for ot. For ot.

And ARM did a lot of that as well. And if you look at ServiceNow and how they're positioning themselves as the business platform for companies, they did the, they did the acquisition of, I believe was Visa, visa, visa to say that, that, so vea, so they've done the acquisition there, they've done the ac they allegedly doing the acquisition of arm. I, so they've got OT covered.

They've made some really smart investments as well into continuous controls monitoring and third party risk. I think there's an op, I think they're trying to, they, oh, it's almost like they're taking a backward approach from where they used to be to GR gain more security relevance and be that big platform for everyone versus the security companies that started like Palo with NGFW and then have expanded out to take on a bunch of different use cases. So I'm curious to see how that all comes together.

Dan, Dan, you got any thoughts on this about the, the merging of these segments, or how does this look to you? I, I, I was gonna go down the path. Jennifer went down.

I mean, this feels to me like ServiceNow kinda, you know, blending the it ot, you know, you look at some of the other acquisitions they've made, they're clearly gonna be big and ag agentic and, you know, kind of building out these business workflows. You know, they've brought, brought in some of the, you know, identity security stuff to, you know, shore up the agent side. I mean, I, these all feel like really highly relevant kind of Bolton acquisitions that extend their platform capability down into these new use cases where they can kind of really reimagine what the business workflow looks like using ai.

So, you know, to me, I think there's a little bit of an element of, you know, they've become a really big software company, and like most big software companies, you, you tend to need a little bit of inorganic alongside the organic to keep the growth machine going, keep Wall Street happy. But if you look at the areas they're picking off, they're highly strategic extensions of their core platform in a lot of ways, right? You can imagine, you know, for the types of things they do in, you know, logistics and shipping and, you know, all of the, you know, kind of it OT processes around those kind of things.

Um, it makes a lot of sense, you know, kinda where they're going. Um, but, you know, these are relatively smaller deals from a revenue perspective, but I think they give them a core capability. And you can bet that there's likely some lead customers who are already trying to stitch this stuff together on their end, working across these suppliers.

Uh, and ServiceNow is gonna help, you know, kind of make the easy button for them on putting all this together. You know, this, this deal reminds me of when ServiceNow bought, was it called Lightspeed? It was the four guys from Google who basically started o hotel open telemetry, um, and it gave, it gave now a catbird seed into that whole observability space.

Um, I, I think this deal is, is a similar back, back to what Jennifer said. And Tom, I I'm interested in your thoughts on this. So are we seeing security going out to the rest of the world, or are we seeing the rest of the world coming into security, right, in terms of Terra force here?

So I, I think it's the second one that you're saying. And, and when you think about what ServiceNow offers, right, they are the software as a service platform, right? Like Jennifer said, if I don't know how to do marketing, I call these people.

If I don't know how to do ticketing, I call these people. What about the, the market that amis is serving, right? They're serving things like hospitals and, and if you thought IOT was a pain in the neck for sensors, wait until you have to treat, keep track of insulin pumps and heart monitors and all that other stuff.

And now you have to keep them secure. And you know, Alan and I had a great conversation about this in a yesterday on a Security Boulevard recording that we did, where we talked about the fact that, you know, old school folks, like, well, frankly, everybody on this call didn't come to security naturally. We came to something else and then picked up security along the way as an adjunct.

Whereas now there is a group of people that are kind of graduating from college, you know, uh, gen Alpha, probably the very beginnings of Gen Z, who are security native, right? Like they can make security their full-time job. But what does that mean for the people who are trying to find those jobs?

It means that companies see the gap there and can say, what if I offer that to you? So you don't have to go out and hire those people. Now, ServiceNow is gonna go out and hire them because they want people who are kind of security native, so to speak.

But as a company, if you're overwhelmed by this, I can just take care of it for you. We'll bump your license cost, I don't know, a couple bucks per seat. You get all this security knowledge and you don't have to worry about paying benefits and all that other stuff.

And oh, hey, by the way, all that money that you save by not having to hire all those new security analysts, 'cause you're paying us to do it. You can invest that in ai. You can invest that in other cool technology that isn't boring and, and kind of ugly like security.

Just, just let us handle that. And then they're super sticky because now that your security has been offloaded to a SaaS company, if you ever drop that company for any reason, or if you ever need to negotiate and make moves to try to reduce your licensing costs, oh, that's gonna be bad because we're gonna lose a lot of security expertise if we do that. So in a way, ServiceNow is kind of positioning themselves for the future when those markets start opening up as more tools are being enabled to become very hyper-focused on those attacks.

Because we've already seen that over the last couple of years where healthcare organizations are becoming targets for ransomware and criminal organizations. Because if you want to get people to pay fast, hit something that's absolutely mission critical. Like an MRI machine, I would just point out that, um, this is kind of like you ever seen that movie Highlander?

Mm-hmm. So whether security is taken over it or it is taken over security, it doesn't really matter. 'cause there can only be one.

It Could be only one and it all Right, that's a good place to end this segment. We've got one more great segment coming back more on m and a news. Nvidia seems to, what a surprise.

Nvidia is active again, you're watching Textron gang. 2026 marks a turning point. Artificial intelligence is no longer just a tool.

It's shaping industries, accelerating innovation and redefining how humans build, create, and solve problems from engineering and medicine to finance infrastructure and everyday life. AI has become one of the most influential forces on the planet. That's why for 2026, we recognize AI as tech Strong's person of the year not for what it replaces, but for what it enables a future built together humans and machines predict.

2026, join us. Hey folks, we're back. And if you watch this show, when we were at CubeCon, we had a whole segment talking about, uh, slum, which is a job schedule, a favored by the folks who build high-end high performance computing systems versus Kubernetes, which is more favored by a subset of the IT community for orchestrating things.

And now NVIDIA has turned around and bought, uh, Schmid, which is the company that kind of drives s SLM in the first place, and they're gonna incorporate that into their portfolio. But Nvidia also has some tools for Kubernetes as well. And then at the same time, NVIDIA's also sending signals now that it intends to become a major provider of AI models itself.

And it seems like it's trying to build this entire stack. Tom, what's your read and what's going on here? This is what I expect Nvidia has to do.

In order to continue to get people to buy all of their chips, all of their GPUs, they have to find a way to differentiate what their GPUs offer. So like you said, the first one is buying schmid. Now listen to what they're saying.

We're, we're still gonna keep lumm open source. You guys can still learn how to use it. Um, implement it on your smaller tasks.

Um, basically the people that Nvidia won't get out of bed to sell to because they don't have enough commas in their revenue. But once you're ready to play ball for real, once you're ready to build your business on this, don't you want to use the, the tools that we own that we may or may not have optimized to work on our GPUs? Yeah, you don't wanna use Google GPUs or a MD GPUs.

They don't work nearly as well with schmid's professional offerings because we've never seen a company do that before. Cough, cough, Microsoft. Um, but more importantly, when they start then producing these models, uh, believe the model variance that you're looking for is nron.

Um, and of course they have, they call them something different, but they're like the tall grande vente model and, and it's very focused on multi-agent ai. Now they're starting to realize that they've gotta pick up with the new trends and run with them as fast as possible because so many companies are making those leaps. You know, we, we joke about the fact that we still can't figure out, you know, how many RSS are in the word strawberry, but that jokes from 2024 LMS are kind of passe now.

Nobody cares about that. Age agentic is what's hot, and now it's multi-agent, as we've even talked about on this call, that people really want to be ahead of the technology curve. And there's so many NVIDIA systems out there that have been deployed.

They've got to find a way to make people want to choose to use them, because this isn't like cloud computing where there's resources out there, and I'll just use whatever's the cheapest today. If you are not the cheapest, which Nvidia is not, they've, there's gotta be a reason for people to want to go through the dropdown box and select that they're using an H 200 or whatever it is. And I think that this is a smart move by them because the hardware itself is not differentiated.

I mean, yes, there's speeds and feeds that will tell you that it is, but it's the software applications that are optimized to run on specific hardware that are gonna make people want to choose that. And if you bought the, the one that everybody wants to use, S slm, that's your end. Mike.

Chris, gimme one sec. I I just, I got a question. The naming conventions here, Schlom Schmid, I thought that Snort, I had seen it all right 20 years ago.

Snort, who comes up with a name like Snort, but you know, Marty did okay with it. Jennifer was there, but Schlom Schmid, have we scraping the bottom of the barrel? No, Man, we we're waiting on slurp.

That'll be the next thing. Look, yeah, I, I came into the, the industry with the scuzzy interface, right? SCSI for the old folks remember that?

And the first thing I did was got the book by the person who wrote the Standard. And as I related, it's pretty damn close. There's like a 17 page introduction that is just him ranting about the fact that he meant it to be pronounced sexy, right?

So don't get me starter with s SLMs and Scuzzy and so forth. But you know Tom's point, right? You know, I, I, I agree, right?

And 25 years ago, I sold billions of dollars of, of Cisco firewalls with Bill McGee and the, and the, and the six and the team by saying exactly that. You're buying a, a Cisco infrastructure, a firewall's a thing. You can buy checkpoint great stuff and everything else, good companies all, but why wouldn't you just, right?

And in the last segment we talked about this with Salesforce, right? You and Jennifer, right? You were talking about, and there's that, and, and that's, those are both true.

But I think, let me argue the counterpoint, right? You know, that's the, the scheduling thing with Nvidia. I get it.

Absolutely. However, that's, that's a big issue. And a as, as all the regulars know, you know, where I'm going down this path, where we're going is narrative, uh, integrity, sovereignty, put it here, run your ais here, you know, the ability to schedule, see, you know, loads across meshes.

The way we look at it is kind of an intrinsic and emergent from the bottom. I think that's where we're going in a lot of things. So Salesforce, Nvidia think absolutely should do that.

Lots of, uh, benefit from that. But I'm interested in the multi-year playout. Is the concentration and control at, at, you know, each of these organizations we're talking about gonna be the big win, or do we get more just distribution and federation?

So I talked to Nvidia about just what is their strategy as it pertains to open source? And Dan, I'm gonna test this your way, but what they were saying was the further up the stack it is, the more open source it is. So if it's a framework for building an application, they're perfectly content to have it open source.

But when you get closer down to the kernel, it's more and more proprietary and that's where the kind of the lock-ins gonna be. But if, if all the tools are free above, do I as a customer, am I gonna care? Or, you know, what is the danger here?

Yeah, no, I mean, listen, I think that strategy makes a lot of sense, right? The more abstracted, you know, the more you know, likely it is to be open source. And, you know, the more you know, kind of specific and down into the, the kernel, you know, the more they're gonna wanna put proprietary, uh, fingertips on it, right?

I mean, you know, I, I do think buying Storm makes a lot of sense. It's, you know, Nvidia is not a hardware company. You know, people call them a software company.

I, I'd call them a platform company. I think that's really what they become. And they've done what platform company do platform companies do, which is when there's another company that builds, you know, something of real value in and around your ecosystem, you know, eventually it becomes important enough that you've want to control it.

You wanna own it, right? And I think that's exactly what happened here, right? They've become slums become the standard, you know, for running, you know, kind of big model tasks on Nvidia hardware.

And, you know, they, they're gonna wanna get control over that because, uh, it's become a critical path for their customers in a lot of way. I, I think the Nron stuff is more interesting. I mean, to me, this is a little bit like the deep seek moment in that, you know, they have kind of activated this, uh, Jevons paradox, you know, uh, you know, kind of principle where, you know, for reasoning tasks, which is kind of really key to getting AgTech to take off, they've made those a heck of a lot more efficient.

And as we make the AI cheaper and easier, you know, easier to do, the more we will consume of it, right? You know, I, I, I haven't seen that takeout there, but it's immediately where my head went on this was you. They have offered something that is materially better, uh, you know, as a reasoning model to, to drive these agents.

And the more we can bring that cost down, the more we're gonna give adoption. We are still very, very early on the adoption curve. I think we forget that sometimes, Tom, you hear, uh, Nvidia used the phrase AI factory, are you ready to leave to live in the AI factory town owned by Nvidia?

What do you say? Yeah, I, I'm a curmudgeon. I'm, I'm not ready to work in the AI factory salt mines any day of the week.

But I'm glad that they're at least looking at these models because they have to provide leadership somewhere. If they are effectively gonna turn the model development over to other companies, they're surrendering any advantage that they might have. Because as soon as I tweak or tune that model to run, you know, basically the same across any hardware, then it becomes a race to the bottom for price.

And we all know that there are companies out there that are willing to cut their prices to the bone to establish a foothold. And that's what Nvidia more or less did kind of at the beginning. They're like, we're gonna give as many of these things away as we can not give away, but, you know, basically sell at a reduced cost so that you become reliant on using our hardware to build these models.

And then once we know that they're the dominant ones in the market, then we can start saying, oh, well the next version that you're gonna have to use, it's gonna cost a little bit more. 'cause we, there's more technology in it, you know, insert business rationale here. So Nvidia really has to kind of be kind of the counterpoint to open AI to say, well, you know, yes, you could run maybe more generalized thing over here, or, you know, maybe what they're wanting to say is, we're gonna take the lead in multi-agent because it runs best on Nvidia.

And if you don't believe that that works, I want everyone who's watching this to go around their house and find a laptop that still has an Intel inside sticker on it, because that was probably the most successful hardware marketing campaign of all time. So, so Mike, I, I think though, we've, you gotta come up and take a 500,000 or a 50,000 foot view of this and admire the genius of Nvidia, right? And I, I say it in all seriousness, listen to me, Jensen Wong and the rest of his team realized what the opportunity is here and where they're, where they're weak or not, where they're weak, but where the potential can, can go wrong.

Right? Now, they've got a generation or two lead over everybody in the market around AI chips, GPU chips, they know that the rest of the world is hot on their tails. It may very well be that China invades Taiwan just for TSMC and, and to make these kinds of chips, right?

Because that, that might be the only way they could catch up. So what they're doing is knowing that they have that finite runway, the time is now for them to establish themselves as the platform for AI and all of its different flavors and permutations. And so you are going to see them with Nron and Lumm and Schmid and, and all these other shimel.

I don't know what else they'll have, but you know, they're gonna establish that platform because now is their time. They, they have this window of opportunity, and once it's gone, th there's a big world out there, even a $5 trillion, there's a lot of people nipping at their heels. They've gotta make it happen now.

And, and so they're moving into this, you know, beyond hardware to, as Dan said, a platform. And they're gonna make that the dominant platform. Not to be overly dramatic about it, but if China does invade Taiwan, the very first missile, the United States fire is gonna be aimed at that TMSC path.

Don't be so, so sure. We were, we we're fire and missiles Uhhuh, but who knows? I mean, but that, I mean, those are like the global stakes here, right?

This is, this is a race, this was, it could be a race that determines who's, what's this next century? What is the 21st century about, and who leads it, right? And I, I think Nvidia has, they're not dumb Jensen, and those guys are smart as heck, and they know they've got the opportunity right now.

They've gotta double down their bets. Yeah. I, I honestly do see them as the John Chambers of, of 2000, you know, they're at exactly that spot.

And, you know, s Cisco at the time, you know, that Chambers did and the team and that everybody involved did a great job, but, you know, life moved on. So they have to be planning for that. And I think they are right.

But I, I'm, I'm with Tom though, right? I think for everything I've said in this, in this episode, I think they'll be fine. I think big infrastructure still works.

However, I think we have an opportunity again, and, you know, history does rhymes. This is similar with the internet or whatever, but the opportunity to move this down and build it from the bottom up, like literally from a tiny little organization in the least resourced who's using AI today, not asking or paying anybody for it, that is gonna have a big play in this as well. Maybe a bigger play in the long run.

I can't help but to think, you know, we, we, we learn from history, right? And going in the way back machine, you know, I think about the market just is the security market's always just kind of in the tech market's, always kind of billowed, kind of like me around the holidays. And so the, um, you know, I think back to my early days in the first, you know, a hundred people at Fortinet and how UTM started to develop because there were all these other technologies out there that weren't necessarily competing for firewall mind share or market share, but they could have taken budget away from a security buyer, Hey, why don't we integrate all this stuff?

Even though half of it at the time wasn't really integrated. Sorry, Ken. Um, and so, you know, and then you saw POW come with NGFW, and then we saw all this consolidation and all these multi technology platforms in the industry, and people would buy up what they might see as competitive, and then they started buying the things that their competitors did.

And then we had a whole bunch of innovation of new types of technologies, and now we're seeing everything kind back together again. And, you know, it's reminding me that Chris said, and Chris and I worked at Cisco for a bit too, after the Sourcefire acquisition. He's like, so who's gonna be there?

You can't get fired for buying Cisco in all of this when we're done. Mm-hmm. That's what I'm really curious about.

This is true. I think people, I mean, I understand Tom's point, but a lot of folks are more interested in the output than they are, you know, the components. And basically, you know, the golden handcuffs are a small price to pay for the output.

Okay. Schmid, Lumm, it's all here. Hey, I know we've got maybe a minute left for some of you on the panel.

This may be your last show we do this year. So I wanted to give you everyone a chance of, you know, what, what, what, what's coming down your way, Tom, why don't you kick off? com.

We already have events scheduled for January. Uh, we are gonna be at RSA this year. It's our first opportunity to have a, a field of extra event at RSA and we are adding new stuff to the calendar all the time.

We're gonna be really, really busy. And in addition to that, I do this, uh, weekly podcast with, uh, some guy named Alan, uh, called Security Boulevard. So make sure you check that out because we have a lot of fun over there too.

It's not just Tom and I though. We also have Fernando Montenegro and Mitch Ashley, future chairman, analyst. So it's great show.

Check that out. Thank you, Tom. Jen, what about you?

I've got lots of stuff coming up. Um, as, as Alan knows, dude, some more doing this, doing some more stuff with Textron, which I'm pretty excited about. Uh, I also started a new job, um, January 5th, the chief marketing officer role.

So putting a lot of focus there. What will I be doing? Where will I be going?

Probably everywhere, because that is the role of the Chief marketing officer at a startup that may or may not be invested in by one of the companies we talk about today. So that's Clear. I'll more on that later.

Absolutely. And and your guest star, was that Hemingway? That was Hemingway.

I am so sorry. He cannot Be, no, don't be sorry. We love, we're a pet friendly kind of show.

As long as he doesn't have any talking lines, we don't have to pay him under, you know, union rules or anything. He had some pretty strong, uh, thoughts about, you know, s slm and I would mad And everything else. Probably quiet down.

Take out one of your snort pigs. Chris, how about you? I, it is been a, a year of building semantic structures and canonical, uh, distributed architectures and ai.

And, and now as I mentioned, you know, we've got things actually deployed in the world and they're happening and we have pilots in Q1 in different sectors will play out, you know, all these, and, and with, with a, with a open source project and a new startup and everything else, all these different hats on. And as, and being like old and looking at these patterns, everything we're talking about here, I'm looking forward the next year playing out. I, I don't think the world's gonna change.

It's not u utopia, but I think if I'm, if I'm right, and if we're right, interesting things start to invert, you know, the control planes we're talking about that are trying to stabilize at this level will be subverted supported by more individuals and enterprises, you know, coming up from the bottom instead of, you know, accepting down from the top. But anyways, I can go on all about that, uh, long for a long time, but I think next year is gonna be fast, I think in, in big issues. Cool.

Dan, you Hey everyone, it's Alan Hummel and welcome back here to Techstrong tv. Excuse me. My next guest is he's frequent guest of ours, Ys Wessling at, uh, Veracode Ys.

I was just checking to make sure I got your name right, but I did. And, uh, every once in a while though, you get that moment of panic, right? Did I get his name right?

Ys, of course, is, uh, with Veracode, as I mentioned, and he's been with Vera Cota. While we will let him introduce himself, Ys, it's great to see you again, I hope all as well. Yeah, it's wonderful to see you again too, Alan.

It's, uh, Those Interesting times. Absolutely. Well, may you live in interesting times, is what they say, right?

For those who, um, maybe have not seen you on here before though, why don't, if you don't mind, give us a little background. Yeah. Um, I'm the, I run up all sort of all architecture and research for Veracode, and I've been there for a while now, and I've sort of kicked off the project, uh, start evaluating the security of large language models, Which is a hot topic, certainly, and of course, wear code being a leader in the AppSec space.

This is something kind of near and dear to them. Um, yeah, it's just before we get into the findings of, of this particular survey and research, um, any idea if I had to ask you what percentage of code being generated today is being generated by ai? Oh, I think it's so hard to tell anymore, but it's certainly a climbing by leaps and bounds day over day.

I think, uh, probably rare to find a lot of developers these days that aren't using AI in some way, shape or form in their development. Agreed, agreed. I mean, I, I saw something that said 90% of developers actually are using AI four.

Yeah. At the 40%. Don't trust it.

I'm surprised it's not a little higher than that, but Yeah. Yeah. Well, 65, 60 5% absolutely believe it.

It, it, it creates instabilities. It's still, it's uh, it is early days yet we'll have to see where things go. I think it's leaps and bounds ahead of where it was six months ago and it mostly wasn't a thing a year ago.

So who knows where it'll be in six months, but I think there's gonna be an upper limit to how effective it can do what it needs to do. And I don't see a world in which you won't need a human verifying that what it's doing is what you want it to do. So The human in the loop.

Well, I mean, and, and that's kind of the subject we're gonna talk about today, but it seems with every new release and between the frontier models, there's new releases, it seems coming out every week or so. Um, it seems with every new release, the the accuracy, the quality of the code that these things are producing is, is improving. So interesting there.

But before we jump into that, kind of the specifics there, Jens, you set the table for us. You, you guys, I mean, Veracode is known for some great reports, a lot of based upon their own customer data that, you know, anonymously anonymized, they've used for, you know, tremendous data sets. Yeah.

What about, so Yeah, about a year ago we started looking over the research on how secure LLM generated code was. And what we discovered is sort of depending on where you looked and who you asked, you get a lot of different answers. And even more than that, they'd give you an answer that's a snapshot in time.

And as you said, every time these models change, the results differ. And what we really needed is something that didn't exist, which is a report that gave a, a very candid, like, description of how well they were performing with respect to security that got updated regularly. So you could actually see whether or not things are moving in a given direction or not.

And we did our first report last spring, and we did our most recent update in October with the latest frontier models and, uh, interesting results. I mean, I think the, the chat GPT reasoning models showed, uh, significant uptick, roughly 10% better than they did prior up until say the low 70 percentile. And basically every other frontier model that had been released over the prior six months didn't actually show any improvement from a security standpoint.

Kind of interesting. So I think Chatt PT did a nice job of sort of focusing on that as one of their key points, and they managed to deliver a probably the largest jump we've seen in the history of the, the reporting data we pulled. Really?

Yeah, Much. This is five one, This is five one. The reasoning models, the, the chat model, the non reasoning model didn't do any better than their prior version really, but, uh, the, yeah, the, the five and the five mini both showed a significant uptick, roughly 20% better than their prior model had done.

What do you, what do you attribute the, the, the regular non reasoning model, if we call it that, not having any improvement versus the big improvement of the reasoning model? I think, so I'll offer some guesses, but I think you'd really have to be in-house at, uh, open AI to know the answer. But reasoning models will take multiple passes through a problem considering sort of different perspectives on how to accomplish their goals.

And if you develop a code generating reasoning model that actually has a past that's concerned with security, it could actually make a significant improvement in how well it does in that respect. And I know they've actually spoken at, at length about how they've invested in security and they put some adversarial models out there and they've tried to improve the quality of what they've done. And I think it shows that that work has paid off and the reasoning model is considering these things now when it's putting together code samples.

But yeah, let me ask the next question, which is, as I, I think I alluded to earlier in our discussion, you know, there's a new model coming or a new version of the models, you know, collectively coming every weeks, every few weeks or whatever. And it seems, well, except for this non reasoning open AM model, each one gets a, at least a little bit better. Is it really just a question of whoever, whoever went last is the best at this point?

Do you see that leveling off at some point? Well, no, I, I mean, I don't think that's the case with respect to application security. I think when we look at the history of these over the last year, the overall improvement in the average has been a few, two, 3% over the last year.

And the open mi there are, there are more recent frontier models than the open AI ones that aren't performing as well as theirs have. So I think it's more than just, we hope it gets better 'cause it's not getting better fast enough. I think you need to make an investment in that as you're producing these models, if you actually want to see improvement and even with all the investment they put in, they're hitting 70% and while that's much better than 50%, which a lot of them are running at, it's still not good enough that you trust it to go to production without actually testing it to make sure that it was secure.

Just, just for reference, what percentage is, let's say human generated code usually? So that's tough. I think it's normally around 60 to 70% humans are also not great and it makes sense 'cause the models are training in human data.

Well then, You know, you're always as good as what you need it. Right? Um, so but what you then that, but that in itself is interesting.

You're saying at this point, eh, they're about as good as a human coder. Yeah. And I think there's a long history of human coders not writing secure code.

And I think the same thing is gonna be true of LLMs. And that's why even if they're, you know, modestly better than humans, it still doesn't mean you get a blank check to release code without being concerned about security. You know, I mean, in all honesty, I don't think anyone sits here and says, we're just gonna release this without first checking it, run it through some security tests and everything.

I think we recognize that I think the question is, how diligent are we with this code and with our testing? How rigorous is the testing? I, I've seen a lot of organizations where they're using one AI to generate the code and another AI to test the code, and there's no human in the loop in that equation.

Right? And, and though I get the, the logic of it, right, this AI looks at it differently than that AI or what have you. It still just doesn't sit well with me, to tell you the truth.

I don't, I think the final validation of security should not be left to anything that routinely hallucinates. Mm-hmm. I I get it.

I get it. But you know, to be fair, the flip side of this is, um, the pressure from above. Even use ai, experiment with ai.

Use ai, get your code out, keep up, move more, publish more, right? Push more. It's, it, it makes it harder and harder to kinda do the right thing, right?

It can be, I mean, I think AI is a useful tool that makes sense for developers to leverage, but like any tool like not indiscriminately, there, there're situations where it's the right tool and situations where it's the wrong tool and understanding the difference is what sort of separates the novice from the master. Fair enough. Jens, Jens, if you don't mind, I'd like to go back to, to the report though.

'cause we, we went off, you and I talking about what, you know, we find it interesting. What else in the report though might be of interest to our listeners watchers? This is the first time we've actually broken out the security sort of by model.

The first one was just sort of the aggregate and we thought it was important to do this time is we did actually for the first time see a pretty significant difference in a few of the, the models. So if security is your top priority, I think the open A models AI reasoning models are probably the, the best in the business right now. And I think if that's where your prioritizing, if you want to have an LM that's reviewing code for security, that would probably be a good choice as opposed to some of the other models that don't even hit 50%.

Hmm, absolutely. Um, the models that don't even hit 50%, do you think that's just a way point on the way to getting better for them? Or do you think those are just inferior models when it comes to security?

I mean, for instance, one of the models that hit 49% was Philanthropics, Claude Opus four one, which just came out, which Considered a good a good coding machine. It's a, it's an excellent model for writing code, but not necessarily for writing the most secure code. And I think when we look at some of these things over time, we don't necessarily see the direction is consistently linear in a positive direction.

Um, I think like Opus four was 50%, four, one was 49%, four five is back to 50%. It's sort of flats. And I think if you don't invest in making that better, it's probably not just going to improve because you've built another model.

You know, a lesson I learned early on in my security career was they'll, they'll make it better when customers demand it to be better. Hmm. And I, I quite frankly, again, I think that's part of the problem here.

Customers want to use these things to generate code. They know the code isn't of the highest security quality. They don't trust it.

A good chip percentage of them, however, they still use it and they're still doing it. So they're not demanding better security. I mean, it's not necessarily worse than what they get from their human developers now.

So I, I understand why they feel the way they do, but I also, and I'm, I'm hopeful because I think the first focus for a lot of these large language models with code generation is getting it to generate the right code correctly and consistently. And I think we're just getting to that point now. And I think once you get to that point, then you can start asking for more.

'cause having code that's more secure but still doesn't function isn't really a win. Agreed. I I, I, I would have to agree with you.

Yeah. Jens, anything in the report or findings that you kinda say, geez, that wasn't on the bingo card? Um, I think that the GPT model just being way out of band in the highest growths we've ever seen was probably the biggest thing.

I think it's interesting seeing that we're seeing some overall improvement in of the language. net seem to be steadily improving with respect to security and less so with some of the other languages that we've looked at. And then I think this model, we actually did a breakdown of reasoning models versus non reasoning models.

And the reasoning models seem to, on average do about five points better than the non reasoning models. That's significant. That's significant.

It is significant, yes. And I think a lot of the AI coding assistance are leaning more towards reasoning models over time. I, I, I do agree with you on that as well.

Um, correlation between how good it is on code versus how good it is on making secure code. Mm-hmm. Right.

Like you, we brought up the anthropic model, right? A lot of developers swear by anthropic for code according to this. They may swear by it, but that doesn't mean it's very secure code.

Yeah. And, and again, I mean the, the, the delta here between the most secure code being, uh, generated and, you know, middle of the pack is, is what I mean the, the big delta's 70 to 50, but I imagine a lot, a lot of 'em are a lot closer. So one of the, the evaluations we is, we looked at how good a job they did at producing syntactically correct code, like just what they produce, compile, and October of 2024, it was roughly 50% of the code they generated would compile successfully and the rest of it wouldn't.

5% for creating syntactically correct code, which they can now do extremely consistently, but only four or 5% better with respect to security. So I I, I've heard this before, you know, and my, my take on it is that narrow use case, whatever you want to call it, of making syntax correct, is something that they cracked the milk, pun intended, they cracked the code on and we, and figured it out pretty well. Hence the, the really good.

But That, I think now that we're at almost a hundred percent, I think people are gonna start asking questions like, okay, now it's intact the correct, it's doing what we want, but is it secure? And I think now they got their first ask, which is, does it work at all? Now it's like, okay, now how do we make it work better?

So I, I'm hopeful that we're gonna see them investing more in the security side of things. So the code they produce is a little more trustworthy. When will you be testing it again?

Uh, it's a bit of wait and see. We sort of wait till another whole swath of frontier models come out again. And I think the latest batch is planning to come out early next year.

So as soon as they do, like we run the, the report regularly, we just generally don't publish the results until we have enough additional data that actually says something new. 'cause there's not much fun for me to get on here with an interview and say it looks exactly the same as last time, The same speed ahead. You know what, speaking of the report though, Jens, where can people get that?

com, we've got a link to the report on the front page and uh, it's a very interesting read. I think, uh, the original report and the extension with the October updates are important reading for anybody that's serious about security and, you know, amongst the 90% that are using AD to help them generate their code. I love it.

Alright. Yeah, we're about outta time here. I want to thank you for coming on as always.

I'm sure we'll be seeing you soon. Maybe R-S-R-S-A is only like three, four months away now. Yeah.

Always a pleasure. Alan. Hopefully I'll run into you at RSA.

Yes. Well, we'll, you know, don't leave it to chance. We'll make it happen.

I someone who sounds Good With us. Yeah. And say hello to everyone at Veracode.

Keep up the great work on this. Hey, we are making progress. I'll, I'll say that.

Yes. And that Things are getting better. I like it.

All right, thanks a and take care. Okay. Jen's Westling Veracode here on text tv.

We're gonna take a break. We'll be right back. Hey guys, thanks for the throw.

We're here with Jonathan Edmonds, who's managing director for Key Data Cyber. And we're having a little chat about, well, the impact AI is having on fraud during the holiday season because it looks like there's been a sharp uptake, which means the bad guys are getting smarter about how to use the latest and greatest tech. Jonathan, welcome to show.

Hey, well thank you. I you for having me. So what is the impact of all of this?

There's always been fraud as long as anybody can remember, and it probably got worse with the advent of online, but now we're entering the age of ai. So what are you seeing? Yeah, you know, it's, uh, it's really twofold.

One, it's, uh, from the consumer side, but uh, also from the retail side, right? Uh, you have, uh, you have more and more, uh, people util utilizing AI to, to gain access. And if you're looking at it from a consumer side, you've always had kind of phishing attacks and all those things to gain access into accounts.

But, uh, you know, what you're seeing is a lot of, uh, deep fakes or even, uh, trying to steal people's phones and, and then gaining, or at least a sim card and gaining access, uh, into, into people's accounts, and then therefore going out and purchasing, uh, mass amounts of, of product, um, through their accounts. And on the retail side, uh, you have organizations that are basically going out and, uh, also doing deep fakes and, and acting as if they are executives or leadership and, and asking them to do things that they normally wouldn't do. Uh, and you're seeing people kind of take advantage, uh, of those things as well.

So coming into the store, say, Hey, I was, uh, was brought to you by, uh, an executive, they told me to do this. Here's, uh, here's a message. Um, and or, uh, doing it through, through e-commerce sites as well.

So definitely see a mass, uh, in uptake of kind of ai, generally ai, but also just regular kind of, uh, fraud in general. So, mm-hmm. Is there something that retailers are doing to thwart these attacks?

Or are they just kind of sucking it up as, you know, the cost of doing business? Uh, I think it's a combination of the two, right? I think anytime you're looking at security, there's always a risk mitigation aspect of it.

It's okay, if I don't do a, B, C, I may have to pay, you know, 1, 2, 3, or something along those lines. And, uh, sometimes you have people who are a little bit, uh, you know, on the side of, Hey, I I I'm worth that risk, right? I'm not a big name, so I'll take that risk.

Uh, but then you have other organizations that are, you know, in the forefront of it that are building their own AI solutions to com combat other, other bots. Uh, but you're also seeing them implement more security, uh, structure. So sometimes it can be perceived as adding more friction, but uh, you need to add friction sometimes to make sure that people are, are protected.

So you're seeing a combination of the two. I would say the majority of organizations today are not utilizing AI enough, um, or even utilizing solutions enough to, to protect their customers and organization. Mm-hmm.

And how would that work and what's involved in putting that together? Because to your point, the alternative seems to be, you know, I gotta put in, you know, five, six factor authentication and nobody will put up with that and they barely tolerate two as it is. So what is AI gonna be able to do for folks who in the retail side to kinda identify these issues in a way that customers won't reject?

Yeah, it's a great question. That's always kind of, I'll say it's the $50 million question everybody's trying to figure out. And the reality is, is data.

You've gotta be able to, to determine trends, you've gotta be able to identify, uh, if somebody is in a geo location that they haven't been before, okay, maybe that's a reason that they, you should prompt them for multi-factor authentication as an example. Uh, or maybe they're purchasing things that just don't seem, um, you know, kind of the norm, uh, of, of what they usually do. So really just building kind of, uh, kind of patterns around a specific consumer.

But even taking to a bigger scale and say, well, if I'm going to this store and all of a sudden I see an uptick of a bunch of people going in and spending a lot of money in the store, there might be something that we may wanna look at and, and use that as a, as a trigger to, um, uh, to authenticate, to revalidate that the user is who they are. Um, but it's, it goes the same thing, even from a product perspective. If you have, I mean, bracketing is, is pretty popular where you go and you buy every single size of a, of a, of a, of a shirt as an example, and the ones that don't sell, you try to return, you return them, right?

So it may not not seem like a big deal to a lot of people that costs a lot of money for, for companies 'cause they have to produce those products. And then all of a sudden you've get, you get overwhelmed with, uh, with returns. Uh, so, so yeah, I mean it's, it's, it's a combination of a lot of things, but to me it's really about, you know, identifying the consumers, identifying trends at, at the store level, uh, and then building policies around that.

There's no perfect solution today, but, uh, you know, starting it out in a way that, you know, you consume the data that's out there and available is probably the best way to do it. Mm-hmm. Do you think consumers are getting a little more wary of where they're shopping and they're kind of narrowing their shopping to places what they perceive have, uh, better security and it's gonna be tougher for smaller companies that don't have that level of security to kinda give that level of assurance?

So over time, are we gonna see some changes in behavior because, well, I may not ultimately lose money. I got other things to do in my life besides sorting out all the fraud that somebody perpetrated using my card or whatever it is. I would like to answer that question, yes, but the reality is no.

Uh, the fact is, is people don't like friction and they're gonna go with simple. They're gonna click on the link from Instagram, don't care where it's from, they click, oh, I really like that. Let me go there.

They don't check it. They, they don't even look to see if it's a a secure website or not, right? They, they just click on links.

So generally I would say no. The answer, the consumers are not educating themselves. They're, I think we do a poor job in, in cybersecurity to educate, uh, people in general.

Uh, but we definitely do a bad job, uh, on the consumer side. So I would say no. But there are, I mean, there is, uh, I would say maybe a little bit of an uptick of people who have gone through, uh, the burden of trying to fix credit or fix, uh, some of these things that happen whenever, whenever your identity is stolen.

So generally, no, but maybe a little bit, Are the, the banks and the credit card companies getting tougher about all this, and are they gonna put in the measures that will ultimately force us all to behave better? Again, I'll answer the same way. I would love to say yes, but again, they, they're, they're money hungry, they're coin operated, so they want, they want access to as much money as possible.

Um, you know, they are putting, they are putting some policies in place and they're making some things a little bit more difficult. Uh, but you have now you've got, you know, buy now pay later, right? So that to me is a very difficult thing for, uh, a credit card company to be able to enforce anything.

Anybody can go buy it now. And then if you don't really validate that that person is who they are, you're never gonna get paid. You're never gonna get paid later.

So there are things that are, are happening that, that actually I think cause uh, more attacks, um, because, you know, they want, they want people to spend more money. Um, but yes, I mean, generally everybody's, every company is trying to put more, uh, more things in place. But I go back to my earlier statement is, customers want less friction, right?

So, uh, there's a reason why Amazon does really well is because they, they have everything you want. You don't have to go to 50 different places to find it. Um, and it's pretty easy to sign in to validate who you are.

Unfortunately, a lot of organizations haven't caught up to that and therefore they're, they're struggling. So who is waking up in the morning and saying, we gotta do something about this? 'cause it kind of sounds like everybody involved is pretty much shrugging.

Uh, it's the, it's, it's the finance department. Uh, primarily it's the, I would say the senior executives, the CEOs, the, uh, CFOs and probably the ones who are, you know, awake at night. 'cause they're the ones who are, uh, tied to both top line and bottom line.

Um, you know, obviously the security teams are always, that's their job is to, is to enforce security. But tho I would say those are the three people who are, uh, probably not sleeping this holiday season. Um, and I mean, you, the people who are only focused on revenue, they're great.

They're like, Hey, we're look at all the revenue we're bringing in. But the people who are really focused on, you know, signing that, signing over those invoices or having to pay the insurance premiums or, uh, you know, have to deal with, uh, any kind of litigation, those are the people who are definitely not sleeping. And to your point about that, does all this fraud eventually just get hidden, or the cost of it gets hidden in the price of the goods?

Because, well, the retailers, ultimately we will decide that, you know, we're gonna pass that cost along and we'll just share it among all the buyers. And, um, you know, the next thing you know, we're kind of all paying for it. I, I think so, um, I think if you look at just the history of retail stores, it's always been kind of baked into the cost of the goods at the store.

Uh, there's always loss prevention. There's always kind of, Hey, we, we just assume we're gonna lose 2% of our, of our stock. That number is gonna go up, and that's definitely gonna be a cost that is passed on the, to the consumer.

Um, is definitely, yeah. So definitely gonna be passed on is gonna cause more and more strife. And I think then you'll start seeing the question of why am my, why are my prices going up?

And then you'll get, well, it's because you're not doing 50 things that we're asking you to do to protect, uh, to protect us. And so, yeah, I see there probably like a little, a bit of a curve where there will be a spike of, uh, this being pushed onto to the consumer, and then there'll be other avenues maybe through like blockchain or other ways that will, that'll kind of balance out that, that increase. Mm-hmm.

What are law enforcement folks doing about any and all of this? I mean, from their perspective, I'm sure they would like to catch a few of these people, but it seems like a, it might be pretty hard and B um, I guess it has to reach some level of threshold before they're gonna go after it, right? Yeah, I think, you know, you look at, like, I'll get Target as an example, but Target is always kind of, I thought very interesting is they've always been, Hey, you can, you can steal up to X amount.

And they didn't really care, but they would, they would track you and they would learn about you, and they'd say they see this person come in over time. And, and when it, when it got to a level, and I don't know if they still do this, but if it gets to a level that is a substantial, then they start to call the police. And I, and I think you're starting to see that the law enforcement do that as well.

It's, it's all about, listen, we understand it's gonna happen. We're gonna track people, and once it starts building to a certain level, um, of fraud or, or theft, then, then we'll come after you. Right?

And, and they're trying to, they're trying to figure out the networks. 'cause it's not just one person, it's just not 14-year-old in the, in the basement trying to steal. It's, I mean, there, there are organizations that have a massive amount of people who are going out and doing this, and I think they're trying to find ways that they can make it difficult for them, uh, but also how to be cracked out on in those larger organizations.

So yeah, you're looking at like, kind of like the FBI, like, they're definitely investing a lot in, in fraud prevention and detection, um, versus, you know, kind of local and local law enforcement, not really as much. Mm-hmm. So just how organized are these rings?

Because, you know, we've seen and heard about them over the years, but, uh, are they global now? I mean, how far does this go? Yeah, I mean, you see, you see both, you see a lot of regional and, and global, I would say they're becoming more global.

They're very sophisticated, and they are unified. I mean, you just look at things like, I remember the flash dances and stuff like that that happened on, uh, on Instagram and TikTok, but people, people would get together. They would go, they'd all use us and do something.

And the same thing happens in these type of attacks is, uh, they have, they have a plan, they have a strategy, they go after it, and then they disappear. And then they all come back up a few months later and they go do it again. Uh, so these, these type of strateg have been planned out for months, Right?

So what's your best advice to folks who are trying to thwart these attacks? What should they be doing that's within reasonable cost structure? Because I think a lot of people are trying to always balance the cost versus security equation, but, um, you know, what do you wish folks were doing?

Yeah, just a couple. I mean, be vigilant. I mean, just be aware of, of what you're clicking on.

The links are the, are are the enemy. Like if you get a link from anybody, just, just don't click on it. If, uh, you know, you get, you get a weird phone call or an email or something and they say, Hey, you know, Val, click on this link to validate or whatever.

I mean, go to the website, call, call them directly. It's much better to do that, especially around this time of year. It's safer.

Um, I mean, simple things, passwords, right? Uh, keeping very difficult passwords to, to remember. I mean, use a password manager if you have to, to, you know, have very sophisticated and difficult passwords.

Don't use the same password across websites. Um, yeah. And I would say, you know, if, if something just seems off, assume it's, it's better to go find another place to purchase something, uh, then where suddenly you feel uncomfortable, it's too good to be true.

It probably is. Hey folks, it's just like shopping in real life. If you went to some part of town where it felt a little sketchy, you might take your wallet, put it in your front pocket and do all kinds of things.

It'd be extra secure just in case. Well, turns out the internet and e-commerce and a lot of these sites, they're sketchy. So be careful out there.

Hey Jonathan, thanks for being on the show. Yeah, thanks for having me. Appreciate it.

All right. And back to you guys and the student. Hey everyone, it's Alan Hummel and we're back here live with our AWS Reinvent coverage.

Hope you've enjoyed what we've been putting up so far. Um, we've got our first non-text strong live guest here today with us. Let me introduce you to Gotham Rao before we get in.

And actually, it's a great segue to talk, well, what exactly is New Bird? New Bird ai? Uh, We, we build an agent at SRE.

Uh, what that is, is a site reliability engineer. Um, so basically, uh, you know, it, operations for enterprises have been, uh, a difficult, challenging, um, you know, uh, part of their operations for a very long time. And the modern, modern compute stack and infrastructure stack is cotton way complex.

I mean, anybody here at Reinvent, which is where we are right now, can can attest to that. So we live in an age of ai. What new Bird AI does is we build, using ai, we solve the problem of complex IT operations.

That's a great, what a great use case. I'm gonna come back to that in a second. But before we do, I always like to give our audience a sense of who they're listening to, who they're watching.

Sure. We didn't even talk about what your role at Newburg is, how you got here. Let, let's hear, let's hear the Gotham story.

How far back do you want to go? Well, you mentioned you're from Brooklyn. I am, yeah.

Which is, that's points in my book, right? Right. Yeah.

From Brooklyn too. But we could, we could probably skip ahead till after college or Something. Sure, yeah.

Let's do that. No, I, um, so I did grow up in Brooklyn. Um, my, uh, dad bought me my first computer when I was, um, barely a teenager.

So I got into computers early on. Short of the long story is I'm an engineer. Um, I am the CEO of New Bird ai, but I'm an engineer.

I write code. I'm passionate about, uh, ai. I am passionate about data science.

Um, and really, uh, the integration of those two things is what new bird AI does. Um, about me, I, uh, did my masters at, uh, the University of Pennsylvania. I grew up, um, uh, well, I grew up in Brooklyn.

Um, grew up in India for a little bit. Moved to Philadelphia for my, uh, graduate, uh, school after that, moved out to California. Um, this is now my fourth startup.

Um, I've been in enterprise software for most of my career. Um, what else can I tell you? I, you know what, I, if you, if you stopped right there, I think everyone will be out there saying, I wish my kid would grow up to be like that.

Right? Think you want that. So, uh, well, it is good and bad with everything, but you know what's interesting?

You were almost apologetic about being a CEO. Hey, I'm not just a CEO, I'm an engineer. I could go God done it.

Yeah. Um, But, you know, there's something, so I, I've done four or five startups myself, venture back startups, and I think there's something about being a founder, co-founder of startups that transcends, whether you're an engineer or a sales guy or a business person or what have you, you, it's passion. You gotta have passion for what, what it is you're doing.

You don't just start a company to start a company. You start a company because I see a problem. I see something that the market hasn't addressed adequately yet.

And I think we could build a better mouse trapp. I think we could build, we could do something that's gonna make someone's life somewhere easier. And that passion, I think, is what separates the engineer from the founder, from the CEOI Think.

So I think, um, you know, look, it, it took me a couple of times to figure this out and, um, first of all, uh, you have to do what you're, uh, passionate about. And, um, look, what we're, what we're doing here at Newburg ai, and, and I'm, I'm here to talk about anything you want to talk about, but specifically, uh, in this company, it's something that I could use, like I've, um, you know, for your audience out there, um, WW what, what is an SRE or W why are it operations hard? Well, what, what the, the truth behind the matter is that, um, you know, you guys use, um, you know, Twitter or Instagram and the complexity of the software and, uh, the hardware and the infrastructure that goes behind all of this, not just delivering the, the content to you, but the AI behind it.

These are very complex systems, and when things break, engineers have to be up. And you're looking at what's called telemetry logs, uh, metrics traces, uh, and you're trying to figure out very com, a, a, a a, a solution to a very complex problem. The short of the long story is I've done that.

I've been up at two in the morning, um, beating my head against the computer and trying to figure out how to fix these things. So, um, I'm passionate about what we're building because it solves that problem. It solves a problem for me, that's the most important thing.

Like, if, if I can build something where I'm happy with it, and I'm like, whoa, this is awesome, then, you know, I hope that there are other people that will benefit from whatever it is that we're building. So, yeah, it's fun building this, um, you're right that I am passionate that I'm an engineer because, um, you know, I'm having fun building what we're doing. And it's, it's, it's like when you're tinkering with something and your hands are dirty and you're figuring out what it looks like and everyday changes, it's a, it's a fun journey.

Absolutely. You know, that, that's not an uncommon fact pattern. Right?

And I call it, I can't be the only one, I can't be the only one with this problem, right? If I could solve this problem for myself, I could solve this problem for everyone who has this problem. And there's gotta be a business wrapped around that, then.

That's right. Um, you know, you, you solve a problem. And if there are enough people that, um, align with the problem you're solving, then there's definitely a business around it.

And, and, you know, the, um, look in, in what we're, and AI is so, um, um, prevalent right now with, or the, and so many startups here at, at least at Reinvent, that are focusing on taking Gen AI and applying it to a very specific domain. Now, what it, what, once you start solving the problem, you start realizing that there are other people also solving the same problem. Okay?

So there's competition out there. And then, um, how do you differentiate yourself from the competition? Well, not two people aren't gonna solve the problem the same way.

The solution is going to look different, and it'll align with people that are, that nuanced in how they want the differentiation to look like. And so, um, then that drives your passion, like, am I, uh, subscribing to a very specific set of customers that want the solution delivered in this, this, in this kind of way? And you engineer toward that, and, um, hopefully you acquire customers that are, uh, aligned with your, uh, mentality and how you wanna solve the problem.

So you're still having fun doing it. It's, uh, it's been a while ride, uh, for the past two years. And, you know, just like, um, looking at, um, talking to all the customers at Reinvent just makes you, um, that much more energized.

I love it. I love it. Let's talk a little, let you know, we, we were up here talking about it from a business point of view, but let's dive into agen ai, AI for SREs.

Sure. Right. com, another one of our sites, SREs have become key members in these communities, right?

The role of the SRE, I think is more clearly defined now than it's ever been. Right? People don't question, is it, is it real?

Is it, you know, what exactly is it? My question to you though is, are we ready for an agent? Is it to replace the SRE to supplement the SRE, augment, Augment, augment.

It's, uh, it's you, you're not gonna replace humans. And people asked us all the time, like, is AI here? There's when the industrial revolution happened.

Mm-hmm. Right? And I'm sure I wasn't alive back then, but, you know, I'm sure people were, were worried about what will happen to our jobs.

And look, we just learned to live with technology. And this is not no different. I mean, it's faster.

It's, um, um, the change of, um, yeah, innovation, the curve is probably a lot more steeper, but ultimately it's something that we will live with. It's something that we augment ourselves with. And, and, and so actually before I answer that question, let's talk about what an agent itself is, right?

Good. Um, Gen ai, you know, could past couple of years, everybody, you know, Chad, GPT came around and people were wowed by it. And how did, uh, AI make gen AI make its way into the enterprise?

And by the way, let's also acknowledge that AI ops itself has been around for a long time, that there's nothing new with AI ops. So why is this wave different? What you asked the question about, um, agent systems, or you made a point about agent systems.

So let's talk about what an agentic system is first. The way the difference between traditional AI and gen AI is that machine learning based, traditional AI based on machine learning is, is engineered for more probabilistic outcomes or known outcomes. So your engine, for an, an example, when you're going to design a system for, um, uh, credit card, uh, fraud detection, right?

You have well-known patterns and you'll engineer for those well-known patterns, gen, ai, the, the, um, the, the number of variables or the space, the complexity, the number of parameters is so large that there's a little bit of uncertainty in what, what its outcome will be. Keeping that in mind, what an agentic system is the following. So we now, we know we have these very large language models, which is the brain, which is the whole core behind gen ai.

How did, how do you use that in enterprise systems? Well, gen AI on its own is generic, has been trained on so much web data out there, not necessarily applicable to enterprise information. So the way it made its way into enterprises is people started with this thing called rag retrieval, augmented generation.

Sure. So I'm gonna provide my enterprise content and let's see what the AI can, um, you know, determine out of this, summarize it, create marketing documents. That's not an agentic system.

That's Re an agent. That's reg. So now, and this has really taken off over the past, uh, 12 months, maybe 18 months, what an agentic system is understanding that these models have a lot of knowledge in them.

You can't just take content and throw it at it for a certain class of problems. An example, we're talking about SREs, right? Site reliability.

Engineering relies on complex telemetry, enterprise data, enterprise application data consists of a lot of logs, a lot of metrics. These are time series data traces, which are very complex graphs to short of this long story is there's just too much information to apply rag. You can't take this information, throw it at the l LM and say, help me.

Mm-hmm. So what is an agent system in ag? And I'll get to how this helps SREs not, not replace, Okay.

I'm letting you run with it. Uh, an age agent system allows the LLMs to figure out what information they need to a access. It's the converse.

Instead of you throwing data at the L LM and saying, help me, you're asking the LLM, you tell me what information you need. I have this ailment, my website is crashing, or this feature is not working. LLM, apparently you have been trained on so many different IT scenarios.

You tell me what to go access. And that's called context engineering. Okay?

And that's where companies like New Bird ai and there are other people solving this problem come in. The point here is that we now believe that these LLMs are so smart, have so much information in them that now the problem that needs to be solved is not making them smarter, but it's about context engineering, garbage and garbage out. If you ask, you can ask an LLM any question and it all come up, always come up with an answer.

And that's the problem. You can't do that with IT systems. You have to be surgically accurate, a hundred percent.

Uh, identifying the problem relies on the right context. So how does this complement SREs? SREs are sitting there under the gun.

They have a problem to solve. If they have a good context engineering solution, they can ask the LLM, I have this problem and here's my context engineering platform. It will help you find the needle in the haystack, and then you help me solve the problem.

This will help make the SREs and the engineer's life a lot easier. They can solve more problems in, in shorter amount of time. And more importantly, they can focus on not firefighting, but innovating and building better product and solutions, which is the bottom line for an enterprise.

Love it. You gave us a whole bunch of stuff here. You can, you guys need to go back and re-listen to this after this.

You watch this, it'll be up in a couple of days because there's so much, I don't want to use the word bedrock 'cause that's a big word over here. Yeah. But there's so much foundational information here between what is an agent agentic ai, what is rag, what are all these things?

Let me pivot a little bit con 'cause we're running on time. Um, AWS announced a whole bunch of agents, or they mm-hmm. You know, they announced three real key agents.

Yeah. One of them though is they're calling it a DevOps agent. In my mind.

I, I don't know if it's really a DevOps agent, but it, it seems to do some of the SRE kind of stuff. Yep. Can be competitive, generally the AWS products, you know, or the 80 20 rule, right?

They're 80% of the functionality, 75% of the functionality. How do you view it? Is it, Hey, use the AWS tool and then when you find out what's missing in your life, come to us, Or No, no, no.

Um, look, um, I, I tell my, uh, customers the following, uh, a story too. We were talking about agent systems. I, uh, I probably rambled on about what it takes to build an agent and context engineering and, and this and that.

Um, a follow up to that story is dealing with an agentic system is different from purchasing software. Um, where in software you kind of have an expectation of what it does, and it's either or. You're either using software from vendor, vendor A or vendor B to solve a problem.

But in dealing with agentic systems, you should approach it differently. It's how you hire people. It is part of your workforce.

It is rooted from a deep, um, um, uh, you know, machine learning. But, um, a, a deep understanding of a variety of different, uh, problems that humans have solved. So what, what do I mean by this?

When you are hiring an employee, do you hire the same type of person again and again? Or do you hire different kind of people? It's about diversity, because you hope that when you ha hire different kind of people, they ha they come with different ideas, different backgrounds, um, different ways of solving a problem.

And so overall, your enterprise is richer. Why am I saying this? I believe the same thing will happen with agentic systems.

You are not going to settle on just one agent. There will be agent diversity agents will work with each other. There are already projects around eight oh, a agent to agent protocols and how agents can access external systems through things like MCP.

So it's great that everybody's has their own agent. And these will solve very nuanced problems. And there potentially, there'll be a framework that unifies all of these things.

And we don't know what that will look like. And I think as the industry matures, we're gonna figure this out. That's my way of answering your question of which is, um, AWS will have an agent, Microsoft announced its agent about, I don't know, uh, eight, nine months ago at their conference call.

Um, you know, the Azure SRE agent? Yes. Datadog, which is a huge partner of ours, has their own agent.

And that's great. And these agents should work with each other. We already have customers that deploy multiple agents.

We, um, did, um, uh, at Microsoft Ignite a couple of, um, uh, weeks ago, we demonstrated how, uh, our agent, which is known as Hawkeye mm-hmm. That's our, uh, agentic, SRE solution, can, um, you know, uh, cooperate with the GitHub copilot agent and the SRE agent. And all these three things together solve close the loop.

What loop is that developers push in code. Invariably, things can break. Our agent can pick it up, uh, at the operations end and saying, I'm seeing this problem submit.

Um, a, um, uh, a request to the GitHub co-pilot agent to go in and write some code to fix it goes all the way back to the developer to say, this looks good, and I'll accept a fix. And that loop shortens how long it would've taken to fix that problem, end to end. So, um, having, um, an enterprise purchasing multiple agents, or working, not purchasing, working with multiple agents and tying these things together will be the future.

Excellent. You know what, we, we didn't mention the URL, how people can contact. Yeah.

ai, N-E-U-B-I-R-D. Okay. Dot ai.

And our agent is known as Hawkeye. Um, we've, um, uh, we G eight actually last year at, uh, reinvent. We have, um, a lot of customers that have been using our systems now.

Um, we have, uh, results on our website that you could go look at in terms of how we have reduced what's called the meantime to incident resolution. Um, in some cases, 90%, uh, time savings. Wow.

And what does that mean? Well, it's, uh, bandwidth and time that the enterprise can get back to work on, um, what they actually want to do. Right.

Building their core products and services as opposed to firefighting. I love it. Gotham, we're outta time, man.

But thank you so much. You know what? Thank you.

Appreciate it. Again, I'm gonna tell you guys something if you want, when this is up on Techstrong tv or the YouTube channel or the OTT channel, go back and listen to what he said. Again, it's a great primer for some, you know, basic concepts that we all bandy about these words, and you may not truly understand what they are.

So go check that out. Thank you for that. Thank you so much.

We're live here at Reinvent. We'll be back in a little bit with our next guest. Stay tuned.

Hey, everyone. We're back here on, uh, text Drunk TV with our AWS Reinvent coverage. You know, as I mentioned, Nick Patience had a run out, so we had him really emphasize the beginning of our conversation.

But I wanna continue our conversation with Mitch. Um, so Mitch, it's good to have you back. You know, you've been podcasting up a storm too, right?

You did something with Brad Shiman. Yes, I did. Agents of Dev, our new Podcast.

I love that one. Yes. You know, it's all about the agents, human agents, computer agents.

Absolutely. Digital coworkers. Yeah.

And then we did a, uh, a a, uh, cyber still still cyber after all these years, baby. That was a, that was a throwback. But you, you spent the day today, I know in a bunch of meetings, keynotes, listening in.

Most of our audience here probably wasn't here in person to watch this. Mm-hmm. Yeah.

What, what could you give them to take home, Mitch? You know, I think, um, I always look at where do things start, because how's this going to set up AWS for the next 12 months? I think Nick made the point about it's not an end of the year conference only for AWS anymore, but what they did is this is gonna help them for three, six months, because things are moving that fast across all the vendor environments.

And they started the conversation with, at least in the pre-brief, with the analyst who said, we're gonna be talking about up here in the stack. We're not just gonna be talking about down here. And they actually started the analyst briefing really talking about kinda what's happening at the user or the user of AWS services layer.

Mm-hmm. And of course, um, Brad teed things up nicely with, with Kiro for me, about that as the, uh, as the AI IDE that AWS has launched back, I guess midsummer and they've standardized internally on, now they've, they've specified this as a spec driven IDE I'm not sure that's a sustainable advantage for, for them, but they'll figure out kind of what their place is with that. The big push was around let's really take agents to the next level, which meant agents that can scale at very large scale.

They talked about billions of agents at some point. Mm-hmm. They talked about long running agents, agents that do work over days Yes.

As well as maybe even weeks or longer. Uh, but they also subdivided that into some new agents that they launched. We talked about the AWS security agent.

I agree with you on the AWS DevOps agent. I think it's little dev large ops. Yeah.

It's an ops agent. Yeah. It's, it's really a DevOps agent.

Mm-hmm. Per se. But, you know, That's okay to start.

So back That up. It's a start Every Exactly. You gotta get there somewhere less specific about what they called the Kero autonomous agent, which is dev work, if you want to categorize that.

They didn't put too much feet on the bone there. So there's a lot to build from. I see this as kind of the scaffolding.

Think of it in development terms. Right. Here's the structure of what we're putting together for the agent framework.

You didn't hear, uh, Q Developer much today. No. Where last year that was huge, all About it.

And then Agent, agent Core came out midyear, uh mm-hmm. About, I guess, earlier this year. Uh, and it is now all about Agent Core as the development platform, if you will.

Of course, Q developer's still there as part of it, but I think that's the new framework that they're building the scaffolding from. And that's where we'll see a lot now, I think a lot of what was introduced here is table stakes. There wasn't anything that was like, oh, that's super innovative.

Nobody else is doing that. Maybe a little bit with Forge about creating your own, your models using fair enough AWS's models with your data. Okay.

That's interesting. But that's okay. This is not a world where you're gonna pull everybody over with one announcement and suddenly the industry goes, oh my God, what are we gonna do now?

This is, But I, I think this is A marathon race where we're watching Absolutely. Quarter mile by quarter mile as this thing's unfold Quarter by quarter is, is a good way of thinking about it. And, and I think it also goes to what we were talking about earlier, and I, and Daniel Newman mentioned this in my interview with him earlier, which is this was a bit of a, of AWS plane catch up.

Oh, it's major catch up. Yeah. Yeah.

I mean, it's not, it's not unknown that people think AWS is behind to the, the other hyperscalers, if you will. And now you can argue whether they're, they have strength. I mean, look, they're a massive company.

They're a huge Customer. They're still an 800 pound gorilla. And they still look, you know, I'm a football fan and, but any sports fan, I'd rather I, my team is the team that's ahead than the team that has to play catch up.

Mm-hmm. Right. It's always nice to be ahead However, or it could be the Broncos and win by one point when the, you know, the guy DOKs the the kick Yeah.

Or the two point track. Right. You know, these Colorado people, they have one good season and they don't, they just kind of work it into every Conversation.

You're gotta be hearing about this for the next 10 years. It doesn't matter how it Turns out. Come on, Mitch.

You want both? Say it with me. You want both.

But, um, that being said, that being, you had to do it three on a week when the steel is like spun the place Out. I kick a guy when he is down. That's what I, that's what, yeah, that's what it is.

But let, let's, let's go back though and, and, and talk a little bit on, on on this agent stuff though, Mitch, you know, it's interesting you said that he opened up the analyst Reefing by saying, we're gonna go at the top of the stack here and talk about what it's like to be, uh, an AWS customer, AWS user. And then everything was about developers. Mm-hmm.

Now it's true. Developers were the lifeblood of AWS early on. Right?

That's, They, that's what built that in the credit card. Right. They, they whipped out their credit.

I was just gonna say, they whipped out their credit card. They spun up a few instances. Mm-hmm.

And off we went shadow it at its best. Is the developer still the engine that drives AWS or is it, is it the infrastructure play? Is it the security folks?

Is it the platform? Is it, you know, ops, SRE and all, all of the rest? Or is still AWS laser focused on the developer?

Well, I learned early in my career, you could tell who drew the diagram by what's at the center. Brad Shiman draws the diagram. There's gonna be a database of data, fabric data, you know something, right?

Mm-hmm. Nick, it's gonna be AI models, it's gonna be AI chips, Mitch, it's gonna be all about the developer. And now you create software, right?

Yep. So, you know, AI is supposed to be the, the death of the developer. And I immediately said, yeah, okay.

You don't know how software is developed if that's what you really think is gonna happen. Developers at the tip of the spear of ai, that's where, that's where the first people are adopting the latest innovations in most cases. And what we're seeing is it, it isn't eliminating the developer.

It's actually emphasizing what developers do because you see AI moving to the command line interface. Oh, I thought, I thought we weren't gonna have to do that anymore. No.

Guess what? We're putting AI at that level. So the develop level, so developers are productive.

We're building in fabric, we're building in control planes for, to be able to manage and put in security guardrails and do things like that. You look at even an IBM tool with their orchestrator, here's the end user interface, the non-pro developer who's the pro developer for the same tool. It's developed and orchestrate agents.

So someone still orchestrates, creates, thinks, drives, innovates. What AI is doing today. It's still very much, you know, developers, I think not in the loop.

I think it's developers driving the loop whether we do all the work, I think we're doing less of The work. Yeah. It, it goes back to something I wrote and I think I did some videos on, you know, to paraphrase, Billy Joel AI isn't starting the fire.

No, I didn't know he said that really Well. Something about starting a fire, wasn't it? Did I tell you Billy Joe lives across the intercoastal from the House?

You have told me that about four times. Yes. Yeah.

Every time we go out there, we wave. We wave. We invited him over the house every Time I ran in your boat as we go by.

Yeah. Well, that's why I'm playing the Billy Joe music really loud. I'm hoping you'll hear me.

But someone started the fire. Mm-hmm. Right?

And I, and I, you know, and then it goes back to AI is is the a tool, it's a developer tool, it's an ops tool. It's a security person tool. But the spark is still the spark of human creation.

And I, I don't think that, not in my work lifetime. I don't think that's gonna change. I mean, look at it.

I mean, we, we we're, we're using tools to create videos that we would've never created before. Right. Just kind of laughable, fun things.

Yep. But useful things too. Um, the agents of Dev podcast I created with one tool, an idea of what we do for our logo.

Brad took that and took to the next level and like, yeah, that's what I want. Because that's very much sort of the comic style. Let's do that.

Mm-hmm. And so I couldn't have drawn that. I couldn't have, you know, even called up somebody and told them what I wanted.

'cause I didn't know what I wanted yet. But through using a creative, you know, mechanism, a tool like ai, look at what you can create. But I, I'll so think back to the first time you started using Photoshop.

And for me, I'll, I'll tell you this, that I'm, I'm gonna go back to 1995. I'm using Corre drawer. Mm.

If you remember, I do remember Corral Corral drawer and layers in corral drawer that allowed you to do Extrusions stuff. My carre draw didn't ended in r it's this corral draw, not drawer. Well, I'm from New York.

Oh, New York. Sometimes we add an R, sometimes we take off an RI can never tell. Can I have an R please?

Alex can never tell. But, um, that a turned what he guess what it says, can I buy a bow? Can I buy a bow?

But we don't have no stinking mouths. But anyway, but, but seriously, Mitchell, this has been, you know, this, this is not new to tech. This is, this is the way, as the Mandalorian would say.

Right? This is the way, This is the way, Like how you got that in there. That's right.

Like when I did that, I worked that right in smooth. Like some things never changed. No, you were, I Was still thinking the R thing, but okay.

Yeah. And I said the R in Mandalorian, You did Mandalorian. Yeah.

Because it's only ours at the end that we have problems with, with, we still can't figure out how it works. You know, I gotta sit here and take abuse from a Nebraska person. Could you believe this?

Um, I Lived in New York. I learned a thing, a few, few thing of two new I learned about you New Yorkers that, but let's get back to AWS reinvent, Mitch. Alright.

Um, you know, Brad said something that I didn't really realize in until he said it. We haven't heard a lot about data and databases and all of that stuff, and data storage. And, and that's always sort of a, no pun intended, but that's always a bedrock of AWS's.

It is, you know, Reinvent. I, I don't know if this is a reason, but we've had a succession of vendor announcements about data fabrics, AI data fabric, AI data fabric, you know, going from Oracle to, you know, you name it. I dunno if that's the reason why they didn't emphasize that this, I think it's more they have to catch, here's my point about developers are still ruling the world.

So software rules the world. They're catching up on the developer, right? They're trying to capture the development environment.

Again, I'm drawing the picture, so this is what I'm putting at the center. Okay. And, and here's why.

I think why Nick talk. Nick talked about the advantage of who has the advantage with models and, and what, what Google has, what Microsoft has isn't just the OpenAI relationship that they're now diversifying with. They've got the massive developer community, right?

Yeah. They have the 150 million developers who use GitHub, all of the IDE tools, all virtually all are based on open source, uh, versions of visual code, visual, yeah. Studio Code.

Um, that's what the competition for is to get those developed, to be creating not just the agents, but the tools, the fabric, the, the scaffolding, the, the next innovations about how we build and create agents. That's what, that's what they need. Now, we're gonna be doing that a lot faster with a lot, lot more innovation from the same group of people, but it's still about creating that software.

Yeah. And it makes the world go around. It's eating the world, but it already ate The world, ate the world, I guess, to see The universe.

Someone else said we, we were gonna be moving from software factories to intelligence engines. That's, that's, yeah, that's phrase, whatever that means. But yes.

Well, Satya said it. We're also don't gonna have sa SAS software. It's still around, but we will see what happens with that.

And I, but I'm not running Microsoft, so who, who knows. That's what I was just gonna say. You are Mitchell.

He Satya. Yeah, I think, I think that he carries a few more sticks in his bag than I do, but yeah. I got a mean three irons, so, Okay.

I played golf with you, Mitch, you are? No, I knew sja. I worked with sja.

You are no sja. I'm no three irons. Anyway, we've gotta get wrapping it up here, Mitch.

We've got two more days. You'll come on in and we'll do actually, we're doing Textron gang early tomorrow morning. Well, You know, as the, uh, this is the philosopher, the Terminator said, I'll be Back.

I'll be back. We'll be back tomorrow with even more from text from Textron on Textron tv from AWS reinvent. We hope you've enjoyed this discussion.

If you can tell, you know, we have a good, great time with the future of analysts. We enjoy, you know, talking shop, just, uh, enjoying, you know, it's a great time. You know, Mike Ard told me it's a Chinese proverb.

I thought it was an Irish proverb, but may you live in interesting times. These are certainly interesting times for now, though. That's gonna wrap it up.

This is Alan Schimmel for Text Drunk tv. Thanks for watching AWS made major announcements at Reinvent, including the Nova two family of models, inclu and Sonic for speech, as well as emerging AI factories featuring cranium custom silicon and S3 vector storage as they try to stake a claim in the AI infrastructure market. Companies like AWS are building deep infrastructure capabilities and platforms like SageMaker and Bedrock are delivering AI powered applications as our companies like Google and Microsoft.

IBM also made waves this week with their announced acquisition of Confluent, which is yet another modern AI application arrow in the quiver of Big Blue. And Nvidia told us a little bit more about how to use a model of mixture of experts designs to increase performance. All that and more on this episode of utilizing ai.

Welcome to utilizing ai, the podcast focused on practical applications of artificial intelligence from the Futurum group. Each episode brings together diverse perspectives to explore news and use cases in the ways in which AI is transforming enterprise IT and the industries it serves, serves. I'm your host, Stephen FoST, president of the Tech Field Aid business Unit here at the Futurum Group.

And before we get started with today's discussion, let's meet who's joining me on the panel today. Stephen, thanks for having me. Hi, everyone.

Brad Shiman. I am an analyst with the Futurum Group looking at data intelligence, analytics, and infrastructure. And I'm Nick Patience.

I'm the AI platform's practice lead at futurum fo focused, uh, solely on AI, really. And as mentioned, I'm Stephen FoST from the Tech Field Day Business unit. I, uh, host the AI Field Day events, and of course, uh, this podcast, I, I have to actually say I am here in at the New York Horological Society, which is hosting me.

And if you're wondering what that is, Google it, it's amazing. So let's dive right in. Um, I'm surrounded by antiquities here, but we've got a lot of cool, uh, modern ultra cool stuff happening here in the future.

Uh, we just got done with AWS Reinvent. Now we're not a news podcast and we don't wanna make it sound like we're, um, you know, kind of reporting the latest. But Nick and Brad, you all have, uh, had some time now to digest the announcements from Reinvent.

Um, perhaps, uh, you can regurgitate a little bit of that knowledge. Uh, I'm gonna not stretch this metaphor anymore. Um, Nick, thank you.

Uh, what was announced at Reinvent and what was interesting to you? So, yeah, we, we, um, Brad and I were there last week and, um, we wrote a, a note for a few from clients. I actually wrote the headline.

And the headline was, um, wrestling Back AI Leadership. And I think that kind of sums up where we are the end of 2025. Um, it's been, I guess, fairly well known that, uh, Google, at least from a narrative point of view, has kind of, you know, taken a bit of a lead and outta the three major hyperscalers.

Uh, and, and AWS is, you know, uses reinvent every year to do, to make its major announcements, obviously. Um, and, you know, this was, this was more about that. So I guess, you know, one of the things were the NOVA two family of models, um, including the, um, yeah, there was Nova, Nova, Nova two Pro for Advanced Reasoning, um, our Omni for, you know, long context, um, workloads.

And then Sonic, which was the speech to speech model, which was pretty impressive, that last one. Um, and 'cause Amazon, um, AWS rather has its contact center applications, which embeds, you know, which some well-known content center apps out there are built on. And I think that that's a, you know, it's obviously a real proving ground, um, for that kind of stuff.

So I think it's, I mean, I, I certainly believe, you know, it's fair to say that a Amazon wasn't at the cutting edge of models. Um, and Nova was released, announced that last year's reinvent. This is NOVA two.

Um, you know, and these weren't, you know, these are, these are strong models, but I think they, yeah, each one of these kind of, um, vendors are gonna find a, a niche. Um, there, I think the, the other interesting thing I thought was the, um, AI factories, which is essentially the, um, Amazon's infrastructure on the, in, in the client's data center. Um, and as part of that, they announced, um, you know, theran Ultra Servers, um, based on training Traum two, um, they also announced Traum three, the, the, the chip and then the roadmap for four.

Um, so it was very much, I think from my point of view anyway, it looked, um, you know, like Amazon on a AWS trying to own the kind of AI infrastructure narrative at least. Um, there's lots of other things and there's lots of other things I announced, so maybe Brad, you can, uh, talk about a couple of the others. Yeah, for me, um, it, you know, just building on what you're talking about with Nova, we all know that, you know, AWS is, is not going to outdo anthropic and even really Google, uh, with Gem, the Gemini family in terms of outright performance.

And I, I, I have to admit that when we were, when we first got to the show and they were talking about everything being Frontier Scale, you know, with a Capital F and air quotes around it, I, I was a little, you know, uh, taken aback and, and thought, nah, come on guys. You're, you're not really doing Frontier scale models, you're just ruining the name and it's really not true. Uh, they're actually building frontier scale models in terms of being multimodal as one checkbox and having super long context windows at a million tokens and other checkbox.

And, and I think that when you look at that, um, coupled with another announcement that came out, uh, and that is their, uh, what they called Amazon Nova Forge, which is this basically a facility instead of tools of what Amazon likes to call recipes, uh, that you can use to fine tune the Nova family of models, and not just in your basic, you know, here's the open weights models, good luck, you know, because we, we've been doing that for years now in terms of fine tuning and instruct tuning and, um, aligning models in the training during the training process and after that. And what they're doing is, is kind of unique in that they're productizing operationalizing those mechanisms, uh, those data science techniques to make them a lot more accessible to the enterprise marketplace. And they're opening up the Nova models to, to actually let you, um, sort of work with those as though they were your own.

So they're letting companies basically grab different checkpoints. Checkpoints are steps along the way to creating this final frontier model so that companies can more readily bring their own data to those models. And that's, that's pretty cool.

And very quickly, uh, I wanna say one other thing that I really caught my attention, and, uh, this is, again, me being dragged forward because, uh, you know, I, I've been, I'm a database guy and I, and I think database management systems are kind of important, and yet, uh, we are seeing a very sizable trend in the industry towards pushing down database functionality to the storage layer itself. And on Amazon, it's the AWS S3 layer, so your object storage, and they released, it's a ga their, um, AWS S3 vectors capability, which is, as you might imagine, a vector database. One that can do, you know, super huge indexing tasks and do so at scale with a comparatively very small cost footprint, which is quite impressive.

And one of the reasons why you might want to push that functionality down to the S3 layer. And if you couple that with an announcement they made earlier in the year around S3 tables, which is basically to bring structured data to the, you know, non-structured na nature of, of object storage, you've got yourself kind of a database, you know, disguised as an, as a file system. So it's, it's really interesting times.

So just picking up Brad on the Nova Forge thing, I think you're right. I think that is at the moment, unique. You could do that, couldn't you?

And like Google Model Garden and, and things like that, you could tie it all together and Microsoft, you could do the same thing and maybe IBM as well, but I think, um, it's, do you think, I mean, I, I kind of think this will be copied by the others by its direct rival pretty within, you know, I'll be surprised if it takes them a couple more than a couple of quarters, but they will do it. But for now, it's the only, it's the, it's the only sort of productized way of doing what they're trying to do, isn't it? Yeah.

Everything else has been data science, you know, open up a note Jupyter Notebook and grab your favorite PyTorch version and have at it. And that's not something that's accessible to every company. So I ask your opinion Experience, uh, yeah, Lemme lemme ask your opinion on the, um, the direction that A-A-A-W-S is going here and, and what we can learn from that.

I see, um, something of a similarity as well. You, you know, you mentioned Google, of course, uh, they're have a similar approach to this. Am I reading this wrong, or are they seeing, uh, you know, a companies like AWS and Google that have a huge infrastructure CapEx investment, are they seeing, uh, models as almost a loss leader to attract people to their environments or to enable, uh, the build out of this, uh, new industry, which I guess that could be a second point?

Or are they seeing these as products that they will make revenue from? I personally think it's the former, I I've long thought voice, or it's the former really, that the models are, are not models, are not, the application models are not the product. Um, they are very much a a means to an end.

They get the attention as, as Brad knows, I mean, every single time any major company releases a model, journalists will always be asking us about that. And sometimes I'm kind of wishing they'd ask us something about something else. Um, but yeah, I think they are very much a, a loss leader, and then you build the value around that, um, either below it with silicon or on top of it with the various tools and then applications.

I, I dunno what you think, Brett. Yeah, I feel the same way. You know, we were just talking about it before we came on air that, you know, whether you can believe this or not, but, uh, it is, has been studied and argued that, uh, perhaps, you know, the further we get with the transformer architecture, the less differentiation we're gonna see between, you know, frontier scale models and actually all, all scaled models simply because they all converge around the same patterns that they're trained on.

So Yeah. Is Nick's totally right? It's it's really just a part of the tool chain and it's, it's an important part, but it is just one.

And we're seeing, we have seen a lot of effort from all the vendors that we've been talking about in terms of trying to at least standardize on how you interact with those models from an API perspective, so that whatever tooling I'm using in that tool chain, you know, I I can bring in the model that's most cost performant, accurate, efficient for me. I mean, that's not to say at the moment, you know, Google, for instance, with Gemini, and that's the only place you can get hold of Gemini, uh, does, yeah. The model that is, um, does have, yeah, that, that kind of, that kind of appeal.

Um, and I think it, but I think it waxes and wanes. I mean, everybody's got a model garden, a model, um, switch switchboard. I like to think, you know, when to, you know, how, how my, I, how may I direct your prompt?

Um, do you want this model, that model, um, so that that ability to, to offer, you know, first and third party models usually isn't much of a differentiator, uh, unless your model is something, you know, quite spectacular. Um, and I think those, the, the kind of window for being spectacular could last days, maybe weeks, um, if you're, if you're lucky, and then it, then it slams shut. So what does this mean for the rest of the industry then, if, if, if models are loss leaders, what does this mean for companies like Anthropic and Open ai?

Um, and, and I have an idea. Um, I think that philanthropic and open AI are trying, they, they don't, so let's, let's back up again. So AWS and Google understand that infrastructure is, has always been where the money is, the revenue, the bulk of the revenue.

And I think that, um, a product like S3, for example, is a very sticky revenue magnet in a way that very few things in our industry are. And so it makes wonderful sense that Amazon would want companies to have more data in S3 to make S3 more AI friendly and attract people to using Amazon's infrastructure to build their applications. I think that open AI and Anthropic, it looks like what they're trying to do is instead build a platform.

You mentioned sort of that a app store kind of approach. I think that's kind of where those guys are going. Or am I missing that, uh, point as well?

And and are Amazon, is Amazon playing there too? Yeah, I, I think, yes, I think they are. I mean, open AI I think is trying to build, um, a completely vertically integrated technology company from chips through, you know, all the software layers we've talked about, um, to devices, um, and everything.

You know, you're working with Johnny, Ivan, all that kind of stuff and everything in between. And then Anthropic, you know, to a slightly less, um, lesser extent, but still, you know, they're building out tools. Um, so yeah, I think they're trying to build new software companies essentially.

Um, whether they, whether they will succeed or not, we will see and we'll be trying to track it very, uh, very closely. But I think that's, um, I think that's, that's what they're trying to do anyway. That's my opinion, Brad.

Yeah, I feel the same way in terms of, you know, these companies are, have been for quite some time trying to build beyond the model itself. And you look at how Anthropic Claude family has evolved in that regard, and you can see that what they've been focusing on is how do you build the attendant tooling around, you know, using their models. You know, how do you get from just a chat bot to a full-fledged agent tech process running in your line of business that's, you know, where they're building for.

And that means, as Nick just said, building out a platform. Everybody's got a platform. So, so that sounds more like a, um, like a, a a a Salesforce or a, a ServiceNow kind of play as opposed to more of a traditional enterprise tech kind of play.

And you asked about AWS are they kind of trying to do that? Yeah, I mean, SageMaker has what hundreds of thousands of, of, of customers and, um, has been around for a very long time. And Bedrock seems to be doing pretty well, um, as well.

So I think they, yeah, they all are doing it. It's, it's obviously leads to, it's, it's sticky, isn't it? It it leads to, you know, getting out of those platforms once you're, once you're in them, um, to that level of depth.

And S3 is the best example, isn't it? I mean, well, the first kind of cloud, well, the first cloud cloud storage product around. And, um, you know, when you talk to, when you talk privately to some, you know, people from some of these companies, it's like, anything we can do to get people to, you know, put more stuff on S3 or Google's case use BigQuery, which is a very successful product, um, is, is an absolute, um, is a, is a kind of mother load for them.

So I think, yeah, I think they're all, they're all trying it Microsoft's slightly interesting. Um, and yeah, maybe, maybe the canniest one in some ways. 'cause obviously outsourced the development of, its of the models to open AI and it's now sitting there as an investor in, on a, you know, potentially a, um, eventually one day some sort of great exit.

Um, meanwhile it's got obviously its own software, you know, stacks and franchises, if you will, which, which dominate, um, you know, you know, the, uh, the corporate world. So it's, uh, yeah, it all takes slightly different approaches, but I think, you know, largely they're all trying to do that as well. It's, it's funny, isn't it, that mi Microsoft has a number of times in its history been that sort of weight and then pounce, uh, kind of approach instead of trying to be the one on point taking the, taking the fire done by Microsoft product until version three type thing.

Sometimes yeah, definitely not 11. Yeah. I mean, But, but you look at what, how far they've come with Azure.

I mean, I, I don't, I'm old enough to remember, and I think you guys are as well, that in the cloud wars Azure was seen as sort of an afterthought. A Oh really? Microsoft, you know, know, yeah, you're gonna bring Windows to, you know, you're gonna bring a knife to a gunfight here.

Well, nobody's laughing now because Azure was so incredibly enterprise and developer focused that they were able to build it into essentially, um, I don't know if it's a bigger business than their traditional Windows business, but it is a monster. And I think they're trying to do the same in ai, right? Yeah, I'd say so.

And I think it's, um, and you can say the same thing about Google, even at the beginning of 2025, people were saying, yeah, that's not a serious enterprise play. And, um, and now, and now look at it and, you know, and we're honest in the space of 1222 months. So yeah, I, I think it is, um, yeah, I think, yeah, there's, it waxes and wanes.

That's what makes it interesting. That's what keeps us in the job, right, basically, isn't it, as well, indeed. And the waxing and waning, by the way, I just wanna pause for a second there, to, to honor the fact that if you do invest in a given model maker, um, that that can be the most, you know, beneficial and frustrating aspect of that at the same time, because it's every time there's a new model that comes out.

1 with OpenAI, for instance, um, it's, it's, the models are a, are a different employee, they become a different, you know, beast that you have to contend with. And you may have a perfectly running, you know, uh, workflows that you've spent, you know, months painstakingly building towards something, you know, that they do exactly what you want and next day they don't. It is, it is crazy.

Sorry, Nick, was that Yes, that you do have something to say about reinvent or No? Yes. That you want me to No, I think, I think we're done.

I think We're done with reinvent. I sense a transition point that will work. So here we go.

So, you know, you talk about companies like Microsoft who are seen as sort of, I don't wanna say stodgy, but sort of, you know, they're coming later, they're more enterprise focused, they're more, you know, more of a traditional, uh, IT vendor. Um, but there's of course a, uh, big daddy traditional IT vendor that made some waves this week as well. And frankly, there again, I feel like the market, um, the zeitgeist isn't right when it comes to IBM because you say those three letters to a lot of people and they immediately go, oh, mainframe.

Well, yes, mainframe still exists and they're still relevant, but that is not IBM and IBM this week, uh, made huge waves if it's possible to make waves, uh, outside of an AI announcement. They made huge waves, um, with the announcement that they, uh, are going to be acquiring Confluent, which is a company that maybe not everybody's familiar with, um, but they're acquiring Confluent. And those of us who are kind of insiders in the industry, we're like, oh yeah, like, I feel like the Kool-Aid man here.

I'm like bursting through the wall saying, this is the, this is such a great move for IBM, but I think most normal people would be listening to this saying What? So, um, um, first off, uh, what's your reaction to confluent IBM and second? Um, this isn't about ai, or is it, It's totally about AI and yeah, $11 billion of any sort is a big splash, is it not?

And the fact that they, they paid this much for a company that, that basically built its business on top of a, uh, very well known, uh, but nonetheless, a piece of open source software in Kafka, uh, which is a, a streaming, um, you know, service, um, says a lot. And what it says is that IBM gets infrastructure and that, that's like my, like top level give, you know, takeaway from that. And you can see this acquisition building on the Hashi Corp acquisition they made earlier this year, and all of that says, infrastructure as code, code is infrastructure.

And if you're going to build ai, um, you're gonna need to, to be able to, you know, architect something that can be, you know, posted, hosted and scaled anywhere, any cloud provider, any premises, et cetera. And you're gonna want something that you can orchestrate in the most effective manner. And by orchestrate, I mean bringing data to ai.

And that's what this Confluent acquisition is all about. You know, we've been building, um, AI systems with, you know, context windows and, and supplementing those, you know, and supplementing the training of the model with context window data, like through rag pipelines and such. And that's great, and it can bring, you know, you're not gonna be indexing that stuff instantaneously, but if instead you could bring data in real time to the models, um, as they are going about the business, and also, you know, bring data out of those models, especially in an agentic workflow, that's gonna mean, you know, whether or not you can actually build and succeed with an idea that you have as a company.

If you're gonna build with ai, you need to be thinking about not just static data that gets fed in through the context window. You need to be thinking about streaming data in real time. Yeah, excellent, excellent input.

This is Brad's area very much. He's the expert here, but, um, I, I, I think it'd be, it will be interesting to see, um, how it kind of get ties into the watsonx, um, AI story. Um, and I think they're kind of, you know, the, that the messaging around there sort of getting, um, you know, it is, you know, getting stronger, but I think yeah, may, may change a little bit, um, and the, you know, and the orchestrate product as well.

So, uh, yeah, it's very much, it did remind me HashiCorp different different use case and different technology. Um, but, uh, but similar, you know, IBM has always had a formidable AI m and a machine, I remember as an analyst event, it must is well over a decade ago, we got to sit with their m and a team, I think I remember and sit there and they were basically telling us like, we're round this table. Let's, let's imagine is if we were gonna buy a company, a made up company, and this is how they go through all due due diligence.

And it's, uh, it's quite a process and it's quite something to see and, uh, you know, could see it's still, it's still executing. Yeah, they're definitely looking ahead once you say Steven, they're, they're definitely looking for the long play here in terms of not being a hyperscaler in terms of, you know, having data gravity, but instead being a hyperscaler in terms of, you know, having an infrastructure that can run anywhere and enable anything. The the thing as well that IBM does so well is, at least in modern times under this current administration, not so much in the past, but in this, this current ad uh, ad administration at the company is, uh, run these things well in a way that doesn't run off customers, um, that is actually accumulating customers and bringing people into the fold instead of excluding them from it.

Um, and I think that we've seen that certainly with Red Hat, uh, we've seen that with, uh, HashiCorp. Uh, another acquisition I wanna bring in here that I think is, uh, parallel here. You're talking Nick, about basically the, the great big acquisition machine.

Uh, all of these acquisitions rhyme, essentially IBM is looking for companies that have just incredible recurring revenue that have, um, you know, customers that all, you know, can, can come into IBM fresh and, and expand within the IBM portfolio of products. But, uh, they recently purchased Data Stacks as well, which is another, um, incredible open source. I mean, maybe not so high profile, but a open source purveyor, um, for enterprise customers from an Apache product, um, Brad, um, data stacks plus, uh, Apache Kafka.

I mean, how, how does this work? Yeah, like, like Nick said, it's, it's all about enabling wa the Watsonx portfolio. So watsonx, ai, wa, watsonx, data, watsonx governance, all of that is benefiting from every one of these acquisitions.

And to my mind, it isn't so much about what these products do, what these technologies do, because you can get Kafka from anybody, you know, I could, I can r up on GCP and, and Azure, you get it for free, It's open source baby, right? But, but I mean, like manage, host it. You know, this, this is what Confluent makes its money on, is manage, host it, right?

And, and yet, what what is really interesting to me about these acquisitions that we're talking about is that each of them have a very well regarded and global ecosystem, um, that is mature. And I would say, like if I was to look back at IBM over the last five to seven years and say, what's their biggest weakness? It would be lack of ecosystem.

I mean, think about what drives the value of Azure that we've been talking about. It's, it's not the greatness of the software, it's the breadth and commitment of the ecosystem that builds on it. And for it, you know, not just the the get and, but it, it, it's right.

It's huge. So if I can, um, break in with a timely quote to those of us old enough to have seen the film, it reminds me of hand solo Luke Skywalker says, you know, that hunk of junk and Han Solo says, who's gonna fly it? Kid you?

Well, that's IBM, right? That's Red Hat. That's, you know, what's going on here?

You know, yeah, you can do this, but who's gonna run it? Well, we are, we are gonna run it in a way that works. And frankly, that has been a very compelling argument for these, uh, for these companies.

Now, another thing that I wanna bring in here, um, hopefully without any more Star Wars quotes, is, um, Nvidia, of course, now I have been really, uh, excited about some of the models that Nvidia has developed. Um, parakeet, absolutely. Rocks, I think I mentioned that recently.

Um, but Nvidia is out now with a, uh, you know, mixture of experts, uh, models. Tell us a little bit more about that, Nick. Yeah, that it's not supposed the mo we were talking about the models and the efficiency of models earlier, but they're talking about techniques in which to optimize mixture of experts models.

And they were, uh, they put out a blog post, um, late last week, I think. Um, I thought it was, it was interesting. They were talking about how, um, 60% of ai, um, of open source AI models released this year are, are MO let's call it MOE, so we don't have to spell out every time, but mixture of experts, models.

Um, and this is kind of taking over from, from kind of dense transformers, but there's problems, um, with those in terms of, um, memory, bandwidth pressure. And this is on their own, on their H 200 systems they're talking about. So on their own, uh, GPUs, um, there are limitations around memory bandwidth, um, from, uh, constantly loading all the expert parameters and then communication latency, uh, when experts are distributed, um, across more than eight, uh, gpu.

So they were looking for a way to, to solve this problem. And is there MV link technology? So they call this the GB 200 MVL 72, which connects 72 Blackwell GPUs, um, and delivers, you know, I'm not gonna go through all the, all the numbers, but a lot of it's very quick.

Um, and it enables, um, you know, to, it enable, gets rid of the bottleneck by distributing the experts, uh, across up to 72 GPUs, reducing the number of experts on each, on each GPU. And that's, that that relieves the pressure on, on the, on the memory. And so, you know, given the, essentially the, you know, Moes are, are the kind of the way that everything's working now, um, or at least, yeah, this is the models that are gonna be, um, you know, rolled out.

Um, they're looking for obviously, you know, ways to optimize the infrastructure on their infrastructure, and of course, you with their MV link technology. Um, actually interesting one on the, um, was it TRA three or four is gonna have, um, the, AWS ones is gonna have, um, uh, NV MV link in it as well. And so I thought, I thought it was just quite interesting, um, how we can, 'cause, you know, in the early days of the transformer models, you know, we're obviously talking about, um, how these models obviously cost an enormous amount of money to, you know, to train to train.

And that's partly because then that the parameters, the way the parameters work, the way the data constraints work, um, and you know, the, the M-O-E-M-O-E is sort of becoming kind of the standard way of building, um, frontier models. Um, and there's obviously, there's very specific reasons why you would use other niche models, and there's like, obviously diffusion and things like this. Um, but I think it's, it's interesting, um, it's obviously, you know, it's self-serving for Nvidia to say, you know, use our, um, infrastructures.

I ihb two hundreds and uses MV link 72. Um, but it's inter I think how they, you know, and they claimed, I should have mentioned, they claimed it was 10, a 10 times performance increase. I should have put, probably put that higher up in the, uh, in my little, uh, ramble.

But I think, I think I thought it was worth mentioning anyway, because, uh, anything that can optimize these things, um, is, is gonna be of interest to, to a lot of, uh, model trainers. And as we were talking about earlier, there's still a lot of those around. Yeah, we're, we're in the scale out era, are we not?

And, and that optimization is, is gonna make or break investments in data center as well as individual projects. And the skeptic in me, by the way, Nick want, wants to see, um, them do this comparison on a chip that isn't three years old. Um, you know, but, uh, so it is, it is, like you said, it is a bit self-serving, but I, I think they're bringing up some really important points that, um, you know, the architecture of these models, you know, really dictates what you can do in terms of scale out and up with these things.

You know, if you have the greatest model in the world, but it sucks so much VRA just to set it up and you can't scale it across clusters, how much concurrency are you gonna get out of it? You know? And by the way, it's not just big models.

I, I know, um, IBM with their, um, granite models, they, it's like a month or so ago, remember that they released a mixture of experts that's like, uh, under 4 billion parameters that is meant to do the same, to bring the same benefits of an MOE to, you know, on device, you know, inferencing. That's awesome. Yeah.

They're doing a lot of work with small language models with IBM, aren't they? That's very, it's very, yeah, yeah. Differentiator for them as they say it.

Well, I've just, uh, accidentally demonstrated the power of expert mixture of experts models by revealing that I didn't know what this story was all about and calling in an expert who did, which is exactly what mixture of, see, I meant to do that. You routed, I routed it properly. Um, you know, I wonder, Nick, does this have anything to do with the, uh, uh, rising price and, um, lowered availability of ram?

Uh, there's kind of a RAM crisis right now. Um, I'm not a, I'm not a RAM expert, um, but I think, yeah, it probably does. It has just, and in, and also just, uh, in terms of, you know, squeezing the most out of the, uh, of the assets that are out there, obviously there's, there's shortages of all sorts of things, including in GPUs themselves.

And so I think it's, uh, you know, they, they were talking, um, specifically about, you know, performance per WA and then being able to, like in Nvidia language, them being able to generate more tokens from your AI factory. These are all the terminology they like to use, but if you can squeeze, if you can get a 10 times, um, increase in performance per wat, that's, that's extremely important in these kind of power constrained and GPU constrained, uh, times in which we, uh, live. Indeed, the data center is limited by the number of watts that it can consume.

And, and by the way, I, I understand that that RAM shortage is, is down, so to one individual named Altman, uh, Mr as in, Are you trying to say that he doesn't have a good brain? Or are, are you trying to say that, that No, That he, he bought up. He bought up.

He all around. I know. I know.

Sorry, his Memory's not what it was. No, he's got the best memory. Um, alright, well, on that note, um, thank you very much.

I'm gonna have to route that query to another expert here at futurum, uh, to find out more about that, uh, hardware crisis. Uh, thank you both for joining us for this week's episode of utilizing ai. I have to say, uh, our producer just ran the numbers and we've got some, some great viewership already with this new podcast.

Thank you everyone for listening. Um, please do, uh, drop us a line if you're watching. Uh, I know you are because we can see the metrics, so, uh, we would love to hear from you.

Uh, you can find me on LinkedIn, uh, as s Foskett. Um, Brad, Nick, where can we find you? Brad?

Yeah, Brad Shiman on LinkedIn and, uh, on the future website itself. Uh, yeah, Nick, patience on LinkedIn and on Twitter X and I'm on Blue Sky, uh, and all of our stuff is published on futurum group com. Excellent.

And, um, and, uh, again, uh, thank you to the Hor Horological Society of New York for hosting me here today in their beautiful, uh, library in Manhattan. And thank you for listening to this episode of utilizing ai. If you enjoyed this discussion, again, please subscribe.

You'll find us on YouTube as well as in your favorite podcast application, and do consider giving us a rating and a review since that helps visibility for all podcasts. Uh, this podcast is brought to you by the analysts and experts from the Futurum Group where Insight meets ai. ai, which is our AI news site, the utilizing AI YouTube channel, or the Textron TV app on your TV or smart device.

Thanks for listening and we'll catch you next week. I am Tom Hollingsworth event lead for security here at Tech Field Day, and here are my takeaways for this special exclusive event with Microsoft Security. We had a great conversation with Microsoft Security around their Sentinel product.

It is something that is being transformed to be a critical part of your security infrastructure. I'd like to take a moment to talk about my three big takeaways from our conversation with Microsoft about what they're doing with Sentinel. My first big takeaway is the evolution of Sentinel from a SIEM to a unified operations platform, Microsoft is fundamentally re-architecting Sentinel.

It has historically been a market leading SIEM or security and information event management tool, and it is becoming a full fledged security platform that powers the Microsoft Defender portal. The goal is to eliminate the swivel chair problem. You know, the one where analysts have to jump between different interfaces to get information.

Sentinel functions are being converged into the Microsoft Defender portal, which serves as a primary interface for security operations. Sentinel is becoming the underlying platform engine that supports other Microsoft portals as well. There are some additional capabilities that you'll see in Microsoft purview for data security as well as Microsoft intra for identity management.

And the platform is designed to be open. It supports OCSF and currently utilizes 350 different connectors to ingest data from third party sources like AWS, Google Cloud and CrowdStrike. My second big takeaway is the data lake.

A major technical and economic takeaway from this event was the introduction of the Sentinel Data Lake. It separates data storage from compute power in order to drastically reduce costs and increased data retention. Previously, customers faced a choice between budget constraints and security visibility.

Things like high volume logs were often way too expensive to ingest into a hot analytics tier. The new data lake functionality built into Sentinel offers a lower cost tier, which is about 6 cents per gigabyte compared to list prices of nearly $4 per gigabyte. It also allows data to be ingested into the hot analytics tier that is automatically mirrored into the data lake at no additional cost that provides a single complete copy of the data.

This architecture allows organizations to store their data for up to 12 years. That's very important for compliance and retro threat hunting capabilities. While the data lake is really considered cold storage, it really supports some high powered analytics.

Users can run high performance jobs using things like Apache Sparks and Jupyter Notebooks directly on the data lake for deep analysis machine learning training, or historical data analysis. My third big takeaway from this special exclusive event was around graph based security and ag agentic ai. Microsoft is introducing new data modalities and AI protocols to change the way that analysts investigate threats.

They're moving beyond simple tabular data to more conversational interactions. The linchpin of this is the sentinel graph. Think about the way your attackers think about your organization because they do think in graphs.

It's mapping the relationships between assets, users, and data to visualize potential attack paths. It allows analysts to quickly calculate the blast radius of a compromised asset and also predict where an attacker could be moving next, based on things like permissions and network connections, the capability can be used for pre-B breach exposure management, such as identifying choke points and post breach investigation. One of the big components that helps this is the MCP server.

You're probably familiar with MCP as model context protocol. It acts like a catalog that allows AI agents to automatically discover and interact with data tools. This can enable things like natural language search, so analysts can ask questions like, tell me what tables are relevant to this password spraying attack, rather than trying to remember the arcane SQL or other database query language to figure out how to get that data.

It also enables Ag agentic workflows where AI can not only read the data, but generate things like Python code, create playbooks, and potentially take actions on them. This AI assisted process could be even be considered something like maybe Vibe Hunting or vibe investigation. When I think about all of the things that Microsoft has introduced to Sentinel, I think about it as maybe a storefront.

That's what Sentinel used to be. It was fast and it was easy to access, but just like all storefronts, you, you had to rent space and it became very expensive. You could only keep your most critical high turnover items on the shelf.

And if you had bulk items or inventory that wasn't moving, you basically had to get rid of it because you couldn't afford the shelf space. But now Microsoft has built this massive warehouse or data lake directly attached to the back of the store. The storage here is well, fairly cheap, and you can keep everything you might ever want to keep for 12 years.

That Unified platform is a single office where you can oversee both the store and the warehouse. Sentinel Graph is kind of like a overarching blueprint that shows exactly where the doors connect to which rooms revealing how someone might break in to the loading dock, and then be able to reach things that are in the back office. An MCP server is basically hiring a team of automated robots that will understand playing English when you tell them, Hey, go find everything in the warehouse related to that shipment from last Tuesday, and then they're gonna run into the warehouse, get all those boxes, and probably even write a report for you.

There are a lot of things that Microsoft is building on top of Sentinel now that they have a robust, very functional platform, and we are gonna be hearing more about them in 2026. As we continue to monitor these things, we hope that you'll head over to the tech field, a YouTube channel to check out the videos from this special Microsoft exclusive security event. And we hope to hear your comments and your perspectives on these technologies.

Thank you very much for watching this episode of Tech Field Day takeaways on the Tech Field Day plus YouTube channel. If you enjoyed it, please make sure you like, subscribe and share your thoughts on Microsoft security in the comments. You can also follow Tech Field Day on X, Twitter, blue Sky and Mastodon for updates, and check out all of our presentation videos on the Tech Field Day website and our YouTube channels.

Our next event is AI Infrastructure Field Day, which is taking place January 28th through the 30th, 2026. Make sure you're tuned in live on our website, on our LinkedIn page, and on Techstrong tv. Uh, hi everybody.

Welcome to this session. We are gonna talk about, uh, AI and how we can protect local AI deployments. Um, as you're probably aware, AI is the, the hottest, uh, trend in the market nowadays.

And, uh, it doesn't just divide, right? We, we know what the numbers are. The numbers are that there is a lot of push towards AI in organizations.

We see upwards of 90% of organizations that are actually able to, uh, think about what they AI strategy is. Uh, not that they have a lot of, uh, advancement there, right? Only 5% of organizations actually do have a clear plan, but everybody thinks that they need to do something with, uh, with ai.

And, uh, what we're seeing though is that security and specifically the governance around AI is lagging, right? We have statistics that tell us that many organizations still don't have a plan on how to tackle, uh, ai, uh, deployments if they're done locally. Uh, we still have, uh, very little runtime controls.

We have very little governance, uh, even though the regulation is starting to creep up. And we have now some frameworks that we can talk about, uh, with regards to how we wanna, uh, posture our a r ai. Uh, but the end result is that, uh, AI is here.

Um, AI deployments are here, there are a lot of use cases for them. And in the next 30 minutes or so, we're gonna break down what are the, um, problems that we wanna solve, and also what are the solutions or how the solutions should look like if we're going to have a good secure AI deployment. But before we get into that, why do we even wanna have AI in, in an organization?

What, what's the benefit, right? Everybody tells you that we need to support it, but what, what, what are really the use cases? So there are actually three kind of main broad based use cases around around ai.

One is the one I think that we are most familiar with, which is the ability to do some chat with, uh, an ai, uh, chat bot. Um, so that would be your chat GT or you know, gr your Gemini, your co-pilot, uh, and so on. And this is between the user and the AI engine.

Um, it, it's a very valid use case. Uh, there's a lot to say about that, but the purpose of this presentation is actually the other two use cases is what happens when we start to develop application that start to make use of AI components. And it could be that the application itself is using AI to do some backend processing, or it could be that the application is actually providing a natural language interface and then users are able to, uh, interact with the software instead of clicking check boxes and, and prompts and, and, uh, and, uh, you know, uh, wizards, we are, uh, actually talking to an application, um, like we would to a human that might need to give us that, that service.

But when we do that, we gotta remember that the basic flow of ai, the way that we see it from our lived experience, you know, talking to a jet chat bot is actually a little bit more complex, right? If we think that we just asking, uh, AI a question, so we give it a prompt. The prompt has a, um, a, uh, uh, inference server that, uh, the information is being given to, and then the, uh, AI model is the one that's generating the response.

Uh, this three step process is actually very, very rudimentary, right? Our lived experience is, is not indicating everything that actually goes on behind the scenes. And, and, and the reason is that the model itself, if we talk about, you know, what, what is an AI model?

It's really like, uh, like a brain in a box, right? It, it knows what it knows, it knows on what it was trained on, uh, but it really doesn't know the specifics of a particular organization or a particular, uh, company, um, unless you train it specific for that company. So if you're gonna use an AI uh, infrastructure, one of the things that you need to ask yourself is, am I gonna, you know, use a generic model and then try to augment the input?

Or am I gonna train the model just for me? And this is really, really important because if you don't train the model and you provide an application, let's say it's a banking application, and you're, you, you're, you're trying to, uh, make the user do something like, you know, transfer money from checking to savings. Uh, if you just give it, if you just give this prompt to a generic AI that hasn't been trained or hasn't been augmented, uh, with the specifics of the, the organization that you're talking to, if you're like, let's say the bank that wanna provide a service, um, you're gonna get a very generic response.

Uh, and if you experienced this as a user, right, in your banking application, if I went to my banking application and, and submitted that prompt and got that response, my experience is not gonna be very good, right? Because it's, I did, I didn't ask how generically to do it. I just wanted the system to do what I asked it to do.

And this is where we gotta understand that the, the, the prompt that we're sending an AI application, even the simple chat bot, even just the generic chat GPT at a private window, uh, is still going to have a lot of decoration and information around it. Because the prompt that you're gonna send in our, you know, fictional application will eventually at the bottom have what the user requested. But at the top, there is a lot of information that is provided to the model around what the user is actually trying to do.

You know, what, first of all defines the, the, um, role of the AI system itself. IQ a banking application, uh, it defines the role of the user. What is the user allowed or not allowed to do?

Um, it is defines what's tools and agents the AI application is, is, uh, able to use. And when we talk to, you know, if you hear the term agentic ai, that is really just AI taking action as if it was a user and not just providing answers. And if you want the AI to take action, we gotta be very, very specific in one action.

We want the, the AI to take, right? So our, our model is actually, uh, getting a lot of information that is not specifically referred to in the prompt, but is inferred by the fact that, that we have our application is the one that, that crafts the prompt. So all the things that, that the model does need, like user data, like, you know, what is the, uh, permitted actions?

What is the mood of the user? What is the intent of the user, what tools it can use, what tools it can't use. All of these have to be, uh, defined in advance in the context of our application.

So when we're building an AI infrastructure, we are really building not just a prompt, uh, to a model. We have a lot of other tools around it that helps us build that model to, uh, the extent that it can be used by the, the application. So that would mean that our prompt might go through a gateway to validate the, uh, identity of the user and tell us what it can do and cannot do.

Uh, there is a lot of context. So if you hear the term, uh, MCP, the, the, uh, uh, context port protocol for models, that is how we augment the prompt. That is how we, uh, provide the prompt with, um, ancillary information that might have changed since it was trained.

Uh, you might hear the word rag. So that is a way to provide, uh, documents and extra, uh, stuff that the, uh, model can draw on in order to provide its answer. Um, there are tools that can provide us with the ability of the, the AI engine to do something in, in the real world.

And then we have the model itself. And that model, if you're doing it internally, and you don't have to host the model internally, but if you even host the model internally, you also have the hardware requirements of of the GPU, right? So it's a little bit more complex than the kind of the three step process that, that we saw earlier.

Uh, but if you wanna deploy all of these components in-house, those are actually gonna translate into quite a lot of, uh, let's face it, Kubernetes deployments, right? At the end of the day, uh, we are really talking about Kubernetes and we're really talking about how we can protect the infrastructure that, that, that you're gonna run, uh, in-house. So if we're gonna translate all of these to a set of Kubernetes deployment, it might look something like this, it might look like a a, a set of deployments that are in Kubernetes.

Some of them are gonna deal with the application. So the top line is probably something that is more familiar to you as people who write or maybe manage applications in, uh, in organizations. And then the rest of it is driving the AI process, right?

Documenting the data, making sure that we have all the information we need in order to successfully, uh, execute on that, that transaction. Now, there's a lot of infrastructure, right? There's a lot of infrastructure that lives in containers.

Um, we know that infrastructure is containers is a target for, uh, bad actors that might wanna do something. But when we are dealing with AI that is wrapped in containers, there is, uh, uh, a, uh, a way that attackers might be able to go into this environment that is outside the normal ways that we know and probably can deal with, uh, when we're dealing with kind of classic security, uh, around, uh, intrusion detection and just protecting the, the infrastructure itself. And, and the fact is that, that, that there are pretty ingenious ways in which, um, ban actors are starting to exploit the way that, uh, AI systems are built in in, in organizations, it means that, um, there could be, uh, prompt injections that, um, make use of tools.

It could be, uh, leaks, uh, of data by the model, right? The, the model, for instance, in, in the example that we talked about, the bank account might return a reply, yes, I've successfully transferred money from checking account to number this and that, to, you know, from the checking to the savings and reveal the account numbers. Uh, now this is an unintended consequence, but we gotta make sure that that doesn't happen, right?

So, so how can you, uh, make sure that, that your model is not divulging information as, as it's doing, its its transactions. And then there are, there are all these, uh, uh, abilities that, that, that we need to have in order to, to safeguard our, our AI infrastructure, uh, from, uh, from attacks. So, what are we missing here?

Right there, there's a big infrastructure that, that's being put in place, uh, both internal and external to the organization. Information is gonna flow between different components of that, that, that, that infrastructure. And it leaves a pretty big gap for security org, uh, organizations to, to fill inside, inside companies, right?

'cause currently there's really very little visibility as to what ai, um, actually does. Where does it look? What, what is it doing in the, in the environment?

Um, there's really no way to, to govern AI in its current state, uh, that it, that it follows any kind of of policy, right? We don't have, uh, still the, the ability to, to impact, like system prompts based on organizational policy. Um, there is no real protection against, uh, uh, prompt attacks that might be, uh, enticing the, uh, AI system to do something that it shouldn't, especially if we're using tools.

Uh, it's very easy to guilt and, and, and have emotional manipulation of AI systems to make them do something, uh, that, that, that they don't want to do. Um, and, and if we're gonna put security in place, because we are running fast, because we have a lot of, uh, very cutting edge technology development taken on in, in many places at once, um, it's, it's really hard to tell developers and the people that, that run the AI infrastructures and, and the businesses that push for the adoption of these AI infrastructures that they need to stop. And, and let's just put a lot of security into this, right?

There's a lot of resistance from the, the business, from the applications team, from the, the, the development teams to, to slow them down and, and introduce more, more things. So there, there are challenges, right? There are challenges in the fact that we we're not sure what to do as, as security professionals, right?

This is very new. Uh, we might not have the tools in place, and, and we might not even have the political will of an organization to, to take the, in my opinion, very necessary pause to see how we actually going go, go, going to absorb this thing. Now, it's all, it's not all bad news, right?

There are, uh, very concrete things that security professionals, um, and cloud native professionals can do in order to stabilize and start to govern those ai uh, uh, deployments. But it, it requires some actions and requires some, some idea about what, what we need to do. So, if you're gonna go over the, the kind of required capabilities of what ai, uh, security might mi might look like, um, there are some questions that, that, that they need to answer, right?

They, they need to answer. First of all, where is AI being used, right? Is it used by users?

Is it used by applications? Where is it deployed? Is it deployed where we wanted to be deployed?

Are the right models in place, right? There's, there's a lot of, just needs to understand what is actually in there in the, in the infrastructure, because most of that infrastructure is gonna run in containerized applications. Our, our first order of business is to make sure that we understand what's in those, those images.

So what, what are we even running? What, what are, what AI components are, are existing in images? Um, are we getting images from AI vendors?

Like, are we getting an NCP for instance from anthropic that might have some vulnerabilities in it? Uh, are we, uh, doing the most secure configuration that we can around those AI services, right? So even when we are building images, when we're building the, the, the building blocks of our service, service by service, we have to understand what are the components that are going in, and what is the level of risk of each of those components.

And when we put all this together and actually start to provide that application to our user base, um, then the question is, first of all, how can, uh, bad AI actors break it, right? What are the limits on that we can put around, around our AI services? Um, how can we avoid, you know, resource overrun?

How can we ensure segregation of data, uh, if an incident happens in an AI system, right? Uh, prompts are very ephemeral. They, they, they can be, uh, uh, uh, issued by a user.

Um, and then, um, you get a, a response, but then the same prompt can be issued by another user and get a completely different response based on context, right? So, so how do you manage an incident where the root cause can be either something that is, uh, uh, introducing risk on one hand, but can also be, um, very few of risk on, on the other hand, right? We have to have, um, some way of, of, of, of dealing with that.

Now, not all these questions have answers today, but I think the purpose of this session is to start to get you thinking about what are the things that you need to, to provide as far as capabilities when you are designing your, your security around around ai. So let's kind of jump right into what's, what's needed, right? So, what's really needed is, first of all, an inventory of the models that are applicable to the organization that wants to use them, what is approved for use in the organization, and really just have an inventory of everything that, um, is running that has AI implications.

Uh, it could be just analytics around users. It could be what tools are being used. Um, we would really like to get an echo of the prompt.

Uh, this is a little bit iffy because it might contain some sensitive information, so we gotta do that carefully. But understanding the prompt is absolutely something that is, is required if we're gonna have good, good AI protection. Um, we gotta conserve resources.

If you're gonna run the inference server, uh, or even the model train internally, you gotta manage GPUs. Those are very expensive, and the time slices, uh, are, are, are very precious, and we wanna make sure that nothing gets wasted. Um, and then the responses, as we mentioned earlier, you know, sometimes the, the model might, might, might divulge some information during the response that that is not really applicable to, to the security and, uh, privacy governance that, that, that is, is, uh, governing what the application is doing.

So all of that are just, uh, a, a way to tell you that we need to have some rules in place, right? Organizations need to decide which AI systems are, uh, appropriate for their environment in what use case, in what capacity. And then we gotta make sure that, uh, the development organization is, uh, absolutely endorsing those and that they have some guardrails that if they go beyond them, somebody would know about it and, and, and, and will, will be able to, to react.

And the way that we do that is, is the way that we have to start to look at how those images that carry the, uh, ai, uh, services are, are manufactured and are, um, are built in the organization. So it has to do with what are the sources of the components, right? Are we, are we running approved operating systems?

Uh, are we running the, uh, uh, models, uh, and the all the other components, you know, cps, inference servers, all that from trusted sources. Uh, and, and we got, we gotta make sure that we have the right, um, uh, SBO m around them so that we have an inventory of everything that goes into those images. Uh, understanding the model, understanding what vulnerability might be at images.

Uh, a good strategy is to try to have leaner images, right? That's just a good security practice, right? Having leaner images without bloat, uh, and, and without too, too many, uh, components that are not required.

And then, of course, if the image is gonna carry any kind of action, it needs to be very contained and, uh, the image configuration it need to be secured, right? There's, there's absolutely no reason to run any image in an ai, uh, capacity or any other capacity as, as a route or privilege user. But with ai, it is actually really, really important because if we have a trick, a prompt to do something that goes beyond what the, um, AI engine, uh, uh, can do, especially if it's using tools or agents, uh, having them run as privileged users is, is a very risky proposition because the prompts are non-deterministic.

And we really don't know what's gonna happen, uh, if, if a prompt is issued. So we really got, got, got, uh, could contain it upfront. So we talk about two things, right?

We talk about policies, we talk about the, the ability to understand what, uh, is required and not required in the, in the environment. And then how we start to control it from the, the, the supply chain side and, and the images. Uh, on the other hand, what's really, really important is once we have AI components being put into play, really right at, at, at runtime, um, we are looking for a, um, we are looking for a way to provide just visibility into everything that the AI engine, uh, provides.

Uh, so whether or not a workload can be accepted into the environment, right? If, whether or not an image is good enough, uh, or secure enough or risk-free enough to, uh, be included in, in our stack, all the way to controlling how we can access the GPU, uh, and then just general security, right? Behavioral detections around, uh, intrusion detection for containers, but then extend that to alerting on dangerous prompts, uh, to understand what executables are being launched by the, uh, entrance server or the MCP or, uh, any other component that is tasked with executing what the AI engine will eventually need to do.

Um, and if we can wrap it up with some network isolation, that's even better, right? Because, because we, we, just because we don't know what the AI system will do, again, prompts are sometimes unexpected. Uh, it's better to surround the whole thing with a little bit of a, of a, of a fence so that we, uh, don't get spillage of, uh, bad actions, just, just because, uh, an n AI engine, uh, was, um, was, was brought in into play.

So to wrap everything together, it really is. And, and if you, if you've done any kind of container security, it, it actually is, is really easy to see how everything kind of fits together, right? Because, um, a AI security is really an extension of container security, especially if you run it your AI in containers.

So just doing the basis right, removing bloat and risk from the images, uh, scanning and, and, and properly gating your development lifecycle. And then just managing images with, uh, the least, least privileges and, uh, and making sure that we have good, good containment around them. Uh, that begins with inventory, it begins with policies for risk.

Acceptance is begins with, you know, secure sourcing over, over the pipeline that then progresses into the runtime controls, um, detections, uh, jailbird detections, uh, guard rails, executions, uh, and so on. And everything has to wrap, be wrapped up with visibility and, and transparency, right? We, we wanna make sure that we understand what AI models are being used, what they are, infrastructure is being used, and then put together the necessary program in order to execute our, our, our controls.

So when we talk about our controls, and this is gonna be the, the last slide, and probably your takeaway into what really needs to be done, um, the controls that we wanna put in place, there's actually kind of four categories of them. One is, is accurate code scanning and identifying what models are being used, what clients are being used, what SDKs are being used, uh, where they're being used, uh, so that we have an, an ability to maybe stop some of those deployments before they go into, into productions. Uh, if they violate our policies or any of the regulatory that are now starting to come up and, uh, and, and present some, some requirements, um, we really have to have security gates.

Uh, without those, everything kind of falls apart. So if we can't stop an image from progressing, first of all, from development, maybe into the registry, and then from the registry into our cluster, um, we won't be able to, to execute, uh, the right controls. So we, we absolutely have to have guardrails in place that, that have some ability to delay or stop the rollout of images that are not, uh, in line with, with, with our security practices.

Uh, and then once those images are running, we really have to identify whether or not, uh, an attack takes place. We need to understand if a, a prompt has, uh, let's say, private information in it, if the response has private information in it, if there is attempts to, uh, you know, ignore all previous instructions. Uh, some models, you know, most models now are resistant to that, but there are very, very ingenious ways, uh, to cause a model to do something that it doesn't wanna do.

Uh, if you ever see a model, uh, an AI chat that doesn't want to do something, just tell them that your deceased grandmother promised on you, promised your grandmother on her deathbed that you, that you would, that the model would do something. And you can actually fool the, uh, an AI model to do something with some emotional manipulation. So, so we, we need to identify those, right?

We need to identify prompts where bad faith actors are, are actually, uh, uh, trying to, to circumvent the system in, in more or less sophisticated ways. And because everything is connected and because we wanna trace back, right? If we had a bad prompt that started some, some incident, how do we trace it back to what actually, uh, caused it?

Uh, the root cause analysis really requires us to connect everything. We have to understand where images are coming from, what components are there, what vulnerabilities are there. So if we have a prompt that tries to do an action that exploits a vulnerability that is because of a certain component in the image, we gotta have visibility into that because we need to solve this very, very quickly, right?

Once AI becomes indispensable in the sense that any disruption in the AI service becomes a disruption in the application, um, then that becomes extremely important. So understanding the root cause requires us to connect everything together. So, as I said in the beginning, right, this is about how applications are using ai.

This is about what stacks are going to be required in order to use ai. And not all the stack might run in your environment. You just might run the client or a gateway or the MCP server, uh, and the model might be elsewhere.

That's fine. The, the ability to control the prompt actually starts from the client application side, which is happily where we usually, uh, uh, have some impact. Uh, and from there, it's really about having good security practices for containers, doing this little add-on of prompt and, and understanding what the flow is of the AI transaction.

And if we do all that, we should be able to have a good secure foundation to run our AI systems. Whether or not you go all the way to the model, or whether or not you just stop at the inference, uh, or anything, uh, um, uh, prior to that, uh, you should have the ability to have confidence that you're gonna deploy your, your applications correctly if you have all these capabilities that we, we outline. So that's my word for today.

I really appreciate you, uh, um, connecting to this, um, to this call. com where we have a whole page about AI risk and how to deal with it. So thank you very much On, don't they know it's Christmas, we're supposed to be slowing down.

Um, what are we kicking off with? You're trying To rush some stuff into the fourth Grade. I guess everybody wants to stick it in there.

What are we starting off with? Well, let's start off with this conversation about the cost of AI agents. And Mark Benioff weighed into this debate saying that at least customers or so he claims are telling him that the agents are gonna be priced maybe, or should be on a per seat basis.

Well, you know, there's a lot of folks who say that a might make sense, and then there's others who say, well, that's not gonna make sense, because each of us might wind up having 10 agents. And right now agents are kind of expensive. And you can start looking at a model where, I don't know, it could be as much as anywhere from 250 to $500 per agent.

So, you know, what's the cost structure look like and how affordable is that? And some folks are even saying, you know, well, this whole current token based model for pricing of AI is also not working either. So Dan, how do you see this playing out?

Because every software vendor and every customer is kind of scratching their head about this issue about, well, how will we, we afford to Alan's point billions of AI agents. Yeah. Uh, I mean, I think this is really getting at the fundamental question, which is, you know, what is the ROI on ai, right?

You know, to date, there's been a, a very small few number of companies that have really made money on ai, right? It's, the kind of joke has been, it's been Nvidia and consultants that have made all the profit here, right? And it's not really wrong in a lot of ways.

Um, you know, obviously the supply chain, you know, infrastructure, you know, there's been, you know, been money to be made there as well. But, you know, the software side has been been tougher. Uh, model companies are starting to make a little money, maybe where they're actually turning the model itself into an application.

Um, but you know, the big enterprise software companies that are really adding ai, you know, kind of embedding ai, um, you know, there's been tension there. You know, uh, if AI really does play out, maybe there's less people. Maybe a seat based model isn't great there.

Um, I think we saw with, you know, the Doge work earlier this year that a lot of enterprise software makes a lot of money for selling software nobody's using, right? So, you know, does, uh, does a a a fixed fee model make more sense? And, you know, there's obviously the case more around, you know, kind of consumption and usage.

You know, do we go with a token model? I think what you're seeing out playing in the market right now is really kind of that tension between the seller and the buyer kind of playing out in real time. Um, you know, I, I think the buyer's looking for some level of predictability on cost.

I think the vendor's looking for, you know, some level of kind of, you know, minimum commitments along with the ability to scale up, you know, with usage, but have a little bit of protection on the, the, you know, the expense side if really these tokens get burned up in a really quick and meaningful way. So, you know, I think we're, we're all wrestling with this fundamental question of we all see immense potential with ai. We know that it is incredibly costly to build and to put in.

Um, but you know, as we're kind of getting to the value, you know, what is that business model ultimately gonna look like? Clearly we haven't answered that question yet, and there's a lot of push and pull in the market. And, you know, I think we're getting closer to that business model, but not there yet.

I'm a simple-minded guy, and I'm gonna throw this over to Tom 'cause I know he is like-minded, but I'm, I don't understand how come we're like setting up with separate price points for AI as opposed to just saying, here's the new cost per seat for using your software and it's gonna be more expensive 'cause we are using ai, and then let's let the market determine what it will bear in terms of its actual cost versus, you know, right now it feels like, you know, the software vendors are trying to like say, all right, we wanna recoup what we're spending on AI and then add on 50% on top of that for our margins and pass that along to the customers. And I think the customers are gonna be smarter than that. Well, you would hope that, and, and if you're gonna break up a paradigm, you can't do it the old way by charging people for stuff they use, you've gotta come up with a creative and novel way to do billing so that Wall Street will reward you with an extra big stock price.

And I was just, uh, looking through my email because I feel like I've heard this conversation before 12 years ago when software-defined networking was all the rage. I know that that was like back in the days of Captain Kangaroo, but really what's going on is that people who want to stick around in a company need to find a way to more closely tie usage to the dollars that they get out of it. And the reason why I bring up this email from 12 years ago is because a, an executive that used to work for a software company then went to a company that was a networking hardware company, and in the middle of a big meeting said, well, why don't we charge people to use software defined networking, in this case, OpenFlow per flow.

And everyone in the room immediately turned and looked at him like he was insane. Because you can't do that in a networking company, right? Like, think about trying to just keep track of the flows that you're gonna be billing, and now you're gonna go to a company and you're gonna say, oh, well, you know, per flow will charge you X amount of dollars or x amount of cents or whatever.

You're quickly going to just be out of pocket with the amount that you're gonna be charging. And Mike, to what you said, that's what the companies who are making this stuff want because they've spent billions of dollars investing in whatever this is gonna be. And we've gotta get our money back out of this.

So we've gotta figure out how to closely tie what you're using to what we can charge that, uh, charge for the amount. Oh, and by the way, if we can continue to make 50% profit off of it, that's even better. Because ultimately we're not beholden to Nvidia or to the analyst.

We're beholden to the shareholders who want their return. Or more specifically, they want you to take those 50% profits and buy back some more of the stock. So the value of my stock goes up.

So, Mike, Tom, Dan, with all due respect, the technology industry has never been known for, for good visibility into billing and why and how. Just take a look at your cell phone bill or your cloud bill and to, to see the state of the art there, right? You, you usually, there's a whole cottage industry, well, it's not even cottage anymore.

It's called FinTech, not FinTech, uh, yes. Finops, you know, to help you get a hold of these things. 'cause they, they do tend to run away with themselves and they often bear no tub.

This is the craziness of it. They bear no connection to actual cost. It's just whatever they can go for.

But here's what I believe we are in a Cambrian explosion kind of, uh, era with, with, with this ai, with Agentic AI and everything else. We may look back on it and say, why did we think eight eye creatures would be better than two eyes or six legs are better than four legs or two legs or what have you. It's, it's a grand experiment.

The market will determine this. Uh, you know what, in 19 96, 97, I started one first company. I started in Tech Tristar Web.

I was getting 49 95 to host a website, a brochure website that did nothing. There was no SSL, there was no real commerce. They were brochures.

People paid me $50 a month. I had 5,000 people paying me $50 a month to store their websites on Sun Ultras Sparks. Two years later, that same website was hosting for $9 and 95 cents, and people were still making money at it.

That nothing's changed. It's, it's gonna be the same thing. However, if 10 agents a person, you're kidding yourself, we're gonna have as many agents as we have passwords today.

I don't know. We'll see. Jennifer, I'd love to get your opinion.

'cause I'm getting ready to run for Mayor of Tech Town on an AI affordability campaign. What do you say? So as, as someone who, and you know, use is a power Salesforce user and my role in Go to market, I was thinking a lot about this.

And what I don't understand is, you know, Salesforce already has a very advanced pricing model. And it's one of the reasons why a lot of companies don't start with them in their early days because they're too expensive. They start with HubSpot, what have you, try to do all their sales and marketing automation, then just move their sales into Salesforce.

'cause marketing is too expensive to do. Marketing cloud's too expensive to do both. But they have a really good structure for different groups and types of users.

And then packages of how much data you can consume or how much data you can process. And I don't know why they didn't do that with, with agents. And also it would make them more competitive for Marketing Cloud because HubSpot does not have great ai, sorry, HubSpot friends out there.

Um, and would allow them to be able to combine workflows of marketing and sales much better if everybody had their marketing and sales under one umbrella. But they could only really do that if they priced, if Salesforce priced similarly to what they do now, which is based on the user, um, understanding that the agents are very expensive, but also so is a lot of the sort of, a lot of the other things that they do. So for me personally, that would affect my buying decision.

Dan. No, there's a lesson there for us, you realize, right? Alright, Chris, you, we have not heard from, we have not heard from Chris yet.

Gimme a second. Yeah, that's rare enough. Um, as I thought of before on the show, this, this is this weekly thing I do with you folks is, is an interesting metric.

And Dan, you know, uh, I think about a conversation we've had and fascinating thing happened this week as we're all talking about this. You know, we helped a, a very, very small business in a very, uh, uh, uh, economically impoverished, war torn area, stand up an agent on an old Windows laptop. And that agent now is operating in this commercial environment, running this business on no hardware whatsoever, right?

Without the GPUs and everything else, with all the information right there, no internet connection and already changing the economics of the situation, better clarity and so forth. And it's all these things we talk about without, you know, Palo Alto and data centers with GPUs out the wazoo, right? You know, as I think about this issue, and Jennifer, everything you were saying, right?

You know, we have these complex systems and they're working just fine, right? You know, everybody has worked really hard, built these wonderful things, but sometimes they reach points. And last week, you know, last week on, on, uh, LinkedIn, both, uh, Rob Lee and Dragos and Mark Weatherford, you know, both publicly posted, uh, comments about different issues that speak to the same thing.

One was the Apple ui, uh, Rud launch, right? You know, the UI died and, uh, in the Apple space. And the other one was, uh, mark was talking about just the popups.

So I took American Pie and rewrote it as, uh, the day the UI died. I think we're speaking to the same thing. We're reaching levels of complexity where we had to do things differently.

So Salesforce, great friends working there, we've done great stuff. However, how do I consume that as a, in this world we're moving into where agents are, right? Yeah.

Spreading out and getting cheaper and modeling out. It's, it's following a certain arc. And, and am I gonna need 10 agents to your point, or will I have maybe four agents that are kind of super agents that are invoking a bunch of backend services?

So I don't need all these, you know, 32 SaaS applications that I'm supposed to buy. I maybe only need five. Aaron, We're, we're gonna talk about that in the next segment on, on, uh, with ServiceNow about SA versus agents.

Look, I I think it depends. Are you talking about persistent agents, alter egos, digital twins, or are you talking ephemeral agents that are kind of disposable, do a job and move on like containers, many containers in a Kubernetes environment? Listen, I, I think we're, I think we're kind of looking a little bit of the symptoms of the bigger problem here with this whole discussion around how we price ai, right?

Like, you know, if I zoom out, you know, the conclusion for me is that, you know, we got used to a world in which SaaS companies were these insanely profitable entities. And the reality is, is as you add AI to software, you massively increase the amount of compute you're gonna consume to run said software. And so therefore, the conclusion should be, being a SaaS company is still a great business model, but it's gonna be lower margin than what you're used to.

And that's probably okay if the AI provides a value above and beyond what you used to get out of it. I, is it still a great business model? Although Satya, Satya says SAS is dead, right?

Uh, I I think that's splitting hairs a little bit. I think when he says SAS is dead, he's talk, he's talking that agents are gonna live. I I I'm blending all of it into this, you know, category of software.

You know, whether it's SA or an agent still software to me, I think he said that to drive the value of the companies he wants to roll up down that, my opinion, Why you think he would do that Anyway, Hey, we're over our 15 minutes on this segment. Let's take a break. We're gonna come back.

You know what? There continues to be crazy news around fundraising and acquisitions and just insecurity, especially cyber. We're gonna talk more about it.

You're watching Textron Gang, You've earned it. The spotlight, the responsibility, the weight of teams, companies, and entire industries fall on your shoulders, lives depend on your decisions, your home life included that work. You are protected physically and digitally.

Nothing gets through your team without a fight. But in a globally connected world, everyone sees you, including those who mean to cause you and your organization harm. And now home your sanctuary attackers see an opportunity.

Your digital front door is wide open. And what compromises your home can breach your boardroom. Because the devil's greatest trick isn't targeting your workplace firewall.

It's convincing you that your personal life isn't at risk. Black clerk, digital executive protection, defending the new attack surface your personal life. Hey folks, we're back and we're talking about some merger and acquisition activity spanning SaaS and cybersecurity.

Uh, ServiceNow allegedly is coming close to a bid for amis that's supposed to be somewhere in the neighborhood of about $7 billion. And meanwhile, we also have a report up on Security Boulevard just talking about the massive amount of money being important to the cybersecurity space, especially outta Israel. Alan, I know we talked about AI agents in the last section, but before we get to that, just kind of set the stage here, it seems like there's a massive amount of VC capital in the cybersecurity space.

So is there just gonna be a wave of these m and a activity? And then how might SAS play into that? So, and, and it's tied into the Israeli story too, right?

Because a bunch of the companies I'm gonna talk about actually come out of Israel, you know what, for the last couple years, and, and we've got three or four security folks on our panel today, right? The, what we've heard at RSA is where's the innovation? Where's the dynamic new stuff?

And there was a lot of dry powder on the sidelines, I think the Google Wiz acquisition for whatever it was, $34 billion. What broke the dam for really bringing this money out to the forefront. Ever since that Wiz acquisition, we've seen a steady drum bait drumbeat of some really eye popping numbers, uh, on, on acquisitions.

Now, Aramis, I, ais, excuse me, ais, I, I first met the co-founders of AIS at an insight, uh, in Insight Ignite event in Iceland six years ago, seven years ago. They were fairly new. They were singularly focused back then on IOT security, and they still have a great IOT security, uh, uh, product service, whatever you want to call it.

But they've expanded since then. They've been hyperaggressive as many of the Israeli cyber companies are, quite frankly. Um, they just did a raise two or three months ago, I think it was a $430 million raise.

1 billion, right? So a $7 billion buy here by ServiceNow in today's market, where a hundred million here, a hundred million, there is no big deal, is not crazy. I think you also gotta look at it in the broad as part of a, this broader pattern that I spoke about, right?

I, the, the, the article you mentioned, Mike, was my article. I, I did it off a report out of, uh, Yel Ventures, Yel Ventures, Jennifer, I know you know them. Our friend Andy Ellis was there for a while.

They pioneered the, what I call the underground railroad of Israeli cybersecurity companies to the us primarily to Boston. They all seem to have moved to Boston, but, um, their report this year just shows that that market is through the roof. I think 40 something deals, I think how many billions of dollars.

What's interesting, it's not just a US based VCs funding. There's, there's homegrown VCs now in Israel. There's this whole, it's not just cybersecurity companies.

It's a cybersecurity ecosystem. And it's, it's fueling that country. It's fueling our industry, right?

And, and we're just seeing, you know, it almost seems like people walk out of unit, what is it? 8100, 8200 and VCs are waiting there for them to start some company. So it, it's, it, it's definitely a real thing.

Here's the part where I think rubber meets the road for ServiceNow, and I'm interested in your all comments. Is this is the second acquisition ServiceNow has done in cyber in the last couple weeks, it seems like. What are they seeing, right?

What are the, are they, are they just, I think, Ann, you said it in the opening, are they just, you know, a big company who's looking to buy organic revenue? But, you know, I, I know those people. They're smart as heck.

Service now. They've been some of the smartest guys in our business for a long time. They see something, they see something there, and I, you know, I'm, I wouldn't where they go.

I'll follow Chris. You got your hand up. Yeah.

So ServiceNow has been very involved in the supply chain for, for a long time, for good reasons, you know, developed some of the, uh, good, good, uh, uh, protocols and, and a lot of work there. And I, I, my read is that they're making it play to be the control plane for, for the, the internet, you know, the, the, the system of record, right? You know, where did things come from and so forth.

And that makes sense. And like everything else, you know, I think it's a logical play. I think it goes right down the path.

You're, you're talking, um, I would I say to sort of what we talked about in the last segment, though. Do we need a single unitary? Is that, is this the your era we're going into?

Or will this be a good play for ServiceNow? So they could be a control plane that makes this easier for people in their space, but we still need to, you know, this has to be like everything else. A dare I say, fully agentic, federated process, right?

Having big players like ServiceNow go down that path indicates this is where we're going. And I would just not read too much in, in into this as being the, it's just a, Jennifer, what's your take there? Because, you know, part of what Chris is sort of alluding to is that while the cybersecurity, as we know it as a separate segment in the industry, just become a feature.

No, I think that might be the way we're heading. Um, not just because the moves with ServiceNow, but some of the moves with, you know, AI that we've been talking about in general. Because what happens, just a side thought, what happens when the AI companies start turning around and building security features, right?

So there's, there's that thread as well. But with ServiceNow, I find it fascinating. So I spent a little bit of time, a couple years of clarity, and we competed with amis and they were all about, at the time, we all talked about was digital transformation for, for ot.

For ot. And amis did a lot of that as well. And if you look at ServiceNow and how they're positioning themselves as the business platform for companies, they did the, they did the acquisition, I believe it was za, visa.

Visa, I'm not sure to say that. So za, so they've done the acquisition there, they've done the AC allegedly doing the acquisition of arm. I, so they've got OT covered.

They've made some really smart investments as well into continuous controls monitoring and third party risk. I think there's an oppor, I think they're trying to, they're, it's almost like they're taking a backward approach from where they used to be to GR gain more security relevance and be that big platform for everyone versus the security companies that started like Palo with NGFW and then have expanded out to take on a bunch of different use cases. So I'm curious to see how that all comes together.

Dan, Dan, you got any thoughts on this about the, the merging of these segments, or how does this look to you? I, I, I was gonna go down the path. Jennifer went down.

I mean, this feels to me like ServiceNow kinda, you know, blending the it ot, you know, you look at some of the other acquisitions they've made, they're clearly gonna be big and ag agentic and you know, kind of building out these business workflows. You know, they've brought, brought in some of the, you know, identity security stuff to, you know, shore up the agent side. I mean, I, these all feel like really highly relevant kind of bolt-on acquisitions that extend their platform capability down into these new use cases where they can kind of really reimagine what the business workflow looks like using ai.

So, you know, to me, I think there's a little bit of an element of, you know, they've become a really big software company, and I, like most big software companies, you, you tend to need a little bit of inorganic alongside the organic to keep the growth machine going, keep Wall Street happy. But if you look at the areas they're picking off, they're highly strategic extensions of their core platform in a lot of ways, right? You can imagine, you know, for the types of things they do in, you know, logistics and shipping and, you know, all of the, you know, kind of it OT processes around those kind of things.

Um, it makes a lot of sense, you know, kinda where they're going. Um, but, you know, these are relatively smaller deals from a revenue perspective, but I think they give them a core capability. And you can bet that there's likely some lead customers who are already trying to stitch this stuff together on their end, working across these suppliers.

Uh, and ServiceNow is gonna help, you know, kind of make the easy button for them on putting all this together. You know, this, this deal reminds me of when ServiceNow bought, was it called Lightspeed? It was the four guys from Google who basically started o Hotel Open Telemetry, um, and it gave, it gave ServiceNow a catbird seed into that whole observability space.

Um, I, I think this deal is, is a similar back, back to what Jennifer said. And Tom, I I'm interested in your thoughts on this. So are we seeing security going out to the rest of the world or are we seeing the rest of the world coming into security, right, in terms of Terra force here?

So I, I think it's the second one that you're saying. And, and when you think about what ServiceNow offers, right, they are the software as a service platform, right? Like Jennifer said, if I don't know how to do marketing, I call these people.

If I don't know how to do ticketing, I call these people. What about the, the market that amis is serving, right? They're serving things like hospitals and, and if you thought IOT was a pain in the neck for sensors, wait until you have to treat, keep track of insulin pumps and heart monitors and all that other stuff.

And now you have to keep them secure. And you know, Alan and I had a great conversation about this in a yesterday on a Security Boulevard recording that we did, where we talked about the fact that, you know, old school folks, like, well, frankly, everybody on this call didn't come to security naturally. We came to something else and then picked up security along the way as an adjunct.

Whereas now there is a group of people that are kind of graduating from college, you know, uh, gen Alpha, probably the very beginnings of Gen Z who are security native, right? Like they can make security their full-time job. But what does that mean for the people who are trying to find those jobs?

It means that companies see the gap there and can say, what if I offer that to you? So you don't have to go out and hire those people. Now, ServiceNow is gonna go out and hire them because they want people who are kind of security native, so to speak.

But as a company, if you're overwhelmed by this, I can just take care of it for you. We'll bump your license cost, I don't know, a couple bucks per seat. You get all this security knowledge and you don't have to worry about paying benefits and all that other stuff.

And oh, hey, by the way, all that money that you save by not having to hire all those new security analysts, 'cause you're paying us to do it. You can invest that in ai. You can invest that in other cool technology that isn't boring and, and kind of ugly like security.

Just, just let us handle that. And then they're super sticky because now that your security has been offloaded to a SaaS company, if you ever drop that company for any reason, or if you ever try need to negotiate and make moves to try to reduce your licensing costs, oh, that's gonna be bad because we're gonna lose a lot of security expertise if we do that. So in a way, ServiceNow is kind of positioning themselves for the future when those markets start opening up as more tools are being enabled to become very hyper-focused on those attacks.

Because we've already seen that over the last couple of years where healthcare organizations are becoming targets for ransomware and criminal organizations. Because if you want to get people to pay fast, hit something that's absolutely mission critical. Like an MRI machine, I would just point out that, um, this is kind of like you ever seen that movie Highlander?

Mm-hmm. So whether security is taken over it or it is taken over security, it doesn't really matter. 'cause there can only be one.

That could Be only one. All right, that's a good place to end this segment. We've got one more great segment coming back more on m and a news.

Nvidia seems to, what a surprise. Nvidia is active again, you're watching Textron gang. 2026 mark's a turning point.

Artificial intelligence is no longer just a tool. It's shaping industries, accelerating innovation and redefining how humans build, create, and solve problems from engineering and medicine to finance infrastructure and everyday life. AI has become one of the most influential forces on the planet.

That's why for 2026, we recognize AI as tech Strong's person of the year not for what it replaces, but for what it enables a future built together humans and machines predict. 2026, join us. Hey folks, we're back.

And if you watched this show when we were at CubeCon, we had a whole segment talking about, uh, slum, which is a job schedule, a favored by the folks who build high-end high performance computing systems versus Kubernetes, which is more favored by a subset of the IT community for orchestrating things. And now NVIDIA turned around and bought, uh, Schmid, which is the company that kind of drives s SLM in the first place, and they're gonna incorporate that into their portfolio, but NVIDIA also has some tools for Kubernetes as well. And then at the same time, NVIDIA is also sending signals now that it intends to become a major provider of AI models itself.

And it seems like it's trying to build this entire stack. Tom, what's your read on what's going on here? This is what I expect Nvidia has to do.

In order to continue to get people to buy all of their chips, all of their GPUs, they have to find a way to differentiate what their GPUs offer. So like you said, the first one is buying schmid. Now listen to what they're saying.

We're still gonna keep s SLM open source. You guys can still learn how to use it. Um, implement it on your smaller tasks.

Um, basically the people that Nvidia won't get out of bed to sell to because they don't have enough commas in their revenue. But once you're ready to play ball for real, once you're ready to build your business on this, don't you want to use the, the tools that we own that we may or may not have optimized to work on our GPUs? Yeah, you don't wanna use Google TPU or a MD GPUs.

They don't work nearly as well with schmid's professional offerings because we've never seen a company do that before. Cough, cough, Microsoft. Um, but more importantly, when they start then producing these models, uh, believe the model variance that you're looking for is nron.

Uh, and of course they have, they call them something different, but they're like the tall grande vente model and, and it's very focused on multi-agent ai. Now they're starting to realize that they've gotta pick up with the new trends and run with them as fast as possible because so many companies are making those leaps. You know, we, we joke about the fact that we still can't figure out, you know, how many Rs are in the word strawberry, but that jokes from 2024 LLMs are kind of passe now.

Nobody cares about that. AG agentic is what's hot, and now it's multi-agent, as we've even talked about on this call, that people really want to be ahead of the technology curve. And there's so many NVIDIA systems out there that have been deployed.

They've got to find a way to make people want to choose to use them, because this isn't like cloud computing where there's resources out there, and I'll just use whatever's the cheapest today. If you are not the cheapest, which Nvidia is not, they've, there's gotta be a reason for people to want to go through the dropdown box and select that they're using an H 200 or whatever it is. And I think that this is a smart move by them because the hardware itself is not differentiated.

I mean, yes, there's speeds and feeds that will tell you that it is, but it's the software applications that are optimized to run on specific hardware that are gonna make people want to choose that. And if you bought the, the one that everybody wants to use, S slm, that's your end. Frank.

Chris, gimme one sec. I I just, I gotta question the naming conventions here. Schlemm Schmid, I thought that Snort, I had seen it all right 20 years ago.

Snort, who comes up with a name like Snort, but you know, Marty did okay with it. Jennifer was there, but Schlemm Schmid, we scraping the bottom of the barrel. No, man, we we're waiting on slurp.

That'll be the next thing. Look, yeah, I, I came into the, the industry with the scuzzy interface, right? SCSI for the old folks remember that.

And the first thing I did was got the book by the person who wrote the Standard. And as I related, it's pretty damn close. There's like a 17 page introduction that is just him ranting about the fact that he meant it to be pronounced sexy, right?

So don't get me started with slums and scuzzy and so forth. But you know, to Tom's point, right? You know, I, I, I agree, right?

And 25 years ago, I sold billions of dollars of, of Cisco firewalls with Bill McGee and the, and the, and the six foot and the team by saying exactly that. You're buying a, a Cisco infrastructure, a firewalls a thing. You can buy checkpoint great stuff and everything else, good companies all, but why wouldn't you just, right?

And in the last segment, we talked about this with Salesforce, right? You and Jennifer, right? You were talking about, and there's that, and, and that's, those are both true, but I think, let me argue the counterpoint, right?

You know, that's the, the scheduling thing with Nvidia. I get it. Absolutely.

However, that's, that's a big issue. And as, as all the regulars know, you know, where I'm going down this path, where we're going is narrative, uh, integrity, sovereignty, put it here, run your ais here, you know, the ability to schedule, see, you know, loads across meshes. The way we look at it is kind of intrinsic and emergent from the bottom.

I think that's where we're going in a lot of things. So Salesforce, Nvidia think absolutely should do that. Lots of, uh, benefit from that.

But I'm interested in the multi-year playout is the concentration and control at, at, you know, each of these organizations we're talking about gonna be the big win, or do we get more dis distribution in federation? So I talked to Nvidia about just what is their strategy as it pertains to open source? And Dan, I'm gonna test this your way, but what they were saying was the further up the stack it is, the more open source it is.

So if it's a framework for building an application, they're perfectly content to have it open source. But when you get closer down to the kernel, it's more and more proprietary and that's where the kind of the lock in's gonna be. But if, if all the tools are free above, do I as a customer, am I gonna care?

Or, you know, what is the danger here? Yeah, no, I mean, listen, I think that strategy makes a lot of sense, right? The more abstracted, you know, the more you know, likely it is to be open source.

And, you know, the more you know, kind of specific and down into the, the kernel, you know, the more they're gonna wanna put proprietary, uh, fingertips on it, right? I mean, you know, I, I do think buying slower makes a lot of sense. It's, you know, Nvidia is not in a hardware company.

You know, people call them a software company. I, I'd call them a platform company. I think that's really what they become.

And they've done what platform company do platform companies do, which is when there's another company that builds, you know, something of real value in and around your ecosystem, you know, eventually it becomes important enough that you wanna control it, you wanna own it, right? And I think that's exactly what happened here, right? They've become, slums become the standard, you know, for running, you know, kind of big model tasks on Nvidia hardware.

And, you know, they, they're gonna wanna get control over that because, uh, become a critical path for their customers in a lot of way. I, I think the Nitron stuff is more interesting. I mean, to me this is a little bit like the deep seek moment in that, you know, they have kind of activated this, uh, Jevons paradox, you know, uh, you know, kind of principle where, you know, for reasoning tasks, which is kind of really key to getting AgTech to take off, they've made those a heck of a lot more efficient.

And as we make the AI cheaper and easier, you know, easier to do, the more we will consume of it, right? You know, I, I, I haven't seen that take out there, but it's immediately where my head went on this was that they have offered something that is materially better, uh, you know, as a reasoning model to, to drive these agents. And the more we can bring that cost down, the more we're gonna get adoption.

We are still very, very early on the adoption curve. I think we forget that sometimes, Tom, you hear, uh, Nvidia use the phrase AI factory. Are you ready to leave to live in the AI factory town owned by Nvidia?

What do you say? Yeah, I, I'm a curmudgeon. I'm, I'm not ready to work in the AI factory salt mines any day of the week.

But I'm glad that they're at least looking at these models because they have to provide leadership somewhere. If they are effectively gonna turn the model development over to other companies, they're surrendering any advantage that they might have. Because as soon as I tweak or tune that model to run, you know, basically the same across any hardware, then it becomes a race to the bottom for price.

And we all know that there are companies out there that are willing to cut their prices to the bone to establish a foothold. And that's what Nvidia more or less did kind of at the beginning. They're like, we're gonna give as many of these things away as we can not give away, but, you know, basically sell at a reduced cost so that you become reliant on using our hardware to build these models.

And then once we know that they're the dominant ones in the market, then we can start saying, oh, well the next version that you're gonna have to use, it's gonna cost a little bit more. 'cause we, there's more technology in it. You insert business rationale here.

So Nvidia really has to kind of be kind of the counterpoint to open AI to say, well, you know, yes, you could run maybe more generalized thing over here, or, you know, maybe what they're wanting to say is, we're gonna take the lead in multi-agent because it runs best on Nvidia. And if you don't believe that that works, I want everyone who's watching this to go around their house and find a laptop that still has an Intel inside sticker on it, because that was probably the most successful hardware marketing campaign of all time. So, so Mike, I, I think though, we've, you gotta come up and take a 500,000 or a 50,000 foot view of this and admire the genius of Nvidia, right?

And I, I say it in all seriousness. Listen to me, Jensen Wong and the rest of his team realize what the opportunity is here and where they're, where they're weak or not, where they're weak, but where the potential can can go wrong. Right now, they've got a generation or two lead over everybody in the market around AI chips, GPU chips, they know that the rest of the world is hot on their tails.

It may very well be that China invades Taiwan just for TSMC and, and to make these kinds of chips, right? Because that, that might be the only way they could catch up. So what they're doing is knowing that they have that finite runway, the time is now for them to establish themselves as the platform for AI and all of its different flavors and permutations.

And so you are going to see them with Nron and Lumm and Schmid and, and all these other Shimel. I don't know what else they'll have, but you know, they're gonna establish that platform because now is their time. They, they have this window of opportunity, and once it's gone, th it's a big world out there.

Even a $5 trillion. There's a lot of people nipping at their heels. They've gotta make it happen now.

And, and so they're moving into this, you know, beyond hardware to, as Dan said, a platform. And they're gonna make that the dominant platform. Not to be overly dramatic about it, but if China does invade Taiwan, the very first missile of the United States fire is gonna be aimed at that TMSC path.

Don't be so, so sure. We we're, we're, we're fire and missiles Uhhuh. But who knows?

I mean, but that, I mean, those are like the global stakes here, right? This is, this is a race. This was, this could be a race that determines who's, what's this next century?

What is the 21st century about? And who leads it? Right?

And I, I think Nvidia has, they're not dumb Jensen. And those guys are smart as heck, and they know they've got the opportunity right now. They've gotta double down their bets.

Yeah. I, I honestly do see them as the John Chambers of, of 2000. You know, the, they're at exactly that spot.

And, you know, s esco at the time, you know, that at Chambers did, and the team and that everybody involved did a great job. But, you know, life moved on. So they have to be planning for that.

And I think they are right. But I, I'm, I'm with Tom, though. I think for everything I've said in this, in this episode, I think they'll be fine.

I think big infrastructure still works. However, I think we have an opportunity again, and, you know, history does rhyme. This is similar with the internet or whatever, but the opportunity for, to move this down and build it from the bottom up, like literally from a tiny little organization in the least resourced who's using AI today, not asking or paying anybody for it, that is gonna have a big play in this as well.

Maybe a bigger play in the long run. I can't help but to think, you know, we, we, we learn from history, right? And going in the way back machine, you know, I think about the market, just the security market's always just kind of, and the tech market's always kind of billowed, kind of like me around the holidays.

And so the, um, you know, I think back to my early days in the first, you know, a hundred people at Fortinet and how UTM started to develop, because there were all these other technologies out there that weren't necessarily competing for firewall mind share or market share, but they could have taken budget away from a security buyer. Hey, why don't we integrate all this stuff? Even though half of it at the time wasn't really integrated.

Sorry, Ken. Um, and so, you know, and then you saw Powow come with NGFW, and then we saw all this consolidation and all these multi technology platforms in the industry, and people would buy up what they might see as competitive, and then they started buying the things that their competitors did. And then we had a whole bunch of innovation of new types of technologies, and now we're seeing everything that kind back together again.

And, you know, it's reminding me that Chris said, and Chris and I worked at Cisco for a bit too, after the Sourcefire acquisition. 'cause like, so who's gonna be the, you can't get fired for buying Cisco in all of this when we're done. Mm-hmm.

That's what I'm really curious about. This is true. I think people, I mean, I understand Tom's point, but a lot of folks are more interested in the output than they are, you know, the components.

And basically, you know, the golden handcuffs are a small price to pay for the output. Okay. Schmid, Lumm, it's all here.

Hey, I know we've got maybe a minute left for some of you on the panel. This may be your last show we do this year. So I wanted to give you everyone a chance of, you know, what, what, what, what's coming down your way, Tom, why don't you kick off?

com. We already have events scheduled for January. Uh, we are gonna be at RSA this year.

It's our first opportunity to have a, a field day extra event at RSA. And we are adding new stuff to the calendar all the time. We're gonna be really, really busy.

And in addition to that, I do this, uh, weekly podcast with, uh, some guy named Alan, uh, called Security Boulevard. So make sure you check that out, because we have a lot of fun over there too. It's not just Tom and I though.

We also have Fernando Montenegro and Mitch Ashley, future chairman, analyst. So it's great show. Check that out.

Thank you, Tom. Jen, what about you? I, I've got lots of stuff coming up, um, as, as Alan knows, doing some more, doing this, doing some more stuff with Textron, which I'm pretty excited about.

Uh, I also started a new job, um, January 5th, the chief marketing officer role. So putting a lot of focus there. What will I be doing?

Where will I be going? Probably everywhere, because that is the role of the Chief marketing officer at a startup that may or may not be invested in by one of the companies we talk about today. So That's Clear.

We'll more on that later. Absolutely. And and who your guest star was that?

Hemingway. That was Hemingway. I am so sorry.

He cannot Not be. No, don't be sorry. We love, we're a pet friendly kind of show.

As long as he doesn't have any talking lines, we don't have to pay him under, you know, union rules or anything. He Had some pretty strong, uh, uh, thoughts about, you know, SLM and I would imagine, and everything Else I haven't quiet down. Take out one of your snort pigs.

Chris, how about you? I, it is been a, a year of building semantic structures and canonical, uh, distributed architectures and ai. And, and now as, as I mentioned, you know, we've got things actually deployed in the world and they're happening.

And we have pilots in Q1 in different sectors will play out, you know, all these, and, and with, with a, with a open source project and a new startup and everything else, all these different hats on. And as, and being like old and looking at these patterns, everything we're talking about here, I'm looking for the next year playing out. I, I don't think the world's gonna change.

It's not u utopia, but I think if I'm, if I'm right, and if we're right, interesting things start to invert, you know, the control planes we're talking about that are trying to stabilize at this level will be subverted supported by more individuals and enterprises, you know, coming up from the bottom instead of, you know, accepting down from the top. But anyways, I can go on all about that, uh, long for a long time, but I think next year is gonna be fascinating in, in big issues. Cool.

Dan, you and Hey everyone, it's Alan Hummel and welcome back here to Techstrong tv. Excuse me. My next guest is, he's a frequent guest of ours, Yins Wessling at, uh, Veracode.

YI was just checking to make sure I got your name right, but I did. And, uh, every once in a while though, you get that moment of panic, right? Did I get his name right?

Ys, of course, is, uh, with Veracode, as I mentioned, and he's been with Veracode a while. It will let him introduce himself. Ys, it's great to see you again.

I hope all is well. Yeah, it's wonderful to see you again too, Alan. It's so For those Interesting times, Absolutely.

Well, may you live in interesting times, is what they say, right. For those who, um, maybe have not seen you on here before though, why don't, if you don't mind, give us a little background. Yeah.

Um, I'm the, I brought up all sort of all architecture and research for Veracode, and I've been there for a while now, and I've sort of kicked off the project, uh, started evaluating the security of large language models, Which is a hot topic, certainly. And of course, wear code being a leader in the AppSec space. This is something kind of near and dear to them.

Um, yeah, it's just before we get into the findings of, of this particular survey and research, um, any idea if I had to ask you what percentage of code being generated today is being generated by ai? Oh, I think it's so hard to tell anymore, but it's certainly climbing by leaps and bounds day over day. I think, uh, probably rare to find a lot of developers these days that aren't using AI in some way, shape, or form in their development.

Agreed. Agreed. I mean, I, I saw something that said 90% of developers actually are using AI four.

Yeah. I though 40% don't trust it. I'm surprised it's not a little higher than that, but Yeah.

Yeah. Well, 65, 60 5% absolutely believe it. It, it, it creates instabilities.

It's still, it's a is early days yet we'll have to see where things go. I think it leaps and bounds ahead of where it was six months ago and it mostly wasn't a thing a year ago. So who knows where it'll be in six months, but I think there's gonna be an upper limit to how effective it can do what it needs to do.

And I don't see a world in which you won't need a human verifying that what it's doing is what you want it to do. So the human in the loop. Well, I mean, and, and that's kind of the subject we're gonna talk about today, but it seems with every new release and between the frontier models, there's new releases, it seems coming out every week or so.

Um, it seems with every new release, the the accuracy, the quality of the code that these things are producing is, is improving. So interesting there. But before we jump into that, kind of the specifics there, Jens, you set the table for us.

You, you guys, I mean, Veracode is known for some great reports, a lot of based upon their own customer data that, you know, anonymously anonymized, they've used for, you know, tremendous data sets. Yeah. What about, so Yeah, about a year ago we started looking over the research on how secure LLM generated code was.

And what we discovered is sort of, depending on where you looked and who you asked, you can get a lot of different answers. And even more than that, they'd give you an answer that's a snapshot in time. And as you said, every time these models change, the results differ.

And what we really needed is something that didn't exist, which is a report that gave a, a very candid, like, description of how well they were performing with respect to security that got updated regularly. So you could actually see whether or not things are moving in a given direction or not. And we did our first report last spring, and we did our most recent update in October with the latest frontier models and, uh, interesting results.

I mean, I think the, the chat GPT reasoning models showed a significant uptick, roughly 10% better than they did prior up until say, the low 70 percentile. And basically every other frontier model that had been released over the prior six months didn't actually show any improvement from a security standpoint. Hmm.

Kind of interesting. So I think chat CPT did a nice job of sort of focusing on that as one of their key points, and they managed to deliver a probably the largest jump we've seen in the history of the, the reporting data we pulled. Really?

Yeah. Which injury? This is five one.

This is five one. The reasoning models, the, the chat model, the non reasoning model didn't do any better than their prior version really. But the, yeah, the, the five and the five mini both showed a significant uptick, roughly 20% better than their prior model had done.

What do you, what do you attribute the, the, the regular non reasoning model, if we call it that, not having any improvement versus the big improvement of the reasoning model? I think, so I'll offer some guesses, but I think you'd really have to be in-house at, uh, OpenAI to know the answer. But reasoning models will take multiple passes through a problem considering sort of different perspectives on how to accomplish their goals.

And if you develop a code generating reasoning model that actually has a past that's concerned with security, it could actually make a significant improvement in how well it does in that respect. And I know they've actually spoken at, at length about how they've invested in security and they've put some adversarial models out there and they've, they've tried to improve the quality of what they've done. And I think it shows that that work has paid off.

And the reasoning model is considering these things now when it's putting together code samples. But yeah. Let me ask the next question, which is, as I, I think I alluded to earlier in our discussion, you know, there's a new model coming or a new version of the models, you know, collectively coming every weeks, every few weeks or whatever.

And it seems, well, except for this non reasoning open AM model, each one gets a, at least a little bit better. Is it really just a question of whoever, whoever went last is the best at this point? Do you see that leveling off at some point?

Well, no, I, I mean, I don't think that's the case with respect to application security. I think when we look at the history of these over the last year, the overall improvement in the average has been a few, two, 3% over the last year. And the open mi there are, there are more recent frontier models than the open AI ones that aren't performing as well as theirs have.

So I think it's more than just, we hope it gets better 'cause it's not getting better fast enough. I think you need to make an investment in that as you're producing these models, if you actually want to see improvement, and even with all the investment they put in, they're hitting 70%. And while that's much better than 50%, which a lot of them are running at, it's still not good enough that you trust it to go to production without actually testing it to make sure that it was secure.

Just, just for reference, what percentage is, let's say, human generated code usually? So that's tough. I think it's normally around 60 to 70% humans are also not great.

And it makes sense 'cause the models are training in human data. Well they, You know, you're always good as what you need it. Right.

Um, so but what you, but that in itself is interesting. You're saying at this point Yeah, they're about as good as a human coder. Yeah.

And I think there's a long history of human coders not writing secure codes. And I think the same thing is gonna be true of LLMs. And that's why even if they're, you know, modestly better than humans, it still doesn't mean you get a blank check to release code without being concerned about security.

You know, I mean, in all honesty, I don't think anyone sits here and says, we're just gonna release this without first checking it, run it through some security tests and everything. I think we recognize that. I think the question is how diligent are we with this code and with our testing.

Mm-hmm. How rigorous is the testing? I I've seen a lot of organizations where they're using one AI to generate the code and another AI to test the code, and there's no human in the loop in that equation.

Right. And, and though I get the, the logic of it, right, this AI looks at it differently than that AI or what have you. It still just doesn't sit well with me, to tell you the truth.

I don't, I think the final validation of security should not be left to anything that routinely hallucinates. Mm-hmm. I I get it.

I get it. But you know, to be fair, the flip side of this is, um, the pressure from above. Even use ai, experiment with ai, use ai, get your code out, keep up, move more, publish more, right?

Push more. It's, it, it makes it harder and harder to kinda do the right thing, right? It can be, I mean, I think AI is a useful tool that makes sense for developers to leverage, but like any tool, like not indiscriminately, they're, they're situations where it's the right tool and situations where it's the wrong tool and understanding the difference is what sort of separates the novice from the master.

Fair enough. Jens gens, if you don't mind, I'd like to go back to, to the report though. 'cause we, we went off you and I talking about what, you know, we find it interesting.

What else in the report though might be of interest to our listeners watchers? This is the first time we've actually broken out the security sort of by model. The first one was just sort of the aggregate and we thought it was important to do this time is we did actually for the first time see a pretty significant difference in a few of the, the models.

So if security is your top priority, I think the open A models AI reasoning models are probably the, the best in the business right now. And I think if that's where your prioritize, if you want to have an LM that's reviewing code for security, that would probably be a good choice as opposed to some of the other models that don't even hit 50%. Hmm, absolutely.

Uh, the models that don't even hit 50%, do you think that's just a way point on the way to getting better for them? Or do you think those are just inferior models when it comes to security? I mean, for instance, one of the models that hit 49% was Anthropics, Claude Opus four one, which just came out, which it Considered a good, a good coding machine.

It's a, it's an excellent model for writing code, but not necessarily for writing the most secure code. And I think when we look at some of these things over time, we don't necessarily see the direction is consistently linear in a positive direction. Um, I think like Opus four was 50%, four, one was 49%, four five is back to 50%.

It's sort of flats. And I think if you don't invest in making that better, it's probably not just going to improve because you've built another model. You know, a lesson I learned early on in my security career was they'll, they'll make it better when customers demand it to be better.

Hmm. And I, I quite frankly, again, I think that's part of the problem here. Customers want to use these things to generate code.

They know the code isn't of the highest security quality, they don't trust it. A good chip percentage of them, however, they still use it and they're still doing it. So they're not demanding better security.

I mean, it's not necessarily worse than what they get from their human developers now. So I, I understand why they feel the way they do, but I also, and I'm, I'm hopeful because I think the first focus for a lot of these large language models with code generation is getting it to generate the right code correctly and consistently. And I think we're just getting to that point now.

And I think once you get to that point, then you can start asking for more. 'cause having code that's more secure but still doesn't function isn't really a win. Agreed.

I I, I would have to agree with you. Yeah. GenZ, anything in the report or findings that you kinda say, geez, that wasn't on the bingo card?

Um, I think that the GPT model just being way out of band and the highest growths we've ever seen was probably the biggest thing. I think it's interesting seeing that we're seeing some overall improvement in of the language. net seem to be steadily improving with respect to security and less so with some of the other languages that we've looked at.

And that, I think this model, we actually did a breakdown of reasoning models versus non reasoning models. And the reasoning models seem to, on average do about five points better than the non reasoning models. That's Significant.

That's significant. It is significant, yes. And I think a lot of the AI coding assistance are leaning more towards reasoning models over time.

I, I, I do agree with you on that as well. Um, correlation between how good it is on code versus how good it is on making secure code. Mm-hmm.

Right. Like you, we brought up the anthropic model, right? A lot of developers swear by anthropic for code according to this.

They may swear by it, but that doesn't mean it's very secure code. Yeah. And, and again, I mean the, the, the delta here between the most secure code being, uh, generated and, you know, middle of the pack is, is what I mean the, the big delta's 70 to 50, but I imagine a lot, a lot of 'em are a lot closer.

So one of the, the evaluations we is, we looked at how good a job they did at producing syntactically correct code. Like 'cause does what they produce compile and October of 2024, it was roughly 50% of the code they generated would compile successfully and the rest of it wouldn't. 5% for creating syntactically correct code, which they can now do extremely consistently, but only four or 5% better with respect to security.

So I I, I've heard this before, you know, and my, my take on it is that narrow use case, whatever you want to call it, of making syntax correct, is something that they cracked the milk, pun intended, they cracked the code on and we, and figured it out pretty well. Hence the, the really good. But That, I think now that we're at almost a hundred percent, I think people are gonna start asking questions like, okay, now it's intact, correct?

It's doing we want, but is it secure? And I think now they got their first ask, which is, does it work at all? And now it's like, okay, now how do we make it work better?

So I I'm hopeful that we're gonna see them investing more in the security side of things. So the code they produce is a little more trustworthy. When will you be testing it again?

Uh, it's a bit of way. And see we sort of wait till another whole swath of frontier models come out again. And I think the latest batch is planning to come out early next year.

So as soon as they do, like we run the, the report regularly, we just generally don't publish the results until we have enough additional data that actually says something new. 'cause there's not much fun for me to get on here with an interview and say it looks exactly the same as last time. Say speed ahead.

You know, it's speaking of the report though, Jens, where can people get that? com, we've got a link to the report on the front page. And, uh, it's a very interesting read.

I think, uh, the original report and the extension with the October updates are important reading for anybody that's serious about security and, you know, amongst the 90% that are using AD to help them generate their code. I love it. All right.

Yeah, we're about outta time here. I want to thank you for coming on as always. I'm sure we'll be seeing you soon.

Maybe RRSA is only like three, four months away now. Yeah. Always a pleasure.

Alan. Hopefully I'll run into you at RSA. Yes.

Well, we'll, you know, don't leave it to chance. We'll make it happen. I someone Sounds good With us.

Y and say hello to everyone at Veracode. Keep up the great work on this. Hey, we are making progress.

I'll, I'll say that. Yes. Things are getting better.

I like it. All right, thanks a and take care. Okay.

Eds Westling Veracode here on text on tv, we're gonna take a break. We'll be right back. Hey guys, thanks for the throw.

We're here with Jonathan Edmonds, who's managing director for Key Data Cyber. And we're having a little chat about, well, the impact AI is having on fraud during the holiday season because it looks like there's been a sharp uptake, which means the bad guys are getting smarter about how to use the latest and greatest tech. Jonathan, welcome to the show.

Hey, well thank you. Thank you for having me. So what is the impact of all this?

There's always been fraud as long as anybody can remember, and it probably got worse with the advent of online, but now we're entering the age of ai. So what are you seeing? Yeah, you know, it's, uh, it's really twofold.

One, it's, uh, from the consumer side, but uh, also from the retail side, right? Uh, you have, uh, you have more and more, uh, people u utilizing AI to, to gain access. And if you're looking at it from a consumer side, you've always had kind of phishing attacks and all those things to gain access into accounts.

But, uh, you know, what you're seeing is a lot of, uh, deep fakes or even, uh, trying to steal people's phones and, and then gaining, or at least a sim card and gaining access, uh, into, into people's accounts, and then therefore going out and purchasing, uh, mass amounts of, of product, um, through their accounts. And on the retail side, uh, you have organizations that are basically going out and, uh, also doing deep fakes and, and acting as if they are executives or leadership and, and asking them to do things that they normally wouldn't do. Uh, and you're seeing people kind of take advantage, uh, of those things as well.

So coming into the store saying, Hey, I was, uh, was brought to you by, uh, an executive, they told me to do this. Here's, uh, here's a message. Um, and or, uh, doing it through, through e-commerce sites as well.

So definitely see a mass, uh, in uptake of kind of ai, generally ai, but also just regular kind of, uh, fraud in general. So, mm-hmm. Is there something that retailers are doing to thwart these attacks?

Or are they just kind of sucking it up as, you know, the cost of doing business? Uh, I think it's a combination of the two, right? I think anytime you're looking at security, there's always a risk mitigation aspect of it.

It's okay, if I don't do a, B, C, I may have to pay, you know, 1, 2, 3, or something along those lines. And, uh, sometimes you have people who are a little bit, uh, you know, on the side of, Hey, I I'm, I, I'm worth that risk, right? I'm not a big name, so I'll take that risk.

Uh, but then you have other organizations that are, you know, in the forefront of it that are building their own AI solutions to com combat other, other bots. Uh, but you're also seeing them implement more security, uh, structure. So sometimes it can be perceived as adding more friction, but uh, you need to add friction sometimes to make sure that people are, are protected.

So you're seeing a combination of the two. I would say the majority of organizations today are not utilizing AI enough or even utilizing solutions enough to, to protect their customers and organization. Mm-hmm.

And how would that work and what's involved in putting that together? Because to your point, the alternative seems to be, you know, I gotta put in, you know, five, six factor authentication and nobody will put up with that and they barely tolerate too, as it is. So what is AI gonna be able to do for folks who in the retail side to kinda identify these issues in a way that customers won't reject?

Yeah, it's a great question. That's always kind of, I'll say it's the $50 million question everybody's trying to figure out. And the reality is, is data.

You've gotta be able to, to determine trends, you've gotta be able to identify, uh, if somebody is in a geo location that they haven't been before, okay, maybe that's a reason that they, you should prompt them for multifactor authentications as an example. Uh, or maybe they're purchasing things that just don't seem, um, you know, kind of the norm, uh, of, of what they usually do. So really just building kind of, uh, kind of patterns around a specific consumer.

But even taking to a bigger scale and say, well, if I'm going to this store and all of a sudden I see an uptick of a bunch of people going in and spending a lot of money in this store, there might be something that we may wanna look at and, and use that as a, as a trigger to, um, uh, to authenticate, to revalidate that the user is who they're, um, but it's, it goes the same thing, even from a product perspective. If you have, I mean, bracketing is, is pretty popular where you go and you buy every single size of a, of a, of a, of a shirt as an example, and the ones that don't sell, you try to return, you return them, right? So it may not not seem like a big deal to a lot of people that costs a lot of money for, for companies 'cause they have to produce those products.

And then all of a sudden you've get, you get overwhelmed with, uh, with returns. Uh, so, so yeah, I mean it's, it's, it's a combination, uh, of a lot of things. But to me it's really about, you know, identifying the consumers, identifying trends at, at the store level, uh, and then building policies around that.

There's no perfect solution today, but, uh, you know, starting it out in a way that, you know, you consume the data that's out there and available is probably the best way to do it. Mm-hmm. Do you think consumers are getting a little more wary of where they're shopping and they're kind of narrowing their shopping to places what they perceive have, uh, better security and it's gonna be tougher for smaller companies that don't have that level of security to kinda give that level of assurance?

So over time, are we gonna see some changes in behavior because, well, I may not ultimately lose money. I got other things to do with my life besides sorting out all the fraud that somebody perpetrated using my card or whatever it is. I would like to answer that question, yes, but the reality is no.

Uh, the fact is, is people don't like friction and they're gonna go, it's simple. They're gonna click on the link from Instagram, don't care where it's from, they click, oh, I really like that. Let me go there.

They don't check anything. They don't even look to see if it's a a secure website or not, right? They, they just click on links.

So generally I would say no. The answer, the consumers are not educating themselves. They're, I think we do a poor job in, in cybersecurity to educate, uh, people in general.

Uh, but we definitely do a bad job, uh, on the consumer side. So I would say no. But there are, I mean, there is, uh, I would say maybe a little bit of an uptick of people who have gone through, uh, the burden of trying to fix credit or fix, uh, some of these things that happen whenever, whenever your identity is stolen.

So generally, no, but maybe a little bit. Are the, the banks and the credit card companies getting tougher about all this and are they gonna put in the measures that will ultimately force all to behave better? Again, I'll answer the same way.

I would love to say yes, but again, they're, they're money hungry, they're coin operated, so they want, they want access to as much money as possible. Um, you know, they are putting, they are putting some policies in place and they're making some things a little bit more difficult. Uh, but you have now you've got, you know, buy now pay later, right?

So that to me is a very difficult thing for, uh, a credit card company to be able to enforce anything. Anybody can go buy it now. And then if you don't really validate that that person is who they are, you're never gonna get paid.

You're never gonna get paid later. So there are things that are, are happening that, that actually I think cause uh, more attacks, um, because, you know, they want, they want people to spend more money. Um, but yes, I mean, generally everybody's, every company is trying to put more, uh, more things in place.

But I go back to my earlier statement, customers want less friction, right? So, uh, there's a reason why Amazon does really well is because they, they have everything you want. You don't have to go to 50 different places to find it.

Um, and it's pretty easy to sign in to validate who you are. Um, unfortunately a lot of organizations haven't caught up to that and therefore they're, they're struggling on them. Alright, so who is waking up in the morning and saying, we gotta do something about this?

'cause it kind of sounds like everybody involved is pretty much struggling. Uh, it's the, it's, it's the finance department. It primarily, it's the, I would say the senior executives, the CEOs, the, yeah, CFOs are probably the ones who are, you know, awake at night.

'cause they're the ones who are, uh, tied to both top line and bottom line. Um, you know, obviously the security teams are always, that's their job is to, is to enforce security. But tho I would say those are the three people who are, uh, probably not sleeping this holiday season.

Um, and I mean, you, the people who are only focused on revenue, they're great. They're like, Hey, we're close with all the revenue we're bringing in. But the people who are really focused on, you know, signing that, signing over those invoices or having to pay the insurance premiums or, uh, you know, have to deal with, uh, any kind of litigation, those are the people who are definitely not sleeping.

And to your point about that, does all this fraud eventually just get hidden or the cost of it gets hidden in the price of the goods? Because, well, the retailers ultimately will decide that, you know, we're gonna pass that cost along and we'll just share it among all the buyers and um, you know, the next thing you know, we're kind of all paying for it. I, I think so, um, I think if you look at just the history of retail stores, it's always been kind of baked into the cost of the goods at the store.

Uh, there's always loss prevention. There's always kind of, Hey, we, we just assume we're gonna lose 2% of our, of our stock. That number is gonna go up.

And that's definitely gonna be a cost that is passed on to the consumer. Um, is definitely, yeah. So definitely gonna be passed on is gonna cause more and more strife.

And I think then you'll start seeing the question, you know, why are my, why are my prices going up? And then you'll get, well, it's because you're not doing 50 things that we're asking you to do to protect, uh, to protect us. And so, yeah, I see there's probably like a little, a bit of a, uh, curve where there will be a spike of, uh, this being pushed onto to the consumer and then there'll be other avenues maybe through like blockchain or other ways that will, that'll kind of balance out that, that increase.

Mm-hmm. What are law enforcement folks doing about any and all of this? I mean, from their perspective, I'm sure they would like to catch a few of these people, but it seems like a, it might be pretty hard and B um, I guess it has to reach some level of threshold before they're gonna go after it, right?

Yeah, I think, you know, you look at, like, I'll get Target as an example, but Target is always kind of, I thought very interesting is they've always been, Hey, you can, you can steal up to X amount and they don't really care. But they would, they would track you and they would learn about you and they'd say they see this person come in over time. And, and when it, when it got to a level, I don't know if they still do this, but if it gets to a level that is a substantial, then they start to call the police.

And I, and I think you're starting to see that the law enforcement do that as well. It's, it's all about, listen, we understand it's gonna happen. We're gonna track people and once it starts building to a certain level, um, of fraud or, or theft, then, then we'll come after you.

Right? And, and they're trying to, they're trying to figure out the networks. 'cause it's not just one person, it's just not 14-year-old in the, in the basement trying to steal.

It's, I mean there, there are organizations that have a massive amount of people who are going out and doing this, and I think they're trying to find ways that they can make it difficult for them, uh, but also how to be cracked down on in those larger organizations. So yeah, you're looking at like, kind of like the FBI, like, they're definitely investing a lot in, in fraud prevention and detection, um, versus, you know, kind of local and the local law enforcement, not really as much. Mm-hmm.

So just how organized are these rings? Because, you know, we've seen and heard about them over the years, but, uh, are they global now? I mean, how far does this go?

Yeah, I mean, you see, you see both, you see a lot of regional and, and global, I would say they're becoming more global. They're very sophisticated and they are unified. I mean, you just look at things like, I remember the flash dances and stuff like that that happened on, uh, Instagram and TikTok, but people, people would get together.

They would go, they'd all use this and do something. And the same thing happens in these type of attacks is, uh, they have, they have a plan, they have a strategy, they go after it, and then they disappear and they all come back up a few months later and they go do it again. Uh, so these, these type strategies have been planned out for months.

Alright. So what's your best advice to folks who are trying to thwart these attacks? What should they be doing that's within reasonable cost structure?

Because I think a lot of people are trying to always balance the cost versus security equation, but, um, you know, what do you wish folks doing? Doing? Yeah, just a couple.

I mean, be vigilant. I mean, just be aware of, of what you're clicking on. The links are the, or are the enemy.

Like if you get a link from anybody, just, just don't click on it. If, uh, you know, you get, you get a weird phone call or an email or something and then say, Hey, you know, Val, click on this link to validate or whatever. I mean, go to the website, call, call them directly.

It's much better to do that, especially around this time of year. It's, it's safer. Um, I mean, simple things, passwords, right?

Uh, keeping very difficult passwords to, to remember. I mean, use a password manager if you have to, to, you know, have very sophisticated and difficult passwords. Don't use the same password across websites.

Um, yeah. And I would say, you know, if, if something just seems off, assume it's, it's better to go find another place to purchase something, uh, than where suddenly you feel uncomfortable. It's too good to be true.

It probably is. Hey folks, it's just like shopping in real life. If you went to some part of town where it felt a little sketchy, you might take your wallet, put it in your front pocket and do all kinds of things that'd be extra secure just in case.

Well, turns out the internet and e-commerce and a lot of these sites, they're sketchy. So be careful out there. Hey Jonathan, thanks for being on the show.

Yeah, thanks for having me. Appreciate it. All right, and back to you guys in the studio.

Hey everyone, it's Alan Hummel and we're back here live with our AWS re Invent coverage. Hope you've enjoyed what we've been putting up so far. Um, we've got our first non-text on live guest here today with us.

Let me introduce you to Gotham Rao before we get in. And actually it's a great segue to talk well, what exactly is New Bird? New Bird ai?

Uh, We, we build an agentic, SRE, uh, what that is, is a site reliability engineer. Um, so basically, you know, IT, operations for enterprises have been, uh, a difficult, challenging, um, you know, uh, part of their operations for a very long time. And the modern, modern compute stack and infrastructure stack has gotten way complex.

I mean, anybody here at Reinvent, which is where we are right now, can can attest to that. So we live in an age of ai. What new bird AI does is we build, using ai, we solve the problem of complex IT operations.

That's a great ed, what a great use case. I'm gonna come back to that in a second. But before we do, I always like to give our audience a sense of who they're listening to, who they're watching.

Sure. We didn't even talk about what your role at Newburg is, how you got here. Let, let's hear, let's hear the Gotham story.

How far back do you want to go? Well, you mentioned you're from Brooklyn. I am, yeah.

Which is, that's points in my book, right? Right. Yeah, I'm from Brooklyn too, but we could, we could probably skip ahead till after college Or something.

Sure, yeah, let's do that. No, I, um, so I did grow up in Brooklyn. Um, my, uh, dad bought me my first computer when I was, um, barely a teenager.

So I got into computers early on. Short of the long story is I'm an engineer. Um, I am the CEO of New Bird ai, but I'm an engineer.

I write code. I'm passionate about, uh, ai, I am passionate about data science. Um, and really, uh, the integration of those two things is what new bird AI does.

Um, about me, I, uh, did my master's at, uh, the University of Pennsylvania. I grew up, um, uh, well, I grew up in Brooklyn, um, grew up in India for a little bit. Moved to Philadelphia for my, uh, graduate, uh, school after that, moved out to California.

Um, this is now my fourth startup. Um, I've been in enterprise software for most of my career. Um, what else can I tell you?

I, you know what, I, if you, if you stopped right there, I think everyone would be out there saying, I wish my kid would grow up to be like that. Right? Uhhuh think you want that.

So, uh, well, it is good and bad with everything, but you know what's interesting? You were almost apologetic about being a CEO. Hey, I'm not just a CEO, I'm an engineer.

I could code God done it. Yeah. Um, But, you know, there's something, so I, I've done four or five startups myself, venture back startups, and I think there's something about being a founder, co-founder mm-hmm.

Of startups that transcends whether you're an engineer or a sales guy or a business person or what have you. It's passion. You gotta have passion for what, what it is you're doing.

You don't just start a company to start a company. You start a company because I see a problem, I see something that the market hasn't addressed adequately yet. And I think we could build a better mouse trapp.

I think we could build, we could do something that's gonna make someone's life somewhere easier. And that passion, I think is what separates the engineer from the founder, from the CEOI Think. So I think, um, you know, look, it, it took me a couple of times to figure this out and, um, first of all, uh, you have to do what you're, uh, passionate about.

And, um, look what we, what we're doing here at Newberg ai, and, and I'm, I'm here to talk about anything you want to talk about, but specifically, uh, in this company, it's something that I could use like I've, um, you know, for your audience out there, um, WW what is an SRE or W why are it operations hard? Well, what, what the, the truth behind the matter is that, um, you know, you guys use, um, you know, Twitter or Instagram and the complexity of the software and, uh, the hardware and the infrastructure that goes behind all of this, not just delivering the, the content to you, but the AI behind it. These are very complex systems, and when things break, engineers have to be up.

And you're looking at what's called telemetry logs, uh, metrics traces, uh, and you're trying to figure out very com a, a, a solution to a very complex problem. The short of the long story is I've done that. I've been up at two in the morning, um, beating my head against the computer and trying to figure out how to fix these things.

So, um, I'm passionate about what we're building because it solves that problem, problem. It solves a problem for me, that's the most important thing. Like, if, if I can build something where I'm happy with it and I'm like, whoa, this is awesome, then, you know, I hope that there are other people that will benefit from whatever it is that we're building.

So, yeah, it's fun building this, um, you're right that I am passionate that I'm an engineer because, um, you know, I'm having fun building what we're doing. And it's, it's, it's like when you're tinkering with something and your hands are dirty and you're figuring out what it looks like and everyday changes, it's a, it's a fun journey. Absolutely.

You know, that that's not an uncommon fact pattern. Right? And I call it, I can't be the only one, I can't be the only one with this problem, right?

If I could solve this problem for myself, I could solve this problem for everyone who has this problem. And there's gotta be a business wrapped around that, that, That's right. Um, you know, you, you solve a problem, and if there are enough people that, um, align with the problem you're solving, then there's definitely a business around it.

And, and, you know, the, um, look in, in what we're, and AI is so, um, um, prevalent right now with, or the, and so many startups here at, at least at reinvent, that are focusing on taking gen AI and applying it to a very specific domain. Now what it, what, once you start solving the problem, you start realizing that there are other people also solving the same problem. Okay.

So there's competition out there. And then, um, how do you differentiate yourself from the competition? Well, not two people aren't gonna solve the problem the same way.

The solution is going to look different, and it'll align with people that are, that nuanced in how they want the differentiation to look like. And so, um, then that drives your passion, like, am I, uh, subscribing to a very specific set of customers that want the solution delivered in this, this, in this kind of way. And you engineer toward that and, um, hopefully you acquire customers that are, uh, aligned with your, uh, mentality and how you wanna solve the problem.

So you're still having fun doing it. It's, uh, it's been a while ride, uh, for the past two years. And, you know, just like, um, looking at, um, talking to all the customers, that reinvent just makes you, um, that much more energized.

I love it. I love it. Let's talk a little, let you know, we, we were up here talking about it from a business point of view, but let's dive into agent ai, AI for SREs.

Sure. Right. com, another one of our sites, SREs have become key members in these communities, right?

The role of the SRE, I think is more clearly defined now than it's ever been. Right? People don't question, is it, is it real?

Is it, you know, what exactly is it? My question to you though is are we ready for an agent? Is it to replace the SRE to supplement the SRE?

Augment, Augment, augment. It's, uh, it's you, you're not gonna replace humans. And people ask us all the time, like, is AI here?

There's when the industrial revolution happened. Mm-hmm. Right?

And I'm sure I wasn't alive back then, but you know, I'm sure people were worried about what will happen to our jobs. And look, we just learned to live with technology. And this is not no different.

I mean, it's faster. It's, um, um, the change of, um, yeah, innovation, the curve is probably a lot more steeper, but ultimately it's something that we will live with. It's something that we augment ourselves with.

And, and, and so actually before I answer that question, let's talk about what an agent itself is, right? Good. Um, gen ai, you know, could past couple of years, everybody, you know, Chad, GPT came around and people were wowed by it.

And how did, uh, AI make gen AI make its way into the enterprise? And by the way, let's also acknowledge that AI ops itself has been around for a long time, that there's nothing new with AI ops. So why is this wave different?

What you asked the question about, um, agentic systems, or you made a point about age agentic systems. So let's talk about what an agentic system is first. The way the difference between traditional AI and gen AI is that machine learning based, traditional AI based on machine learning is, is engineered for more probabilistic outcomes or known outcomes.

So your engine for an, an example, when you're going to design a system for, um, uh, credit card, uh, fraud detection, right? You have well-known patterns and you'll engineer for those well-known patterns, gen, ai, the, the, um, the, the number of variables or the space, the complexity, the number of parameters is so large that there's a little bit of uncertainty in what, what its outcome will be. Keeping that in mind, what an agentic system is the following.

So we now we know we have these very large language models, which is the brain, which is the whole core behind gen ai. How did, how do you use that in enterprise systems? Well, gen AI on its own is generic, has been trained on so much web data out there, not necessarily applicable to enterprise information.

So the way it made its way into enterprises is people started with this thing called rag retrieval, augmented generation. Sure. So I'm gonna provide my enterprise content and let's see what the AI can, um, you know, determine out of this, summarize it, create marketing documents.

That's not an agentic system. That's Reg an agent. That's reg.

So now, and this has really taken off over the past, uh, 12 months, maybe 18 months, what an agentic system is understanding that these models have a lot of knowledge in them. You can't just take content and throw it at it for a certain class of problems. An example, we're talking about SREs, right?

Site reliability. Engineering relies on complex telemetry, enterprise data, enterprise application data consists of a lot of logs, a lot of metrics. These are time series data traces, which are very complex graphs to short of this long story is there's just too much information to apply rag.

You can't take this information, throw it at the LLM and say, help me. Mm-hmm. So what is an agent system and ag, and I'll get to how this helps SREs not, not replace, Okay, I'm letting you run with it.

Uh, an agent system allows the L LMS to figure out what information they need to a access. It's the converse. Instead of you throwing data at the LLM and saying, help me, you're asking the LLM, you tell me what information you need.

I have this ailment, my website is crashing, or this feature is not working. LLM, apparently you have been trained on so many different IT scenarios. You tell me what to go access.

And that's called context engineering. Okay. And that's where companies like New Bird AI and there are other people solving this problem come in.

The point here is that we now believe that these LMS are so smart, have so much information in them that now the problem that needs to be solved is not making them smarter, but it's about context engineering, garbage in, garbage out. If you ask, you can ask an LLM any question and it all come up, always come up with an answer. And that's the problem.

You can't do that with IT systems. You have to be surgically accurate. A hundred percent.

Uh, identifying the problem relies on the right context. So how does this complement SREs? SREs are sitting there under the gun.

They have a problem to solve. If they have a good context engineering solution, they can ask the LLM, I have this problem and here's my context engineering platform. It will help you find the needle in the haystack.

And then you, you've helped me solve the problem. This will help make the SREs and the engineer's life a lot easier. They can solve more problems in, in shorter amount of time.

And more importantly, they can focus on not firefighting, but innovating and building better product and solutions, which is the bottom line for an enterprise. Love it. You gave us a whole bunch of stuff here.

You can, you guys need to go back and re-listen to this after this. You watch this, it'll be up in a couple of days because there's so much, I don't want to use the word bedrock 'cause that's a big word over here. Yeah.

But there's so much foundational information here between what is an agent, agent ai, what is rag, what are all these things? Let me pivot a little bit con 'cause we're running on time. Um, AWS announced a whole bunch of agents, or they mm-hmm.

You know, they announced three real key agents. Yeah. One of them though is they're calling it a DevOps agent.

In my mind. I, I don't know if it's really a DevOps agent, but it, it seems to do some of the SRE kind of stuff. Yep.

Can be competitive. Generally the AWS products, you know, or the 80 20 rule, right? They're 80% of the functionality, 75% of the functionality.

How do you view it? Is it, Hey, use the AWS tool and then when you find out what's missing in your life, come to us Or no? No.

No. Um, look, um, I, I tell my, uh, customers the following, uh, a story too. We are talking about agentic systems.

I, uh, I probably rambled on about what it takes to build an agent and context engineering and, and this and that. Um, a follow up to that story is dealing with an agentic system is different from purchasing software. Um, where in software you kind of have an expectation of what it does, and it's either or.

You're either using software from vendor, vendor A or vendor B to solve a problem. But in dealing with agent systems, you should approach it differently. It's how you hire people.

It is part of your workforce. It is rooted from a deep, um, um, uh, you know, machine learning. But, um, uh, a deep understanding of, of variety of different, uh, problems that humans have solved.

So what, what do I mean by this? When you are hiring an employee, do you hire the same type of person again and again, or do you hire different kind of people? It's about diversity, because you hope that when you ha hire different kind of people, they ha they come with different ideas, different backgrounds, um, different ways of solving a problem.

And so overall, your enterprise is richer. Why am I saying this? I believe the same thing will happen with agentic systems.

You are not going to settle on just one agent. There will be agent diversity agents will work with each other. There are already projects around eight OA agent to agent protocols and how agents can access external systems through things like MCP.

So it's great that everybody's has their own agent, and these will solve very nuanced problems. And there potentially there'll be a framework that unifies all of these things. And we don't know what that will look like.

And I think as the industry matures, we're gonna figure this out. That's my way of answering your question of which is, um, AWS will have an agent, Microsoft announced its agent about, I don't know, uh, eight, nine months ago at their conference call. Um, you know, the Azure SRE agent?

Yes. Datadog, which is a huge partner of ours, has their own agent. And that's great.

And these agents should work with each other. We already have customers that deploy multiple agents. We, um, did, um, uh, at Microsoft Ignite a couple of, um, uh, weeks ago, we demonstrated how, uh, our agent, which is known as Hawkeye mm-hmm.

That's our, uh, agent at SRE solution, can, um, you know, uh, cooperate with the GitHub co-pilot agent and the SRE agent. And all these three things together solve close the loop. What loop is that developers push in code.

Invariably, things can break. Our agent can pick it up, uh, at the operations end and saying, I'm seeing this problem submit. Um, a, um, uh, a request to the GitHub co-pilot agent to go in and write some code to fix it goes all the way back to the developer to say, this looks good, and I'll accept a fix.

And that loop shortens how long it would've taken to fix that problem, end to end. So, um, having, um, an enterprise purchasing multiple agents, or working, not purchasing, working with multiple agents and tying these things together will be the future. Excellent.

You know what, we, we didn't mention the URL, how people can contact. Yeah. ai, N-E-U-B-I-R-D.

Okay. Dot ai. And our agent is known as Hawkeye.

Um, we've, um, uh, we G eight actually last year at, uh, reinvent. We have, um, a lot of customers that have been using our systems now. Um, we have, uh, results on our website that you could go look at in terms of how we have reduced what's called the meantime to incident resolution.

Um, in some cases, 90%, uh, time savings. Wow. And what does that mean?

Well, it's, uh, bandwidth and time that the enterprise can get back to work on, um, what they actually want to do, right. Building their core products and services as opposed to firefighting. I love it.

Gotham, we're outta time, man. But thank you so much. You know what, thank you again.

I'm gonna tell you guys something if you want, when this is up on Text Strung TV or the YouTube channel or the OTT channel, go back and listen to what he said. Again, it's a great primer for some, you know, basic concepts that we all bandy about these words, and you may not truly understand what they are. So go check that out.

Thank you for that. Thank you so much. We're live here at Reinvent.

We'll be back in a little bit with our next guest. Stay tuned. Hey, everyone.

We're back here on, uh, Textron TV with our AWS Reinvent coverage. You know, as I mentioned, Nick Patience had a run out, so we had him really emphasize the beginning of our conversation. But I wanna continue our conversation with Mitch.

Um, so Mitch, it's good to have you back. You know, you've been podcasting up a storm too, right? You did something with Brad Shiman.

Yes, I did. Agents of Dev, our new Podcast. I love that one.

Yes. You know, it's all about the agents, human agents, computer agents. Absolutely.

Digital coworkers. Yeah. And then we did a, uh, a, a, uh, cyber still cyber still cyber after all these years, baby.

That was a, that was a throwback. But you, you spent the day today, I know in a bunch of meetings, keynotes, listening in. Most of our audience here probably wasn't here in person to watch this.

Mm-hmm. Yeah. What, what could you give them to take home, Mitch?

You know, I think, um, I always look at where do things start, because how's this going to set up AWS for the next 12 months? I think Nick made the point about it's not of end of the year conference only for AWS anymore, but what they did is this is gonna help them for three, six months, because things are moving that fast across all the vendor environments. And they started the conversation with, at least in the pre-brief with the analysts, he said, we're gonna be talking about up here in the stack.

We're not just gonna be talking about down here. And they actually started the analyst briefing really talking about kinda what's happening at the user or the user of AWS services layer. Mm-hmm.

And of course, um, Brad teed things up nicely with, with Kiro for me, about that as the, uh, as the AI IDE that AWS has launched back, I guess, midsummer and they've standardized internally on, now they've, they've specified this as a SPECT driven IDE I'm not sure that's a sustainable advantage for, for them, but they'll figure out kind of what their place is with that. The big push was around let's really take agents to the next level, which meant agents that can scale at very large scale. They talked about billions of agents at some point.

Mm-hmm. They talked about long running agents, agents that do work over days as well as maybe even weeks or longer. Uh, but they also subdivided that into some new agents that they launched.

We talked about the AWS security agent. I agree with you on the AWS DevOps agent. I think it's little dev large ops.

Yeah. It's an ops agent. Yeah.

It's, it's really a DevOps agent. Mm-hmm. Per se.

But, you know, that's okay. It's To Start back that up, you know, To start Exactly. You've gotta get there somewhere less specific about what they called the Kero autonomous agent, which is dev work, if you want to categorize that.

They didn't put too much meat on the bone there. So there's a lot to build from. I see this as kind of the scaffolding, think of it in development terms, right.

There's the structure of what we're putting together for the agent framework. You didn't hear Dev, uh, Q Developer much today. No.

Where last year that was Huge, all about it. And then Agent, agent Core came out midyear, uh mm-hmm. About, I guess, earlier this year, uh, and is now all about Agent Core as the development platform, if you will.

Of course, Q developer's still there as part of it, but I think that's the new framework that they're building the scaffolding from. And that's where we'll see a lot now, I think a lot of what was introduced here is table stakes. There wasn't anything that was like, oh, that's super innovative.

Nobody else is doing that. Maybe a little bit with Forge about creating your own, your models using fair enough AWS's models with your data. Okay.

That's interesting. But that's okay. This is not a world where you're gonna pull everybody over with one announcement and suddenly the industry goes, oh my God, what are we gonna do?

Now, this is No, but I, I think this Is a marathon race where we're watching Absolutely quarter mile by quarter mile as this thing's unfold Quarter by quarter is, is a good way of thinking about it. And, and I think it also goes to what we were talking about earlier, and I, and Daniel Newman mentioned this in my interview with him earlier, which is this was a bit of a, of AWS playing catch up. Oh, it's major catch up.

Yeah. Yeah. I, it's not, it's not unknown that people think AWS is behind to the, the other hyperscalers, if you will.

Now you can argue whether they're, they have strength. I mean, mean, look at, they're a massive company. They a huge Customer.

It's still an 800 pound gorilla. And they still look, you know, I'm a football fan and, but any sports fan, I'd rather I, my team is the team that's ahead than the team that has to play catch up. Mm-hmm.

Right. It's always nice to be ahead Or could be the Broncos and win by one point when the, you know, the guy donks the, the kick. Yeah.

Or the two point try. Right. You know, these Colorado people, they have one good season, and they don't, they just gotta work it into every Conversation.

You gotta be hearing about this for the next 10 years. It doesn't matter how it turns out. Come on, Mitch.

You want both? Say it with me. You want both.

But, uh, that being said, that being you had to do something on a week when the steel is like, spunk the place Off. I kick a guy when he is down. That's what I, That's what, yeah, that's what it is.

But let, let's, let's go back though and, and, and talk a little bit on, on, on this agent stuff, though, Mitch, you know, it's interesting you said that he opened up the analyst briefing by saying, we're gonna go at the top of the stack here and talk about what it's like to be, uh, an AWS customer, AWS user. And then everything was about developers. Mm-hmm.

Now it's true. Developers were the lifeblood of AWS early on. Right.

That's what they, that's what built that in a credit card. Right. They, they whipped out their credit.

I was just gonna say, they whipped out their credit card. They spun up a few instances. Mm-hmm.

And off we went shadow it at its best. Is the developer still the engine that drives AWS Or is it, is it the infrastructure play? Is it the security folks?

Is it the platform? Is it, you know, ops, SRE and all, all of the rest? Or is still AWS laser focused on the developer?

Well, I learned early in my career, you could tell who drew the diagram by what's at the center. Brad Shiman draws the diagram. There's gonna be a database of data, fabric data, you know something, right?

Mm-hmm. Nick, it's gonna be AI models, it's gonna be AI chips, Mitch, it's gonna be all about the developer. And now you create software, right?

Yep. So, you know, the AI is supposed to be the, the death of the developer. And I immediately said, yeah, okay.

You don't know how software is developed if that's what you really think is gonna happen. Developers at the tip of the spear of ai, that's where, that's where the first people are adopting the latest innovations in most cases. And what we're seeing is it, it isn't eliminating the developers.

It's actually emphasizing what developers do because you see AI moving to the command line interface. Oh, I thought, I thought we weren't gonna have to do that anymore. No.

Guess what? We're putting AI at that level. So the develop level, so developers are productive.

We're building in fabric, we're building in control planes for, to be able to manage and put in security guardrails and do things like that. You look at even an IBM tool with their orchestrator, here's the end user interface, the non-pro developer, here's the pro developer for the same tool to develop and orchestrate agents. So someone still orchestrates, creates, thinks, drives, innovates, what AI is doing today.

It still very much, you know, developers, I think not in the loop. I think it's developers driving the loop, whether we do all the work, I think we're doing less of the Work. Yeah.

It, it goes back to something I wrote, and I think I did some videos on, you know, to paraphrase, Billy Joel AI isn't starting the fire. No, I didn't know he said that really Well. Something about starting a fire, wasn't it?

Did I tell you Billy Joe lives across the Intercoastal front? You have told me that about four times. Yes.

Yeah. Every time we go out there, we wave. We wave.

We invited him over the House cup every time I ran in your boat as we go by. Well, that's why I'm playing the Billy Joe music really loud. I'm hoping he'll hear me.

But someone started the fire. Mm-hmm. Right?

And I, and I, you know, and then it goes back to AI is is the a tool, it's a developer tool, it's an ops tool. It's a security person tool. But the spark is still the spark of human creation.

And I, I don't think that, not in my work lifetime. I don't think that's gonna change. I mean, look at it.

I mean, we, we we're, we're using tools to create videos that we would've never created before. Right. Just kind of laughable, fun things.

Yep. But useful things too. Um, the agents of Dev podcast I created with one tool, an idea of what we do for our logo.

Brad took that and took to the next level. And like, yeah, that's what I want. Because that's very much sort of the comic style.

Let's do that. Mm-hmm. And so I couldn't have drawn that.

I couldn't have, you know, even called up somebody and told them what I wanted. 'cause I didn't know what I wanted yet. But through using a creative, you know, mechanism, a tool like ai, look at what you can create.

But I, I'll so think back to the first time you started using Photoshop. And for me, I'll, I'll tell you this, that I'm, I'm gonna go back to 1995. I'm using Corel drawer, if you remember, I do remember Corll Corll drawer and layers in Corre drawer that allowed you to do Extrusions stuff, stuff.

My carrell draw didn't ended in r it was this Corre draw, not draw. Well, I'm from New York. Oh.

In New York, sometimes we add an R, sometimes we take off an RI can never tell. Can I have an R please? Alex can never tell.

But, um, that a turned, well, guess what it says? Can I buy a valve? Can I buy a valve?

But we don't have no stinking mouths. But anyway, but, but seriously, Mitchell, this has been, you know, this, this is not new to tech. This is, this is the way, as the Mandalorian would say, Right?

This is the way, This is the way, Like how you got that in there. That's right. Like, the way I did that worked that right.

In smooth. Like some things never changed. No, you were, I was still thinking the R thing, but okay.

Yeah. And I said the R in Mandalorian, you Did Mandalorian. Yeah.

'cause it's only ours at the end that we have problems with, with, I still can't figure out how it works. You know, I gotta sit here and take abuse from a Nebraska person. Could you believe this?

Um, I lived in New York. I learned a thing. A few, few thing or two new I learned about your New York.

But let's get back to AWS reinvent, Mitch. Alright. Um, you know, Brad said something that I didn't really realize in until he said it.

We haven't heard a lot about data and databases mm-hmm. And all of that stuff, and data storage. And, and that's always sort of a, no pun intended, but that's always a bedrock of AWS's.

It is, you know, Reinvent. I, I don't know if this is a reason, but we've had a succession of vendor announcements about data fabrics, AI data fabric, AI data fabric, you know, going from Oracle to, you know, you name it. I dunno if that's the reason why they didn't emphasize that this, I think it's more they have to catch, here's my point about developers are still ruling the world.

So software rules the world. They're catching up on the developer, right? They're trying to capture the development environment.

Again, I'm drawing the picture. So this is what I'm putting at the center. Okay.

And, and here's why. I think why Nick, to talk, Nick talked about the advantage of who has the advantage with models and, and what, what Google has, what Microsoft has isn't just the OpenAI relationship that they're now diversifying with. They've got the massive developer community, right?

Yeah. They have 150 million developers who use GitHub. All of the IDE tools, all virtually all are based on open source, uh, versions of visual code, visual, yeah.

Studio Code. Um, that's what the competition for, is to get those developers to be creating not just the agents, but the tools, the, the fabric, the, the scaffolding, the, the next innovations about how we build and create agents. That's what, that's what they need.

Now, we're gonna be doing that a lot faster with a lot, lot more innovation from the same group of people, but it's still about creating that software. Yeah. And it makes the world go around.

It's eating the world, but it already ate the world. It ate the world, I guess, to see The universe. Someone else said we, we were gonna be moving from software factories to intelligence engines.

That's, that's, yeah. That's, that's satcha, that's phrase, whatever that means. But yes.

Well, Satya said it. We're also don't gonna have sa SAS software. It's still around, but we will see what happens with that.

And I, but I'm not running Microsoft, so who knows. That's Why. I was just gonna say, you are Mitchell.

He Satya. Yeah, I think, I think that he carries a few more sticks in his bag than I do, but yeah. I got a mean three irons, so, Okay.

I play golf with you, Mitch. You are. No, I knew Satya.

I worked with Satya. You are no Satya. I'm no three irons.

Anyway, we've gotta get wrapping it up here, Mitch. We've got two more days. You'll come on in and we'll do, actually, we're doing text on gang early tomorrow morning.

Well, you know, as the, uh, this is the philosopher, the Terminator said, I'll be back. I'll be back. We'll be back tomorrow with even more from text, from text on Textron tv from AWS reinvent.

We hope you've enjoyed this discussion. If you could tell, you know, we have a good, great time with the future of analysts. We enjoy, you know, talking shop, just, uh, enjoying, you know, it's a great time.

You know, Mike Ard told me, t it's a Chinese proverb. I thought it was an Irish proverb, but may you live in interesting times. These are certainly interesting times for now, though.

That's gonna wrap it up. This is Alan Schell for Text Drunk tv. Thanks for watching AWS made major announcements at Reinvent, including the Nova two family of models, inclu and Sonic for speech, as well as emerging AI factories featuring Tanium custom Silicon and S3 vector storage as they try to stake a claim in the AI infrastructure market.

Companies like AWS are building deep infrastructure capabilities and platforms like SageMaker and Bedrock are delivering AI powered applications as our companies like Google and Microsoft. IBM also made waves this week with their announced acquisition of Confluent, which is yet another modern AI application arrow in the quiver of Big Blue. And Nvidia told us a little bit more about how to use a model of mixture of experts designs to increase performance.

All that and more on this episode of utilizing ai. Welcome to utilizing ai, the podcast focused on practical applications of artificial intelligence from the Futurum group. Each episode brings together diverse perspectives to explore news and use cases in the ways in which AI is transforming enterprise IT and the industries it serves.

I'm your host, Stephen Foskett, president of the Tech Field Aid business unit here at the Futurum Group. And before we get started with today's discussion, let's meet who's joining me on the panel today. Steven, thanks for having me.

Hi, everyone. Brad Shiman. I am an analyst with the Futurum Group looking at data intelligence, analytics, and infrastructure.

And I'm Nick Patience. I'm the AI platform's practice lead at Futurum focused, uh, solely on AI, really. And as mentioned, I'm Stephen FoST from the Tech Field Day Business unit.

I, uh, host the AI Field Day events, and of course, uh, this podcast, I, I have to actually say I'm here in at the New York Horological Society, which is hosting me. And if you're wondering what that is, Google it. It's amazing.

So let's dive right in. Um, I'm surrounded by antiquities here, but we've got a lot of cool, uh, modern ultra cool stuff happening here in the future. Um, we just got done with AWS Reinvent.

Now we're not a news podcast and we don't wanna make it sound like we're, um, you know, kind of reporting the latest. But Nick and Brad, you all have, uh, had some time now to digest the announcements from Reinvent. Um, perhaps, uh, you can regurgitate a little bit of that knowledge.

Uh, I'm gonna not stretch this metaphor anymore. Um, Nick, thank you. Uh, what was announced at Reinvent and what was interesting to you?

So, yeah, we, we, um, Brad and I were there last week and, um, we wrote a, a note for a future from clients. I actually wrote the headline. And the headline was, um, wrestling Back AI Leadership.

And I think that kind of sums up where we are at the end of 2025. Um, it's been, I guess, fairly well known that, uh, Google, at at least from a narrative point of view, has kind of, you know, taken a bit of a lead and outta the three major hyperscalers. Uh, and, and AWS is, you know, uses reinvent every year to do, to make its major announcements, obviously.

Um, and, you know, this was, this was more about that. So I guess, you know, one of the things were the Nova two family of models, um, including the, um, yeah, there was Nova, Nova, Nova two Pro for Advanced Reasoning, um, our Omni for, you know, long context, um, workloads. And then Sonic, which was the speech to speech model, which was pretty impressive, that last one.

Um, and because Amazon, um, AWS rather has its contact center applications, which embeds, you know, which some well-known content center apps out there are built on. And I think that that's a, you know, it's obviously a real proving ground, um, for that kind of stuff. So I think it's, I mean, I, I certainly believe, you know, it's fair to say that a Amazon HA wasn't at the cutting edge of models.

Um, and Nova was released, announced that last year's invented, this is NOVA two. Um, you know, and these weren't, you know, these are, these are strong models, but I think that yeah, each one of these kind of, um, vendors are gonna find a, a niche. Um, there, I think the, the other interesting thing I thought was the, um, AI factories, which is essentially the, um, Amazon's infrastructure on the, in, in the client's data center.

Um, and as part of that, they announced, um, you know, theran Ultra Servers, um, based train, train two, um, they also announced train three, the, the chip, and then the roadmap for four. Um, so it was very much, I think from my point of view anyway, it looked, um, you know, like Amazon on a, a Ws trying to own the kind of, of AI infrastructure narrative at least. Um, there's lots of other things and there's lots of other things I announced, and maybe Brad, you can, uh, talk about a couple of the others.

Yeah. For me, um, it, you know, just building on what you're talking about with Nova, we all know that, you know, AWS is, is not going to outdo anthropic and even really Google, uh, with Gem, the Gemini family in terms of outright performance. And I, I, I have to admit that when we were, when we first got to the show and they were talking about everything being Frontier Scale, you know, with a Capital F and air quotes around it, I, I was a little, you know, uh, taken aback and, and thought, nah, come on guys.

You're, you're not really doing Frontier Scale models, you're just ruining the name and it's really not true. Uh, they're actually building frontier scale models in terms of being multimodal as one checkbox in having super long context windows at a million tokens and other checkbox. And, and I think that when you look at that, um, coupled with another announcement that came out, uh, and that is their, uh, what they call Amazon Nova Forge, which is this basically a facility instead of tools of what Amazon likes to call recipes, uh, that you can use to fine tune the Nova family of models, and not just in your basic, you know, here's the open weights models, good luck, you know, because we, we've been doing that for years now in terms of fine tuning and instruct tuning and, um, aligning models in the training, during the training process and after that.

And what they're doing is, is kind of unique in that they're productizing operationalizing those mechanisms, uh, those data science techniques to make them a lot more accessible to the enterprise marketplace. And they're opening up the Nova models to, to actually let you, um, sort of work with those as though they were your own. So they're letting companies basically grab different checkpoints.

Checkpoints are steps along the way to creating this final frontier model so that companies can more readily bring their own data to those models. And that's, that's pretty cool. And very quickly, uh, I wanna say one other thing that I really caught my attention, and, uh, this is, again, me being dragged forward because, uh, you know, I, I've been, I'm a database guy and I, and I think database management systems are kind of important, and yet, uh, we are seeing a very sizable trend in the industry towards pushing down database functionality to the storage layer itself.

And on Amazon, it's the AWS S3 layer, so your object storage, and they released it to ga their, um, AWS S3 vectors capability, which is, as you might imagine, a vector database. One that can do, you know, super huge indexing tasks and do so at scale with a comparatively very small cost footprint, which is quite impressive. And one of the reasons why you might want to push that functionality down to the S3 layer.

And if you couple that with an announcement they made earlier in the year around S3 tables, which is basically to bring structure data to the, you know, non-structured na nature of, of object storage, you've got yourself kind of a database, you know, disguised as an op, as a file system. So it's, it's really interesting times. So just picking up Brad on the Nova Forge thing, I think you're right.

I think that is at the moment, unique. You could do that, couldn't you? And like Google Model Garden and, and things like that, you could tie it all together and Microsoft, you could do the same thing and maybe IBM as well.

But I think, um, it's, do you think, I mean, I, I kind of think this will be copied by the others by its direct rivals pretty within, you know, I was surprised if it takes them a couple more than a couple of quarters, but they'll do it. But for now, it's the only, it's the, it's the only sort of productized way of doing what they're trying to do, isn't it? Yeah.

Everything else has been data science, you know, open up a note Jupyter Notebook and grab your favorite PyTorch version and have at it. And that's not something that's accessible to every company. So if I can ask your opinion Experience, uh, yeah, Lemme, lemme ask your opinion on the, um, the direction that A-A-A-W-S is going here and, and what we can learn from that.

I see, um, something of a similarity as well. You, you know, you mentioned Google, of course, uh, they're have a similar approach to this. Am I reading this wrong, or are they seeing, you know, companies like AWS and Google that have a huge infrastructure CapEx investment?

Are they seeing, uh, models as almost a loss leader to attract people to their environments or to enable, uh, the build out of this, uh, new industry, which I guess that could be a second point? Or are they seeing these as products that they will make revenue from? I personally think it's the former, I I've long thought voice or is the former really, the, the models are, are not models are not, the application models are not the product.

Um, they are very much a a means to an end. They get the attention as, as Brad knows, I mean, every single time any major company releases a model, journalists will always be asking us about that. And sometimes I'm kind of wishing they'd ask us something about something else.

Um, but, you know, I think they are very much, um, a loss leader and then you build the value around that, um, either below it with silicon or on top of it with the various tools and then applications. I, I dunno what you think, Brad. Yeah, I feel the same way.

You know, we were just talking about it before we came on air that, you know, whether you can believe this or not, but, uh, it has been studied and argued that, uh, perhaps, you know, the further we get with the transformer architecture, the less differentiation we're gonna see between, you know, frontier scale models and actually all, all scaled models simply because they all converge around the same patterns that they're trained on. So Yeah. Is next totally right.

It's, it's really just a part of the tool chain and it's, it's an important part, but it is just one. And we're seeing, we have seen a lot of effort from all of the vendors that we've been talking about in terms of trying to at least standardize on how you interact with those models from an API perspective, so that whatever tooling I'm using in that tool chain, you know, I I can bring in the model that's most cost performant, accurate, efficient for me. I mean, that's not to say at the moment, you know, Google, for instance, with Gemini, and that's the only place you can get hold of Gemini, uh, does, yeah.

The model that is, um, does have, yeah, that, that kind of, that kind of appeal. Um, and I think it, but I think it waxes and wanes. I mean, everybody's got a model garden, a model, um, switch switchboard.

I like to think, you know, when to, you know, how am I, how my, I, how may I direct your prompt? Um, do you want this model, that model, um, so that, that ability to, to offer, you know, first and third party models usually isn't much of a differentiator, uh, unless your model is something, you know, quite spectacular. Um, and I think those, the, the kind of window for being spectacular could last days, maybe weeks, um, if you're, if you're lucky, and then it, then it slams shut.

So What does this mean for the rest of the industry then, if, if, if models are lost leaders? What does this mean for companies like philanthropic and OpenAI? Um, and, and I have an idea, um, I think that Anthropic and OpenAI are trying, they, they don't, so let's, let's back up again.

So AWS and Google understand that infrastructure is, has always been where the money is, the revenue, the bulk of the revenue. And I think that, um, a product like S3, for example, is a very sticky revenue magnet in a way that very few things in our industry are. And so it makes wonderful sense that Amazon would want companies to have more data in S3 to make S3 more AI friendly and attract people to using Amazon's infrastructure to build their applications.

I think that open AI and Anthropic, it looks like what they're trying to do is instead build a platform. You mentioned sort of that app store kind of approach. I think that's kind of where those guys are going.

Or am I missing that, uh, point as well? And, and our Amazon, is Amazon playing there too? Yeah, I, I think, yes.

I think they're, I mean, open AI I think is trying to build, um, a completely vertically integrated technology company from chips through, you know, all the software layers we've talked about, um, to devices, um, and everything. You know, you're working with Johnny, Ivan, all that kind of stuff and everything in between. And then, you know, to a slightly less, um, lesser extent, but still you, they're building out tools.

Um, so build software companies essentially, um, whether they, whether they will succeed or not, we will see, see and we'll be trying to track it very, uh, very closely. But I think that's, um, I think that's, that's what they're trying to do anyway. That's my opinion, Brad.

Yeah, I feel the same way in terms of, you know, these companies are, have been for quite some time trying to build beyond the model itself. And you look at how Anthropic Claude family has evolved in that regard, and you can see that what they've been focusing on is how do you build the attendant tooling around, you know, using their models. You know, how do you get from just a chat bot to a full fledged agentic process running in your line of business that's, you know, where they're building for.

And that means, as Nick just said, they're building out a platform, everybody's got a platform. So, so that sounds more like a, um, like a, a a a Salesforce or a, a ServiceNow kind of play as opposed to more of a traditional enterprise tech kind of play. And you asked about AWS and are they kind of trying to do that?

Yeah, I mean, SageMaker has what hundreds of thousands of, of, of customers and, um, has been around for a very long time. And Bedrock seems to be doing pretty well, um, as well. So I think they, yeah, they all are doing it.

It's, it's obviously leads to, it's, it's sticky, isn't it? It like it leads to, you know, getting out of those platforms once you're, once you're in them, um, to that level of depth. And S3 is the best example, isn't it?

I mean, well, the first kind of cloud, well, the first cloud cloud storage product around. And, um, you know, when you talk to, when you talk privately to some, you know, people from some of these companies, it's like anything we can do to get people to, you know, put more stuff in S3 or Google's case use BigQuery, which is a very successful product, um, is, is an absolute, um, is a, is a kind of mother load for them. So I think, yeah, I think they're all, they're all trying it Microsoft's slightly interesting.

Um, and yeah, maybe, maybe the canst one in some ways. 'cause obviously outsourced the development of, its of the models to open AI and it's now sitting there as an investor in, on a, you know, potentially a, um, eventually one day some sort of great exit. Um, meanwhile it's got obviously its own software, you know, stacks and franchises, if you will, which, which dominate, um, you know, you know, the, uh, the corporate world.

So it's, uh, yeah, it all takes slightly different approaches, but I think, you know, largely they're all trying to do that as well. It's, it's funny, isn't it, that mi Microsoft has a number of times in its history been that sort of weight and then pounce, uh, kind of approach instead of trying to be the one on point taking the, taking the fire. Don't buy Microsoft product until version three type thing.

Sometimes Yeah, definitely not 11. Yeah. I mean, but, but you look what, how far they've come with Azure.

I mean, I, I don't, I'm old enough to remember, and I think you guys are as well, that in the cloud wars Azure was seen as sort of an afterthought. A Oh really? Microsoft, you know, yeah, you're gonna bring Windows to, you know, you're gonna bring a knife to a gunfight here.

Well, nobody's laughing now because Azure was so incredibly enterprise and developer focused that they were able to build it into essentially, um, I dunno if it's a bigger business than their traditional Windows business, but it is a monster. And I think they're trying to do the same in ai, right? Yeah, I'd say so.

And I think it's, um, and you can say the same thing about Google, even at the beginning of 2025, people were saying, yeah, that's not a serious enterprise play. And, um, and now, and now look at it and, you know, and we're honest in the space of 12 to 18 months. So yeah, I, I think it is, um, yeah, I think, yeah, there's, it waxes and wanes.

That's what makes it interesting. That's what keeps us in the job, right, basically, isn't it, as well, indeed. And the waxing and waning, by the way, I just wanna pause for a second there, to, to honor the fact that if you do invest in a given model maker, um, that that can be the most, you know, beneficial and frustrating aspect of that at the same time, because it's every time there's a new model that comes out.

1 with OpenAI, for instance, um, it's, it, the models are a, are a different employee, they become a different, you know, beast that you have to contend with. And you may have a perfectly running, you know, uh, workflows that you've spent, you know, months painstakingly building towards something, you know, that they do exactly what you want and next day they don't. It is, it is crazy.

Sorry, Nick, was that Yes, that you do have something to say about reinvent or No? Yes. That you want me to No, I think, I think I'm done.

I think We're done with reinvent. I sense a transition point that will work. So here we go.

So, you know, you talk about companies like Microsoft who are seen as sort of, I don't wanna say stodgy, but sort of, you know, they're coming later, they're more enterprise focused, they're more, you know, more of a traditional, uh, IT vendor. Um, but there's of course, a, a big daddy traditional IT vendor that made some waves this week as well, this, and frankly, there again, I feel like the market, um, the zeitgeist isn't right when it comes to IBM because you say those three letters to a lot of people and they immediately go, oh, mainframe. Well, yes, mainframe still exist and they're still relevant, but that is not IBM and IBM this week, uh, made huge waves if it's possible to make waves, uh, outside of an AI announcement.

They made huge waves, um, with the announcement that they, uh, are going to be acquiring Confluent, which is a company that maybe not everybody's familiar with, um, but they're acquiring Confluent. And those of us who are kind of insiders in the industry, we're like, oh yeah, like, I feel like the Kool-Aid man here. I'm like bursting through the walls saying, this is the, this is such a great move for IBM, but I think most normal people would be listening to this saying What?

So, um, um, first off, uh, what's your reaction to Confluent and IBM and second? Um, this isn't about ai, or is it, It's totally about AI and yeah, $11 billion of any sort is a big splash, is it not? And the fact that they, they paid this much for a company that, that basically built its business on top of a, uh, very well known, uh, but nonetheless, a piece of open source software in Kafka, uh, which is a, a streaming, um, you know, service, um, says a lot.

And what it says is that IBM gets infrastructure and that, that's like my, like top level give, you know, takeaway from that. And you can see this acquisition building on the HashiCorp acquisition they made earlier this year, and all of that says, infrastructure as code, code is infrastructure. And if you're going to build ai, um, you're gonna need to, to be able to, you know, architect something that can be, you know, posted, hosted and scaled anywhere, any cloud provider, any premises, et cetera.

And you're gonna want something that you can orchestrate in the most effective manner. And by orchestrate, I mean bringing data to ai. And that's what this Confluent acquisition is all about.

You know, we've been building, um, AI systems with, you know, context windows and, and supplementing those, you know, and supplementing the training of the model with context window data, like through rag pipelines and such. And that's great, and it can bring, you know, you're not gonna be indexing that stuff instantaneously, but if instead you could bring data in real time to the models, um, as they are going about the business, and also, you know, bring data out of those models, especially in an agentic workflow, that's gonna mean, you know, whether or not you can actually build and succeed with an idea that you have as a company. If you're gonna build with ai, you need to be thinking about not just static data that gets fed in through the context window.

You need to be thinking about streaming data in real time. Yeah. Excellent, excellent input.

This is Brad's area, very much easy expert here, but, um, I, I, I think it would be, it will be interesting to see, um, how it kind of get ties into the watsonx, um, do AI story. Um, and I think that kind of, you know, that, that the messaging around there is sort of getting, um, you know, is, you know, getting stronger, but I think, you know, may, may change a little bit, um, and the, you know, and the orchestrate product as well. So, uh, yeah, it's very much, it did remind me HashiCorp different, different use case and different technology.

Um, but, uh, but similar, you know, IBM has always had a formidable AI m and a machine. I remember as an analyst event, it must, it was well over a decade ago, we got to sit with their m and a team, I think I remembered and sit there and they were basically telling us like, we're around this table. Let's, let's imagine as if we were gonna buy a company, a made up company, and this is how they go through all due due diligence.

And it's, uh, it's quite a process and it's quite something to see and, uh, you know, could see it's still, it's still executing. Yeah, they're definitely looking ahead once you say Steven, they're, they're definitely looking for the long play here in terms of not being a hyperscaler in terms of, you know, having data gravity, but instead being a hyperscaler in terms of, you know, having an infrastructure that can run anywhere and enable anything. The the thing as well that IBM does so well is at, at least in modern times under this current administration, not so much in the past, but in this, this current ad uh, ad administration at the company is, uh, run these things well in a way that doesn't run off customers, um, that is actually accumulating customers and bringing people into the fold instead of e excluding them from it.

Um, and I think that we've seen that certainly with Red Hat, uh, we've seen that with, uh, Hashi Corp. Uh, another acquisition I wanna bring in here that I think is, uh, parallel here. You're talking Nick, about basically the, the great big acquisition machine.

Uh, all of these acquisitions rhyme, essentially IBM is looking for companies that have just incredible recurring revenue that have, um, you know, customers that all, you know, can, can come into IBM fresh and, and expand within the IBM portfolio of products. But product, uh, they recently purchased Data Stacks as well, which is another, um, incredible open source. I mean, maybe not so high profile, but a open source purveyor, um, for enterprise customers from an Apache product, um, Brad, uh, data stacks plus, uh, uh, Apache Kafka.

I mean, how, how does this work? Yeah, like, like Nick said, it's, it's all about enabling wa the Watsonx portfolio. So watsonx, ai, wa, watsonx, data, watsonx governance, all of that is benefiting from every one of these acquisitions.

And to my mind, it isn't so much about what these products do, what these technologies do, because you can get Kafka from anybody, you know, I could, I can r up on GCP and, and Azure, you get it for free, It's open source baby, right? But, but I mean, like manage, host it. You know, this, this is what cofluent makes its money on Yeah.

Is manage host, right? And, and yet what what is really interesting to me about these acquisitions that we're talking about is that each of them have a very well regarded and global ecosystem, um, that is mature. And I would say, like if I was to look back at IBM over the last five to seven years and say, what's their biggest weakness?

It would be lack of ecosystem. I mean, think about what drives the value of Azure that we've been talking about. It's, it's not the greatness of the software, it's the breadth and commitment of the ecosystem that builds on it.

And for it, you know, not just the the get and, but it, it, it's right. It's huge. So if I can, um, break in with a timely quote to those of us old enough to have seen the film, it reminds me of Han Solo.

Luke Skywalker says, you know, that hunk of junk and Han Solo says, who's gonna fly it? Kid you? Well, that's IBM, right?

That's Red Hat. That's, you know, what's going on here? You know, yeah, you can do this, but who's gonna run it?

Well, we are, we are gonna run it in a way that works. And frankly, that has been a very compelling argument for these, uh, for these companies. Now, another thing that I wanna bring in here, um, hopefully without any more Star Wars quotes, is, um, Nvidia, of course, now I have been really, uh, excited about some of the models that Nvidia has developed.

Um, parakeet, absolutely. Rocks, I think I mentioned that recently. Um, but Nvidia is out now with a, uh, you know, mixture of experts, uh, models.

Tell us a little bit more about that, Nick. Yeah, that it's not supposed to. The mo we were talking about the models and the efficiency of models earlier, but they're talking about techniques in which to optimize mixture of experts models.

And they were, uh, they put out a blog post, um, late last week, I think. Um, I thought it was, it was interesting. They were talking about how, um, 60% of ai, um, of open source AI models released this year are, are MO, let's call it MOE, so we don't have to spell out every time, but mixture of experts, models.

Um, and this is kind of taking over from, from kind of dense transformers, but there's problems, um, with those in terms of, um, memory, bandwidth pressure. And this is on their own, on their H 200 systems they're talking about. So on their own, uh, GPUs, um, there are limitations around memory bandwidth, um, from of constantly loading all the expert parameters and then communication latency, uh, when experts are distributed, um, across more than eight, uh, gpu.

So they were looking for a way to, to solve this problem. And is there MV link technology? So they call this the GB 200 MVL 72, which connects 72 Blackwell GPUs, um, and delivers, you know, I'm not gonna go through all the, all the numbers, but a lot of it's very quick.

Um, and it enables, um, you know, to, it enable, gets rid of the bottleneck by distributing the experts, uh, across up to 72 GPUs, reducing the number of experts on each, on each GPU. And that's, that that relieves the pressure on, on the, on the memory. And so, you know, given the, essentially the, you know, Moes are, are the kind of the way that everything's working now, um, or at least, yeah, this is the models that are gonna be, um, you know, rolled out.

Um, they're looking for obviously, you know, ways to optimize the infrastructure on their infrastructure. And of course, you know, with their MV link technology, um, actually interesting one on the, um, was it TRA three or four is gonna have, um, the AWS one is gonna have, um, uh, Nvidia MV link in it as well. And so I thought, I thought it was just quite interesting, um, how we can, because, you know, in the early days of the transformer models, you know, we're obviously talking about, um, how these models obviously cost an enormous amount of money to, you know, to train to train.

And that's partly because then that the parameters, the way the parameters work, the way the data constraints work, um, and you know, the, the MO the MOE is sort of becoming kind of the standard way of building, um, frontier models. Um, and there's obviously, there's very specific reasons why you would use other niche models, and there's like, obviously diffusion and things like this. Um, but I think it's, it's interesting.

Um, it's obviously, you know, it's self-serving for Nvidia to say, you know, use our, um, infrastructure use I hb two hundreds and use this MV link 72. Um, but it's inter I think how they, you know, and they claimed, I should have mentioned, they claimed it was 10, uh, 10 times performance increase. I should have probably put that higher up in the, uh, in my little, uh, ramble.

But I think, I think I thought it was worth mentioning anyway, because, uh, anything that can optimize these things, um, is, is gonna be of interest to, to a lot of, uh, model trainers. And as we were talking about earlier, there's still a lot of those around. Yeah, we're, we're in the scale out era, are we not?

And, and that optimization is, is gonna make or break investments in data center as well as individual projects. And the skeptic in me, by the way, Nick want, wants to see, um, them do this comparison on a chip that isn't three years old. Um, you know, but, uh, so it is, it is, like you said, it is a bit self-serving, but I, I think they're bringing up some really important points that, um, you know, the architecture of these models, you know, really dictates what you can do in terms of scale out and up with these things.

You know, if you have the greatest model in the world, but it sucks so much VRA just to set it up and you can't scale it across clusters, how much concurrency are you gonna get out of it? You know? And by the way, it's not just big models.

I, I know, um, IBM with their, um, granite models, they, it's like a month or so ago, remember that they released a mixture of experts that's like, uh, under 4 billion parameters that is meant to do the same, to bring the same benefits of an MOE to, you know, on device, you know, inferencing. That's awesome. Yeah.

They're doing a lot of work with small language models at IBM, aren't they? That's very, it's very, yeah, yeah. Differentiator for them as they say it.

Well, I've just, uh, accidentally demonstrated the power of expert mixture of experts models by revealing that I didn't know what this story was all about and calling in an expert who did, which is exactly what vaccine. See, I meant to do that. You routed, I routed it properly.

Um, you know, I wonder, Nick, does this have anything to do with the, uh, uh, rising price and, um, lowered availability of ram? Uh, there's kind of a RAM crisis right now. Um, I'm not a, I'm not a RAM expert.

Um, but I think yeah, it probably does. It has just, and in, and also just, uh, in terms of, you know, squeezing the most out of the, uh, of the assets that are out there, obviously there, there's shortages of all sorts of things, including GPUs themselves. And so I think it's, uh, you know, they, they were talking, um, specifically about, you know, performance per WA and then being able to like an NVIDIA language, them being able to generate more tokens from your AI factory.

These are all the terminology they like to use, but if you can squeeze, if you can get a 10 times, um, increase in performance per watt, that's, that's extremely important in these kind of power constrained and GPU constrained, uh, times in which we, uh, live. Indeed, the data center is limited by the number of watts that it can consume. And, and by the way, I, I understand that that RAM shortage is, is down, so to one individual named Altman, uh, Mr as in, Are you trying to say that he doesn't have a good brain?

Or are, are you trying to say that, that No, That he, he bought up. He bought up. He bought all around.

I know, I know. Sorry, His memory's not what it was. No, he's got the best memory.

Um, alright, well, on that note, um, note, thank you very much. I'm gonna have to route that query to another expert here at futurum, uh, to find out more about that, uh, hardware crisis. Uh, thank you both for joining us for this week's episode of utilizing ai.

I have to say, uh, our producer just ran the numbers and we've got some, some great viewership already with this new podcast. Thank you everyone for listening. Um, please do, uh, drop us a line if you're watching.

Uh, I know you are because we can see the metrics. So, uh, we would love to hear from you. Uh, you can find me on LinkedIn, uh, as s FoST.

Um, Brad, Nick, where can we find you? Brad? Yeah, Brad Shiman on LinkedIn and, uh, on the RUM website itself.

Uh, yeah, Nick, patience on LinkedIn and um, Twitter X and I'm on Blue Sky, uh, and all of our stuff is published on futurum group com. Excellent. And, um, and again, uh, thank you to the Horror Horological Society of New York for hosting me here today in their beautiful, uh, library in Manhattan.

And thank you for listening to this episode of utilizing ai. If you enjoyed this discussion, again, please subscribe. You'll find us on YouTube as well as in your favorite podcast application and do consider giving us a rating and a review since that helps visibility for all podcasts.

Uh, this podcast is brought to you by the analysts and experts from the Futurum Group where Insight meets ai. For show notes and more episodes, head over to Textron ai, which is our AI news site, the utilizing AI YouTube channel or the Techstrong TV app on your TV or smart device. Thanks for listening and we'll catch you next week.

I'm Tom Hollingsworth, event Lead for Security here at Tech Field Day, and here are my takeaways for this special exclusive event with Microsoft Security. We had a great conversation with Microsoft Security around their Sentinel product. It is something that is being transformed to be a critical part of your security infrastructure.

I'd like to take a moment to talk about my three big takeaways from our conversation with Microsoft about what they're doing with Sentinel. My first big takeaway is the evolution of Sentinel from a SIEM to a unified operations platform. Microsoft is fundamentally Rearchitecting Sentinel.

It has historically been a market leading SIEM or security and information event management tool, and it is becoming a full fledged security platform that powers the Microsoft Defender portal. The goal is to eliminate the swivel chair problem. You know, the one where analysts have to jump between different interfaces to get information.

Sentinel functions are being converged into the Microsoft Defender portal, which serves as a primary interface for security operations. Sentinel is becoming the underlying platform engine that supports other Microsoft portals as well. There are some additional capabilities that you'll see in Microsoft purview for data security as well as Microsoft Intra for identity management.

And the platform is designed to be open. It supports OCSF and currently utilizes 350 different connectors to ingest data from third party sources like AWS, Google Cloud and CrowdStrike. My second big takeaway is the data lake.

A major technical and economic takeaway from this event was the introduction of the Sentinel Data Lake. It separates data storage from compute power in order to drastically reduce costs and increase data retention. Previously, customers faced a choice between budget constraints and security visibility.

Things like high volume logs were often way too expensive to ingest into a hot analytics tier. The new data lake functionality built into Sentinel offers a lower cost tier, which is about 6 cents per gigabyte compared to list prices of nearly $4 per gigabyte. It also allows data to be ingested into the hot analytics tier that is automatically mirrored into the data lake at no additional cost that provides a single complete copy of the data.

This architecture allows organizations to store their data for up to 12 years. That's very important for compliance and retro threat hunting capabilities. While the data lake is really considered cold storage, it really supports some high powered analytics.

Users can run high performance jobs using things like Apache Sparks and Jupyter Notebooks directly on the data lake for deep analysis, machine learning training, or historical data analysis. My third big takeaway from this special exclusive event was around graph based security and ag agentic ai. Microsoft is introducing new data modalities and AI protocols to change the way that analysts investigate threats.

They're moving beyond simple tabular data to more conversational interactions. The linchpin of this is the sentinel graph. Think about the way your attackers think about your organization because they do think in graphs.

It's mapping the relationships between assets, users, and data to visualize potential attack paths. It allows analysts to quickly calculate the blast radius of a compromised asset and also predict where an attacker could be moving next, based on things like permissions and network connections, the capability can be used for pre-B breach exposure management, such as identifying choke points and post breach investigation. One of the big components that helps this is the MCP server.

You are probably familiar with MCP as model context protocol. It acts like a catalog that allows AI agents to automatically discover and interact with data tools. This can enable things like natural language search, so analysts can ask questions like, tell me what tables are relevant to this password spraying attack.

Rather than trying to remember the arcane SQL or other database query language to figure out how to get that data. It also enables Ag agentic workflows where AI can not only read the data, but generate things like Python code, create playbooks, and potentially take actions on them. This AI assisted process could be even be considered something like maybe Vibe Hunting or vibe investigation.

When I think about all of the things that Microsoft has introduced to Sentinel, I think about it as maybe a storefront. That's what Sentinel used to be. It was fast and it was easy to access, but just like all storefronts, you, you had to rent space and it became very expensive.

You could only keep your most critical high turnover items on the shelf, and if you had bulk items or inventory that wasn't moving, you basically had to get rid of it because you couldn't afford the shelf space. But now, Microsoft has built this massive warehouse or data lake directly attached to the back of the store. The storage here is well, fairly cheap, and you can keep everything you might ever want to keep for 12 years.

That Unified platform is a single office where you can oversee both the store and the warehouse. Sentinel Graph is kind of like a overarching blueprint that shows exactly where the doors connect to which rooms revealing how someone might break in to the loading dock, and then be able to reach things that are in the back office. And MCP server is basically hiring a team of automated robots that will understand playing English when you tell them, Hey, go find everything in the warehouse related to that shipment from last Tuesday, and then they're gonna run into the warehouse, get all those boxes, and probably even write a report for you.

There are a lot of things that Microsoft is building on top of Sentinel now that they have a robust, very functional platform, and we are gonna be hearing more about them in 2026. As we continue to monitor these things, we hope that you'll head over to the tech field, a YouTube channel to check out the videos from this special Microsoft exclusive security event, and we hope to hear your comments and your perspectives on these technologies. Thank you very much for watching this episode of Tech Field Day takeaways on the Tech Field Day plus YouTube channel.

If you enjoyed it, please make sure you like, subscribe and share your thoughts on Microsoft security in the comments. You can also follow Tech Field Day on X, Twitter, blue Sky and Mastodon for updates, and check out all of our presentation videos on the Tech Field Day website and our YouTube channels. Our next event is AI Infrastructure Field Day, which is taking place January 28th through the 30th, 2026.

Make sure you're tuned in live on our website, on our LinkedIn page, and on Techstrong tv. Uh, hi everybody. Welcome to the session.

We are gonna talk about, uh, AI and how we can protect local AI deployments. Um, as you're probably aware, AI is the, the hottest, uh, trend in the market nowadays. And, uh, it doesn't just divide, right?

We, we know what the numbers are. The numbers are that there is a lot of push towards AI in organizations. We see upwards of 90% of organizations that are actually able to, uh, think about what their AI strategy is, uh, not that they have a lot of, uh, advancement there, right?

Only 5% of organizations actually do have a clear plan, but everybody thinks that they need to do something with, uh, with ai. And, uh, what we're seeing though is that security and specifically the governance around AI is lagging, right? We have statistics that tell us that many organizations still don't have a plan on how to tackle, uh, ai, uh, deployments if they're done locally.

Uh, we still have, uh, very little runtime controls. We have very little governance, uh, even though the regulation is starting to creep up. And we have now some frameworks that we can talk about, uh, with regards to how we wanna, uh, posture our a our ai, uh, but the end result is that, uh, AI is here.

Um, AI deployments are here, there are a lot of use cases for them. And in the next 30 minutes or so, we're gonna break down what are the, um, problems that we wanna solve, and also what are the solutions or how do solutions should look like if we're going to have a good secure AI deployment. But before we get into that, why do we even wanna have AI in, in an organization?

What, what's the benefit, right? Everybody tells you that we need to support it, but what, what, what are really the use cases? So there are actually three kind of main broad based use cases around around ai.

One is the one I think that we're most familiar with, which is the ability to do some chat with, uh, an ai, uh, chat bot. Um, so that would be your Chad g, pt or, you know, grok, your Gemini, your copilot, uh, and so on. And this is between the user and the AI engine.

Um, it, it's a very valid use case. Uh, there's a lot to say about that, but the purpose of this presentation is actually the other two use cases is what happens when we start to develop application that start to make use of AI components. And it could be that the application itself is using AI to do some backend processing, or it could be that the application is actually providing a natural language interface, and then users are able to, uh, interact with the software instead of clicking check boxes and, and prompts and, and, uh, and, uh, you know, uh, wizards, we are, um, actually talking to an application, um, like we would to a human that might need to give us that, that service.

But when we do that, we gotta remember that the basic flow of ai, the way that we see it from our lived experience, you know, talking to a jet chat bot is actually a little bit more complex, right? If we think that we're just asking, uh, AI a question, so we give it a prompt. The prompt has a, um, a, uh, uh, inference server that, uh, the information is being given to, and then the, uh, AI model is the one that's generating the response.

Uh, this three step process is actually very, very rudimentary, right? Our lived experience is, is not indicating everything that actually goes on behind the scenes. And, and, and the reason is that the model itself, if we talk about, you know, what, what is an AI model?

It's really like, uh, like a brain in a box, right? It, it knows what it knows, it knows what it was trained on, uh, but it really doesn't know the specifics of a particular organization or a particular, uh, company, um, unless you train it specific for that company. So if you're gonna use an AI uh, infrastructure, one of the things that you need to ask yourself is, am I gonna, you know, use a generic model and then try to augment the input?

Or am I gonna train the model just for me? And this is really, really important because if you don't train the model and you provide an application, let's say it's a banking application, and you're, you, you're, you're trying to, uh, make the user do something like, you know, transfer money from checking to savings. Uh, if you just give it, if you just give this prompt to a generic AI that hasn't been trained or hasn't been augmented, uh, with the specifics of the, the organization that you're talking to, if you're like, let's say the bank that wanna provide a service, um, you're gonna get a very generic response.

Uh, and if you experience this as a user, right? And your banking application, if I went to my banking application and, and submitted that prompt and got that response, my experience is not gonna be very good, right? Because it's, I di I didn't ask how generically to do it.

I just wanted the system to do what I asked it to do. And this is where we gotta understand that the, the, the prompt that we're sending an AI application, even the simple chat bot, even just the generic chat, GPT in a private window, uh, is still going to have a lot of decoration and information around it. Because the prompt that you're gonna send in our, you know, fictional application, we'll eventually at the bottom have what the user requested.

But at the top, there is a lot of information that is provided to the model around what the user is actually trying to do. You know, what, first of all defines the, the, um, role of the AI system itself, I banking application. Uh, it defines the role of the user.

What is the user allowed or not allowed to do? Um, it is defines what's tools and agents the AI application is, is, uh, able to use. And when we talk to, you know, if you hear the term agent ai, that is really just AI taking action as if it was a user and not just providing answers.

And if you want the AI to take action, we gotta be very, very specific in what action we want the, the AI to take, right? So our, our model is actually, uh, getting a lot of information that is not specifically referred to in the problem, but is inferred by the fact that, that we have our application is the one that, that crafts the prompt. So all the things that the, that the model does need, like user data, like, you know, what is the, uh, permitted actions?

What is the mood of the user? What is the intent of the user, what tools it can use, what tools it can't use. All of these have to be, uh, defined in advance in the context of our application.

So when we're building an AI infrastructure, we are really building not just a prompt, uh, to a model. We have a lot of other tools around it that helps us build that model to, uh, the extent that it can be used by the, the application. So that would mean that our prompts might go through a gateway to validate the, uh, identity of the user and tell us what it can do and cannot do.

Uh, there is a lot of context. So if you hear the term, uh, MCP, the, the, uh, uh, context port protocol for models, that is how we augment the prompt. That is how we, uh, provide the prompt with, um, ancillary information that might have changed since it was trained.

Uh, you might hear the word rag. So that is a way to provide, uh, documents and extra, uh, stuff that the, uh, model can draw on in order to provide its answer. Um, there are tools that can provide us with the ability of the, the AI engine to do something in, in the real world.

And then we have the model itself. And that model, if you're doing it internally, and you don't have to host the model internally, but if you even host the model internally, you also have the hardware requirements of, of the GPU, right? So it's a little bit more complex than the kind of the three step process that, that we saw earlier.

Uh, but if you wanna deploy all of these components in house, those are actually gonna translate into quite a lot of, uh, let's face it, Kubernetes deployments, right? At the end of the day, uh, we are really talking about Kubernetes, and we're really talking about how we can protect the infrastructure that, that, that you're gonna run, uh, in-house. So if we're gonna translate all of these to a set of Kubernetes deployment, it might look something like this, it might look like a, a set of deployments that are in Kubernetes.

Some of them are gonna deal with the application. So the top line is probably something that is more familiar to you as people who write or maybe manage applications in, uh, in organizations. And then the rest of it is driving the AI process, right?

Augmenting the data, making sure that we have all the information we need in order to successfully, uh, execute on that, that transaction. Now, this is a lot of infrastructure, right? So a lot of infrastructure that lives in containers, um, we know that infrastructure is containers is a target for, uh, bad actors that might wanna do something.

But when we are dealing with AI that is wrapped in containers, there is, uh, uh, a, uh, a way that attackers might be able to go into this environment that is outside the normal ways that we know and probably can deal with, uh, when we're dealing with kind of classic security, uh, around, uh, intrusion detection and just protecting the, the infrastructure itself. And, and the fact is that, that, that there are pretty ingenious ways in which, um, ban actors are starting to exploit the way that, uh, AI systems are built in, in, in organizations. It means that, um, there could be, uh, prompt injections that, uh, make use of tools.

It could be, uh, leaks, uh, of data by the model, right? The, the model, for instance, in, in the example that we talked about, the bank account might return a reply, yes, I've successfully transferred money from checking account number, this and that to, you know, from the check into the savings and reveal the account numbers. Uh, now this is an unintended consequence, but we gotta make sure that that doesn't happen, right?

So, so how can you, uh, make sure that, that your model is not divulging information as as it's doing, its its transactions. And then there are, there are all these, uh, abilities that, that, that we need to have in order to, to safeguard our, our AI infrastructure, uh, from, uh, from attacks. So what are we missing here?

Right? There, there's a big infrastructure that that's being put in place, uh, both internal and external to the organization. Information is gonna flow between different components of that, that, that, that infrastructure.

And it leaves a pretty big gap for security org, uh, organizations to, to fill inside, inside company, right? 'cause currently there's really very little visibility as to what ai, um, actually does. Where does it look?

What, what is it doing in the, in the environment? Um, there's really no way to, to govern AI in its current state, uh, that it, that it follows any kind of of policy, right? We don't have, uh, still the, the ability to, to impact like system prompts based on organizational policy.

Um, there is no real protection against, uh, uh, prompt attacks that might be, uh, enticing the, uh, AI system to do something that it shouldn't, especially if we're using tools. Uh, it's very easy to guilt and, and, and have, uh, emotional manipulation of AI systems to make them do something, uh, that that, that they don't want to do. Um, and, and if we're gonna put security in place, because we are running fast, because we have a lot of, uh, very cutting edge technology development taken on in, in many places at once, um, it's, it's really hard to tell developers and the people that, that run the AI infrastructures and, and the businesses that push for the adoption of these AI infrastructures that they need to stop.

And, and let's just put a lot of security into this, right? There's a lot of resistance from the, the business, from the applications team, from the, the, the development teams to, to slow them down and, and introduce more, more things. So there, there are challenges, right?

There are challenges in the fact that we we're not sure what to do as, as security professionals, right? This is very new. Uh, we might not have the tools in place, and, and we might not even have the political will of an organization to, to take the, in my opinion, very necessary pause to see how we actually go, go, go, going to absorb this thing.

Now, it's all, it's not all bad news, right? There are, uh, very concrete things that security professionals, um, and cloud native professionals can do in order to stabilize and start to govern those ai uh, uh, deployments. But it, it requires some actions and requires some, some idea about what, what we need to do.

So if you're gonna go over the, the kind of required capabilities of what ai, uh, security might, might, might, might look like, um, there are some questions that that, that they need to answer, right? They, they need to answer. First of all, where is AI being used, right?

Is it used by users? Is it used by applications? Where is it deployed?

Is it deployed? Where we want it to be deployed? Are the right models in place, right?

There's, there's a lot of, just needs to understand what is actually in there in the, in the infrastructure, because most of that infrastructure is gonna run in containerized applications. Our, our first order of business is to make sure that we understand what's in those, those images. So what, what are we even running?

What, what are, what AI components are, are existing in images? Um, are we getting images from AI vendors? Like, are we getting an NCP for instance from anthropic that might have some vulnerabilities in it?

Uh, are we, uh, doing the most secure configuration that we can around those AI services, right? So even when we are building images, when we're building the, the, the building blocks of our service, service by service, we have to understand what are the components that are going in and what is the level of risk of each of those components? And when we put all this together and actually start to provide that application to our user base, um, then the question is, first of all, how can, uh, bad AI actors break it, right?

What are the limits on that we can put around, around our AI services? Um, how can we avoid, you know, resource overrun? How can we ensure segregation of data, uh, if an incident happens in an AI system, right?

Uh, prompts are very ephemeral. They, they, they can be, uh, uh, uh, issued by a user. Um, and then, um, you get a, a response, but then the same prompt can be issued by another user and get a completely different response based on context, right?

So, so how do you manage an incident where the root cause can be either something that is, uh, uh, introducing risk on one hand, but can also be, um, very few of risk on, on the other hand, right? We have to have, um, some way of, of, of, of dealing with that. Now not all these questions have answers today, but I think the purpose of this session is to start to get you thinking about what are the things that you need to, to provide as far as capabilities when you are designing your, your security around around ai.

So let's kind of jump right into what's, what's needed, right? So what's really needed is, first of all, an inventory of the models that are applicable to the organization that wants to use them, what is approved for use in the organization, and really just have an inventory of everything that, um, is running that has AI implications. Uh, it could be just analytics around the users.

It could be what tools are being used. Um, we would really like to get an echo of the prompt. Uh, this is a little bit iffy because it might contain some sensitive information, so we gotta do that carefully.

But understanding the prompt is absolutely something that is, is required if we're gonna have good, good AI protection. Um, we gotta conserve resources. If you're gonna run the inference server, uh, or even the model train internally, you've gotta manage GPUs.

Those are very expensive, and the time slices, uh, are, are, are very precious and wanna make sure that nothing gets wasted. Um, and then the responses, as we mentioned earlier, you know, sometimes the, the model might, might, might divulge some information during the response that that is not really applicable to, to the security and, uh, privacy governance that, that, that is, is, uh, governing what the application is doing. So all of that are just, uh, a, a way to tell you that we need to have some rules in place, right?

Organizations need to decide which AI systems are, uh, appropriate for their environment in what use case, in what capacity. And then we gotta make sure that, uh, the development organization is, uh, absolutely endorsing those and that they have some guardrails that if they go beyond them, somebody would know about it and, and, and, and will, will be able to, to react. And the way that we do that is, is the way that we have to start to look at how those images that carry the, uh, ai, uh, services are, are manufactured and are, um, are built in the organization.

So it has to do with what are the sources of the components, right? Are we, are we running approved operating systems? Uh, are we running the, uh, uh, models, uh, and the all the other components, you know, MC ps, inference servers, all that from trusted sources.

Uh, and, and we got, we gotta make sure that we have the right, um, uh, SBO m around them so that we have an inventory, everything that goes into those images. Uh, understanding the model, understanding what vulnerability might be at images. Uh, a good strategy is to try to have leaner images, right?

That's just a good security practice, right? Having leaner images without bloat, uh, and, and without too, too many, uh, components that are not required. And then, of course, if the image is gonna carry any kind of action, it needs to be very contained and, uh, the image configuration it need to be secured, right?

There's, there's absolutely no reason to run any image in an ai, uh, capacity or any other capacity as, as a route or a privileged user. But with ai, it is actually really, really important because if we have a trick, a prompt to do something that goes beyond what the, um, AI engine, uh, uh, can do, especially if it's using tools or agents, uh, having them run as privileged users is, is a very risky proposition because the prompts are non-deterministic. And we really don't know what's gonna happen, uh, if, if a prompt is issued.

So we really got, got, got, uh, could contain it upfront. So we talk about two things, right? We talk about policies, we talk about the, the ability to understand what, uh, is required and not required in the, in the environment.

And then how we start to control it from the, the, the supply chain side and, and the images. Uh, on the other hand, what's really, really important is once we have AI components being put into play, really right at, at, at runtime, um, we are looking for a, um, we are looking for a way to provide just visibility into everything that the AI engine, uh, provides. Uh, so whether or not a workload can be accepted into the environment, right?

If, whether or not an image is good enough, uh, or secure enough or risk-free enough to, uh, be included in, in our stack, all the way to controlling how we can access the GPU, uh, and then just general security, right? Behavioral detections around, uh, intrusion detection for containers, but then extend that to alerting on dangerous prompts, uh, to understand what executables are being launched by the, uh, entrance server or the MCP or, uh, any other component that is tasked with executing what the AI engine will eventually need to do. Um, and if we can wrap it up with some network isolation, that's even better, right?

Because, because we, we, just because we don't know what the AI system will do, again, prompts are sometimes unexpected. Uh, it's better to surround the whole thing with a little bit of a, of a, of a fence so that we, uh, don't get spillage of, uh, bad actions, just, just because, uh, an n AI engine, uh, was, um, was, was brought in into play. So to wrap everything together, it really is.

And if you, if you've done any kind of container security, it, it actually is, is really easy to see how everything kind of fits together, right? Because, um, a AI security is really an extension of container security, especially if you run it your AI in containers. So just doing the basis right, removing bloat and risk from the images, uh, scanning and, and, and properly gating your development lifecycle.

And then just managing images with, uh, the least, least privileges and, uh, and making sure that we have good, good containment around them. Uh, that begins with inventory, it begins with policies for risk. Acceptance is begins with, you know, secure sourcing over, over the pipeline that then progresses into the runtime controls, um, detections, uh, jail break, detections, uh, guard rails, executions, uh, and so on.

And everything has to wrap, be wrapped up with visibility and, and transparency, right? We, we wanna make sure that we understand what AI models are being used, what AI infrastructure is being used, and then put together the necessary program in order to execute our, our, our controls. So when we talk about our controls, and this is gonna be the, the last slide and probably your takeaway into what really needs to be done, um, the controls that we wanna put in place, there's actually kind of four categories of them.

One is, is accurate code scanning and identifying what models are being used, what clients are being used, what SDKs are being used, uh, where they're being used, uh, so that we have an, an ability to maybe stop some of those deployments before they go into, into productions. Uh, if they violate our policies or any of the regulatory frameworks that are now starting to come up and, uh, and, and present some, some requirements, um, we really have to have security gates. Uh, without those, everything kind of falls apart.

So if we can't stop an image from progressing, first of all, from development, maybe into the registry, and then from the registry into our cluster, um, we won't be able to, to execute, uh, the right controls. So we, we absolutely have to have guardrails in place that, that have some ability to delay or stop the rollout of images that are not, uh, in line with, with, with our security practices. Uh, and then once those images are running, we really have to identify whether or not, uh, an attack takes place.

We need to understand if a, a prompt has, uh, let's say private information in it, if the response has private information in it, if there is attempts to, uh, you know, ignore all previous instructions. Uh, some models, you know, most models now are resistant to that, but there are very, very ingenious ways, uh, to cause a model to do something that it doesn't wanna do. Uh, if you ever see a model, uh, an AI chat that doesn't want to do something, just tell them that your deceased grandmother promised on you, promised your grandmother on her deathbed that you, that you would, that the model would do something.

And you can actually fool the, uh, an AI model to do something with some emotional manipulation. So, so we, we need to identify those, right? We need to identify prompts where bad faith actors are, are actually, uh, are trying to, to circumvent the system in, in more or less sophisticated ways.

And because everything is connected and because we want to trace back, right? If we had a bad prompt that started some, some incident, how do we trace it back to what actually, uh, caused it? Uh, the root cause analysis really requires us to connect everything.

We have to understand where images are coming from, what components are there, what vulnerabilities are there. So if we have a prompt that tries to do an action that exploits a vulnerability that is because of a certain component in the image, we gotta have visibility into that because we need to solve this very, very quickly, right? Once AI becomes indispensable in the sense that any disruption in the AI service becomes a disruption in the application, um, then that becomes extremely important.

So understanding the root cause requires us to connect everything together. So, as I said in the beginning, right, this is about how applications are using ai. This is about what stacks are going to be required in order to use ai.

And not all the stack might run in your environment. You just might run the client or a gateway or the MCP server, uh, and the model might be elsewhere. That's fine.

The, the ability to control the prompt actually starts from the client application side, which is happily where we usually, uh, uh, have some impact. Uh, and from there, it's really about having good security practices for containers, doing this little add-on of prompt and, and understanding what the flow is of the AI transaction. And if we do all that, we should be able to have a good, secure foundation to run our AI systems.

Whether or not you go all the way to the model or whether or not you just stopped at the inference, uh, or anything, uh, um, uh, prior to that, uh, you should have the ability to have confidence that you're gonna deploy your, your applications correctly if you have all these capabilities that we, we outline. So that's my word for today. I really appreciate you, uh, um, connecting to this, um, to this call.

com where we have a whole page about AI risk and how to deal with it. So thank you very much.