Will AI Agents Accelerate Cloud-Native Adoption? | TSG Ep. 958

Hey, everybody. Happy Monday. And are you ready for Q coupon?

Well, you will be in a minute. We'll be right back. Welcome back, everybody.

Today's edition of Textron Gang is gonna have our usual, some of our usual stars starting with Mitch Ashley. Mitch, how you doing? Good to see you.

Good, good morning. Good morning. Also joined by Kate Scarsella and Jack Poller, who are gonna lend their insights into all kinds of fun stuff.

But first we're gonna start with, well, what's going on with Cloud native? Because not only did we host a Cloud Native now event last week, but cube con's coming up in a week or so, and we have some articles over on Cloud native now talking about how, uh, maybe AI is gonna drive more people to build cloud native applications. 'cause well, maybe it'll get easier, but Mitch, I know you were at the, uh, cloud Native NOW event and you gave a presentation there.

What's your overall assessment right now? Where are we on this March to cloud native? Well, we're all watching the, it's like, like an F1 race, and we're sitting at one corner, right?

You kind of see it zoom by for a minute and they're like, okay, what's happening in the next lap? It just goes so fast. The, um, the, the market is starting to, to make the next shift, and this is the very leading edge of it where you see vendors that are, that are introducing, um, agent control planes and management of multiple agents doing development work, coding, coding agents.

And there's a great article that, uh, we had on, on, uh, cloud Native now that also tied this in. I think it was na, Nathan, Eddie that wrote it. Really good perspective on it.

But I think what we're seeing, and it's not just there, but it's mostly on kind of the developer front end. And then we see it with code reviews. We see some security guard rails starting to be implemented.

Of course, uh, GitHub had their, uh, universe, uh, conference last week and, and made big, big announcements about Agent HQ and are trying to be sort of the neutral platform to bring your agents, bring your model, we'll, we'll run it on our control plane, et cetera. Um, cursor came out with their own kind of similar, we have our own coding model called, called Compose. And, uh, we're also will help you run multiple agents in parallel.

Uh, OpenAI came out with, um, a, uh, a vulnerability repair capability within, within their model. So you're starting to see them, see the vendors sort of try to take it up to the next level. And I think it's a lot of, it's just a race for the, for the ground, right?

Everybody's wants to plant the flag. And while the Oklahoma rush is going on, Kate, I don't think it's any secret that building these apps is hard and deploying them on Kubernetes isn't always a joy. But do you think, or do we have hopes that maybe AI agents will make this easier and more accessible and maybe we will build more of these applications faster?

What do you say? I personally think, yes, we need to embrace this technology. We need to, of course put some sort of guardrails, which presently we, I, we don't have a lot of guardrails around that, but absolutely, I, I think that we need to embrace this technology and we need to understand it so that we're able to start to do a better job at securing it.

But yes. All right. Jack, any thoughts here?

Do you hope that this comes about or do you think that this is just gonna be more trouble than it's worth? Uh, I don't think it's more trouble than it's worth. What I'm trying to understand is how much of the effort is being driven by AI improving the developer's, uh, life and or ai or the end user application that the developer is building, right?

And I think right now we're sort of seeing a lot of focus on developer activities and how do we improve the developer activ and hopefully as a result, the quality of the code they output. And more importantly, from my perspective, the security of the code they output, I think it's still gonna be a while before we get to the point where they're able to really embed AI and particularly agent AI into the final applications. Um, let's see.

I think there's two things to unwrap there, and I'll go to Mitch for the first one. Um, I think most of the AI applications that people are building these days is run on Kubernetes, and I think most of them are cloud native by definition, because well, they're so large that there's no other way to kind of get after this thing other than to break it up into a bunch of microservices. So do you think that as that occurs, that that just naturally pulls more people into building cloud native applications per Jack's point?

Well, first of all, um, we, we have some data in my research from buyers that indicates, it says AI is the number one workload that they're working to put onto Kubernetes. Um, but the point of it is actually that everything's going on Kubernetes cloud native applications, but everything else too, database as you name it, it's, it's the, it's the workload platform now, kind of just above the operating system, uh, by default. Uh, to your, to your point about cloud native applications, there's a sort of a natural affinity between microservices and agents.

They kind of had look a little similar, um, but they don't function the same way. But if you can imagine, you know, a group of agents doing different, uh, tasks, a part of completing some process, just like a group of microservices do now, are those agents running under Kubernetes themselves? Are they running in a, in a different control plane?

Um, like in an Agent HQ or something like that? Is is a different story that we'll see how that shakes out. But I gotta believe under underlying all of it, there will be Kubernetes there.

And I think we'll see this real mix of, uh, cloud native microservices and sitting right beside them, you know, agents that are doing some, some workloads. Increasingly to Jack's point, it's like, it's not like turn the page and it's all agents now. It's gonna slowly, uh, be put into production over time.

Mm-hmm. Okay. I'm trying to figure this one out.

Are people just gonna take their legacy stuff and just toss it out the window and deploy something new that feels like a cloud native microservices application? Or will they try to maybe use AI to take those applications apart and peel 'em off into a set of microservices that, you know, might take 'em a few years to do, but they're gonna keep the core app in place? Or is it just time to get rid of the stuff?

Well, from previous podcasts, you probably have heard me say like, yes, we need just to get rid of it. That's my own personal opinion. Um, because sometimes it takes a heck of, you know, it's like if you think about doing a, a project in your home, it can sometimes take on, if you're trying to save all these, you know, items, it, it can, it, it's painful.

Sometimes it's just better just to take the wrecking ball, um, not to the White House, but in other situations. Um, in this case, it's really, I believe that it's built on blocks, um, that perhaps have been, that is not secure. We're getting some new areas that we can really do this right.

And I think to do this right, we need to think differently and we need to put it in place and, and build the right trust frameworks around it. I, I think when we're building this, now, let's do the right foundation. We know what's wrong, we've seen it, let's do this.

Right. So yeah. Mitch, do you agree with that?

Because a lot of the legacy applications were written by people who I hope are now sitting on a beach somewhere probably laughing their asses off and running. This stuff was documented, and I don't think AI is gonna actually go in and maybe fully document all those things. So why not just start over?

Well, my, uh, one axiom I've, I've come to agree with or put together part of my career is no technology ever actually dies. It's all still out there in somewhere some form someday. And, uh, you know, we say, Hey, that's the, you know, the, the old, uh, IAM and the old, uh, IIBM databases, those things will go away.

It's all gonna be relational. No, no, they're, they're out there too. Um, but I think to what you're talking about that you were asking, Kate, is there are efforts to specialize AI to help you modernize the applications, not just virtualization of it, but actually the app itself, and in my experience in my career is you rarely wholesale replace a whole application.

That's usually a career ending move. Um, you, you decide a different strategy. And so for example, IBM has something they call Project Bob.

I call it Bob's your uncle. Now it's Bob's your coding agent, but it, and it its specialty is understanding and help you modernize code base. The big issue for modernizing today is there's so much job out there and trying to maintain, uh, consistency with the new versions of Java is a lot of code to update.

So I, my, one of the things I did earlier in my career as about 10 years ago is the head of monolith application that we really wanted to, to do something with. Um, but the people who built it were on the beach, right? And they were not coming back.

We could not bring 'em back. So we actually carved out a part of it, a small part of it, 'cause we didn't understand most of the app, and then re rebuilt that in a way that then we could actually with microservices and Kubernetes, and then we could enhance it and do that kind of thing. I think AI will help us speed those kinds of things up.

So you may peel off part of an app and say, great, let's, first of all, let's understand it. 'cause nobody can understand that code anymore. Nobody knows it.

AI will help us with that too. The only thing I'll add is, you know, working for IBM, um, from 1999 into 2000, remember the whole Y 2K thing? Oh yes.

You know, Do you know that we actually would shut down, um, mainframes and if we didn't get a call, we'd be like, okay, we're good. And that's how, that's how we, um, basically, uh, went through getting rid of things that were no longer needed. I mean, it's, it sounds crazy that we did it like that, but, you know, sounds, Sounds like streaming services at my house.

No, no. Just haven't called yet. So I guess nobody uses that anymore.

True too. I think microservices are similar. If you turn those off, people go, oh, I guess we're not using that one anymore either.

Right? So, big applause jar. If the security people had a vote in this, where would they vote?

Would they vote for trashing the legacy code and just starting over again and maybe, maybe this time building security in from the beginning? What do you say? Well, I think most security people would actually be for trashing the, uh, the existing code and not starting over again because we wanna reduce the attack surface.

But since not starting over, Thank you. It's not an option. We have to do something.

I think security people would really like to start with security by design, right? Is if you're going to do something, don't bolt on security after the fact. Think about security as you're architecting and designing the thing.

And you know, that's true. Whether you're re-architecting it or redesigning it from scratch, or, Hey, we need to make this modification. If we're gonna make this modification, how are we gonna secure it?

Right. And, you know, I am, I, I don't know, I can't beat that drum loud enough, right? Think about security, Jack.

Think about the market. If some model maker, some product maker came out with, I call it security OG at the point of origin, right? Right.

Where we create the code, we combine it with other things, we configure it, we set up the environment, uh, using ai. If, if a company came out with that and really made some big progress, man, talk about that would be, that would be the kind of the golden egg that the goose laid it, it would be. And I think, you know, as, as we think about AI and AI in the Kubernetes environment, there is an, it is very easy for us to conflate security and safety, right?

And I wanna make that distinction. Mm-hmm. When we think about AI in the world, safety to me is all about things like, does the AI follow your guardrails and prevent you from, you know, if, can you, can you force the AI to tell you how to build a bomb when the AI is there as a coding agent?

That safety and security is really about how does it interact with the rest of the world? Can somebody exploit it to attack your environment? Which is sort of two separate things, or use, you know, is it, you know, expanding your attack surface in the Kubernetes environment when we're using AI in terms of coding assistance and how do we accelerate our development process?

There's a tremendous amount of advantage we get, but we also should think about how do we use that AI to improve our security, to reduce the attack surface at the same time. And that's really, I think where we can, you know, companies that think about it that way and look at it and say, this AI that's gonna be a coding assistance is always, it's prime directive is always gonna be to generate secure code or to, uh, analyze your code that you'll look, that you've created for security as well as just for traditional, what we call bugs, right? And I think if you are, you're developing, uh, tools for the developers that help them security, which is a very complex and hard to understand topic, then you're gonna be, you know, you'll have a winner on your hands.

What's interesting, Jack, is you brought up, when you started to, um, speak about safety and security, which is definitely an OT pros, uh, position, operational technology, POI position. And traditionally from an IT perspective, we were always on the, you know, CIA, you know, confidentiality, integrity, and availability. Yeah.

So I find it interesting that going forward we are thinking more, more from a critical infrastructure point of view from safety and security. Um, and I think that that's important as we move forward. If we were to put that type of thought process in our heads as yeah, as we, as we, you know, define this path Well, and, and I, I like your, your concept of the OT versus the it right?

In a different perspective. And I actually think we need to take it one step further and we need to think about everything in terms of the benefits and risks to the business overall. So from a, you know, from a risk to the business, security is very, very important.

But if you're developing an application that includes an LLM or an agent of some form or another, then the other, the safety aspects is also critical. The classic case we all sort of hear about in the press is the AI agent that, uh, promised free flights to somebody when it should not have, right? And, and that's the safety aspect of it, is there's no, it wasn't a security issue.

The user of the LLM didn't exploit it incorrectly, just the LLM gave the wrong answers. But that's still a risk to the company, both a financial risk, a reputational risk, possibly compliance risk, all of those types of things. So as we make it easier, uh, for developers to include AI capabilities, particularly AI agents which have free reign to wander around through the, all the data of a company, safety and security become more and more of a concern as opposed to bare basic functionality and bugs, at least in my opinion.

Here's my whole problem with the modernization storyline, and Mitch, correct me if I'm wrong here, but basically before ai, it was, uh, yeah, we're gonna come in and modernize your app, and here's these five really bright people that are gonna do it. And then, you know, when the day came, the bus showed up and about 50 graduate students moved in for about a year or more and basically crashed your house. And now we're saying progress.

It's great. Those same students are gonna show up and they're only gonna stay for six months because of ai. It just doesn't sound very appealing.

Tell me, Believe it or not, I was one of those graduate students showed up at a bank, we're gonna rewrite all your banking systems. So we did. But it's pretty amazing.

It, it's, it's, it is a familiar model. Maybe it is, maybe it's not, you know, 50 graduate students, maybe it's 10 of 'em, and you know, a season Kate or Jack or you or somebody, you know, directing that, that knows the domain. And that's what we're looking at.

I mean, that's, that's, I think that's what people expect to happen. It's not necessarily a great thing for people starting their career, but also I think there's a path for them too. It isn't just this, the, uh, indentured servant approach that, that I took through the consulting companies.

There you go. All right, folks, I'm gonna leave this one here. We will be at Cube Con, we'll be on the show floor, so come on by and say hello at the very least.

And also, there's an article by Alan Shimmel over on the cloud native now talking about all the great things that are happening in Cube Con. So you should check that out because, well, there's just a lot to do down in Atlanta, and we look forward to being there. And also by all means, if you can't make Q con or if you're just a glutton for the content, check out our little virtual event from last week.

It's unavailable online, and we expect you to guys get a huge amount of content. Mitch has a whole presentation there that's worth checking out for sure. And we'll be pack in a minute.

You've earned it. The spotlight, the responsibility, the weight of teams, companies, and entire industries fall on your shoulders, lives depend on your decisions, your home life included, that work your protected physically and digitally. Nothing gets through your team without a fight.

But in a globally connected world, everyone sees you, including those who mean to cause you and your organization harm. And now home your sanctuary attackers see an opportunity. Your digital front door is wide open.

And what compromises your home can breach your boardroom. Because the devil's greatest trick isn't targeting your workplace firewall. It's convincing you that your personal life isn't at risk.

Black clerk, digital executive protection, defending the new attack surface your personal life. Hey folks, we're back. I'm gonna have a little chat about cloud outages again, because now it's Microsoft's turn.

First it was AWS and then Microsoft had an outage. And maybe these things are just gonna become regular routines and we gotta figure out how to learn to live with all this stuff. But Kate, what's your take on what's going on here?

Is this just gonna be something that we have to live with because, well, these things are not too big to fail and they have some sort of single point of failure in 'em, and we just gonna randomly discover this from time to time. Well, so first, um, just to review the, there was an eight hour Azure outage this week, um, knocked out services worldwide, which we all know. And as you said about AWS same thing.

What I continue to think about is I come to hate the word and even being the cyber security person that I am in resilient. And I, and I've thought about why do I hate this word? So it just takes me back to the beginning of, um, you know, we used to talk about five nines.

If we were to think about building our cloud in a five, nine more from a redundancy framework mindset, I wonder if it would change, um, us away from the idea of resilience. And I, it sounds crazy, like we're talking about redundancy resilience, but I do think that if I think about five nines I notice in my life, typically I have a backup to a backup to a backup. And it seems to be across the board, it, it, it happens to me personally as well.

I'm like, oh, I have a backup and I have a backup. You know, that's the problem. When we, when we look at cloud, when we look at cloud, we, we don't think of redundancy.

I think we think as we program, oh, it's redundant because it's in the cloud, but it isn't. And we see that with, with both, you know, big cloud. And I know I've sort of gone off script here, but it's just this, you know, morning type of, uh, um, idea of what are we doing?

And when we think about resilience, we can't just think cloud, like cloud is our resilience. It isn't, we have to think of it from a redundancy perspective. We have to think if this, if, if east part goes down, you know, why do we not have it globally, um, in such a way that if one part of this goes down is it's gonna be somewhere else, we should be doing five nines and not think of this from a resilient perspective.

So that's my take. You Know, Kate, it kinda reminds me of CrowdStrike where pushing out a change, which is apparently some configuration change, is what caused this in the content distribution. Uh, the whole staging of rolling it out, right?

You know, push it all out at once. I don't know how this is rolled out, but usually those are the, those are the circuit breakers, right? They'd like trip like, well, wait a minute, this didn't come up, or, you know, it worked fine at three, but now we've put on 30 and it's not working anymore.

So something in that, you know, it's, it's good to say we want it to be resilient. And also of course, you know, back up and redundancy, but also sort of like, don't change everything. Don't paint your whole house until you've looked at what color you're gonna use, right?

I think Jack, to Kate's point, those extra nines in that five nine equation are frigging expensive. And I think people know that. And they, one of the reasons that things are not as resilient as they should be, as people don't wanna pay for it.

It's just, you know, the level of redundancy, the backup, the ability to switch networks on a dime is just get out all pricey. And people look at that and they go, well, you know, that's nice, but I'll take the risk. And if it goes down, you know, somebody will yell at me later, but I don't have the budget for five nine.

So how do we make this whole thing affordable? Well, I think I, I think that's actually the root cause and the reason we went to the cloud, and we've forgotten what our life was like before the cloud, is that maintaining five nines on our own infrastructure is very expensive, right? It's much more expensive than relying on the cloud service providers who are able to distribute that cost across their thousands or hundreds of thousands or millions of customers.

In the days before we had the cloud or the cloud was so prevalent, all the services used to be down all the time. And there's this wonderful site, uh, down detector, and if you look at it, you can see who's up and who's down. And we don't talk about that anymore because for the most part, people don't go down.

This is a very, very rare situation, right? I mean, once the last time we've had a major outage like this, it's been a couple of years. So I think people lose a little bit of perspective and think about it at, when it goes down, a lot of services go down.

But on the other hand, how many of those services did maintain five nines or, or four nines, or even three nines or two nines, yeah, before then, right? There were a lot of services around that we relied on that were 95% uptime, not 99% uptime, and that would have be down for multiple outages over the course of a year. Um, so distributing that cost, you know, by the, through the cloud service providers, I think is how we ameliorated the, the issue.

But you can program for it to be distributed, right? Not to just show up in, you know, I mean, yes, you're right. And cost absolutely was a factor.

And I think though that, and one of the things that we saw with, um, CrowdStrike is that the cost becomes a human cost and it matters. We have to understand, especially as, um, AI becomes, you know, augmenting humans, that the cost is no longer a financial cost. It's actually human cost and this matters.

And so when we think about, you know, we have to think, the developers have to think that, um, the way that we distribute, um, across cloud and even having multi-cloud, it becomes important. And it, it's, is it not? I mean, is it that expensive?

I mean, I I honestly it, you know, from a multi-cloud perspective, you know, is it that five nines? Yes, that was extremely expensive, but is multi-cloud expensive? I don't know.

I'm asking, I think Go ahead, Jim. I was, I I was gonna say it's, well, again, that's sort of, there's the human cost involved in that is architecting your environment to span across two different cloud service providers is actually fairly challenging, right? If, you know, there are many organizations that are multi-cloud, but they have, every single app is single cloud, right?

They just, right. They might have something in AWS and in Google and in Azure and in Oracle and, and, and, and they're spread across all of that. But most applications are, um, monolithic across a single cloud because there's enough differences between the clouds and there's not a universal API and not a universal interfaces for this stuff.

So it is becomes very challenging to build an app to make a single app resilient because it spans clouds. And the only other question, I'm sorry, that I would have then is when we used to architect for five nines or whatever, um, you know, we used to have, uh, data centers in different sites, especially in Florida, you know, being hit by hurricanes and et cetera. Why don't they just have like a, a backup ready to go, like IBM even used to offer that as a service, you know, like a, A hot spear.

Yeah, it's, I dunno. Yeah, I, I, well, my understanding is, I, on, on these particular issues, the network routing, the, the, the network information was the cause of the failure. Mm-hmm.

So if you had another location you could switch to, it was dependent on being able to get the network information correctly, and because that part of it wasn't working, then you can't switch to whatever your yeah. Alternate site is, right? That's, that's a, that's, you know, there's, it becomes very, very difficult to engineer out every single, single point of failure in the system.

Yeah. That's always the network guys. While at the end of the day, I think it's not the software developer system.

I didn't say that. I'll tell 'em all at network field day next this week. So yeah, that Mike, So, so Mitch, which is the better situation for an IT person, right?

So to Jack's point, right? Uh, so we had things on premise and they probably crash more often than they do in the cloud, but not everybody, you know, the employees knew it, but the whole world didn't know it. And now I have a cloud scenario where it doesn't crash as often, but when it does, the whole world knows it.

So which one do I want 'em have? Well, it's kinda like when the air traffic control system goes down and now every plane is, you know, trying to figure out how to get where they need to go. It's that level of an outage when you have a big outage.

Um, I, I can remember the day of pick your application, the invoice processing system's down again, it's down again fourth time today. They're trying to get it back up and figure out what's going on with it. You know, those days don't, those don't happen much to, to Jack's point, like, like they used to.

And I think it's 'cause we're architecting systems better, building a more redundancy, not relying as, as monolithic of code. But from a, from an, from my standpoint, I was happy to move it outta my own data center and put it in the cloud. Um, and part of it is I just to deploy networks in, in co-locations and in central offices and and service providers.

So I was pretty comfortable with it. But you know, the thought of, okay, the air conditioners, the chiller's got a problem. I do, I really wanna mess with that.

Is that what this was that what life is about? Is that what I got a CS degree for? Is the fricking chiller to keep, keep the computer room cold?

No. So, you know, it, I, to me, I think it's a blessing not to have it on site when you can do that. Um, doesn't mean there aren't pains moving up to the cloud too, but you have a lot more flexibility.

Um, you have less control, but you have a lot more flexibility of which you can do, Kate, is there a certain amount of comfort to be taken in the fact that when it does happen, it happens to everybody all at once and there's lots of misery to go around, I guess, uh, the adage, right? Misery loves company, so, uh, we'll go with that. We'll see how it goes.

Um, Jack, just one question here about the cost of all this stuff. I mean, do you think that it's that the providers haven't figured out a way to deliver that capability of five nines in a way that's cost effective? Or is it just that the IT people and the apps folks don't really want to pay for it even though they haven't figured out that maybe the cost of downtime far exceeds the cost of that extra capability, especially for certain applications that are driving Remy?

It's a little bit of both. I mean, but I'll go back to what Kate said in I think the previous segment about the human cost, right? And, and part of the challenge here is that we can build as much redundancy into a system as we want, and we can spend as much money as we want, but at some point, a human's gonna trip over a cable and knock out the power to everything.

And there's, that's really hard to avoid that, right? And so, you know, I think we're approaching the, the point of diminishing returns. We can spend infinitely large amounts of money to approach six nines or seven nines.

And the ROI of that investment is very, very small, if anything. And that's where we are today. I mean, as I, as I said, you know, there was a point where, and we forget now, but there was a point when failures were so common that we had custom failure pages that became memes.

Does everybody remember the Twitter fail whale, right? I mean, their failures, failures were so often that you had a, uh, a graphic that you designed specifically for your failure case. Nobody does that anymore.

So I think we're at the point where we've gotten, you know, we've, we've spent what we believe is the appropriate amount of money for the appropriate reliability and resiliency, you know, from a customer of those services perspective, what gets concerning is this is the third failure by fill in the blank. My, this vendor in the last 30 days or last 60 90 days. Oh, then I'm starting to get concerned, right?

What's going on here? Are they, do they have their act together, but the fact that they are rare, um, and aren't service affecting to what you run? Uh, this says, says that they do a pretty darn good job at it.

The only thing that I'd like to add is that, um, the AWS outage happened. It was an east coast outage, right? Mm-hmm.

And that should be, um, rectifiable, meaning, you know, not everyone should, like, if I had to develop, um, whatever, I wouldn't go to AWS East because the last two times it was AWS East, um, I or make it multi-cloud and that, I mean, there's, I think there's a checkbox if I'm wrong, that you can just, you know, multi-cloud that. And so that I think we're not blaming the network people. So Mitch, you're gonna be off the hook and we're gonna be blaming the developers on that one.

So Take it to GCP in Iowa. There you go. There You go.

I like it. Alright, I think we have some agreement here. I think, you know, we need, resiliency would be nice if it was cheaper, but right now it's a little bit expensive, so you gotta figure out whether it's worth it for your application.

And if it's not, suck it up buttercup. We'll be back in the morning. Discover Techron group, the epicenter of tech innovation.

We are your go-to for reaching IT leaders and practitioners worldwide. Our secret impactful content that sparks awareness, engagement, and top quality leads with us. You'll access editorial websites, streaming videos, virtual events, custom content analyst research, and more.

Join our satisfied clients. Let's revolutionize your tech journey. Contact us today and tell your story to the world in the most powerful way with Textron Group.

Hey folks, we're back in this next segment. I think personally I found it fascinating, but the bad guys have figured out how to avoid the takedown. So if, I don't know if you've been watching some of these things, but law enforcement officials around the world have been kind of taking down servers and in Elliot ness kind of fashion, they're short of actually having the cameras on hand, but they basically break into these warehouses and, you know, taking ax to the beer battles, beer barrels, and then they grab all the servers and away they go and it makes for great theater.

But Jack turns out the bad guys are getting smart and using something called blockchain and ether riding. Please explain. Well, this has got to be the most diabolically evil case of hide in plain sight I've ever seen or heard of.

And, uh, the bad guys are basically using a feature of blockchain called con, uh, contracts, software contracts where a blockchain is basically an immutable ledger. Everything that gets stored on the blockchain is uncorruptible. You can't be modified, uh, except in very specific use cases, which we'll get into in a moment.

And everything is publicly, it can be seen and accessed publicly. And most importantly, the blockchain is distributed, it exists on millions of computers across the world. So the law enforcement guys can't go in, crash the doors, grab the server because that thing on the blockchain, that piece of data is replicated everywhere across the world.

And so they'd have to subpoena and grab millions or billions of computers, which is just never gonna happen. Now what the bad guys are doing is they're taking advantage of something called a smart contract, which includes, uh, in it a data field or a set of data fields in that smart contract, the data fields in the contract itself can be modified through an API and the bad guys basically corrupt a machine. And then once they have the initial compromise, they're able to go and fetch their code that exists in the data fields of the blockchain so they can modify that code and update it and change it as they want, and therefore use the blockchain as a distributed repository for their malware, which is really evil and is very, very hard to get rid of.

And it is very hard to detect because they're not usually using the usual methods of command and control servers to get to this. Mm-hmm. So Kate, do you think this will become kind of the standard setup for these guys?

I think it's both. Uh, it's two North Korean groups that kind of pioneered this thing and, uh, but everybody else is gonna look up and go, well, thanks for the notes and crib it and away they go. Right?

Absolutely. And from a cybersecurity perspective, and it's not just because, you know, Jack is right in front of my face, but I will say that this article that he wrote is pro, I would say is one of the most important articles, and I'm, you know, very serious here when it comes to, um, cybersecurity, um, moving forward. Like this is something cybersecurity people, um, architects, engineers really need to think about because this methodology moving forward is highlights, uh, this distributed architecture, which is gonna be very important for us moving, um, as we, um, as we grow.

But absolutely 100% everything that, that Jack wrote in this article is a blueprint, um, on, on what's happening and what we need to do, um, and how do we start to, um, fight this specifically. So great job, Jack, and really it's phenomenal. Mitch, what's your take on this from a DevSecOps perspective?

Because, you know, as Jack noted in his article, they were targeting WordPress sites initially because, well, you know how great the security is of those WordPress sites. So it seems like, you know, a natural distribution vehicle, Lockdown gold standard, that's for sure. Um, you, you know, in a way it's a supply chain attack, right?

Just like, um, with SolarWinds attacking the, the build process and in the, in the software creation DevOps processes here, you're doing it in a payload that's going everywhere to, to Jack's point, it it is everywhere and you can't get rid of it. Um, I, it you're usually in DevSecOps are thinking about application security oftentimes, but I think that's why I said supply chain, because now we're talking about something that we're relying on for other uses, you know, in this case, um, the blockchain to be the mutable ledger, uh, for whatever uses that it is crypto or whatever, um, and then it, it's getting hijacked, um, no pun intended, Jack, um, by the nation state guys in this case. So I don't, you're not gonna, you're not gonna solve this by scanning your code more often.

This is a different kind of problem. Um, and, and I think most software developers would say, I don't even know what you're talking about. I know which mean using an API to get to something when it's, you know, when it's down I get, but you know, they're not blockchain experts.

So it's, it's a, it's really a tricky problem. I I'm not sure how we solve this, Jack, what are we supposed to do about all this? Well, the, the, I think there's sort of one silver lining here is that while the code repository that the bad guys are using, they're treating, essentially treating the blockchain as a code repository, and that is a distributed environment, but there is some centralization involved in their accessing and changing that repository.

So we, from a global perspective, a law enforcement or a, you know, people searching for the bad guys and what they're doing perspective, there is some hints that we can have to look at it from a how do we protect our environment from prevent from getting infected, um, you know, we need to start developing new concepts of what our, our attacker tactics, techniques and procedures TTPs are. And we need to start developing new rule sets and new heuristics to see when people are going to the blockchain and trying to understand what is legitimate activity from our site of corporate environment. How often do people go and access the blockchain and should they be accessing blockchain stuff?

And, uh, you know, and trying to see about that to detect when we've been compromised and how to, you know, defend against that. Basically you think we'll see some security, uh, enhanced around the API to get to that smart contract. That seems to me the only way to, you know, get getting it in there, you're not gonna prevent because people can do it on the blockchain themselves, but whether some process on your server is now accessing that API should it be and what's it using it for?

And no, I don't know what that is. So, you know, lock it out. I, I think we'll see that.

And we will also maybe, uh, if the people developing, you know, the, the blockchain owners, the Ethereums and the Bitcoin, you know, whoever it is, if they have some global moral concepts, then maybe they will start trying to figure out how they can detect bad people from using their environment. But a lot of those, you know, those environments, their whole concept is anonymity and privacy, and we don't care what happens on the blockchain is because it has legitimate uses as well as illegitimate uses. So it's, you know, that's the real challenge.

And you know, it's, we have seen similar attempts at hiding activity, say, through DNS, where people have used DNS to exfiltrate data in a DNS query. And again, it's the, from a, a defender point of view, it's trying to figure out what does legitimate uses of this technology look like versus illegitimate uses, and how can we differentiate between the two? And this may be a case where AI can help us, I don't know, but, you know, it's, it's becomes, you know, it's essentially, like I said, it's hiding in plain sight.

So how do you differentiate the, the legitimate from the illegitimate is the challenge. I, and I think, you know, as a person who, you know, developed, you know, worked with, um, sox, you know, security operation centers, I would say, um, we could look at the chain telemetry, um, and feeding it into threat intel feeds. So that would be one way that I would look at it.

Uh, you know, just stuff Like 10:00 PM where's your blockchain been? Yeah. All right folks, I think we're gonna leave this here.

Um, I would just say to everybody, you know, I was sobering, happy Monday and that, uh, knowing feeling in the pit of your stomach is definitely not indigestion, but hopefully we'll get smart about all this and fix this sooner than later. Hey, I wanna thank everybody for sharing their knowledge and their insights today. As usual, great show.

Folks, I wanna thank you all for spending some time with us once again, and please stay tuned for the rest of the tech strong TV lineup, which is gonna be equally awesome. And we'll see you all the mark.