Techstrong Gang – July 2, 2024

Happy Tuesday to you. What's going on at the post office? I don't know.

We're gonna discuss that. Cybersecurity and more. You're watching Text Drunk Gang.

Hi everyone, happy Tuesday. It's Alan Shimo for Techstrong and the Textron gag. We've got a full lineup of some really interesting topics to discuss, and we've got some of our best gang members with us.

I love our Monday lineup. Let me introduce 'em to you real quick here. First of all, joining us from out in the West Coast, it is our shiny new penny of an editor, uh, John Swartz.

Hey John, welcome. Hope all is well. Thank You all Is well.

Good to be Well with you all. Good to have you. Also a, a regular on our Monday show.

He is the founder and leader of the Tech Field Day, gestalt IT team. On the Futurum group, it's our friend Steven Foskett. Hey Steven, welcome.

Uh, core bli. Me. I am ready to speak English.

Beautiful. We're going to, okay. Um, joining us on the East Coast.

She is, well, she's from Mire, but she's from a lot of other things. She's got her hand in. It's our friend Chase Bannon.

Hey, trace, how are you? I am just right, excited about our topics today. Excellent.

And then finally, he's not down in Boca. He's, he's up in Lake Placid, hoping to, uh, cover the Winter Olympics a little late, but, uh, nevertheless, he's with us today. He's our Chief content officer, Mike Baard.

Hey, Mike. Hey, if I was gonna cover the Winter Olympics here, I need to be here now to get a ticket, but they're not coming anyway. No, not anytime soon.

I, I don't know if it made. Well, there's a lot of reasons for that. Anyway, guys, we, we've got a, a lot to cover today.

Let's jump right into it. Mike, we've got a bit of a post office scandal going on here. Why, why don't you kick that off?

Yeah. There's this scandal been playing out in Britain now for several years, and it's been playing out in slow motion, essentially. But at the core of it, um, a lot of the agents who handle the mail in local communities or independent contractors, and there was a new application put in by Fujitsu.

And apparently some of the money that was recorded didn't get recorded by these folks. And then they all were basically hauled in the court by the post office for trying to steal money. And some of them wound up doing jail time and some of them, you know, went home and, you know, basically ended their lives because they couldn't handle the scandal and the pressure.

And, uh, but it turned out it was all a big giant IT mistake and involved, uh, the, the an inability to record transactions. And there was intermittent. So nobody seemed to realize what was going on.

And nobody kind of correlated all these different instances might be the same thing, and it might be an IT problem. And now they're still playing out the trial. And the head of, uh, IT for Fujitsu who built the system, just testified and more IT folks are expected to follow him.

And it's become this fascinating little scandal. But Steven, I know you've been following it. What's your sense of, um, what went wrong here?

Well, everything went wrong here. I think that the, the, I've been watching this story for a long time. I'm so excited to have this on Textron gang.

Um, it is a case study, a textbook case study. And the crazy thing is, you know, you say that a new IT system, you know, for a few years, uh, this is an old IT system and it has been going on for a long time. So, um, this is the kind of thing I might have studied in college.

And incredibly, this story begins two years after I graduated. So in 1996, the UK decided to do a new public benefits system. Um, it was so, such a boondoggle that they canceled it in 1999 and instead of actually canceling it, they handed it to the post office system.

And so the post office actually has been implementing this system literally for the entire 21st century. And throughout that time, the system has been plagued with bugs because, you know, big accounting systems and some of the bugs are particularly egregious. There was one where, um, one of the bugs, the bugs are all named after the individual post offices, which makes the whole thing, like I said, you know, speak in English.

It, it's all musical amusing, uh, English, uh, post office names, which would be wonderful if it didn't affect so many people so horribly. So for example, one of the bugs was when the screen froze, if you kept pressing submit, it would just record those as multiple transactions and then the post office would be on the hook for payments that were never made. So basically the, the recipient only gets one 'cause they use this system for, uh, pensioners, uh, benefits and that sort of thing.

Uh, so the pensioners standing there saying, why isn't this thing working? And every time they hit enter, they're basically creating ghost transactions. So they get their regular money, but then the post office is on the hook for the fact that their accounting doesn't line up because there was like 10 extra transactions, things like that were going on all over the place.

People tried to report it literally for 20 years. People tried to report this thing and instead of actually getting, uh, the, this, the privatized post office system to do anything about it or the private maker of the system to fix the bugs properly, instead they just deny, deny, deny, oh, the system is great. There's nothing that can be done.

And it looks like what happened. And again, I don't know for sure, but it looks like what happened was eventually the, the contractor responsible for the system, they, they caught on. And so they started getting in there and tampering with people's, uh, the branches, uh, implementation.

So they'd be like, oh man, we gotta cover this up. So they were like deleting extra transactions and stuff. This guy that you mentioned that was the architect, well, he was called as a, an expert witness.

So he was actually called to be some sort of expert in IT systems and in this system and said, uh, but of course he works for the contractor, so what is he gonna say? Anyway, the whole thing came, uh, crashing down. There was a, um, a, um, really amazing, um, uh, uh, by BBC program about it.

Then they did a docudrama about it, and that finally blew the whole thing up. And now here we are with like the biggest it story of the uk and it all comes down to basically people covering up software bugs in this system. That's an in, I mean, it's an incredible story, Stephen.

It's just said this kind of, and it all goes back to this old axiom of the coverup is worse than the crime. I mean, essentially this, if you look at it, and it, this goes back maybe back to 2005, and I see it various dates. It's like a slow motion Watergate of it worthy of a Harvard business school study.

I mean, this kind of underscores an over-reliance on it. And this, this guy, uh, Jenkins, I believe who is from, for Jiujitsu, keeps popping up and either saying he doesn't recall something or he doesn't remember getting something or, or does a Asia coverup. In a sense, it's kind of like this, uh, dangers of unflinching faith in a, in products and, and depending on one person.

And, and that just the irreparable damages done to people over the years, including just like some horrendous things involving individuals. It's, it's definitely the anatomy of a train wreck. So this is definitely worth a study, right?

We're gonna see this like the, the Rite Aid in economics. You study the Rite Aid model. We'll be studying the post office example here.

What's what's interesting to note is to, or, or that we need to research a little bit more, is to understand contractually what were some of the incentives to keep things quiet. Because there are incentives in some of these contracts, especially when you're dealing with governments, um, where you are, uh, paid by the line of code you're paid for, um, your, their demerits for defects. I've been, uh, historically been around situations where there were two bug logs.

There was the bug log that, that the organization knew about and it was relatively clean. And then there was the other bug log that the contractors had that was much more prolific and they would prioritize what they would allow to be seen in the major log in the, in the, in the transparent and formal log. So do you guys, Steve, you've been looking at this for a bit.

Do you have any ideas if there was some kind of additional, and why would anybody want to ignore these bugs? Why would they want to hide the, you know, how poor the quality was? What was the incentive to not say something?

Well, I would guess that the incentive to not say something was the fact that this was just a giant boondoggle government contract. And I, you know, I mean, I think we've all been in it projects like this. I mean, to kind of bring this home to our audience here on Techron Gang, most of us have been involved in huge, uh, enterprise software development projects and, and rollouts where we know that the product isn't all that it's cracked up to be.

In fact, in many cases, most of the, uh, developers are under the surface just like panicking over the fact that they know that the system doesn't work very well. They know that the pro project is massively o over budget, off schedule, et cetera. And I would definitely think that the, um, the incentives are there for people to cover things up.

Like I said, it looks like some of the contractors from the IT subcontractor here, it looks like they were literally going in and by hand trying to fix bugs that the, in the accounting ledger that the system had created. And instead of actually doing that, uh, you know, having this be public and instead of even, you know, addressing the core problems, the bugs, they were desperately trying to kind of, you know, plug holes in the, in the, in in the thing as it's leaking. And again, I think that a lot of us in it can probably sympathize with this.

Um, eventually there was enough, uh, you know, whistleblowers, um, by the way, my favorite weird detail of that, that story is that the chief whistleblower, uh, for the BBC, his name was Richard Roll. That's right. Rick roll.

Rick Roll, roll. Rick Roll. I like it.

I like it whistle on this story. That's deep Throat, but I mean, it is just, it is not at all surprising, number one. But the other thing is it just shows how this stuff can go rolling down the hill.

I'm sorry to put, make a pun, but rolling. You know, it it under its own weight. And, and instead of actually somebody saying, wait a second, this thing is just totally screwed up.

They covered it up for like 20 years and they prosecuted all these people, hundreds of, of sub of, of, of individual postmasters were prosecuted for financial fraud, for, you know, accounting irregularities. As you said, some of them even took their own lives because nobody would stand up in it and say, this thing is well until rickroll and, and this thing is just broken. It makes you wonder if there are other instances like this that exist out there.

And one thing I was gonna mention, really cool comment about Jenkins was he spent his entire career at one company and by his own admission he said he dealt better with systems and people. Okay. You know, this is look guys Shades of the Phoenix project, right?

Mm-Hmm. com. We obviously are big proponents of Gene Kim's work and the Phoenix project, but this is the, the biggest problem with these kinds of things is everyone's afraid to tell the emperor they have no clothes.

Exactly. Exactly. And, and so they, instead of just saying, Hey, we've gotta the full stop get this right, we, we just sort of, you know, spit and bubble gum our way through these things, knowing that it's a, it's a train running off the tracks until finally, you know, the proverbial you know, what hits the fan, which seems to be, in this case, it took a long time for that stuff to hit the fat.

But it, it seems here now, I can't wait to see all these people who, whether I mean all the way through suicides and have had their life ruined and have had jail time as a result of this, you know, someone's gotta do right by these people. There's gonna be a lot of civil lawsuits, right? Absolutely.

Oh, I got a, a nightmare that will extend 15 It, Tracy, can I ask you a question here? Um, Sure. Why wasn't there, do you think, maybe this is an example thereof where the Fox's garden, the hen house.

So why wasn't there a third party testing auditing of these systems by somebody who wasn't part of that Contract? Like an IG office, an IG office, IVNV? Usually, oftentimes in this kind of situation, uh, depending on the contract there is, there should be IVNV, right?

Independent verification and validation. There normally are, but every contract is different than it actually brought to mind, Mike, that, um, you know, the incentive may have been to just fix it enough, just fix enough. So things, over the years they've been making fixes, they've been modernizing and changing things as they went.

But if you don't fix it all, there's also a tail to come back and do more work. So there could be a naughty incentive here that they didn't want to fix it because they were continuing to make a dramatic amount of money, 20 years, 20 years worth of contract here. Not, not, not five years, 20 years.

So if you don't fix it the whole way, if the mechanic doesn't fix the car the whole way, and you gotta keep, here's that next little thing you gotta take it back for, and this next little thing. If they've got that kind of an umbilical, that could have also been a, a really bad but to relevant motivation. And there's another weird little twist here, and that is that because this is the post office, they are able to act as a public prosecutor of their own case because of a weird quirk of the history of the UK postal system.

So again, that's not really gonna be relevant to beyond this, but they were able to basically trample over everybody 'cause they didn't wanna admit what was going on. Well, I guess maybe they didn't know what was going on. But it, it, and again, just a another strange aspect to this story.

I, I wonder how much sort of, you know, hoho the if upper lip of the British worked against them here. gov didn't work when it first came out, or it was working but not working and all it was bloody murder. They were screaming and until, you know, it got done and Then they got it working Just a few and they got it working relatively quickly because there was a spotlight shined on it, and it, and people yelled and screamed and stamped their feet and, and you know, no one had to go to jail, thank God.

But we got new, new contractors in and, and it was fixed. Is is the British civil Service sort of mantra and, and you know, way they do things, their heritage there, did it work against them here? And and it may very well be that right, because they, they have a long distinguished, uh, heritage there.

And within the, the British Civil Service that, by the way, in the, uh, Singapore, many of the former British colonies have inherited that into their civil service. And not that it's bad British civil service, civilized a lot of the world, but for these kinds of things, I, I don't, it might have worked against them. Yeah, let's hope as we go forward that a lot of the things that we are all reporting, right, whether it's, whether it's Jean Kim, whether it's, uh, Dr.

Phil Lalo, regardless as to who it is, we're talking about psychological safety so that people are able to come forward. So we're talking about not keeping a generation, this is a generation of developers that were cycling through this, giving them the opportunity to say, this ain't right. This isn't right.

So in teaching people these different capabilities, right? If imagine if they had modularity, if it was a decoupled design, imagine if they had more modern patterns, design patterns that they could apply to this so that they could get after those bugs so they could actually address the issues. Let, let's hope that all the things that we're helping people to learn about start to apply, if not to this system, to other ones that are in similar situations.

Let me ask you one more question, Steven. Shouldn't there be some sort of code of ethics that it people live by where they know when they see this kind of thing and like doctors do no harm? It seems like we're kind of missing something here.

And, and with so much writing on these systems, do we have some ethical responsibilities here? It doesn't work for doctors, but go ahead, Steve. I I actually, I I think Tracy may be better equipped to answer that question than me, but I I absolutely think that there should, and it kind of reminds me of some of the stories that we've heard recently about the, uh, for example, the AI whistleblowers that we covered here on, uh, Textron gang, I think last week.

Um, there needs to be protection in place for whistleblowers in it, especially as these systems become more and more important and more and more invasive. And especially with the rise of ai, we need to have the ability for people to, to say this, there's a problem here. And it's not that hundreds of postmasters across the UK are stealing money.

Mm-Hmm. Guys, I gonna take the last word on this and then we've gotta go to break. I think the important thing to, to remember here is there is a, a, a left to right continuum of, of the severity of blame.

You know, we start at the far left, bugs happen, right? Mm-Hmm. It's just the natural state of writing software.

You're gonna have bugs. You find vulnerabilities, bugs happen as we move further, right, though, right from left to to right here, and we shift right a little bit at some point. The amount of bugs, the severity of bugs, the way we're dealing with these bugs crosses over from just innocent, you know, best practices to negligence and negligence, especially the British common law system invented the concept of it negligence is when you act unreasonably, right?

So you are unreasonable in writing the code and testing the code and correcting defects. And then as we continue, right, we shift right further, we move over from negligence to willful negligence, right? Which is, look, I know I'm negligent and I'm doing it anyway, that's, you know, can start crossing now into criminal.

And then we continue to the far right of this where it's, it's not even willful negligence. It, it's criminal negligence and it's coverups and everything else that the kind of stuff, right? We we're trying to cover up past negligent by doing criminal acts.

This seems to have gone all the way to that far right extreme. And that generally involves not just civil suits, but criminal penalties and, you know, the, the worst that we can do as a, as a government and a society. And I think that this is, when you cross to that level, it, it's bad.

It's a bad thing. And, and I hope the, the authorities take the, the right actions here. But we've gotta take some action here.

We're gonna take a break on text on gang when we come back, you know, in the constant whack-a-mole of what vertical is the next target of the cyber warfare people or the cyber gangs, criminals. It could be your, your friendly automotive dealer. Stay tuned.

We'll be back in a minute. All right, folks, we're back onto our next segment, which involves an outfit called CDK, which was targeted with malware by a bunch of bad guys that took their SaaS application offline. And in so doing, made it almost impossible for, uh, hundreds of dealers to sell cars.

And I think everybody's been kind of tracking this story a little bit in the, um, popular press, shall we say. But it's interesting in that the government, in the form of the FTC and a few other agencies was warning car dealerships before this happened, that they were gonna be targeted and they had some issues. And I guess my first question comes down to, well, did that alert tell the bad guys to launch the attacker?

Was the attacker already there and it was inevitable and they just didn't get in front of it? Alan, what do you think? So, in regard to your specific question, I would think that the government probably put that warning out because they had already seen some threat intel that said the, they were being attacked.

So I, I'm not gonna blame the government for this sort, but here, here's, you know, I, I learned a little bit about the way car dealers work. You know, vis-a-vis software and operating systems. We had a salesperson who worked for us here at Techstrong, who actually came from a CDK type of company.

So most of these dealerships, and remember, you know, we have a, it's kind of an archaic system. Tesla tried to blow it up and they did it. But we have an archaic system of the relationship between car manufacturers and car dealers.

But over the years, what's happened here is certain families or businesses have, have started, you know, its franchises, I guess, basically. But, you know, you can have AutoNation's, the biggest example, Wayne Hega rolled up a whole bunch of dealerships, and it's the AutoNation, and that's probably the biggest in the country now. AutoNation kind of has enough size and girth to, to create their own software, their own operating systems, their own processes.

However, the vast majority of dealerships are still mom and pops, who may be operate one, two, or three dealerships. And they, like so many other verticals joined the SaaS revolution or evolution years and years ago, because they can't have their own, they don't, they don't have the girth to create their own sales systems, their own inventory systems, their own, you know, IT infrastructure. And the, surprisingly, the car manufacturers don't give that to the dealers.

The dealers have to get that. And so they have companies like CDK and, and others who create turnkey SaaS operating systems that these car dealers use just the same way. Many doctors' offices have similar kinds of systems, and legal offices have similar kinds of systems.

And if you use Shopify for your e-commerce stuff, right? We rely on these SaaS based providers of systems, systems that are, that are, you know, customized for a particular vertical. So what it really created was a big fat honeypot for the bad guys to go after the automo, you know, compromise one one of these SaaS providers, and you're in into dozens if not hundreds of car dealerships.

And that's exactly what happened here. Um, now is it ransomware, right? Or, you know, because you gotta say why, why are the bad guys doing this?

Well, to make money, usually, because it's not like shutting down your car dealership is a critical infrastructure. Uh, there's ransomware, there's, you know, all all different ways of monetizing this, but I'll, I'll throw it out to the gang. What do you guys think?

It just seems like a natural target of mm-hmm. Automotive industry, right? You're handling a bot, a lot of sensitive customer data, large financial transactions, and these are fairly immature or cybersecurity systems.

There was one thing that, that jumped out at me, I think, I don't know if Steer was looking at into this or saw this, probably wrote about it with this one report from last year that revealed nearly half of the dealerships reported a cyber attack or incident of some sort that resulted in a financial or operational loss over the previous 12 months. That's like a staggering number to mm-Hmm. It is staggering.

Consider this, the CDK issue has, has hit 30,000 dealerships at this point. So what is, what is the, the financial impact of it? Yes, they're looking for money, right?

There's, there's a ransom aspect to this, but what's the additional financial impact of not being able to sell a car? Mm-Hmm. If you're a dealership and you're not able to sell those cars, it will be interesting to see if we see other types of attacks like this where they're not looking to get money, but they're looking for, to damage, to damage the dealerships.

If I, you know, do instead of going after the car dealerships, do I go after coal and gas? Do I go after something else? Critical infrastructure, right?

From a political perspective. Well, cisa, uh, you know, has identified 16 critical infrastructure areas, uh, and we're working with them. Mitre is working with them.

A number of organizations are working together to address this right now. Um, you know, as we were talking about this earlier, I wanna tie this back from a cybersecurity perspective to our conversation about post office and people being neglectful. There is, right now work being done by the National Society of Professional Engineers.

So they're in charge of all engineers, mechanical, chemical, electrical, et cetera, software as well, software and systems. The other engineering professions have certifications. And I don't mean you went and got your AWS certification or your Google Cert from a vendor.

I mean, you have legal responsibility to assert that the thing that you've designed meets certain standards. They're working, the National Society of Professional Engineers is working with SE and they are in beta of a test for system integrators in the United States. That if you're a software and systems integrator, that you're going to need to have so many people on your staff to certify that you've addressed all of these different types of things that you've thought about, the financial impact and the vulnerability that you've thought about, the common threats you've thought about social engineering and the IO OT aspects.

It, it has the potential to really help everybody, but what's that do for us right now? Right before, there's something there before that, safety net's there. What do we do, Steven?

Neglect is a word that we keep hearing all day today. At what point is an organization, we know there are bad guys out there, we know they're launching attacks. At what point are we just responsible to build something that's more resilient?

And we're all talking about resilience these days, but at what point does that just become a requirement versus an aspiration? Well, I think it's already a requirement. And, uh, to Tracy's point, um, you know, this, the, the vulnerable in industries, if you look at that list, it's basically every industry.

Um, because think about it this way, you know, bad guys don't have to take down the electric grid to, uh, basically, you know, cause chaos in society. They could take down every grocery store, you know, they could take down the gas pumps, they could take down all sorts of things that everyday people rely on. And I think that, you know, I I, I'm not gonna defend CDK in this case because clearly CDK, um, it, well, it, it sure seems to me that they screwed this up and that they continue to screw it up.

Uh, we should probably talk about their remediation efforts as well. But that being said, it actually makes sense for somebody like a car dealership to outsource Mm-Hmm. IT operations to a company like CDK.

So, I mean, CDK actually started in 1974. So I, I was a baby when CDK was, they should know how to do this. They should know how to put together IT systems, how to manage them, how to defend them against ransomware.

And, and frankly, if you've been involved in automotive dealers or, uh, retail, or food or gas, I mean, in these industries, most of them are not big it, uh, organizations. And in fact, they kind of shouldn't be. Uh, our automotive dealers, as you mentioned, Alan, at the top, there are kind of a, um, a special breed in the US because they're required to be independent.

They're required almost to be small businesses. But as you pointed out, uh, some of them aren't, uh, Penske and, uh, AutoNation, but those businesses shouldn't really be in the business of it. They should be in the business of the business that they're in.

And so it makes sense for them to trust a service provider like CDK to manage IT operations and to be ready, uh, when and if the bad guys knock on the door. Uh, so it really, in my mind, falls on those organizations. I feel like that's how it should be.

And I feel like they really let us down, as I said, with the remediation effort, which is just a complete face palm. But there are two parts of that. One part I'm gonna, and I'm gonna be devil's advocate on both sides of this.

There has to, there's a financial incentive, right? In the end, if your clients are asking you to do something, you are more likely to do it. So, one piece of this needs to be that the dealerships need to be saying, well, this needs to be secure, but it, it's gonna track me to the other side of this.

I don't wanna hold, I don't wanna say that they're the vic that they caused this themselves. They didn't. But there's, companies will do the things that make money for them, and they won't do the things that they don't think people see of value.

Now, on the other side of this, I think we're at the point where if you are, uh, signing up for a sas, if you're signing up for this, you have to understand your bill of rights as someone who's, what's in that EA what are you agreeing to in those SLAs? I think that needs to be much more transparent. And these security ramifications need to be there.

'cause the SLA needs to be more than, you know, and that's the, your, your service level agreement needs to be more than, well, you're up for 99% of the time, right? It needs to be more than that. It needs to be, this is your resiliency target.

This is your security mark, and here's how you are going to address emerging threats, because we're always gonna have emerging threats. So what's the responsibility of your provider when there's an emerging threat? So maybe getting after something that gives you a, a bill of rights or that ability to empower organizations to say, Hey, if I'm gonna get this SaaS from you, can, can you make sure all these things are handled?

I I need to know that it's secure. I need to know that it's resilient. I need to know how you're gonna handle the next issue.

So, Alan, Alan, are we blaming the victim here? Like, yes, but, but you're not without blame. Mm-Hmm.

Right. Look, you know, I think what we're dealing with is, this is kind of just a weak point in this whole business model. It's the same thing basically with the cloud too, right?

Mm-Hmm. If you are, if you are gonna run a business where basically you are becoming the focal point, the nexus for a rich bounty of, of data, of, of it, you know, a choke point, if you will, part of your responsibility is to make sure that because you've put a target on your back, you're also taking extraordinary measures, or better than reasonable, better than ordinary measures to protect your flock. You, if you will, right?

These, these assets that you're gathering. It was one of the big things around cloud security. When cloud first came out, 2005, 2006, it's the same thing in all of these SaaS, multi-tenant Mm-hmm.

Type of environments, right? By gathering multiple customers into one location, into one choke point, you make yourself a target. And you've gotta take better than reasonable ordinary measures to make sure that you have secured that target.

com, uh, days, we were at a SP application service provider. And one of the lessons I learned there was, if it is not core and critical to your business, you're definitely outsourcing it. If it's core, but not critical or critical, but not core, you may very well outsource it.

I would say it falls into critical, but not core to their business, though. Look, mark Andreessen said it 10 plus years ago, every company's a software company. And so at the very least, these dealerships need to have plan B in place, right?

I, I had this, and I don't know if it was part of this or not, but I had to bring my car in for service, what an AutoNation dealership about two weeks ago. And I got in and it was chaos. They had cars lined out into the street.

I said, what's the matter? I have a, I have an eight o'clock appointment. And they said, oh, the computers are down the whole system, all of AutoNation is down.

And I said, well, that's great, but I gotta get to my office. I'm gonna leave my car here with the keys, have someone I'm in the office and call me when you get this straight. And it was, it was a simple oil change.

They didn't call me till about three, four in the afternoon. They said, everything's back up and running your car's, Stu. Um, that's what happens here.

These people don't have plan B in place. They don't have a contingency plan. And maybe, hey, some smart entrepreneur out there.

There's your new business. So you're telling a contingency plan. You're telling me that in this day and age, we can't figure out how to change oil without a computer.

Isn't that a sad state? No. You wanna know the truth?

They changed. They did. They changed my oil.

They changed my oil. They just couldn't update my records or figure out what to bill me, or if it was included or anything. This is like the fallout of every company wants to be considered a tech company, and they're going through some sort of digital transformation, which is sort taking over use phrase, but they just aren't equipped already.

And sometimes maybe they shouldn't go down that path. And it just seems like a cautionary call You and critical, right? If it's not core and critical, they might outsource it, you know?

Anyway, There's a, there's a thing called pen and paper. You can write stuff down and hold on to it and update it later. But can you, that's the thing.

Like, how many of us have gone into a store or a dealership or a restaurant or whatever, and the answer is, I'm sorry, I can't do anything. The computers are down, right? That's, that's like universal these days.

Mm-Hmm. I mean, how, how many, um, old fogies have jokes about the young people these days not being able to do, to make change without the cash register? You know, that's cruel.

It's also kind of true the way the world accurate works these days. I mean, absolutely. Yeah, absolutely.

You can cash out groceries without the register. Absolutely. You can change the oil without the DMS being up and running.

But the question is, are you going to do it? And the answer is no. And, and Steven, my computer is down, is followed closely by the computers are slow today.

Yeah, exactly. You know, crazy. Anyway, hey, we, we've, uh, we, we've gotta take a break is what we've got to do.

We are, you know, running a little over today. We're gonna be back with time to get memory safe. Uh, you are watching Textron Gang Cloud native now is the web's leading resource for the growing cloud native ecosystem.

com is your destination for news, thought leadership, features and webinars on cloud native architecture, Kubernetes, serverless, cloud native application development, microservices, service mesh, cloud native security, and more. Stay on the cutting edge of modern application development at cloud native now. All right, folks, and now we're on to our final segment, which is cisa and some of the other law enforcement folks out there have issued a guidance saying that there is usage of unsafe, um, languages that are using, um, the wrong kind of memory code.

I'm trying to simplify this as much as I can for everybody, but, um, it that may not come as a surprise given the fact that most of the legacy programming languages are using this same methodology. Tracy, let's start with you. What's your sense of a why bother even picking on the open source community to issue this alert when I think everybody's got the same problem?

Um, I think because open source is an easy, easy target right now, but also because of our dependency on open source, the number of open source packages that are being used, uh, in the average organization, uh, I forget what the sonotype numbers are on this, but they were pretty incredible. Uh, so we need to hold people accountable. Uh, it comes down to who's paying to make the changes.

So if you wrote this and you did it as a, a labor of love, uh, and you're not necessarily financially backed by a bigger organization, some of the open source maintainers are small shops. They're individuals. Uh, one of them, I just read about this this morning.

There was a, uh, I think it's IP node, this maintainer shut down and essentially turned his open source project to read only because somebody logged a CVE against him, which sent everybody into a panic, which made dramatic, uh, implications on what they had to change and what they had to do. Um, so I think it's the, and he, so he shut himself down. It's read only, but you can't make a pull request.

You can't ask for changes to be made. It's essentially, and it's used to, uh, I think it has 170 million downloads and use uses against. So it's a big one.

It's a, it's a, it's one of those well used tiny little pieces of utility code. So why go after the open source folks? Well, because it's open source, it's free, it's out there for you.

So if people are going out and they're grabbing things that are free and they aren't aware of the memory, but that's, there is a need to alert people to say, if you're using this free stuff, make sure you're looking for these challenges. It doesn't do away with at all. To your point, Mike, it doesn't do away with the fact that we have to be very cognizant.

Why are we moving to newer languages? Why are we looking at Russ? It's so that we can get to that memory safe, uh, area, right?

We have to make sure that our code is, is memory safe. And there's a lot of, a lot that we have to do with migrating from old code to new, from old languages to new languages. We have to fix the code that we already have.

But the, you know, this entire issue about bringing it up for, from an open source perspective is because of their usefulness to so many. And it's not just impacting my organization, it's impacting anybody who's leveraging that particular package can get hip of the, You know, I look at it as, you know, open source is a victim of its own success, right? Back in the, the early nineties, early to mid nineties, you know, Mac was so much safer than Windows.

Well, yeah. When you only have 4% of the market, you're not really a target from malware. It was a lot.

And, you know, not to say that Windows was locked up so tight, but it represented 95% of the market. Of course, that's where we're gonna go. Mm-Hmm mm-Hmm.

Well, when open source components make up 70% or whatever the, a majority of the code within any given application, that's, that's where I'm aiming for if I'm a bad guy. And so, while I'm a big open source fan and a supporter and pro open source, mm-hmm. It, it can't just, we can't say, oh, poor, poor, open source is being picked on.

Poor, poor, open source has become the dominant form of coding today. And if it's going to be, it's going to be a target and we've gotta do what we need to do to lock it down. Now, Tracy mentioned rust and some of the newer, uh, languages that are designed to offer, you know, better memory protection.

Mm-Hmm. And it'd be great if we can magic wave a magic wand and, and turn everything, you know, into rust or a similar, or have everything written in rust or similar, you know, modern languages is, it's not gonna happen, right? You, you might have greenfield projects where that happens, but not brown buddy fields and not in our lifetime.

So I think buyer beware, caveat mTOR is, is the, is the mantra here. Let's open to what you guys think It's CNC plus plus that are the, the, the most dangerous right? Now when it comes to having things that are memory unsafe.

A good thing is that Java, it can be memory safe if you're doing your smart coding, if you are thinking about applying your DevOps principles that you're doing, your scanning, that you understand any kind of packages that are outside your control that you're pulling in. 'cause not everybody uses open source, but regardless as to who or what you're building, you have to understand your entire lineage. You have to be in charge of and understand if you are a C or a c plus plus shop.

I hope that they're assessing what their security posture looks like right now in assessing what their memory issues are right now. That's the only thing you can do. To your point, Alan, is it time to rewrite?

Gosh, that's a big question. It depends on what domain, what field you're in right now. Maybe, maybe not.

Maybe it's time to let that just eventually deprecate and die out. I like what you said though, that open source is, its, is a victim of its own success. Um, I've always been an advocate of open source, but I have been very concerned about it for about five or six years now with the immense growth of using, without necessarily understanding what's under the covers in that open source.

And that's another piece of the puzzle is being smart and understanding anything you pull into your organization to use open source or not, you need to understand how it works and what the vulnera potential vulnerable areas are. Steven, if I downloaded a piece of open source software and it is maintained by two people, and I stuck it in a production environment, who's the fool? I know you're being facetious here.

Uh, but the truth is that you're not downloading an open source package. You're downloading a package built on dozens of open source projects and a whole heritage of, uh, software development, because that's the nature of open source, and that's why open source is so incredibly valuable. But I do feel like I need to point out that, uh, c and c plus plus is not just a problem for open source.

I know, I know you guys have already said this, but, but, but keep it in mind that, um, again and again and again, um, we've seen that what everything that I've seen says about 70% of exploits are due to buffer overflows pointers, you know, basically memory unsafety. And, and, and there are a lot of projects out there to convert things to rust. Uh, I'll point out by the way, and on the max side, that SWIFT is a inherently memory safe, um, programming language.

And most of Mac OS and, uh, MAC OS applications are, are moving towards SWIFT or being changed to that, uh, language. Uh, same thing is happening, but we're never gonna get everything. And so I think that it's important that we recognize this.

There are tools that can help us check, uh, garbage collection, check, make sure that we're not, um, causing more problems with, uh, with, with changes. But ultimately it is going to need to move to new operating systems or new or to new languages before this thing. This problem gets fixed and that's just not gonna happen.

It'll take a long time. But we also have a responsibility, and we talked about this in some of our other topic areas, developers, engineers, architects, quality engineers, IBNB folks need to understand, they need to be testing, right? Need to design for security.

Anytime I make a fix, I need to be thinking about cybersecurity. I need to be thinking about SAS and das, right? My different types of scanning so I know what's going on.

Um, I think that that's a, a huge piece of this is the ongoing education. We haven't always put the emphasis on the quality. We've always been worried about getting something into the hand of the end users.

And it just needed, you know, when we talked about this from the post office perspective, they cared enough, but not, not enough, right? They cared that the end users could use the system, but they didn't truly care about quality, memory issues, open source, security. These are all parts of part of quality, how you evaluate the quality of software.

So I think we have some education to do secure in depth secure education. Mm-Hmm. Yeah.

I mean, 'cause Tracy, to your point, right? I I think one of the kind of fallacies about open source software is, hey, it was vulnerable. Vulnerable, of course it was.

You have an obligation to go look at that source code and you could have sort it yourself. Mm-Hmm. That's the beauty of open source software.

I'm giving you the source code. Why didn't you look at it? Malarkey is some political candidates would say, right?

Um, the, the fact of the matter is probably less than 2% of all open source users ever bother looking at the source code of the open source they're using. Well, I did a make Sure the beast, I did a capstone recently. It was about two, it was in 2021, and I was using React, and I was using GraphQL, and I think I was in the neighborhood of 18,000 different things getting updated.

So when you did the NPM, when you're pulling down all of the different package, just this beget that be forget the next beget, the next how much code. I know little bits of we're not talking about. All these things are not massive and human.

Whoa. It's a snip. They're, they're, they're, they're tiny little snippets.

There's this four, this four lines and this 32 lines, like these tiny little things have been cobbled, I don't know, glued, duct taped together. Um, nobody has the time or ability to look at it in the depth that we're talking about. So another piece of it is the house of cards that we've built.

Open source on open source and open source on open source. You know, something that, uh, Steve mentioned earlier, Now, SBOs, right? SBOs in a way might be able to help us here because mm-Hmm.

All of these little snippets that are Frankenstein together should be contained within my sbo M and mm-Hmm. A lot of people just say, okay, great. I've got an sbo.

It's like my, my tag on the mattress that if I pull off it's a federal offense or whatever, but no, it's a living, breathing kind of thing. And I think what some security providers and some of the SBO management providers are now understanding is, once I have my SBO M and it's up to date, and I keep it up to date sort of automatically, but the next logical step is I should be testing all of those dependencies contained in my SBO m to make sure we have the latest version. We don't have any known vulnerabilities.

There's no warnings or bulletins or something else out, right, that allow us to keep my software up to date. I, I think what we're going to see over the next five years is that the SBO becomes the vehicle. Mm-Hmm Mm-Hmm.

To keep your software memory free, defect free or what have you. If we could start automating, you know, taking that basic just tag and, and making it come alive. And I think that's the real beauty of best bonds that a lot of people don't recognize.

It's, it's, it's going to help. It absolutely is going to help. There's still a lot of pushback about creating SBOs.

Industry is not sold on creating SBOs. There's a belief that if I leveraged, um, let's say I used Alan's secret, uh, his, his open source product, and I put just a thin layer on top of it, and I sell it as a product. If I give you an SBO and you now see that I just am fronting what Alan has, even though it was my special sauce, my wrapper, my fronting them, my proxying, um, people are worried that they're gonna be found out that.

And there's also a concern, if I let you know everything that's within my, uh, lineage, you now have it. Um, will you keep it Safe? They gave away the recipe.

I Gave away the recipe. And can you keep it safe? There's one of the pushbacks that industries had against giving SBOs to the government.

Love to give you my sbo, but don't know if you're gonna be able to keep it safe from the bad guys. You know what? Chen Wang did a great presentation on this at one of the DevSecOps events we did at RSA conference.

Not this year. Might have been two even three years ago. Tracy, I think you were there for it.

Mm-Hmm. I think so. Where we, we did a lot of SBO stuff.

We had, uh, Alan Friedman and, and some other folks from CSA there. Mm-Hmm. Anyway, guys, One last, one last quick question Here.

Go ahead. Uh, anybody can answer it. Do you not think that the cyber criminals are kinda looking at all this and just laughing their asses off?

Well, they gotta be in between Royalty. The bank can be is How everything works. It's like, you know, I mean, you know, in the movies you sit down at the keyboard and the green screen and you, and you and you type the thing, and then you break into the system.

In reality, you do a buffer overflow. That's how it all works. Well, the fact that so many of these issues are not new.

The fact that we have the oasp top 10 and nine of those items haven't changed in a decade as being the most prevalent. We ain't not fixing the things that we know are broke. There's an issue.

We're not educating people that, hey, these things need to not be in the code. We need to make sure that we're scanning for all these things. There's a lot of responsibility that goes into the state of how things are right now.

We have a responsibility to, to educate people. We have a responsibility to be scanning these things. And to, to Alan's point, we do need to, um, be more aggressive about sbo m All right, over to you, Alan.

The theme for the day was responsibility And negligence and reasonableness. But let me, let me end it on a positive note. As someone who's been in cybersecurity 25 plus years, in spite of everything we just said, I will say that our software today, open source and closed source is probably more secure and harder to hack than it was 20 years, 25 years ago.

It's just that this is a, an an an arms race and the bad guys are not dumb, and they've got a lot of resources. And, and so there's this constant one-upmanship. But overall, there are a lot of security people, a lot of software developers, a lot of folks out here who spend their lives trying to make software as safe as they can, given the restraints they deal in and kudos to them and keep up the great work.

We will do better. We have to do better. But for today, that's it.

On Textron Gang, I'm Alan Shimel, and we're out.