AI Can Find Every Vulnerability — So Why Can’t We Fix Them?

Hey everyone, welcome back here to Techstrong TV. Look, everybody's talking about this Mythos and Glasswing and is it a vulnerability apocalypse? Is it a big nothing burger?

I couldn't think of a better person to talk about this with than my friend Jeff Williams. You know what, Jeff, I'm going to let you introduce yourself if you don't mind, but don't be shy and don't be modest. Tell them who you are.

Thanks, Alan. I'm Jeff Williams. I'm the CTO and founder at Contrast Security.

I've spent the last 30 years working on application security problems, helped to start OWASP, and now we sell application security platform driven by instrumentation and runtime security. Excellent. And Jeff, for those who aren't familiar with Contrast, right, Contrast started life as I think many AppSec companies did, doing scanning, DaaS, SaaS.

You guys had some of your own twist on different kinds of scanning, but I guess it's more than a year ago already, you started not pivoting, but recognizing that the mission might change here and the mission might grow and started looking at runtime security, which for a lot of people in AppSec was kind of blasphemy, to be on that side of the horizon and a lot of different things other than just pure scanning and finding vulnerabilities. Yeah, it just makes sense when you see organizations struggling for really decades with vulnerability analysis and remediation and building security into their products. At some point you have to say, "Well, okay, what are we going to do about the vulnerabilities that we know are in production?

Are we going to see if anyone's attacking us? Are we going to stop those attacks? " And then I think what's the most important is how are we going to create a feedback loop from production back into development so that we can prioritize correctly and fix the things that need to be fixed and not the other 95% of stuff that really can't be exploited.

Exactly. You know what? I want to talk about Mythos and Glasswing, but Jeff, while we're here, let me give you a chance to give us an update on what's happening with Contrast.

Yeah. So we spent the last couple years bringing together the development view of application security and libraries and supply chain and all that with the production view of those same things. And what we found is that it really changes the game because you can get the prioritization and the context from production to make sense of the vulnerabilities that you have in development.

And it's interesting, when a new problem comes out, really, organizations should start two workflows. Like if there's a new vulnerability like Log4Shell or something, when that comes out, there's really two workflows. There's how do you respond, like incident response in production to that, and then how do you fix the underlying problem in development and get that into production as the fixed version.

And so we think that the right way to solve both of those problems is with runtime instrumentation to watch what's actually happening in the running applications. I love it. People maybe want to get a little more information on Contrast.

com? I don't remember. That's it.

Yep. That's what I thought. Anyway, go check it out.

Jeff and his team, they're always kind of on the leading where what you'll see a lot of AppSec and security vendors doing next year. You could go get a clue into it right now. Jeff, of course, we're all the security industry, and I know you have a lot of friends in it, as I do.

We're all over the place with this Mythos, huh? Yeah. It's super exciting to me as a security researcher and a long time pen tester or threat modeler.

It's exciting. AI's doing some things that we thought that really required human expertise to do. There's a few questions around exactly what did it do and how did it do it, and how expensive it was, and is this just a marketing stunt?

But I think if you take a step back, there is some real progress happening here. There is, no doubt about it. " I don't know if this is going to be the end of the internet or the end of...

But it's also not something to trivialize or take lightly either. At the very least, it's going to fundamentally lead to more vulnerability reports. The big question is kind of the tokenomics question, like- Mm ...

imagine this made vulnerability discovery free. Well, then, sure, there's going to be an explosion of vulnerabilities. Imagine it makes it, like you can find vulnerabilities, but it's really expensive.

Well, does that really change anything? I don't know. And the estimates that I've seen seem to make it look like it's going to be roughly the cost of a human pen tester to find- Oh, really?

like this is going to be like 40 or 50 times more expensive than using traditional AppSec tools and techniques. Mm-hmm. So I don't know if that's going to change that much.

And when you think about there's millions of open source libraries out there, are we going to all of a sudden find all of those? I think it's important to think of it likeLike Bitcoin mining. There's some that you can find right away, then there's harder ones and harder ones and harder ones.

But there's always going to be vulnerabilities out there because you're never going to get to the ones that are super, super, super expensive to find. Agreed. My suspicion is that this doesn't change very much in that world.

It will create more risk for companies that now they're finding more vulnerabilities in their code. There's more open-source vulnerabilities at some scale. So that's going to leave them with an exploit window to deal with.

So here's the thing. It depends on whose wallet this is, right? If you've got nation-states involved, well, except for maybe North Korea, they'll do it on a shoestring, right?

But for most nation-states, the tokenomics of it are not as important as the political ramifications, if you will, or the political- Sure. They don't need to find all the vulnerabilities. They just need to have an arsenal of exploitable vulnerabilities that they can use as weapons.

And quite frankly, I wouldn't doubt if the NSA or the Chinese Communist Party or the Russian, who used to be the KGB, they probably are still sitting on a bunch of zero-days they haven't used. So- What people don't understand is that the number of vulnerabilities that we've found so far in history, it seems like a big number, and it doubled last year. But it's a tiny fraction of the vulnerabilities that are out there.

Most vulnerabilities are latent. Yeah. We don't know this for sure, but I'm pretty sure there's tons of them out there because you think about what is our system for finding those things today?

It's like a ragtag team of volunteer researchers doing this in their free time. And there's this huge security industry built on the shoulders of these geniuses doing this work. But if AI comes in, maybe that calculus changes.

Agreed. I feel the same way. But here's the other thing, and you mentioned it when we were talking off camera.

This is really the first domino, right? All right, so now the AI, let's put tokenomics to the side. AI finds a lot of vulnerabilities.

Okay. We know that, I don't know, 98%, 99% of them are not exploitable, not reachable, or are big nothings, right? They're- Yeah, there's no critical asset, that it's not connected to the internet.

There's a bunch of ways that these things aren't exploitable. But so this is an old saying. But now AI could maybe write exploit code, so it does a good job of making some percentage of them more exploitable.

So maybe we go from 99% to 95%. People out there saying, "Well, it's still 95%," but that would be a huge amount. I don't know if we'll ever get to that high, to that big a jump.

But nevertheless, let's assume it makes some jump. Right, so now this is like the book The Goal or The Phoenix Project, the theory of constraints. Now we move to the next bottleneck.

Right. The next bottleneck is, okay, we got to fix them. How are we going to fix them?

Well, we could do them by hand. We could do them by AI, maybe. Yeah.

We could automate them. Now we fix the vulnerability. But Jeff, we go round and round the merry-go-round here, round and round the maypole.

But at the end of the day, when do we get to writing better, more secure code? Yeah, that's the thing. It's kind of a failure of imagination here.

You know it- We're trying to solve yesterday's problem with AI. So they're using it, the current mode of people's app sec programs is find vulnerabilities and fix them, and find them and fix them, and find them and fix them. And we all know that we should be doing things like secure by design and threat modeling and security architecture and assurance work, but nobody does that because they're too busy feeding the hamster wheel.

But I think the real interesting opportunity here is how do we keep toppling those dominoes, and we use AI to do some of those activities that we were never able to get to because we didn't have enough experts. We couldn't scale them. They weren't part of our process.

But that's the real opportunity here is using AI to generate a positive assurance case automatically for the software that you're building. And if you read the people that are really doing software development with AI, they're talking about dark software factories where you feed it a spec and it runs in the dark with the lights out, and a few hours later, it spits out a fully complete, fully tested, a working application. And look, that's on the cutting edge here.

It's not perfect yet. I'm not saying that exists. But it's going to exist.

And we can make security part of that, but we got to be working on those dominoes, the next five dominoes that have to fall. How do we do threat modeling with AI and feed that into a spec? How do we do security architecture and feed that into the spec and so on.

That's how we get there. Let me ask you a question. So you've been at this for more than a day, right?

You ever wished, if you could go back 25 years ago and tell Jeff 25 years ago what was happening here, what do you think Jeff at 25 years ago would be saying? I'd probably make a-You know what I would do? So I would make a really dumb security product that checks a box, and I'd sell it like crazy.

Sell it like hotcakes. For the last 20 years, that's what has succeeded in the market. That's the security industry.

And that's terrible. I probably wouldn't do that. But like it- No, I know you wouldn't ...

it would make a lot of money, but trust me But I hear what you're saying. But let's be real for a minute. As much as, and you know a lot of security people, I do too.

We want to make the world better. That's right. We want more secure software.

We'd like to see developers develop better, higher quality software. We want to make it easier for people not to be hacked- Yeah ... and security instances.

However, we're very much a A-to-B-to-C kind of world of industry, right? And you look at this as a visionary kind of person that you are, Jeff, and say, "Wow, I could see beyond the first three dominoes. " And this might get us to some sort of Promised Land.

Yeah. But the average Joe, he doesn't. He sees going from domino A to domino B to domino C.

And you can't blame him, right? Right. His bread's buttered there, and their bread is buttered there.

But so how do we get people to see that big picture, to see the end game, to move to that and not get stuck, like in the trees? Yeah. Forest.

I think the more I mature and the more I understand the market better, I think it's pretty obvious to me that the market is broken. And I've talked about this for 20 years, but the market doesn't really demand security, and a lot of that comes from the sort of the deregulatory, the big tech lobby and the deregulatory kind of environment that we have in government right now. I thought we were making progress with some previous cybersecurity orders, but the latest ones are really not pushing that hard.

And without regulation- There's a concept of a plan Yeah. So that's going to be really tough to change. We have rules in this country that say money is speech, and so big money buys policy.

And, without policy change, that's going to be tough. So we've got to work within the constraints that we have, which is that organizations are required to do very little. The compliance requirements are, they seem burdensome, but- Ah, no.

They were always least common denominator requirements, but they're even worse. They're getting worse now. I agree with you.

Look, this Mythos thing is, and the response to Mythos is a perfect example, right? The CSA came out with their Mythos Ready report. A lot of people you and I know are involved in that, including the folks at RSA and a lot of industry.

But little to any public government involvement. Well, here's the thing. It's going to take a long time to change for the reasons that you're citing.

Security people, they want to do the right thing, but they're also very conservative. And I think I'm pretty aggressive that way. I want to blow things up and change them because I don't think what we've done is working.

And when you look at the stats in AppSec, the average app still has 25, 30 vulnerabilities. That's the same number that it was when we wrote the OWASP Top 10 in 2002. And it takes six months, on average, to fix vulnerabilities.

There's just all the stats are really- Oh, and that's part of this. Okay, so I already have this many, let me put my hands up, this many vulnerabilities that are just sitting there. Now I'm going to have this many vulnerabilities that are just sitting there.

What changed? Yeah, right. It doesn't change.

Mm-hmm. And I feel like we know that the existing process doesn't work very well. To me, that's the bar we have to exceed, but- That's a pretty low bar, my friends.

It's not for nothing It's a low bar, but it's very difficult to get people to change the way they do things. They think that AppSec testing has to be a certain way. It has to be scanned.

They think that security architecture is something you do every once in a while. That compliance is the big driver, and I don't know, it's weird. People don't use runtime protection.

That's crazy. The one thing you can actually do that really lowers your risk dramatically is put some runtime protection in place around whether it's an AI app or a regular app or API. You shouldn't run without that Some of that is because of the silos in security.

If it ain't AppSec, it don't mean crap, and so in AppSec is all about runtime, and, oh, that's someone else's job. That's a different people in security. But that's the biggest risk is the people- Oh, absolutely ...

the processes, and the culture. Those are going to take a decade to change. Yes.

Yes. Even with AI. Agreed, and I wish I could paint a rosier picture for you and disagree.

But yes. Right? Now, if we start saying, "Okay, if we're going to speed up vulnerability remediation, we got to automate remediation," well, you got a big problem in the IT department there about that because they're not going to go for that.

I learned this lesson in my days of doing Stohs Secure when we had intrusion prevention. We went from detection to prevention. We went from vulnerability finding to remediation.

No one wanted to turn that on for whatever reason. Yeah. And people are suggesting that, oh, well, we can discover the vulnerabilities with AI and we can fix the vulnerabilities with AI, but there's a massive difference there.

People are much more reticent to accept an AI-generated fix into their code base than they are to use it to find a vulnerability. Yeah. And even if they're using AI to build the code, they're still not comfortable.

The security people aren't comfortable with using AI to fix the code, which is kind of crazy, but- But it is. But it is what it is. That's why I love this time.

I mean, it might be crazy or not. So here's the thing, though. I do think we're going to pass through a bit of a crucible, right?

Just the sheer weight of vulnerabilities and these other things, I think it's going to, I don't want to use the word force, but force people to reevaluate a little bit how we do things. And I think we do come out the other side better for it. Yeah.

It's just getting there. Yeah. It would be incredibly helpful if government would put some weight behind it.

I know in the EU, they're pushing the Product Liability Directive. Absolutely. It's super exciting and changes everything.

It's a really simple change, actually. It just says software is now a product like every other product. Yep.

And you can be held liable. No, here, we're trying to preempt states. Let's not get into the politics of it.

Let's just say that I do hope that one day, maybe after this next election cycle or the one after that, we kind of come to our senses a little bit and realize- Unless the market changes ... it has to be a partnership. Unless the market changes, nothing's really going to change.

We might have some changes on the margin, but it's going to be still mostly about not getting fired and meeting compliance. And that's all good. And so doing good security is going to come third, even though that's what everybody wants.

No, it's the A to B to C people who, they won't get to C without doing B. Anyway, Jeff, we're over time, but I enjoyed talking to you, man. I'm sorry I didn't get a chance to see you at RSA this year.

We were busy. I know you were doing different things with Contrast, but don't be a stranger, man. Oh, of course not.

Thanks. Especially now. There's too much good stuff going on.

We should talk more. Well, now that you're in the 20th century with your fiber hookup, we can talk more frequently, maybe. Absolutely.

Always on. com, here on TechstrongTV. We're going to take a break.

We'll be back with more.