AI Security Remediation Needs Continuous Human Validation

Hey guys, thanks for the intro. We're here with Kara Sprague, who's the CEO for HackerOne, and we're having a chat now about, well, there's all these awesome capabilities that the frontier AI model people are providing for cybersecurity these days, but how do you operationalize that, and how do you turn that into something that you can actually leverage? Well, that remains to be seen.

Kara, welcome to the show. Thank you. Appreciate you having me.

We have seen that the providers of these more advanced AI models have given limited access to many cybersecurity companies, but how does that get passed down to the average cybersecurity organization, and what can they benefit from, and how do they take that and operationalize it in a way that gives them something that looks like an actual benefit? Great question, and it varies depending on the frontier lab and also depending on the program. And we've experienced with a few of them.

I can speak to our participation in Glasswing as an example. We joined that, and as part of our participation in that, the intent is, and we are given access to use Mythos, as well as the next generation after Mythos, to test our own defenses and use that to shore up HackerOne's own platform and enterprise security. I want to say up front and just be very clear, as part of this work with AI, HackerOne does not, nor do we permit any third parties to use confidential information to train, fine-tune, or otherwise improve the models.

But we are using these models extensively from discovery phase all the way through validation and ultimately into remediation, so that we can shorten the time between finding issues and fixing them. Well, how do we fix the issues? I think that what we have seen so far is that it's pretty clear that the number of patches that are being created has exponentially increased, and a lot of these things are becoming more difficult to deploy and keep up with.

So what have you seen people do to operationalize all this in a way that allows them to keep up with this? Because otherwise, we're playing whack-a-mole with patches, and the patches are more than we can handle. Yeah.

And I think that is a real challenge that we are going to be facing for quite some time. I think I've heard different folks around the industry talk about how long this deep clean period might last. And my current thinking is it's looking it's going to be at least 12 months, up to two years, and there may very well be organizations that are still doing deep clean long after that.

And we are going through an intense period right now where the bugs getting found, we just keep finding more and more. I remember just a couple of weeks ago, a post from Daniel Stenberg over for the Curl program, sharing that he had done more runs on Curl and Mythos and the latest GPT had come up with nothing. And it felt like just a couple of days later, the IO team went and ran their harness against Curl and found, I think, a number of additional vulnerabilities.

So even the most hardened, most widely deployed software out there is still going through a bit of a deep clean. And so what does that mean if you're a CISO or an organization that's thinking about deploying these things? Well, from HackerOne's own experience, if you get access to these advanced cyber-capable models, make sure that you take advantage of the access and the credits that the labs are offering for using them.

Point it at as much of your environment as possible to understand what it finds, because the window between when you have that access and when an attacker will get access to some sort of similarly capable model is very short. And so find the issues, then you have to have a team work through them and figure out which ones are truly externally exploitable and assign risk to them, and then ultimately shovel them through to your engineering teams with enough context, and enough clarity on the prioritization that your engineering teams can get to remediating the issues. I would say another thing is these models are finding a lot more.

They are also finding a different class of vulnerabilities or issues that I've seen from other automation. As an example from our own work with Mythos, this not only did kind of a snapshot scan across our repositories and our code bases to identify issues, it went back in history. And so looking at multiple commits against the code base and across dependencies, across authors over time, to find issues where each individual commit itself may have seemed harmless and safe and passed the unit testing and the functional testing.

But over time, it accumulated into an actual vulnerability, and a relatively severe one. And so, that's an example of how these advanced cyber models are actually surfacing issues that other tools were not finding. Everybody's kind of expecting this tsunami of attacks based on all these vulnerabilities that are being discovered.

Have we seen that yet, and if so, what do they kind of look like? I think we are seeing evidence of that. And I'll just point to the amount of ransomware activity that is getting done over the last several months is up significantly.

Threat actors in that space are having a heyday at going after organizations. You saw from Verizon's DBIR report that now for the first year in many, many years or the first year ever, that breaching a company via way of a vulnerability is now the number one cause of breaches. Previously, I think phishing was the number one cause, and so that is definitely something that has changed.

And so the frequency of cyber attacks is up, and the frequency at which those attacks translate into a breach via a vulnerability in code is also up. 5. And, well, like it or not, they seem to do a bang-up job discovering vulnerabilities, even with something that's not quite as advanced as Mythos or Fable.

So are they really standing around waiting to gain access, or are they already at work? They're already hard at work. And you're absolutely right, and just the reality is that a lot of our software is just really, really insecure.

What will be the role of the folks who are the contractors that people can hire through your platform, and how do they kind of play in this whole equation? Because for the most part, I don't know many organizations that have the resources to go deal with all of this on their own. Yeah.

Our researchers, and you're exactly right, we have tens of thousands of them that are actively hacking on programs every day, and they are identifying and using AI tooling. So many of them are very excited about testing out the latest model capabilities that get released, and regularly talking about how one model has improved over another. But we benefit from having that diversity of that independent security researcher ecosystem, combined with the diversity of them adopting every model from open-weights models through to those that are released by the frontier labs, and their unique kind of methodologies and perspectives that they bring to their own security research and pointing at those assets of our customers.

And so, they continue to find, and the submissions on our platform are up over 100% year on year. The researchers are now finding a higher percentage of critical and high-severity issues than they have been historically. So the number of valid issues and the number of high-severity valid issues found by researchers is up significantly.

As we go forward here, are we going to have to just change the way we build software? And I ask the question because historically it was kind of like we all focused on the features and that was the new capability. And maybe once a month we sat around and worked on patches, and then we'd ship those over to somebody who may have spent some time testing that.

So it was maybe a 30-day cycle by the time the patch got applied, if we were lucky, sometimes not at all. Do we need to just rethink that whole DevSecOps workflow? We absolutely need to rethink that entire workflow.

And that's what the whole concept around continuous security really is about. It's about continuously testing and continuously testing through a layered set of defenses, including automated scanning of your code bases. It includes continuous black box testing, both automated and with humans, such as with bounty hunters.

It includes continuous validation, so those findings are continuously being prioritized and identified as which are the ones that are truly exploitable given your own deployment circumstances, and then continuous remediation. And what that will call for, it calls for from security organizations, is a fundamental retooling of that whole discovery to remediation workflow, and using as much automation as possible, but also including the layer of human accountability and human attestation around it. And for engineering teams and engineering directors, it also means, at least through this period of deep cleaning, that we're putting a lot more prioritization and resources against remediation efforts.

Mm-hmm. The other thing people are trying to figure out is, is the code being generated by these AI tools better, worse, or roughly the same when it comes to the number of vulnerabilities that might be included? Because while there's certainly a lot more code, but I'm not quite clear if the ratio of vulnerabilities is any higher or lower than when humans were doing it.

Yeah, there is certainly a lot more code. And at least the research reports published up until around March this year, I haven't seen the most recent, but up until March, the reports were still coming out that code that is getting shipped by AI is generally less secure, includes more vulnerabilities than code shipped by humans. Now, that may have flipped because there's been a lot of movement on the front of secure coding agents.

And a lot of enterprises are now adopting those. And so again, I don't know if currently the flow of code getting deployed is more or less secure than it has been historically. But if you think about this, there's a gigantic amount of deployed technology out there, both software, firmware, hardware, that was pre-AI, and that was very insecure already.

And then we've been publishing a whole bunch more technology out there up till at least March of this year, and that was also pretty insecure. And so we've got a huge amount of security debt to work down. The other thing is, of course, people are using AI tools themselves to go discover vulnerabilities in their code.

But I also hear folks saying that the number of false positives generated by those tools is substantially higher than previous generations of tools. So what's the role of the human in all of this? Where does that tester fit in, and how much do I lean on AI, and how much do I lean on the tester?

Yeah. So our own experience a little bit different. So relative to the old automation like SAST tooling, where we would frequently hear from customers that they were seeing a 95%-plus false positive rate, our experiences with these frontier cyber capabilities in doing white box testing by decode is a much lower false positive rate.

But still enough that you do need a human, which is reviewing each of the findings, and again, the role of that human is, one, to confirm that this is an exploitable issue in your environment because you don't want to tap your development resources on issues that are not actually exploitable externally. And then secondly, to assign a risk score to it so that you're prioritizing which ones matter most for your few dev resources or your dev agents to go fix. And so the role of the human here is really, that's a part of the workflow that we call validation.

So it flows from discovery or report of a finding through this validation, and validation is still something that is very reliant on human judgment and context of the deployment, and then through to remediation. And at the remediation side, we're getting a lot more advanced in terms of you can carry context through that entire workflow and get to feeding through the various workflows that engineering teams consume. For example, Git or Jira or other types of workflow tools, and get the right context and the right information to the right consumer of that all the way to, in some cases, even publishing a PR automatically.

So we're starting to see full automation through that entire loop. It's very exciting. What do you see folks doing that just makes you shake your head a little bit and go, "Well, I think we might want to be a little bit smarter than that"?

I do see a lot of security teams and organizations are-- There's so many new tools and capabilities and dozens probably of companies that have been funded or entered into some sort of continuous testing or autonomous spin test category in just the last year. And I do see, unfortunately, a lot of security leaders are caught in a bit of a vendor evaluation cycle or getting excited about benchmarking model X is better than model Y on this particular thing. And where I think the focus really needs to be very urgently is on burning down the backlog of known un-remediated risk.

And so at some point, yes, cost should be an optimization around how we use these tools, and we want to make sure that you're not breaking the financials of your organization trying to fix this problem. But we also have to recognize that this deep clean period is exposing a ton of risk and huge amounts of cyber risk that needs to be closed, and closed very expediently because the attackers are not waiting. Do you think at the end of all of this, with perhaps more pain than anybody cares to admit, but are we going to get to some point where the applications that we're deploying are finally a whole lot more secure than they have been historically, and we might actually have a net positive at the end of all this?

There's different folks say different things about this. I am more of a pragmatist here. And while I think there will be parts of our tech stack, like maybe, for example, just classic web applications where we get to a point where the vulnerabilities are few and far between, we are innovating on technology layers so quickly and deploying new technology stacks.

So, for example, everything to support an AI model or an AI system, and agents even, those are entirely new frontiers of security. And so I think as we deploy more and more of those, there's going to be more areas for us to find new vulnerability classes and more innovation. That's why I think the job will never be complete for human security researchers who need to be at the frontier and are the ones who are providing this kind of attestation or assurance layer around your security posture.

And that's also why I don't think there's going to be rainbows and kitty cats on the end of this. I think we'll have safer web applications and safer kind of foundational infrastructure tech. At the same time, continued innovation and continued areas to shore up in the new deployments.

All right. Well, folks, you heard it here. Yes, everything now in software engineering is happening at machine speed.

The challenge now, of course, is trying to figure out where does the human fit in all of that because, well, the machines can't do it all. Hey, Kara, thanks for being on the show. Thank you.

And back to you guys in the studio.