Navigating Security in the Age of AI with Paul Davis | Black Hat 2025

Hey everyone, this is Alan Hummel from Textron tv, and we are here at Black Hat on the show floor. We were lucky enough with our media passes to sneak in early before the floor gets too crazy, and I stopped over here at our friend's booth, uh, uh, the Frogs Jfr to check in on them and what's happening here at Black Hat on the Jfr front. And I've got my friend Paul Davis, who's field CSO at Jfr.

If you've watched Text Drunk tv, Paul's been a frequent guest. Hey, Paul, welcome back to Tech Drunk tv. Thank you.

Glad to be here and glad to be back in Vegas, a black hat. So we came to the swamp or the, the black hat Swamp. Yes.

Yes. It is green around here. Yes, yes.

There's the frog. Um, so Paul, a lot of my audience out here is saying, gee, we know Jay Frank. They're a DevOps company.

Yeah. Yep. They're a DevSecOps company.

Yep. But aren't they a black hat security company? So yes, they are.

Right. This is one of the interesting things. If you imagine today's world, you can't push anything product, software out into production unless you're dealing with security.

And security has many different aspects. Yes, you've got the CVEs, you've got the vulnerabilities, but you've also got the compliance, the regulations, and also making sure you've got trusted software out there in production. So to do that, that's a security thing.

And security leaders, that's what we worry about, you know? Sure. Because if we, if we can't say yes, it's good.

That's what wakes us up at three o'clock in the morning when we get the phone call saying something's happened to our software. So it's a bit of a nightmare. That One.

Absolutely. Um, you know, Paul, obviously the, the theme and no surprise, the theme of this year's black hat is ai Oh, yeah. There's AI this, there's AI that Yes.

Here in a ai. They're in AI everywhere. In ai.

Yep. Um, I, I heard Caleb SEMA talk yesterday in a session from Cloud Security Alliance. Yep.

And he called, he talked about AI washing Ai AI washing. Okay. Yeah.

Just slapping AI on Yeah, yeah. You know, whitewashing everything with ai. Yep.

Let's talk a little bit about that. Yes. About how Jfr views ai, ML ops as well, how this is all coming together in terms of the frogs.

So machine learning, ai, generative ai, these are, it's, as you said, it's almost like you can't have a real product in today's world unless you've got an AI tag somewhere in there. Every brochure, everything happens. The problem is, is that it's accelerated so fast that we're losing control.

And what we've discovered is, is that many of our customers have multitude of AI and ML projects going on, but they haven't got the control. So the first thing is, is that, interestingly enough, when we talk about the attack surface around AI and machine learning, there's two sides to it. There's the development side, and then there's one's running in production.

From that perspective, you really wanna have a strong foundation. So we started last year, first of all, we discovered that they are actually attacking the data scientists. They're attacking the data scientists going after them.

So if you download like a model, you'll actually try and compromise your endpoint, your workstation, right? They'll also do all sorts of tricks. And especially like with MPC, with this new, uh, tech, we've recently just published some research saying, Hey, look, if this actually cans Q code and in infect your system, so that's a newer attack surface.

And we've, we've always worried about, Hey, let's put endpoint protection on the workstation. Let's do security. But we didn't think about data scientists, data engineers, and this is a whole new world around that.

And so really what we are seeing is, is companies saying, we've lost control of ml. We need to start putting control points in place. We need a central place to do storing the mls.

We need to do deep scanning. Because many organizations, I was re uh, watching a Gartner, uh, presentation yesterday, 80% of companies are now investigating how they're going to scan their ML models for malicious code. That's either stuff they brought in from the outside world or stuff that's been introduced accidentally.

You know, Paul, one of the problems though is that themes, things seem to be moving so quickly. Oh yeah. So fast.

Oh yeah. And it's, it's full speed ahead. Damn.

The torpedoes. Right? And, and so we look at something like you mentioned MPC.

Yeah. MPC was like invented in January, basically. Yeah.

Here we are in August. It's already the defacto standard. Yep.

Everybody's coming out with an MPC server and no one is stay stepping back and saying, what about the security? Yes. And, and you know, unfortunately, as someone who's been in security for 30 plus years, this is a familiar pattern.

Yes. Right? Yes.

Yep. We're always the afterthought. Yes.

Oh yeah. Security. And, and so I worry, yes.

I worry a lot about that. You know, Gartner, Gartner's saying 80%, I'll be honest with you these days, I don't know how much I believe Gartner, maybe we should file fire their head, uh, static statistics keeper. Yeah.

But do you really think 80% of companies are, are scanning their ml? They're not. They want to, they're planning.

Oh, they're planning. Oh, Yeah. Yeah.

So the actual reality, we just published the state of the union report a couple of months ago, and I think it's like over 70% of the executives think that we're using AI with scanning developers is just a little bit over 50. So they disconnect them. Now, what's interesting is, is that some people are realizing, especially record compliance EU with its AI laws, they're the tough cookie.

They're the people who have teeth over here. We are slowly getting there. Well, around the rest of the world, we're starting debt.

So the regulatory pressures are starting to push down the executives and guess who they look at the security leaders. Yeah. Hey, how are you gonna handle this risk?

And it's like, what do you mean? Well, you, you responsible for IT security. I mean, You don't think you're getting more budget?

No. Oh, no, no, no, no. More or less.

You know, but No, but the thing is, is that what is interesting is they're now bringing in the CISO for those conversations and that I'm increasingly having conversation with CISOs about how do I get control of this? How do I start streamlining? And the interesting thing is that the lifecycle around building ML kind of aligns with what DevOps, because at some point you take that model and it's got to be integrated.

And then one of the most frustrating things I find is I go and talk to something, it says, well, what about your open source? Well, we don't deal open source, we just have likes. Of course You do.

Yes. They use open source for cleansing the data. Use open source for Well, No, there applications are built on open.

Exactly, exactly. So there's almost this denial in education going on where we have to tell them that it's ignorant there. Yeah.

But basically what you've gotta have is now control points to make sure you are actually understanding what's ending up in production. And that's what's really scaring me is this thing of ML models are just being integrated in and the, it's being integrated. All these applications, applications you build, applications you buy.

How do you get control of that? That's what's really scaring me nowadays. I I will tell you, it is scary.

There's two things here. It's the velocity Yes. And the volume.

Yes. Two vs. Oh, yeah.

And I, as, as counterintuitive as it may sound, I think the only way security folk can can get their arms wrapped around this Yeah. Get their heads wrapped around it, is to use ML and AI Yeah. Themselves.

Yeah. To combat the issue. Yeah.

So you gotta, in essence, fight fire with fire. Yep. Let's talk about what Jfr G's doing around that.

So what's interesting, we actually just released a a beta of an MPC. Really? Yes.

Go through, which works with J Froog, but it's really clever. 'cause you can ask your question saying, Hey, what vulnerable package do I exist? Et cetera.

For vulnerability management, since we have ton, we have a thing called catalog. It is like the gold mine of vulnerability data around open source and LLMs. And you can go in there and I've got some companies are saying, this is our default path before we do a, when we do a triage, we're gonna look at that.

So we have, first of all, we have brought a company called Quack, and we've turned that into frog ml. Yes. And that is basically how do I get a consistent building path for how I actually build out my ML models?

Whether it's the data, whether it's building out the feature stores, whether it's experiments, whether it's actually rolling it out in production. We've made that whole journey. So jfr is very much focused on the process of building a secure foundation.

Because if you've got a secure foundation, then when you're having to monitor it in production, it's got less attack surface, it's got less vulnerabilities. And you can get proactive on that. So from a, from the AI perspective, we've got catalog, we have frog ml, we have our advanced security, we have the ability to store ML models centrally.

And that means we can get access, we can generate audit logs. Really? Yes.

We can show who is downloading or trying to use that ai. So you're fighting fire with fire? Well, I wouldn't say fire.

We are a car. We are like the, uh, Pepto bmo. So the calming first terrible analogy, but we, it's really trying to put the fire down so it's manageable.

So it's less painful. You don't get burnt from it. Because as I say, the more visibility we have into the process of how something ends up in production, the better.

Yep. I wanna bring up another subject with you. Yes.

Lot of stuff going on around platform platform engineering. Yes. You DevOps and platform engineering working together for the internal developer platforms.

Yes. And you know, the developer becomes the customer, if you will, of the product. How does that play into all of this ML ops and, and Right.

You know, the, the base jfr offering actually. So it is all about streamlining. Most executives want to streamline what, to simplify the process without compromising their security.

The problem is, is that at the moment, because everybody has their all favorite tools, they don't, they can't streamline it. Yeah. But we, we can't beat all things to all people.

So the platform play from the perspective of a DevOps platform, from designed all the way into production. We support that. And we have also the ability to actually, uh, use evidence files as gates to prevent something into production if it hasn't passed a test to link is sexy.

Yeah. Right. But from the point of view of the, the, the developer and enabling them, that's a big thing for us.

Shift left. We've been doing shift left and focusing for years. The thing for me is I want to turn developers into Security Warriors.

I want to give them the information at their fingertips. So you say, Hey, this is rarely a problem. It's like, we rarely need to have traceability.

We need to have Is this rarely a real threat to your software? There's terrible saying, I have a use it multiple times. One bad function doesn't make a bad package if you're not calling the bad function.

I don't need to worry about it. I don't wanna tell the about, Hey, you've got a problem 'cause you're using a bad package, but I'm not calling the bad function. Why are you bugging the hell outta me?

I just wanna write code. Yeah. And you talk about, interesting enough, the impact of this across the whole organization.

When a developer makes a mistake and it includes a bug or a secret or whatever, that ripple effect goes to the AppSec team. The AppSec team, they might miss it or they might, they might get through. Then it goes through to IT ops to production and then SecOps.

So you have all these IT ops, BizOps. So we actually call it every ops, but I've got to grips other than that, every ops, because the impact of making mistake at the beginning ripples through you double the cost of deploying somebody deploys above Double No, I think it's more than double. Yeah, it's exponentially.

Yeah. Yeah. It's just crazy.

So if I can fix it earlier, just like the kill chain, the sooner the fixer, the cheaper it is. And as I said, the fact that in the AI world, the CSOs teams are being brought in on the design, I still worry about data. I don't think we've got data governance under control yet.

No. But as far as the actual binaries, the open source, the, that sort of stuff, I think that's sexy. Excellent.

Um, just about a month out from now. Yes. We will be in Napa at the Lumbo.

We'll, oh yes. Excited for that. We'll be there actually filming live.

Wow. Brilliant. Yeah.

So we'll be there. I'll probably see you there and we'll talk some more. Oh Yeah.

I'm actually doing a training calls there. Very cool. Okay.

So they've got me recorded, like as Morpheus from the, um, the Matrix you guys sold. Oh yeah, exactly. You got, right.

Because I've got a training course about compliance and showing how you can automate easiest way of security to automate. But yeah, it's a great event. I went there last year for the first time.

It's rarely like the old fashioned security communities where you actually get a chance to sit down and talk. Yeah. Actually, it's not just No, I, I've going to swamps for years.

Amazing. Great Community Event. Yeah.

So you build long lasting connections there. So thank no doubt about it. Yeah.

Looking forward to it. So that starts, I wanna say it starts the eighth, September. Yes.

No, it's ninth and 10th. Yeah. Yep.

You're testing me now. I'll have Yeah, no, it'll be there. The ninth and 10th filming.

Yep. Paul, thank you so much for giving us a peek into j Rog here at Black Hat. Enjoy the rest of Black Hat.

Thank you so much. All righty. Okay.

Paul Davis Field CSO for J Rog here at Black Hat. We're going to continue our coverage on the show floor. So stay tuned for now though.

That's out. That's it. We're out.

We'll be back.