60% of Code Is AI-Generated—Are We in Trouble?

Hey, everyone. Welcome back here to Textron tv. Our next guest has been on with us before.

He is my friend Jonathan Rende. Jonathan is Chief Product Officer at Check Marks, and we'll get into this in a second. Let's first welcome Jonathan to the show.

Jonathan, happy new year. It's great to see you. Happy New Year, Alan.

Great to see you, and great to be here. So, Jonathan, uh, you know, for people who are not familiar with yourself, I mentioned you're the CPO over at Check marks, but why don't you give them kind of a sense of your journey? Yeah.

Well, as many people know, check marks is in security, and I started in security, uh, many years ago at the beginning of DAST and Sast years ago. And, um, left security for a while. For the past nine years, I've been at PagerDuty delivering solutions to developers.

Um, and there's a lot of similarities between, you know, the urgency of major events that happen and, and the, uh, identification and the, the security vulnerabilities that are found. So, a lot of, a lot of similarities brought me back to security and brought me back to check marks, uh, just at the beginning of this past year. It's been a great journey so far.

Beginning of pa last year. Yeah, the beginning of this past year. Sorry, I know we're 20, 25.

Yeah. Year. Happy.

I just wanna make sure. Happy New Year. It always messes me up, Jonathan and I always need to kind of make sure I get it straight.

Um, and of course, we, we knew you had PagerDuty, our, our friend Damon Edwards was there as well. And Covered page. I I covered PagerDuty, I think from when they launched.

Yeah, right. Yeah. Um, anyway, we're here to talk about check marks.

So Check marks is a company, it really needs no introduction to our audience. We do a lot with check marks, but maybe there's some folks out here, Jonathan, who are not familiar. Yeah.

If you wouldn't mind, give them kind of the check marks story, if you will. Absolutely. Well, check Marks is established really a leadership position over the past many years, uh, in application security.

And we've always been primarily focused on app security, so security before production. Um, we integrate with all the, the CNA, all the, the production vendors out there, but have been very focused on really two audiences. Um, the application security teams under the CISO that work and partner with development, and over the last two, two and a half, three years, many in the audience may not know.

We've made huge, huge progress in delivering really seamless experiences to developers. As we all know, from the beginning of, uh, security, uh, for developers. If there's any kind of friction out there, developers are gonna resist using a security product.

So it has to be a part of their workflows, has to be a part of what they do. Um, so that's been a journey we've been on and made really great progress there. And additionally, like, who can't talk about whether it's the life cycle and how it's being disrupted or, uh, how applications are being built and how they're being secured than to talk about kind of, um, AI and the disruption it's creating.

And so we've over the past year have delivered, um, a set of agents, uh, that work along the life cycle to augment both development and AppSec. So it's been a really exciting time, a lot of change happening, and a lot of just marquee customers out there who are continuing to use our product and engaging even more like using the product even more, given kind of the anxiety level going up with ai. Absolutely.

There is an, you know, I was just in the last interview we was with the Cloud Security Alliance. I'm sure you're familiar with Jonathan. Yeah.

They, they have a, a new study out around AI and governance and security, and the, there is a tremendous amount of anxiety, a lack of trust in some cases, uh, just anxiety around the whole thing. Um, now, Jonathan, you know, we've all seen some of these numbers. How much code is actually being generated by AI today?

And, you know, it used to be, oh, it's just in test systems, don't worry. It's just in the dev environment. Well, no, it's in, it's in, it's in production.

There's a lot of code out there. It creates a lot of anxiety. It's scary.

Now, check marks recently made an acquisition, right, to help bring or graft some, let's call it AI security, uh, DNA onto the check marks, you know, uh, organism. Talk to us about that if you can. Yeah.

So we look at, you know, what is happening for all of our customers today in a couple of ways when it, uh, when in the lens of ai, there's AI for security, how can we apply AI to help those using our products, developers and AppSec teams. And then there's security for ai or what many of us refer to as AI security, which means the supply chain of, of applications, how applications are built. There's a lot of new elements out there.

There's models, LLMs that are getting built in. There's agents that are getting built in. There's things called CPS that are, um, you know, getting built in and six months ago who was talking about this.

So things have changed very rapidly to your point of kind of the anxiety level of, of CISOs going up very, very quickly. And I think two things are happening. The dynamic I see, one is development teams due to the promise, you know, with great, um, power comes great responsibility as they say, and I can generate, I can create more, I can be more productive.

But with that comes a responsibility to make sure that's secure. And so the development teams and the leadership are running fast. The CISOs as, as we talk to them, uh, very much feel that anxiety you're referencing.

And we started this plan, uh, and delivered products midpoint of this past year, 2025, uh, with the first set of agents to help, you know, developers and AppSec. We call that the assist family of agents that sit on top of our SaaS platform, check marks one. And, and that's been very successful.

It's, um, delivered a lot of value to our customers today. Uh, changing like from a, a reduction of cost, reduction of time to identify and remediate for developers like upwards of two and a, uh, you know, two thirds. So 60% of the time that it took before manually using our agents, we can reduce that time.

So huge savings, uh, both in time and cost, and then ultimately reduction in risk. Now we launched that, that was really successful. And then around, uh, November, December, we started talking with some other vendors in the, in the market and, um, decided to join forces and acquire a company by the name of Tromso.

So tromso is a pure play a SPM vendor. Uh, we have an A SPM solution that sits on top of check marks one, and one of the reasons we're attracted to them, actually two of the reasons. One is, um, um, harsh it, uh, PARIC, Harshel Paric, their CEO and co-founder, um, is has been a CISO in the market before he started his company.

He's done a great job of building out a set of agents that will help in triage and remediation of issues, which is like at the core of both what deve developers and, um, AppSec teams want to do when they're looking holistically across all the repos. So a lot of expertise in some of the products to help accelerate our agenda. Um, and then secondly, the, the talent of the team bringing them in.

So accelerate our agenda on the products and accelerate our acquisition of talent. So it's been really a, a great marriage so far, and we're ready to start releasing our new set of agents. Love it.

And you know what, as we sit here in the new year, Jonathan, you know, 2025 was gonna be the year of agent ai. In many cases it was, but I, I think a common thing I hear from people is these agents aren't so great yet, right? They gotta get better if we're gonna be using 'em.

And I think that's gonna be a key piece of 2026, is getting these agen, and it's not just security related, but agentic AI in general, making these agents easier, more useful, working the way we think they should work. Um, but I, turning back to AI generated code, Jonathan, I, I need to, you know, I I, I've spoken to a lot of people on this subject. I read a lot of surveys, seen a lot of data.

Here's the thing, when we look at the security of human generated code, the line is kind of flat, right? In terms of the, the, the amount of vulnerabilities, you know, per 100 lines or whatever you want, however you want to judge it, it, it, it's not really going down a lot, but it's not necessarily going up. It, it's kind of flat.

You're getting, I forgot what it was, 30 vulnerabilities for have X lines or whatever. In the case of ai, that line is going down. It's not the hockey stick line, it's the, you know, it's going down pretty drastically as time goes on.

We're seeing less and less vulnerabilities in AI generated code than in generation before gener. And the generations come right after the one, right after the next. From what I've told is that we're roughly at a point now where we're approaching AI generated code, having the same amount of vulnerabilities in know roar.

I'm talking now from human generated code. They're, they're roughly equivalent almost now. But the thing about it is the AI code keeps getting better in terms of it.

And the human code is kinda flatlined, as I said, unless you're gonna apply AI to it, to, to kind of goose it up, if you will. Now, are we holding AI generated code to a higher standard than we do human generated code because we trust it less? Or is there something is, you know, are we rooting against AI generated code is, as it, I think comes out in your own study, 60% of code is generated by AI today, something like this.

Um, it it, you know, is it only going to get better, meaning more secure? And are we, you know, are we pining to get 100%? Is it even possible to have 100% secure code?

Yeah, it's, it's a, it's an ongoing debate, Alan, in the, in the industry. And, um, one of the, uh, one of the objective, um, websites out there and a group of experts, um, are testing every new model as they come out. com.

So if you go out and look at that, you'll see, um, testing of every new model from a security lens to see, um, each new model. How secure are they? And to your point, absolutely, um, the models are getting better.

Um, they're learning, uh, and they're still generating insecure code to be sure, like in the 30, 35% range. And, and to your point as well, it's starting to rival the, um, the, the, the amount of vulnerabilities that a human would, um, put into the market, but there's still vulnerabilities there. I think what's interesting over time is if you take this out to its kind of continuum, like out into the future, um, at some point, will the generation of code and that code being secure be good enough?

In some ways it will be. In some ways it will be. And that's a good thing for all of us.

And that's a good thing for developers. Um, I think the nature of the, uh, of the vulnerabilities though that are there is what's interesting. I, I think that that changes.

So, you know, if one of the things that can be and likely will continue to be prevented, avoided are some of the basic, um, many of the basic type of vulnerabilities that a human would, um, inject into their systems. I think the harder things will be, uh, not the SQL injections and the cross site scriptings, but the logic challenges that become very transitive, very, you know, kind of, um, one step removed. Those will be harder and harder and harder to manage, and AI can assist in those areas.

Um, uh, but it won't solve all of them. And so I I I think that we will get to a place where generated code is absolutely more secure. But will there be, if you take that to its, you know, kind of final state, uh, will you need application security or testing for security?

I know I'm a suspect being a vendor in the space, but a hundred percent yes. And I can see the nature of that, just like the nature of development is changing, the nature of security is changing. And so maybe the, the, the traditional static rules of the past may not apply, but dynamic becomes very important in this kind of a world dynamic testing.

Um, many other aspects, uh, can be, can be added to augmented to make sure, again, with, uh, with more vulnerability and more surface area becomes, uh, way more, way more, uh, threat and risk. And, you know, it's, we just got everything under control when it comes to the world of supply chain, open source libraries. If you think about how applications are built today, you know, 60, 70, 80% of applications come from components that weren't developed, right?

And along comes AI and the new AI supply chain agents and LLMs and models, these need to be secured now. So, you know, the more things, uh, change, the more they stay the same. There's, there's more threats all the time.

Absolutely. And, and I think the answer here, Jonathan, is we need, we need to improve the security of code no matter who wrote it or what wrote it or how it got here. If we're putting code in production, we should have a high degree of, of belief that it is, or high degree of certainty that it is in fact secure.

And that that's really a, whether it's AI or not, you know, when, when AI is generating a majority of the code, this, this artificial distinction between what was generated by machine versus human is just that artificial or code needs to be secure just like all rights and women's rights or whatever Hillary said back in those days, right? And, and I think that that could be a mantra here. Um, I think we're about outta time, but Jonathan, where can people get more information to stay on top of check marks?

Yeah, thank you. So, uh, we have a lot of thought leadership, a lot of material that can help, uh, your audience out there. com, but as a part and on that website, uh, we have one of the biggest, most well-known research groups that are constantly figuring out and identifying new vulnerabilities and the best practices of how to address them.

And we freely share that on something called Check Marks Zero. It's kind of a play on words from check marks one, check mark Zero is before check marks one, that's our research team. Very cool.

Um, and so there's a blog out there for them where everyone can go there. Um, we're trying to, to your point, make sure systems are secure with the kind of the ever-changing landscape. So we codify that into the product, obviously, but, but that's for the good of all and for the benefit of all.

Love it. Jonathan, it's great to have you here on Text Drug tv. Don't be a stranger.

Thank you. Thanks Alan. Really appreciate it.

And uh, again, happy New Year. You too. Maybe we'll see out.

You're gonna be out at RSA. I will be there. I'll see you there.

All right. For sure. Jonathan, Randy, chief Product Officer, check marks here on Textron tv.

We're gonna take a break. We'll be back.