AI’s Impact on DevOps and DevSecOps | RSAC Virtual 2024

All right. We're in the very early stages, in my opinion. Think about when did copilot come out February of this year.

So, you know, when you start looking at AI and you start looking at what AI can do, I mean, we're really at the very beginning of what AI is now, there's a lot of definitions. What really is ai because machine learning is, is just a component of ai. And so when you start looking at models and large language models and you start looking at the overall, I mean, really for the most part, we're at the very beginning.

I'll go a little bit deeper and I'll let the guys speak. Yeah, sure. So, um, I agree.

We're at the very beginning. Uh, there is one aspect of this whole systems thinking problem that it we're not at the very beginning of. We're still early stages, and that is in terms of using AI to write code.

And so, so that part, we sort of now start to have some information about GitHub copilot. I, I'm a active developer. I write code every day.

I'm the primary author of a dozen open source projects, one of which gets a million downloads a month. I have a full team of folks working on that project in particular. And, um, we, we started using copilot almost two years ago now at this point, GitHub copilot.

And so, uh, the productivity of those folks is two to four times, uh, greater, which aligns with the Microsoft talk just a, a few minutes ago. So what does that mean for us? That means that there's more code that could potentially have vulnerabilities in it if the, if the density of the vulnerabilities is the same as before, that's a two to four times increase in the problem.

But reality is, in the research that we've, we see starting to come out today is that GitHub copilot generated code, or assisted generated code actually has a higher density of defects than non generated code, the prior code. Now, we don't have research that says it has a higher density of vulnerabilities, but what is a vulnerability but a defect? They're highly correlated.

So you can actually expect that to be an increase in the density. So, so you could be a two to four time problem X pro, uh, of the problem before could be more of a, like a three to six x, uh, of the problem before. And so that's, uh, on the problem side.

Now, we, we we're at the early stages on using AI to actually solve the, the problem. And, and that's very promising. And hopefully it keeps up with the fact that the problem side has gotten so much worse.

But we'll see. Yeah. So, uh, I actually agree with, um, uh, all the sentiment.

So I think that, uh, we are at the beginning, uh, maybe not the very, very, very beginning because we are see, starting to see some traction. Um, and I think my rule of thumb for understanding whether we're at the beginning of something or we're kind of like well through it, is how I can imagine the reality being eventually, and the imagine the, the imagination I can put for what we can do. And I hope that, that that's what we're gonna explore in this, uh, conversation, um, with, uh, gen AI and how it will, uh, empower us as security professionals, as also as infrastructure people.

Um, it's way beyond what we have now as available. Um, and that's why I believe we're very, very much at the beginning. I think that's, that's the, the plus side.

And also, uh, referring to, uh, the, the last, uh, statement, um, again, I don't, I don't have, uh, a lot of data about this. Uh, but the way I see it as LLMs, um, they, they will of course generate a lot of coding. Kind of makes sense, what you say.

Um, and the reason for that is that the way they work, and again, to, to my understanding is they, they just generate the next most likely, uh, text to come mm-Hmm. Um, and they're optimized to what eventually the developer will accept. Now, what eventually the developer will accept does not necessarily have to be the most secure version.

Um, it does have to be, I think what will eventually work, which is, again, I think that us as security people know that that could be potentially very, very dangerous. Um, so we are going to see a lot more code and as adoption picks up and more and more code is generated, and that code is, you know, based on, again, your stats or, uh, the sentiment that you, uh, spoke about might be more densely uh, flawed. And that is, that is a huge challenge that we need to look into.

And, and one more thing that I'm gonna throw in there too is, you know, think of Chet GPT and take SQL injection, plug it in, run it 20 times, you're gonna get 20 different answers. And what I mean by that is some of the answers are gonna be, Hey, here is how to clean the data, you know, go through this method, clean the data. That's the wrong answer.

And the problem is, is some of the AI is just gonna say, here it is. And you need to be smart enough to know that is a good answer or not. And so if you're a student that just graduated college, you're now in the market and you're running, the problem is you think this is the right information and it's wrong.

And, and for those who, you know, know, we, we know, you know, parameterization and, and commands are the way to go. But the problem is, is, you know, you ask it 20, you know the same question 20 times, you could get nearly 20 different answers. Some right, some wrong, and a variation in between.

Thank you, Chris. Yep. I'm curious everybody's opinion about, so the code that we generate through LLMs, we can generate through co co-pilots and such.

Do we need specialized LLMs that know how to generate secure code? Or do the LLMs know what secure code even looks like given that the output is non-deterministic? You won't get the same answer all the time.

I know you're doing a lot of development. What are you seeing happening, Larry? Yeah.

Yeah. So I, I, I don't think you're going to get anytime soon specialized LLMs that that generate code, but only in a secure way. But what you will get, I think is sort of a staged process where, where the, the, the code that was generated will be like normal code and then, and then code that will look for vulnerabilities will also use gen ai.

And, and that's maybe not the bottleneck, though. The code to resolve the vulnerabilities that are found is the bottleneck, though. And so you can automatically patch that code with an eye towards security.

So I think what you'll see is essentially this pipeline change. So that's why I take a systems, uh, you know, system, systems thinking approach to, you know, theory of constraints and what's the bottleneck. I, I won't go into that now, but if you wanna poke at that a little bit.

Tuck, it's the guard of DevOps, right? It's the heart of DevOps. Exactly.

It's the flow feedback and the culture of experimentation and learning. Right? Well, flow has been recently essentially renamed systems thinking.

'cause it was always meant to be systems thinking flow is just a shorter hand uh, word for that. But yeah, so if you're not, if you're not optimizing for the bottleneck, you're not making all of your improvements at the bottleneck, you're basically being wasteful. And so you ha it, you have to understand where the bottleneck and the process is.

Well, yeah, the world is changing so fast right now with gen ai, that that's a, that's a very rapidly moving target. And you're gonna have to do some experiments and measurement and, and, you know, even empirical sort of qualitative, uh, uh, uh, study in order to understand where the bottleneck is in your own process. And when I say bottleneck, I mean like for the process of, of making sure you're producing valuable code that's free of defects.

There's a lot of steps in in doing that. So yeah. Good.

Anybody Else? Sure. Um, so think about it this way.

Come sit down. If I take a database table structure, plug it in and say, create me an API endpoint, it's gonna create it, but it's not gonna know, Hey, this is 15 car. Well, it should, this is 15 characters, because by definition of the database structure should be, but the problem is it doesn't create the validation scripts.

It doesn't create all this stuff that says, Hey, has this been tainted? The information coming in, it does not look at or, or know the whole thing. And so, I, I love when, when I do a lot of these, I love asking questions for a show of hands.

So show of hands, if you were to have an automated process that would automatically say, Hey, guess what? This is bad. I'll automatically create the poll request.

And you just simply accept it, or we'll just auto accept it to create it. How many people would actually do it? No hands.

No hands. Thank you guys, because that is Right answer, right? I'm gonna challenge that a little bit though.

Okay. Let's a conflict on here. So there are some tasks that gen AI already does significantly better than humans.

And data cleaning is one of those tasks where, right, where the gen AI can do it 60% better and a thousand times faster, right. Than humans can do it. So, so you have to, you have to be a little more nuanced with, with that and saying, you know, we're not gonna accept what the gen AI fixes necessarily.

I, I, I have a question from a different department, but I think it's, uh, relevant to what we're talking about. Who here would ride in a car driven by ai? Who has a Tesla?

Uh, still, still not a lot. I expect, I actually expect a lot more. I expect that to be like a more mature, uh, thing that we have, uh, envisioned.

Like smart group of people over here. Exactly. I I, I think so.

Um, Cautious group of people. Maybe It's a security conference. It's A Security conference, Right?

Right. What is the definition of a security person? Someone standing across history shouting, stop, you know, don't do that.

So, so here's a question and, and we are generating code now, right? With code copilot and other capabilities like that. How do we, as security professionals say, should that be treated differently than if it's a human generate code, if it does generate more errors?

And proportionally some of those errors could also be vulnerabilities. Are we just setting our ourself up for more security issues? Do we need to be doing things either as the development team or security organizations to say, good stuff, let's keep doing it, but here's how we get better at it so we don't introduce more problems as opposed to get better at what we do?

Yeah, I think it depends a lot on the stakes. Yeah. I mean the, the stakes are a very important factor because in, uh, in a high stake situation, I, I I, I don't see it happening without a human in the loop, uh, very much, um, in a low stake situation or no stake situation.

Uh, definitely I think would be like interesting to experiment, like to see what, what, what we can, what we can get there. So the stakes being, you know, on a, on a scale of things, human lives are, are higher value than, than business profitability. Because, because you can imagine a world where you would intentionally accept, or products where you would intentionally accept lower security in exchange for more features and getting them out faster and getting more users.

I, I think Zoom at the beginning of Covid is a great example of this. They got dinged for some choices. They made that favored user experience over security.

And you know, they won though the zoom, you know, uh, you know, work from home thing. So, so they made a smart decision. So from a financial perspective, yeah, but if, if you're talking about medical devices or military, uh, weapons controls or aircraft controls, then yeah, I think, I think you're right.

You're gonna be much less likely to, to, to accept. Uh, I'm gonna go a little different. So when I've done ai, when I've run teams and we were looking at things, I look at AI as a friend, a partner, a helper, co-pilot per se.

And I treat the data, the information, kinda like from Stack Overflow. You take it, you look at it, you say, does this meet, does this work? Does it do what I need to?

You can use AI to make your your stuff better. You can take nested for loops and say, give me a link, query it, will it, it does a great job. And, you know, you need to look at it in that way right now.

I mean, we are progressing at the light of speed. But one of the other things too, that a lot of people, and this is a question for you guys, how do you look at it from a senior tech review? Because a lot of people look at the code and they think, well, this is machine generated, so therefore it's correct, it's right, it's secure, it's got all the parts and pieces.

It works. Yeah, it could work, but it's not, right. And so from, you know, a senior tech review portion, you know, what are your Thoughts?

So, so currently the way I look at, uh, gen AI and it's curtain state of the art, and that's, I think that the best metaphor I've heard for it is it's sort of like that, uh, kind of an eccentric employee that you have, that it's very, very talented, is very, very talented. He or she, I don't know what pronouns to use, um, but they, they have access to all this information. They do things very, very fast.

But every once in a while they're like, they would do something that's kind, kind of like out there. They would invent, invent things. Uh, they would say things that are utterly wrong and do it with extreme confidence.

And that is how you look at it. Like, this is, okay, I have to review this. I have to make sure it's right, but I can get exceptional things from it.

So are hallucinations your developer on drugs? We've got those for a while, and some Of 'em are pretty good, right? Yeah, I, I, I haven't experimented with it, so I don't, no, no.

I don't know. No. Well, so, so here's a question I'm now drawing from today.

One of my takeaways from today is we need AI to secure ai. We need AI to test and train ai. We need ai.

And, and arguably you could say logically at some point, humans can't do it 'cause it's too large of a problem, too quick of a problem. You could do it faster, et cetera. Is that something we should kind of a security professor professionals be looking at to say, well, what should be training what so that we know what to trust, which models should be training?

What kind of models in what situation? 'cause we just kind of turn that loose to everybody and say, train your model. Use whatever you want.

We're kind of introducing yet another variable of security or lack of security introduced into the models that we're training. I mean, I like the idea of ai AI training or securing ai, but it seems like that's yet another attack vector. Feel free to disagree if you don't agree.

Yeah, I I I, I I think that's, you know, AI to, to follow up after AI written code with, with AI tools to detect vulnerabilities and then another stage of AI to remove those vulnerabilities. I think, I think that's gonna happen. I mean, that, that's already happening today.

You know, so I think at some point these very straightforward things will be acceptable. Like, like nobody would get behind, get in a car that was self-driven five years ago, maybe 10 years ago, maybe. I don't know what the, the timeframe is, but we had, we had a handful of hands go up here.

And I think if we were not at a security conference with, you know, folks across a thwart history, yelling, stop sort of being the tendency, um, you, we would have even, even more. But you go back in time. So if you go forward in time from this point about it, you'll have more people willing to do that.

But that is not really much different than the entire history of software engineering. So the progress of software engineering over the years, it essentially comes in one of two forms, and it comes in leaps in one of these two forms. It either lowers the cost of coordination dramatically like CI does, or it serves as a cognitive prosthetic, what was fortran, but a cognitive prosthetic for writing assembly code.

What was higher level languages? What were four gl What are what, what, these are things that just sort of help you write code faster. And the design intent is the part that, for a while at least, is still a ways off.

And, and that's what the developer and the engineer role needs to shift to, is to be the, the head designer and, and let the things that are a little more mundane, like writing code even be done by the, the, the tools more and more as time goes Well. And, and one thing I want to add to this is, you know, don't put your hand your head in the sand when it comes to ai, because I mean, the benefits are, are definitely amazing. And, and when you look at the different tools and the different things that are out there, especially from an IDE standpoint, you know, you have Visual Studio that's nearly predicting your next line of code, almost accurate, to the point where it really does speed up your development process or other languages that, that are out there.

And so, you know, it we're, and that's why I said we're kind of at the early stage because, you know, as we progress, you know, I, I was on a panel before this where, you know, the question was, where do you see AI in five years? And you know, the answer, the answer, you know, and I agree with what was said on the panel is, you know, if, if I knew where AI is gonna be in five years, you know, that's where I put all my money because I'd, I'd be rich. But the reality is we're moving at such a fast pace that right now there's a lot of bumps in the road, but things are gonna get better, smoother, we're gonna be more actionable.

You are gonna hit the point where when you look at your overall pipeline, you know, from the, the, you know, the development standpoint and, and the tools that are gonna go, Hey, that's SQL injection here, let me give you a code example of fixing it right here, right now. Hey, do you accept it? Great.

And then you start moving down the line to the right, where you get to the point where something is going on and, and you know, when you're doing development and you're doing a microservice that has 30 pieces and you're only focused on two or three, and when it all comes together on the build, now you've got the whole picture that AI can see the whole picture and say, Hey, wait a second, your little parts and pieces by themselves are great, but once you start putting everything in the context of the entire application, now you have stuff. And for the simple things to what where you were talking about, absolutely it can correct it, it can clean it up for you, it can give suggestions. I'm not there personally where I would be willing to accept a poll request blindly, but I mean, they're getting better.

Definitely better. I, I, I think an important thing here is the business value and what you make of it. And, you know, you spoke about the different kind of models that there are in the air for this and the air for that there, for this, I, I think it's very important to focus on what we make of it, because we don't want AI for the sake of ai.

Um, we want AI for doing things that we do today, and we can harness it in a responsible and an effective, uh, way. And I think that, um, there are, there are like, I think three kinds of approaches, um, to that. Um, basically, um, so there are people that are very much overwhelmed with ai and they said that they even like kind of mystify it.

They don't understand it. They don't want, they don't want anything to do with it. Just, you know, when when it's ready, I'll, I'll use it.

Um, and there are the other kinds of people on the other end that, uh, what they say is, oh, it's the shiny new thing. I gotta have it. It's the, it's the silver bullet.

It's gonna fix everything. That's, of course, that's the other end. And I think that the truth, of course, and think the healthy approach is to think about a use case, to think about what it is that you need from it, and then to harness it in the most responsible way possible.

And as we discussed to understand its fallacies to understand its benefits, to understand what it can and cannot do, how much you can rely on it and then move, uh, along, uh, based on experimentation, based on measurements and stuff like that. And if you apply, uh, this kind of an approach, you will make the most of it and eventually it will get better. I think.

I mean, I think that's, Yeah, I think, I think that's right. So how many of you, show of hands, have heard of the Gartner Hype Cycle? Okay, good, good, good portion here.

So Gartner recently put Gen AI at the peak of inflated expectations. That's the, the peak there. What comes after the peak of inflated expectations?

Does anyone know Traffic disillusion the trough of Disillusionment? Yeah. Yeah.

So, so here's my survey to the group here. Uh, how many of you think that, um, ai, I'm gonna give you three options. How many of you think that that AI will go down into the trough of disillusion meant and never come out?

Show of hands, one person. Interesting. I, I thought that about, about, um, blockchain and it turned out to be true, but I haven't thought that about many things.

I've always been wrong though. You're right. Right?

So yeah. Yeah, we did. But, um, how many folks think that it will go down into the trough of Disillusionment and come out and, and, and that's, that's sort of about three quarters of the people.

How many people think, and this is, this is sort of like anti Gartner hype cycle. How many people think it's never going to really significantly go down? It might level off for a while, but it'll always keep generally going up.

Okay. And that's the remaining quarter, essentially. I, I, I'm in that latter camp.

This is one of those really significant changes to tech that is not going to go into the trough of disillusionment, in my opinion. It is. It, it, we're gonna figure it out and we're gonna struggle with it, but it's always gonna keep going up and up.

The capability is gonna get better. The value it delivers is gonna continue to rise. And, uh, you know, it might be some changes in sort of overinvestment in areas like, like NVIDIA's stock do dove after sort of the big surge of buying, um, AI machinery, uh, went down.

So you might see a little of that, but I don't think from a technology perspective, I don't think we're ever gonna go down, is my opinion. I look, I look at it as like an iPhone moment. You know, a lot of us have the old Nokia phones and we just, yeah, we got a phone, right?

And look at the iPhone today. I mean, look at what all it can do. It was a game changer.

That's what AI is. It's a game changer. And it's only going to get better over time.

We're, again, it, it's very rough in its early stages. It's like the, the original Google phone G one with the little roller track ball. I mean, what was that about?

Right? But the reality is, is it evolves and, and the phones have evolved and they've gotten better and they've gotten stronger. And now, you know, you can open up an app on your cell phone and you can take a picture and then you can click a button and, and remove an item or an object or a person out of a picture.

And, you know, it's only gonna get better over time. If we have any, if we have any questions, please go up to the mic. Oh, do I have somebody here?

Oh, we have a gentleman over here that's already at the mic. Sir, would you ask your question? So I think we all, we're all here because we agree the AI's great and it's gonna change the industry.

It is changing the industry. It has been with copilot. I think we also, there's probably general consensus that AI is great so long as it remains a copilot.

I think the fundamental problem with the two questions you posed earlier was you asked, would, would you let a car drive for you? I think most of us would actually say yes. The, the, the point is, is when you take the driver out of the loop, right?

So long as I'm behind the wheel, I'll let it drive for me. 'cause I can intervene. Uh, I think a lot of us are probably gonna raise our hand about, you know, uh, AI making pull requests.

We do that with depend bot and a number of other things today. The problem is, is when you said, would you let it auto submit that pr no. Right?

So long as the humans in the loop things I think are gonna be fairly positive. We were called, the big fear most people have irrationally about AI is, is that it's gonna kill us all. And the reason it kills us all is because it does a logic test and decides that the best way to solve human suffering is to just eliminate the humans, right?

So really, I think multiple Star Trek episodes about that. It's been going on forever. So it fundamentally, what we have to be concerned about, I think, uh, from a security and, and, and uh, uh, infrastructure standpoint is how do we continue to keep humans in the loop so that the humans are in control and not the ai?

That brings us back to sort of core DevOps principles, which is if we have a GI ops type flow and everything is going through code mm-hmm. Prs provide us a really excellent opportunity to interact in an audible controllable way with ai, right? Whether it's in the editor with it making, uh, code suggestions that are then later reviewed by a human being right in a pull request review.

Um, or also as a mechanism for other ais to interact with that PR and do their own reviews to add to it so that we're ultimately keeping a human in the loop. So my concern is, you know, we have sort of two mechanisms today. One, do everything through a PR mechanism so that we have that, that history Mm-Hmm.

And that inter that way of interacting or have a policy to not do stupid things. With, with ai, there's someplace in the middle, right? We still have click ops going on, we still have lots of manual interaction with things going on.

How do we either create mechanisms to secure those other channels or really get everyone to double down on doing everything all the time in code? Okay, there's a question there. Yeah.

So, um, yeah, lots of parts that larger parts there, but I'm gonna go back to, to one of the things you said that I I, I'm gonna challenge it a little bit. I, is that, you know, how do we keep the human, make sure the human is always in the loop. And I, I don't think that we want to always do that.

I wanna be able to take a nap or read a book while I'm riding in a car eventually, someday, not, you know, before I die. I want that to be possible to have happen. And there are things today where I am, this is actually not ai, but, but there's, there are things today where I'm willing to auto accept pr.

So I have projects with close to a hundred percent test code coverage, and I will allow, uh, a bot to figure out if a dependency is off in a, uh, uh, the second digit of December, or the third digit of em that patches the third digit. Um, the, um, uh, I forget what the second digits called, but, but, um, uh, if it's in either of those two, it's not in the first digit. I will auto allow it to commit the poll request for some of my projects if it runs the test suite and it passes the entire test suite.

And so, I mean, you can imagine circumstances where you make smart decisions about when to the AI is to the point where you can trust it and you're willing to take that risk. And, and I, I don't think, I don't, I think that will, that that case where you're unwilling to trust it will get smaller and smaller at time to the point where it's very rare. Well, we won't trust it to be able to automatically do.

It might take years or decades, but I think we're on that path, and I don't think we should be fighting to always keep a human in the loop. I think the main idea here is choice. It's the choice of whoever ho owns the platform, as long as they do things lawfully, of course, um, and they don't, you know, interfere with other people's autonomy.

Let's put it this way. Um, and the more you trust it, the more you can give it, the more you can give the ai, uh, its agency. Um, and it really is a matter of how much you grow to trust it.

And based on, uh, past experience and as we mentioned, the stakes, um, and what can potentially happen. Uh, I, I don't think there's a way, I don't know if the, the talk is going towards, uh, talking about regulation or talking about, um, I don't know any other kind of things other than the technical aspects of it. Um, I'd like To switch to the attacker side of it, if we could go for it.

Yeah. Uh, oh, well, okay. I Mean, You I'll, I'll start.

You'll start. You start, I'll start. Um, because I played with some of these tools that are out there.

So, you know, let, let me ask you a show of hands, right? There are ethical AI hacking toolkits. How many of you, by show of hands, how many people have heard of those?

Okay, great. My, my next question to you, what makes them ethical? The intent, the User.

It's the, as as as they sit in Maverick, it's the person in the box or the person at the keyboard. Because the reality is some of these tool sets are amazing. They can do great things.

I, I get in front of CISOs frequently and I always asked, where do you see things going in the next two years? Or better yet, you always have these tools that, that are, you know, coming up behind. And these tools are making kids able to go do things that they should never be able to do.

And, you know, a script kitty can now go in and, and own a network by a couple of clicks or through automation. When you look at some of these toolkits and the brute force attacks for emails or accounts or, you know, being able to find ways through things, it, it's absolutely fascinating. But what makes it ethical?

It's the person at the keyboard and, you know, it, it's, it's just, you know, it's, it's just one of those things where it's amazing but scary at the same time. Yeah. So, so AI in the hands of attackers like you're describing is scary.

But let me give you an even scarier scenario that you may not have even thought of. So, uh, that one project that I mentioned of mine that gets a million downloads a month, it's considered critical infrastructure by the US government. They send me a check every year.

Essentially, the check is to assure that I notify them if I ever want to hand over control of the project to somebody else. Because that's a way to essentially, you know, inject, by the way, every cryptocurrency exchange and wallet uses that project that I'm talking about. So my retirement plan is to, is to add a vulnerability that will mine, um, their, their Bitcoin.

And, and I will, I will first have moved to a, nobody's laughing. That's What, It's a joke. It's A joke.

I believe this is being recorded, by the way. It's a joke. It's a joke.

I'm not with him. But, And, and that purchase of that, of that, of that home on that island that has no extra dish, that's a coincidence as to do you know. Anyway, um, but so imagine this DeepFakes are a much scarier aspect of gen AI than anything else we've talked about here already today.

When human beings lose the ability to discern truth, then you can trick anyone into anything, pretty much. And we are, we are at the point where we've, we've gone, you know, we're standing a, a th history yelling stop. Uh, but we have essentially learned how to identify the things we want to yell stop to.

Well, what if somebody is able to fake all the characteristics of something you would normally say don't stop to, then you're gonna approve it. So if I get a pull request from somebody and it looks like somebody I know and I've been working with for years and, and I trust them, but it isn't that person because gen AI has tricked me into thinking, uh, that they're different than who they are, then I'm likely to approve that. So you could also trick someone into launching a nuclear war with gen AI DeepFakes.

And so think about that. And, and I don't need to like be a total Debbie Downer here, but, uh, that's really a very much more significant problem than what we're talking about here. Yeah, Exactly.

Well, and, and there was a recent story that that came out where somebody had actually recorded the CEO talking on a YouTube, and then they turned around and called the accountant with that voice and said, I need to withdrawal a, a direct deposit, um, at this convention. Please send it. And it was convincing 'cause it was the voice, it was everything.

And the person did it. And the CEO found out and said, what, um, the, the thing, and I shared this on the last one and I'm sharing it here, have a code word, guys have a code word, because, you know, just like you were saying, deepfake a picture, that's all it takes is a picture, a little bit of audio. Now they can mimic you on Zoom.

They can mimic your voice, they can do everything. And if you don't have a password for your family, you know, and, and something happens to, oh, I got pulled over and I'm jail, please help me send, you know, gift cards from, you know, best Buy or whatever, you know, the reality is, if you have a special word for your family that you share, or even at work, I mean, you know, you have staff underneath you and what if you're, you know, if somebody calls with your voice and everything and says, this is going on, I need you to do X, Y, and Z for me without those special keywords or those, those things. I mean, they're gonna do 'em if they really think that it's you, A panic room keyword or whatever.

It's, yeah. Let me, let me change directions a little bit here. A couple of questions that we asked the audience, uh, to respond were, were, do, does your organization have a strategy or plan for how you're going to utilize AI in DevOps or in DevSecOps over the next 12 to 24 months?

And the results were about the same for both. Um, roughly 22% said, yes, I have a plan for how we're using ai. 14 to 22% said, I'm gonna rely on the vendors to do that for me.

Around 38, 6 to 40% said, not, not yet, but we're working on it. Another 18 to 24% said we, not at this time, we're not working on it. Should we, how much should we rely on our technology suppliers to guide us into the AI future versus you were suggesting get some hands on, learn it, kind of figure out your strategy.

Why don't you start Sure. How much money you guys have. I mean, it's not enough.

Yeah, I mean really the thing is, vendors are over promising what AI can do. They're coming out, they're trying to say, Hey, our tools can do this and we can do that. And, and the reality is, you really need to understand AI in general and, and the basics, and then grow from there.

Because every vendor's gonna come to you and say, we can, we can answer these questions, we can do it. And you guys have absolutely nailed, you know, the nail on the head with the hammer and, and not one, you know, LLM can do everything. And so you really have to look at the specialization and, and, and knowledge.

Now the question becomes, you know, if a vendor is, is doing this AI and it's looking good, what if it accidentally gets poisoned or something malicious happens to it? You know, what are you gonna do then? Because the biggest thing, and and I'll go down a rabbit hole real quick and I promise I'm back up, but the question is, is what the human, if something happens, I'm gonna ask you, how'd you get there?

How'd you come to that conclusion? How'd you get to where you are now? And the problem with AI as it is right now, some kind of, they're getting there, but they're not there.

They can't tell you exactly how it could got, how it got to that conclusion and the data and the, the facts that got it to where it is right there for that answer. I'm gonna take a different tack. Not disagree with what you're saying though, but, and, and this might be a little, a little bit, uh, sort of, uh, counterintuitive based on what we've been talking about so far and, and sort of the type of conference we're at.

So the state of the art and, and I've always been on the leading edge of the state of the art. I, I like to sort of understand the best way that the best people are doing all the difficult things. And, and I'm always sort of a student of that and try to learn that.

And, and that's what I, I heard you say is that you should be doing that. I would take a completely different approach with the vast majority of you though, I would say focus much, much, much more on closing the gap between the state of the art and the state of the practice. And, and that gap is maybe a decade behind in, in the software world in general and maybe two decades behind in the cybersecurity world.

And what I mean by this is, you know, what the right thing to do is, but you're just not able to get everyone in your organization to do that. And so what I'm talking about is essentially change management concepts, not not cybersecurity concepts. I'm talking about understanding developer or engineering psychology.

I'm talking about understanding how engineering organization sociology works and to leverage those things rather than fight them to overcome organizational pathologies, sicknesses that would prevent you from making rapid changes when you make changes. People's jobs are on the line there, there jobs disappear. Their, their responsibility, the number of direct reports they have, the domain they built their entire career up will be demolished and disrupted and they will fight that change.

And so this is coming whether you want it or not, and it's gonna come from the outside, from the attacker side. And if you don't change rapidly, if you don't figure out how to alter your organization to be able to change rapidly, you won't be able to keep up. So focus on those softer skills, I would say.

Yeah. So I, uh, think I want to add something to that. So, um, I, first of all, I stand by my, uh, recommendation from before to find, uh, effective use cases that you can drive business value from.

I think this is the most, uh, important catalyst for implementing any kind of technology. Um, and I do agree that we as practitioners need to, uh, utilize this technology more, um, in, in, in case we haven't noticed we are in an arms race with the attackers. And for this technology to be used effectively for defense, it has to be in the field, it has to be utilized, there needs to be a feed, a feedback loop, um, and it needs to be seen also by the industry that it makes sense to invest in it.

And so we utilize it so it's effective and its usage in the field is very important in, um, in that aspect. Um, and that's, uh, that's pretty much it. That's what, Yeah.

So to a quick one to summarize, sort of bring it back to your question. Um, trust your vendors to provide the tech, but you have to do the hard work of actually getting it to work, right? Which I think was what you were saying a little bit and I think meets your, your question, Shared responsibility means it's still your responsibility.

Exactly. Okay. So one of the, the last question we asked in our survey in the poll question was, I believe AI will cause us to fill in the blank how we secure software development life cycle in the next 12 to eight, 18 to 24 months options.

Were completely overhaul how we do things substantially rethink, cautiously evolve and gradually improve. Mm-Hmm. I'd like each of your opinion about where do you stand, what do you think we're gonna be doing?

We're gonna really have to seriously rethink or maybe even completely overhaul how we think about security in our terms of our software, or set a gradual improvement or cautiously evolve with it. Who'd like to go first? Last question.

Worth 500 points and for the game, I, I'll go, I, uh, I would say cautiously evolve because your large enterprises do not turn on a dime. Your large enterprises are really horribly slow to implement and make changes. And with what you were saying, you, you are absolutely spot on, you know, with, you know, the cycle and, and the communication because as you build the trust and you build the, the backbone of AI for, you know, the industry, it's gonna start getting accepted.

It has to be in the field. The small companies are gonna take it. They're gonna run with it because they're gonna see the value immediately where the large companies are gonna be like the Titanic and they're gonna be slow to change, but when they do change it, it's gonna be a very positive change.

Okay. Who's next? Yeah, I, I, I agree with that, that that's the answer to the question of what is likely to happen with the vast majority of large organizations.

But I would say that if you aren't, so that's in the, the second half of his options cautiously evolved was the, the second to last one gradually improved was the, the last one, the first two were completely overhaul and substantially rethink. And if you're not altering your organization to be more agile so that you can be closer to completely overhaul in 1218 to 24 months, it doesn't say in, in six to to 12 months, it says 18 to 24 months. So even large organizations can completely overhaul their processes in 18 to 24 months, at least for parts of their organization at a time.

And so I think you've gotta be pushing to make that be the bucket that you are in. If you're in that ladder bucket, I think you're gonna lose. So I, I would say cautiously evolve, but I wanna put one asterisk on it, um, because something can change it, and that is some kind of killer application.

If there's gonna be some kind of killer application that can be very confidently applied very easily, like it can be very vi viral, it can really, uh, change things from end-to-end. Um, of course if I knew what it was, I would, uh, yeah, when you figure that out, I would, I would share it. Of course, we, We'll be back here next year to talk About that Prediction.

I'm gonna weigh in, in my opinion. And I think AI is not, not necessarily just an evolutionary, I think it's a revolutionary technology. It's evolutionary right now about how we're trying to understand it and figure it out.

I think we're gonna have a Netflix moment or a DevOps moment, um, where we say somebody has figured out how to the velocity increase the velocity substantially. We're all gonna say that we have to compete with that. That's the new model to compete.

So I think we're, we're racing in, in a pretty fast pace to see what the next bar is gonna look like and there may be multiple of those after that. So I think ultimately we're gonna be at a minimum, substantially rethinking, if not completely overhauling. Think about, you know, this is 18 to 24 months.

Certainly past that, I think we're overhauling. So, okay. Great.

Thank you guys for this. Um, I'm gonna, we're gonna have Mark come up here and close this out. I'd like to thank all of you for coming to, from, uh, the tech strong, uh, side of things, uh, working with Mark.

It has been a real pleasure in putting this together and certainly wanna thank our sponsors. If you please gives, please give a big round of applause to men, contrast, security, and Tenable for putting this on with text.