DevSecOps in the AI Era – Greg Ellis, Digital.ai

This is Techron tv. Hey guys. Thanks for the throw.

We are here with Greg Ellis is the general manager for cybersecurity for digital ai, and we're talking about DevSecOps and the challenges that are inherent as people attempt to implement the best practices in this space. Greg, welcome to the show. Thank you.

I'm glad to be here. We've been at this for a while with, uh, arguably some mixed success, but what are you seeing from folks out there? What are the common mistakes people make and what could we all collectively be doing better?

I'd say one of the, the first things that I've seen a, a trend recently is as we move more to the low-code, no-code, we're getting into situations where, uh, many of the developers haven't been trained in, in the security side. And so it's putting more of a burden on us as security providers to really help them adopt that as part of their overall workflow. So a lot of the things that we've been focusing on this year is not just, uh, how we are improving our security product and the protections with that, but how we integrate that into the rest of our DevOps, uh, pipeline, uh, with the rest of our platform.

And so really trying to make it easy for that to be adopted and what we would effectively call shifting left in that entire process, uh, to be able to, you know, effectively help citizen developers, uh, more easily adopt security. Aren't we asking too much of developers in general? There are some folks that say the cognitive load is simply too high and we gotta make the whole thing simpler than it is.

ai, we've had ai, you know, kind of in our, in our name prior to the whole chat, G P T, and a lot of the AI explosion. Um, so we've obviously been focused on it for a while. From our standpoint, it's, it's almost this kind of story arc from, you know, a manual to, uh, informed to automated to autonomous is, is kind of what we're trying to do with a lot of our products, both independently and, and as a whole.

And what I'm seeing from a lot of our other, uh, folks in the industry is they're doing very much the same thing. And, and we're all at different phases in that, in that journey of how we can do that for ourselves. And again, how do we make that easy, you know, to reduce, to your point, the cognitive load on the developers.

How do we just help them move from what has traditionally been a manual process, ultimately getting to this autonomous process with, uh, with AI or, or augmented intelligence. Mm-hmm. Do you think that we're also highly focused these days on developer productivity, and we already have issues where they're spending maybe only a quarter of their time writing code, and that may be because, well, the environment is too complex or they just might be playing games or whatever it is, but for whatever the purpose is, they are not writing as much code as they could be as quickly as they might.

So how do we kind of balance productivity and security? And so, so I think there's a couple things going on there. On the productivity side.

You know, one of the things that obviously is happening is with the generative AI is, and again, citizen developers, they're not doing as much actual, you know, writing of new code. A lot of times they're, they're using either, you know, co-piloting or, or generative ai, or, you know, much like, uh, many of us have done over the years as we're going out to stack overflow and finding, uh, similar challenges and, and, and adopting those for, for where we're at so that we can try to increase that productivity. The, the challenge that we see with that is, uh, you know, from a testing standpoint, you know, if you're introducing all this, this code that became from this generated, uh, code that the developers are doing to, to try to improve their productivity, it's, it's really no different than what you might see from a junior developer.

Um, it could be buggy, you know, it's, it maybe needs a little bit more introspection. And so what what we are trying to do is put that focus both from a security standpoint and a testing standpoint to highlight that using some of our an, you know, an analytical lenses and predictive analysis, uh, lenses to allow developers and testers to better understand where they might need to introspect. And the example I always use is if I put a, a new summer intern on, on one of my teams, you know, by, its, by the very nature of it being kind of a junior developer, I do need to check up on them a little bit more.

Not that I don't trust 'em, but as move stuff to production and as they mature and, and get more experience, they're going to become better developers. And so again, this, this generative AI that's supposed to be helping the productivity, we have to kind of treat it the same. And so I think what it's going to do is it's actually going to allow developers where they're not having to do as much code development and freeing them up to do more creative, more innovative thinking about the next architecture, the next design, as opposed to just doing kind of run of the mill, um, development.

So will that increase their productivity? We hope so. Uh, I think if nothing else, it will, it will ha you know, give a better sense of feeling.

For, for developers, we, we all get tired of doing, you know, bug fixes and maintenance. We always wanna work on those new things. And so I think that, you know, some of this will help us go do that.

And again, as we, as we augment, you know, that whole workflow with security, we're really trying to push it where it's not a separate step to go apply security, but it just becomes inherent in the process. And so we're doing a lot of things to try to link again that, that overall DevSecOps pipeline from phase to phase. Another example where that's possible is on our release orchestration.

Um, we're actually using effectively like a manifest file that comes along with, uh, the development. Maybe you're including, you know, software bill materials or something of that nature as well as my protection been applied. Am I putting, you know, an unprotected application out there that hasn't, uh, hasn't been hardened yet, and potentially letting it get out into the wild, you know, where I don't want it.

And so there's just some, some simple things that we can do and pass that along to the release orchestration phase that they can tie that to open policy agent as an example, and do some additional regulatory or, or governance type things. And also provide those artifacts on the back end for any of the auditors that need to go do that. So it's just, again, trying to make this kind of seamless flow as we progress through the stages of the cycle.

Are you worried things might get worse before they get better? 'cause some folks are saying these copilot tools are interesting, but they're using code bases that are general purpose that they found anywhere and everywhere. And then a lot of that code bases has vulnerabilities in it.

So now we're just seeing more code written with more vulnerabilities, at least in the short term. Is that an issue we should be concerned about? Yes, I think it is.

And, and not only that, but the other piece of that is the data privacy, you know, portion and, and frankly, some of the other legal concerns around was it copyrighted, you know, code that that got pulled in. So there's a, I think a variety of things that are happening that we really need to be paying attention to, not just from a vulnerability standpoint, but the rest of what kind of business risk does that introduce? We've actually seen a number of enterprises that obviously have, have asked their developers or shut off access to some of these, you know, generative ai, um, tools, just because we're all kind of taking a step backwards from a, from a legal standpoint and from a security standpoint to really understand how it needs to be incorporated.

And so we're not really unlocking that, that value yet. So is it gonna be worse before it gets better? I think that answer is yes until we can kind of get some of these other issues like data privacy and the vulnerabilities and understanding how we modify the tools that, that are already there.

Things like the SASS and das tools, pen testing tools that, that exist today and ensuring that they have the ability to kind of, you know, do the deep dive into this generated code, just like any other code that's, that is, um, that's, that's, um, developed, but potentially maybe doing that prior to it getting incorporated into the baseline. Do you think we'll see more domain specific large language models for writing code that have been trained on basis of code that are vetted? So we don't run into this issue as often?

Yes. I, I think, uh, not only that, but I think, um, specific industries will have, um, potentially, uh, large language models that they use as their own enterprise, but also potentially across, you know, a family like let's say pharmaceuticals. Obviously there's a lot of competition in the pharmaceutical industry as an example.

Um, but there should also be some co-option as well as, as they have the ability to potentially go to, you know, clinical trials faster. Um, or if we use financial services as, as another example, how do we kind of collate fraud data that's happening, you know, across that industry and being able to share that across the different financial services and, and using, you know, the, the, that data as well as any of the input models that come with that. 'cause again, we want this, this full feedback loop, um, back into the, this DevSecOps process.

So I think there will be these large language models that are both, you know, enterprise specific and industry specific that come into play. Uh, and again, I think as we're seeing, you know, a lot of the, uh, whether you call it, you know, hallucinations or whatever else, like how do you just not have biases getting introduced into that? And how do you have good clean data sets, you know, that are being used to develop those so you're not just propagating, you know, uh, maybe not vulnerabilities, but either biases or errors.

That's always my favorite thing when a machine is wrong and hallucinates and when you and I are wrong, we're just flat out lying. But who knows? Um, as you kind of think this through, though, clearly with the introduction of ai and right now you could say AI is benefiting the developers more than say the folks running the pipelines, but it will come to the pipeline.

So as you look forward, how do you think DevSecOps pipelines are gonna evolve in the age of ai? What should the people start thinking about will be automated and where should they refocus their efforts? Yeah, I think, again, you know, the things that we're looking at are, we may do all this generated code, um, from the generative AI pieces at the development stage, but if we haven't done the rest of the downstream things and the testing and the release and the deployment phase to be able to do that, we're effectively just gonna have a bottleneck at the next stage.

And so, um, really what we're looking at is, again, how do we streamline stage to stage as that, as that progresses through and being able to take advantage of not only the AI pieces and let's say continuous testing, but also, uh, taking kind of x of these, these, uh, lenses across the top in our case that are both backwards, looking on the analytical side and forward looking on the predictive analysis side so that we can kind of say, for example, um, maybe we're gonna have on a protected application from a security standpoint, we've added additional security that's going to have a performance impact. We don't want to be all the way downstream in our testing phase and realize that after we've gone through all the other tests, we want to kind of catch that upfront. And then, you know, if there's something going on with that performance or it looks like there's something going on with that performance, go ahead and just kick it back a stage before letting it progress through.

Uh, another example would be, you know, on, on the development stage, as you mentioned with the vulnerabilities, just the normal, um, sast and dast and, and some of the other, uh, pen testing that happens. Again, catching that stuff earlier in the process and ultimately, again, training, you know, these AI models to be looking for those patterns and to only effectively management by exception, if you will, um, to where you need to get a human involved to go look at a specific issue, right? But otherwise, just let it kind of progress through and, and cycle through in the background automatically or autonomously, you know, as, as we move forward.

Mm-hmm. What do you think is gonna be your best advice to folks about how to go forward from here? I think everybody kind of gets the idea that there's some impending change, but how do you prepare for that?

Or, or do we just, is it full steam ahead and we'll figure it out when we get there? Or is there a kind of a rational way to start thinking about this? I, I think, you know, much like we've seen other, you know, architectural changes or tool set changes in the last, you know, 20, 30 years for as long as, or as long as, uh, you know, programming is, programming has existed, there's always these kind of evolutionary changes.

And I use the analogy right now with where we're at on the a i side, much like the bubble explosion was, you know, 10, 15 years ago and starting to see how we needed to change how we were doing development to support those things. So to answer your specific question, I think it's important to go do some pilot programs, some sandbox stuff, do it in a controlled fashion, understand how it's going to affect, you know, an individual team, but more importantly on these enterprise levels, how is it going to scale and being able to do that without just, you know, going all in on, on a particular technology. Um, but, but taking a very methodical approach to that so that we understand, you know, where those pitfalls are at, understanding how it's going to, you know, affect kind of the downstream piece, but also then looking for, you know, tool vendors such as ourselves that can help provide that, uh, infrastructure or, or, uh, as our c e o likes to kind of use the analysis, like the train tracks right there, maybe all the trains coming and being built, but if there's no tracks for them to run on, you know, it's, it's gonna be a problem.

And so that's some of the things that we're looking for is to really enable, you know, that that flow across the entire process. So if you can take these pilot programs and then start to scale 'em out and not just work with the development team, not just work with the testing team, but as I mentioned earlier, some of these data privacy concerns, some of the copyright concerns, you know, you need to get other functions involved, you know, across the business, whether that's legal, uh, the IT group, operational group on the, on the cloud services side. So that I think has to be this very holistic approach as you're looking at adopting any of these technologies.

Business leaders have always complained that it is too slow for them. Um, are we maybe approaching a point where soon we'll be delivering applications and updates faster than the business can absorb 'em? Well, so yes and no.

Uh, I think that that's, that's the goal, right? Is that we are delivering them faster than we can absorb and we get to be choosy about which ones we adopt. Maybe we start to skip some, some patches because they're coming out, you know, every day as opposed to, you know, patch Tuesday or, or whatever, right?

But, uh, I think that the, the goal again is in the past we've had, it has kind of been off on their own. They haven't been engaged with the development teams at the r and d teams. I've seen a shift to where it is coming to the table, you know, IT, and operational technology as well, coming to the table with these r and d teams and operating more as, as an integrated, you know, team moving forward.

So I think just that, um, sharing of the information with each other on how it can be deployed and how it needs to operate with how it's being developed, I think that will help, uh, on some of that in terms of the speed to getting to deployment of speed of adopting it in a business as opposed to it just being kind of thrown across, you know, over the wall and a team just expected to go use it. Um, so again, a lot of this is, is not just tools, but it's processes and its culture on how we actually go do change management for, for adopting some of these new technologies. Of course, everybody who comes into contact with the topic of AI has the same question.

Ultimately, are we all about to be replaced? So what's your sense of what will be the relationship between DevOps engineers and AI as we ultimately start this journey? Yeah, again, you know, I'll go back to my earlier comment around freeing us up to be more creative and more innovative.

I would much rather spend my time not on the trivial things or the mundane or the, you know, mind-numbing, whatever you wanna call it, things that have to be done in my daily life. I want to go do things that are fun, creative, exciting, innovative that, you know, I get, you know, a lot of, uh, passion from and not just, you know, banging away the keyboard or trying to take care of something. You know, the kind of the real world example where we're at today is the self-driving cars.

I, I'll, I'll be the first one that when they're, you know, everything's self-driving, I'm gonna be a happy, a happy man and I hope it happens in my lifetime. 'cause I would much rather be sitting there and catching up on the news or, you know, talking to a friend of mine as opposed to having to worry about getting from point A to point B as an example. So again, are we going to be replaced in certain areas?

Sure. That I think that happens. If you go back and look at any of the industrial revolution, there's always been examples where, you know, tools or machines have replaced humans.

But I think we've adapted to that, you know, as time has gone on and found other creative outlets for how we want to try to, uh, employ our, our time. All right, well folks, you heard it here in some ways. If you just start thinking about all the tedious things you hate about your job, well, that's the things to throw AI at.

And the rest of it is keep to yourself. Hey, Greg, thanks for being on the show. Appreciate it.

Have a nice day. All right. And back to you guys in the studio.