Next Gen DevOps – Transforming Software Delivery Through Applied AI with Tracy Ragan at Techstrong Con 2024

Hello everybody, and hey, thanks for attending. Uh, Textron Con. You know, there's so many, uh, events out there.

We really appreciate that you attend these, and I think Textron has got some really interesting stuff going on, and I'm super happy to be here presenting today. And what are we gonna talk about today? We are gonna talk about what I like to call Next Gen DevSecOps, and how we can start thinking about using applied to ai, because there's all kinds of AI that we could apply to this solving this puzzle to really transform how we deliver software and include security as part of that transformation.

So imagine if we could just imagine this amazing DevSecOps pipeline, this next gen DevOps pipeline, that it could achieve this unprecedented level of decision making, security remediation, and threat analysis across all stages of the pipeline. I, I think about this. This is stuff that I think about when we start talking about ai, and I believe this dream is much closer than any of us, uh, realize.

We just have to our minds to it. So, I am Tracy Reagan. I am the CEO of a company called Deploy Hub.

I have served, uh, in many open source communities, including the open SSF. I've served on the board of the open SSF, the Continuous Delivery Foundation. I'm on the TO, the Technology Oversight Committee for the Continuous Delivery Foundation, and I got to help start the Eclipse Foundation.

So I'm not new to open source or technology for that matter. Um, I am, uh, the co-founder, again, and CEO of Deploy Hub. I founded a company years ago called, uh, open Make software that used a rules-based system to build a, uh, build automation solution.

And I am one of the co-hosts of Techstrong Women tv and join me there because we have some amazing conversations with amazing women doing technology. So I hear about AI in almost everything. Uh, in fact, in recording this, we just had a, a new button from Zoom about, um, having some kind of AI thing turned on.

So everybody's talking about artificial intelligence. It is the buzz, but how far along have we really gotten in this world, in the dev, in the DevOps world? We haven't gotten too far.

Uh, but we have started. And most of the work, if we think about development, security, and operations, most of the work right now is around the dev phase. This is where what we're, we're really seeing, this is where chat GPT has become important.

I don't know if I even go a single day anymore without touching chat. GPTI have to admit, I use it for everything. I love it.

So, code Generation is becoming a, a popular, uh, use of ai as we all know. Um, I've seen some of the, uh, the tools, DevOps tools, CICD pipeline starting to use, uh, AI to identify code reviewers, uh, in Git. So if you're trying to get your code review, you could use AI to say, who's the, who's the best person to review my code?

Uh, and of course, identifying, uh, potential vulnerabilities in code. We've been scanning code for a long time. So starting to use AI from, from that perspective is becoming more of a conversation, um, and code explanation in natural language.

Being able to say, this is what I want to generate code for you is pretty, pretty, a pretty cool feature. And then test generation, generating test, and then threat modeling. And some of the tools, some of the, you know, tools that are currently doing this is chat, GPT, of course, um, Microsoft's, Bard and Bert.

And then there's something called pasta process for attacks, uh, simulation and threat analysis. So we're beginning to see these tools applied to a broader section of the, uh, the de the development, security and operations pipeline. But for the most part, we're pretty basic.

It's really basic, and it's pretty much focused around solving the, the dev state, basically code generation. And we have to go quite a bit farther than that if we really wanna start improving our DevSecOps process. I, So we are hearing more discussion in the DevSecOps process.

We're lo we're hearing a lot about open source security governance and SBOs. Now, this is an area I believe that AI can be, um, can be applied to. And we have two kind of converging problems.

We have a need now to better track the software supply chain around open source security and third party objects that we are starting to consume. And we also have digital transformation or moving out of, uh, a monolithic structure of architecture to a decoupled architecture. And when we talk about a decoupled architecture, what it means is that for every single container we're building, we're having a pipeline.

So navigating this process, because we have so many more workflows, executing, creating smaller components, is becoming con increasingly complex. And then at the on top of that, IT teams are being asked to, Hey, you better start managing your software and what you consume better and start exposing these hidden vulnerabilities in these open source and third party libraries, which every container that we create, every build image, create, uh, that we create consumes these packages. So now we have 'em duplicated in many, many locations.

So the demand for smarter DevOps is, and streamline DevOps is starting to soar. People are starting to realize we do need to have better steps. We do need to do a better job of our DevOps and start and start doing more security around DevOps.

So that's why we call it DevSecOps. But we also need to think about how to automate that better, uh, and start using the data to do better things. And that's sort of, that's where applied AI can come in.

So applied ai and what I'm talking about is taking any kind of AI to solve the problem. It's not just, uh, you know, generative ai. It's not just, um, machine learning and ML workflows, but it's looking at a broader, uh, picture and looking at the different use cases that we need to solve.

And how AI can be used to do that, to create this next Gen DevSecOps pipeline. It really does. You know, and I wrote this, it heralds the dawn of a new generation for DevOps.

It creates a whole new way of, of, of thinking about how to solve the lifecycle puzzle. And we've been working for quite some time to automate, basically builds and deployments, builds and deployments. Once in a while, we start talking about testing, adding testing into the pipeline.

And for many companies, they have done that. And now we have to start adding security into the pipeline and security at all these different levels. So the integration of of AI into some of these steps will help us refine the process, empower decision making, and enhance overall efficiency through the pipeline.

And May one, one of those examples that I use, something similar, who's the best person to approve a pull request that is enhancing overall efficiency? Because you don't have to go to three or four people and they're saying, well, no, I really don't understand that code. It says, this person's the best person to do it, and it, and it gives you that information.

That's what we're talking about when we talk about refining processes and empowering decision making and enhancing overall efficiency. How do we, how do we do that? And what do we need to do?

How, what, what pieces of the AI puzzle do we bring in to the whole DevSecOps pipeline to make it a better world? So what really I'm talking about is extending AI beyond code generation. How do we bring it into not just dev, but into development, into security processing, and to operations?

Because our DevOps pipelines can do a lot more than we're asking them to do. And, uh, I think many of our problems, and if you've heard me talk before, it's one of my pet peeves. Many of the problems with our devs sec, our DevOps pipelines, our CICD pipelines, as we focused on build and deployments, we focused on writing build and deployments scripts, and now we have thousands, literally millions of Jenkins workflows that are specifically designed to do builds and deployments.

But we need to be able to expand those to do more than just, uh, build and deployments because build and deployments need more now because they need more security reviews. So how can AI extend to the full DevSecOps process, um, and focus on securing software supply chains, which we hear about every day. It's the new buzzword, I think.

Um, remediation of vulnerabilities. If you find a vulnerabilities, do I even, do I need to deal with it? Does it actually touch my code?

Does it touch, is it ever executed in, in a production runtime environment? How can I define zero trust policies? You know, do, is there a way that somebody can automatically tell me I shouldn't consume this particular package?

And how do we start watching the, and controlling the flow of, of open source into our, uh, into our, our digital assets, our corporate assets? It would be really nice to see that flow and to understand what packages are coming in, what packages are popular, which ones are, are high risk, are we consuming them? Are we not?

What is happening? Because you can't see it now. So let's take a look at a few use cases that I think about would be interesting to add to that pipeline.

All right, number one, this use case high risk repositories. So what, when we think about what's happening in the open source kind of security realm, one of the first areas that is being closely looked at are high risk repre repost is your, is your repository secure? Uh, are the, is the owner of that repository, have they implemented security steps around that repo?

So we already know that there's some work being done around that. There's data out there about a high, what a high risk repository could be. So AI could determine if a repository is high risk.

Well, if it is high risk, maybe we wanna prevent the use of an associated package coming from that, um, that high risk, uh, repo don't consume it if the repo's high risk. Another num, another item that we could add to those is how many maintainers is in that open source community? If it's one, maybe there's, that's a sign that we probably shouldn't, shouldn't consume it.

While log four J in particular was maintained by one person, it just meant that, and everybody's using Log four J, it meant it was high risk. Uh, even though that one person did the best they could to, to support thousands and thousands of applications that consumed it. Something like a lack of signing and bra and branch protection, if we see that there is, is no signing and branch protection, maybe AI could tell us that's the case, and we not allow the consumption of that particular open source, uh, package.

So potentially AI algorithms could be defined to locate a replacement package from a repository that it's low risk. So you have a PA package that you wanna use, you find out it's a high risk package, and AI says, Hey, guess what? Here's four other packages that does the same thing, except they're from repositories that are, that are lower risk.

That is an easy, really easy application right now of an AI that could be added to the process. Number two, improve high risk repositories. So while we know that a, a repo might be high risk, there's no reason why we couldn't help them make their repo, um, lower risk.

So AI could determine why a repository is high risk and produce the needed security tools to fix the issues. They, we could automatically add signing, for example, and bra branch protection if we knew the, the repo is high risk. Or add more things like depend abott or renovate, uh, for, you know, for dependent for the dependency tool part or auto configure, uh, a dependency tool that that's not configured appropriately or add fuzzing or linting, um, to be configured as part of the repository.

If we're code generating, we can also generate these steps. So something like chat, GPT could be utilized to perform the auto generation of these pieces. So it's not that far away, it's just a way of how we think about what needs to be done as part of the process.

Now in this area, if you think about, you know, my, I started my discussion talking about DevSecOps. We're still kind of in the dev world, but it's the, it's, it puts us as close as possible to the problem of consuming components into the build that might be bad. How do we stop that?

How do we fix it? Number three, up to date package management. Okay, this is a pet peeve that we talk about often at Play Hub.

Um, when you have old versions of open source and third party packages, they, uh, that's a risk. Anytime you run a build, the most recent packages should be brought in. But because builds are often static, that does not happen.

So AI could be used to auto update those old versions and kind of ignore what static, maybe statically listed someplace in a build script and say, just go and use the latest release of the package. Super, super simple step that solves many, many problems, because for the most part, vulnerabilities are more, more common in old versions than they are in new, even though the old version's working quite fine for you, you might still wanna bring up the new version. I mean, you know, you use, the other day I was trying to call somebody and my phone decided it needed to update right then and there.

Why? I don't know. But it did, it forced me to update that, it, that my phone's operating system.

So why can't we do that with open source packages? Well, I believe we can, and I think we need to. Okay, the holy grail, the absolute holy grail auto remediation.

All right. So if we're in the DevOps pipeline, we know where the repo is, we know how to build the, the, the, the image and we know how to deploy it. Why can't we go and say, I'm gonna look for vulnerabilities If there's a high risk vulnerability, I'm gonna go ahead and look at the, uh, the, um, the tactics for remediating that, which could be bringing down a new version of the package, go ahead and update the build and redeploy it, or at least create a poll request that says, this has been updated.

We don't need to spend 227 days remediating vulnerabilities, which is according to jfr, the average time it takes to get every single container or built image out there updated with a new version of a package. When a vulnerability is found, that takes too long. We can automate this through the DevSecOps pipeline.

We have the information, we have the ability to do it. The, the challenge is to go find out how to, to figure out where that new that new package is and bring it in. Do we do it through a pull request or we just do it automatically?

I think we do it automatically. And then some other possibilities to talk about. Imagine if we had historical data that we could start using to evaluate threat models.

So, you know, we sit in a room, we talk about potential threat models for our application, but are they real? Are those threat models? Real is pot.

There's a potential of, with historical trend analysis, that we could start looking at the flow of open source and start evaluating our threat models and actually even generating new ones. Um, that's what, uh, pasta is all about. It's, it's about generating threat models and determining the accuracy and, uh, uh, of those threat models.

So, work's already being done in this area, but it could be brought into to the DevSecOps pipeline as well. Why not? We're doing build, we're doing deployments.

Why can't we also add threat modeling and evaluating threat models on a regular basis if we had that historical data? How about analyzing companies, company-wide usage of open source? What's the most commonly consumed open source project across my organization?

If every single component, every single container bill that includes log for J, maybe that's a, that makes it high risk because we have a lot of exposure to it. These are the insights we don't have generally, but we could have it and we could do something with it, um, if we knew it. So for example, maybe if we have an overused, uh, package, we start, AI could give us some recommendations on how to spread the, the exposure instead of everybody using this one logging, uh, utility, everybody doing Java using this one log logging utility.

There's other utilities that could be used to spread the risk. Sort of like, you know, when there's an earthquake in California insurance that is, comes from all over the United States, so not just the, the California in insurance companies are covering the cost of an earthquake. That's kinda what I'm talking about.

AI could be used to do that. And then an area that we make the mistake on so many time is to just include every bit of every package that we think we might need and put it in our, our image build. AI should be used to report and remove unused packages from the build unused packages.

If they're, if they're not needed, don't put 'em in there because they just, they just add to the risk you're putting in packages that you're not using that could have vulnerabilities. Now you gotta address the vulnerabilities even though they're not used, because you may not understand if they're used or not. So artificial intelligence has many, has many benefits.

I think that the historical data, the use of open source across the organization and really understanding the, uh, if, if packages are being used or not, these three are, are areas that we could really build upon to make our DevOps pipelines to fortify the DevOps pipeline, create these next gen dev SEC pipelines that are doing more than just build and deploy. But they're starting to, they're starting to be smart and have some decision making built into 'em at every step in the process. So let's just talk about the challenges in training it.

The ai. So one of the, one of the challenges that are pretty common is the, the problem with, um, the data. We don't have large comprehensive data sets of DevOps pipeline transactions with the security in IT to do the machine learning.

This is an area that we need to, to, to work on, and it it, it's the reason why we don't, we lack historical trends because we don't have enough information, uh, from historical data to do the threat modeling and the assessment of those models, because we're not watching it happen. We're not looking at behavior. We have no way of seeing that without data.

Data is the, the backbone of ai. It's where everything comes from. Um, and in, in essence, you would need, I would say, 50 to 60,000 transactions to start looking at, start looking at models and behavior.

So we need to start tracking that data. And the other thing is the DevOps pipelines themselves. If we wanted to, let's say, start doing a generation of better pipelines, we have to have better pipelines to generate them from.

And because our pipelines are scripted, they're not very templated. We don't have a good accurate data to start generating better pipelines. So not a good source of starting data.

That is the problem with creating these next gen DevSecOps pipelines. Alright, another problem is all the data is fragmented. Every workflow that we execute, every single one we execute is based on a one deployable object.

Now, in the monolith world, we'd wanna build, and the whole, the whole application would be compiled. But in our digital transformation, in our need to be able to spin up certain, uh, certain services to support more people, um, to be more flexible and to be more agile for our businesses, to be more agile, we started moving into a decoupled architecture. Well, that means there is an SOM and A CVE and an impact report and a version history for every single microservice, every single, uh, decoupled artifact that we move out to our production environments, it's fragmented.

So we have a bigger problem. We, we only see the results, the security results, the, uh, the success or failure rate of a deployment for one component at a time. We don't see it for an application, and we don't have a good understanding of a domains security posture because the data is completely fragmented.

So we've been thinking about this pretty heavily at the Orillia project. Um, Orillia is a project that we support, deployed hub, uh, contributed about 80% of the code to the continuous delivery foundation to solve some of these problems. So what ORUS does is it looks at the fragmented data at every single workflow that's being executed across these decoupled environments.

It gathers that, and it, and it tracks where the deployments are. It gathers that and it consolidates it, creating reports and visibility based on, um, that's usable. Let's just call it that.

That's usable. A logical application, an environment, an entire domain, something that's being managed at a director level and everybody underneath it, or for the entire company, and starting to create these, these views and, and collect data to make the data me the data meaningful. So that was our first goal, is just to make the data meaningful.

Now, with ai, we can use that data to start achieving some of the use cases that we talked about. And it's an incubating at the CD Foundation, and certainly we'd love to have you be a contributor. The open SSF is working pretty hard to, to build solutions.

And many of this is way on the left hand side. It's really about, uh, the repose right now is worried about some, to some extent builds, um, vulnerabilities, but the information that they're starting to gather is critical, and it should be part of the DevOps pipeline, and it should be gathered and it should be acted upon. So tools like, uh, security scorecard, there's a whole list of basically checks that you need to do against your, your git uh, GitHub, um, actions in your GI repo to sort of audit the repository.

That's where I was talking about in one of those use cases, we could look at the open SF s scorecard if, if the, if the repo had the scores and determine if it, what, what it's risk score is. If it's not doing that, we gotta add it. So we, we can have that information.

And we do it through a poll request, especially for the open source communities. SALSA is trying to give us an idea of how to better improve our build process and our build standards. Well, some of that data we could look at and determine if a container has been built, how successful it's kind of our new door metrics, what our SALSA metrics are for that build.

Is that build secure? Is there something we should be doing to make it more secure? Uh, you know, SPDX and, um, it, it, that's, it's a SBO m standard SBOs have to be added to the, to the pipeline.

They just have to, I know there's millions of pipelines if we look at it from an open source perspective that we have to update, but they have to be updated. And things like OSV Dev being able to bring in this information about vulnerabilities and, and associate it to the SOM and be able to bring it in on a regular basis. These are critical steps.

Companies like Sift and, uh, or, or ncor or, or are also I improving SBO M generation. And they have things like SBO M management and SBO m sharing. So where does, uh, Orillia get the security data then?

It's from here. It's these tools, these open source tools. It's an open source project, and Orillia is starting to, to bring in more and more of this, of this data.

And when we do, we can, we have a, we have a really good starting point, right? We have a really good way of starting to identify the application security posture across individual components and aggregate that back up to the logical application and release. Every time one of these components get updated, you have a new version of your application, which means you could have a different, uh, a different profile for your application.

So what arterius is doing is it's creating on a regular basis these new views, these new postures at the component level, at the application level, and at the, uh, at the higher domain levels built into the DevOps pipeline to try to help build that next gen DevSecOps, uh, pipeline. It's also has the data, it hasn't applied any AI to the data yet, but it has the potential to do that. And it's gory.

The data, I admit, is gory. Um, but it's interesting if you like to geek out on things, everything from, you know, the owner, uh, uh, the, the owner of the component, um, if the SBO m uh, is valid, if the signing is done, the SonarCube information, the, or the Veracode, whichever one you might be using or a different tool, doesn't matter actually, um, is a repost or the repos have bra branch protection. Um, are the, uh, do we have signed releases?

So this is the level of data that the ORUS community is starting to gather, and many of it's already been, um, achieved. So the more we gather, the more data we have, the better view we have into the health of a, uh, of a, of a particular component or application or domain. And we can do that all through the DevOps pipeline, starting at the lowest level, the component level, and that level, that information is brought into ours and then aggregated up to the higher levels.

And this ultimately is what will give us the, the, the, the data to apply AI into in these different scenarios. So some things to look at to, you know, if you say Tracy's dreaming, she's crazy, well, here's some tools that may just get us to where we need to be. SWE agent, it's a really interesting tool coming out of, uh, Princeton.

And it's all about applying, uh, bug fixes. And from, uh, and issues in GitHub repositories. This is a way we could automatically update a vulnerability, update, a package that has a high risk vulnerability hugging face.

The more we start working with these, um, ML models for DevOps, the more we can share them, the faster we'll get there. Of course, there's, there's different kinds of cogen, but the, this one from Salesforce is, is kind of interesting. I would encourage you to take a look at it.

You know, it's a large language model engine, um, for doing cogen, but it's really specific around cogen and it could be applied to the DevOps, uh, process and any na any of the natural language toolkits are important to, to, to learn about. Um, most of them are done in Python. NLTK is one that I would say would be an interesting one to look at if you're interested in this area.

And then pasta, the process for attack simulation and threat analysis. Um, take a look at, uh, at pasta as well. It is possible.

It is possible. I am not crazy. It is totally possible.

So thank you and find TIUs. Uh, again, if this is an area you're interested in, we'd love to have you join the project and start working on machine learning around some of these data points. Uh, if you wanna chat with me about it, you can find me on LinkedIn at tracy dash reagan dash oms, and you can connect with us at GitHub, um, uh slash orus.

And there is a link to the Discord channel. If you are fast enough to type that in, into your, into discord at the moment, you would be able to find us. Thank you.

Thank you so much. Love all of you out there. Love talking about DevOps, and especially DevOps and ai.

Hope to see you soon. Bye-Bye bye.