John Willis – Automated Goverance

It's hard to introduce someone who needs no introduction, but I will give it a shot. I am extremely honored to introduce John Willis from our sponsor RedHat today. John is a DevOps legend.

He's built 12 different start ups. He's written almost as many books, including Beyond the Phenix Project. And today he's going to talk to us about how to automate governance.

Take it away, John. Hey, hello, everybody, this is John Willis doing a presentation today called Automated Governance an Overview. I actually work for a Red Hat, as you can see, part of the global transformation office.

And so a little bit about myself. For people who don't know me, I go by the Twitter handle @botchagalupe. I've been in this industry for quite a long time, 40 plus years.

I've written about 10 or 12 books over the years. But these are the most recent ones. Coauthor of DevOps Handbook, coauthor of a book that I collaborated with Gene Kim on the Phenix Project.

I've also done this really interesting foreign paper, which really is is a lot of the material that are going to cover in here today. In fact, those two books the DevOps automated governance and the automated cloud governance are a lot of the stuff I'm going to cover today. com.

Quick thing about my background. I'm not going to tell you about my 40 years. It would take the whole presentation, but sort of from left and right.

You know, today I'm at Red Hat and I'll talk about what I'm doing at Red Hat. But more recently, I sold a company to Docker. A few years back, I was a networking, so I spent some time at Docker, sort of the eye of the hurricane I was early in, not the actual founder, but part of the founding group of Chef.

Back when we began, I was the eighth or ninth person there. I had a company that was a multi cloud management company that I sold to Dell. I've done a lot of startups have done also about 10 or 11.

I lost count start ups over the years, most of them miserable failures, by the way. But the last five or six years have been very good to me. I also did the one of the first private clouds.

This is pretty open stack, at Canonical I was an executive in residence in between startups at one point for KeyBank, and I started my career at ExxonMobil and also was involved in some of the original discussions of DevOps days. In fact, Andrew Clay Shafer, Damon Edwards and myself are really the progenitors of the first DevOps day in the US. And I was the only American at the first DevOps they in in Ghent.

Yeah. And you know, some of the things about the Global Transformation Office, I've introduced the players here, but we came on board at Red Hat and typically the four of us were not traditional Red Hat, you know, sort of rail people. As I said earlier, we've been heavily involved in the shaping of the DevOps movement, not not by ourselves, I mean with the shoulders of giants and lots of people.

But the idea was how could we infuse? And so one of the things that we talked about, we came in in October last year is, you know, this at 2020 you know, at the end of 2019. Right.

Like we've done a great job. DevOps is good. I would say the glass is half full.

We should really I mean I'm not kidding. Pat ourselves on the back. We've changed commerce, but the question is then all the goodness we did in 10 years, what are we going to do for the next 10 years?

And you know, if we're talking about GitOps and we're still you know, if you look at CI/CD four or five years ago, that was novel. Today it's table stakes. So a lot of what GTO is about is helping the industry.

And RedHat, it's given us this great opportunity with, you know, with all, you know, Open transmation, all the great things that RedHat has. You know, you know, if you haven't looked lately, you know, we have know really good distribution, open X of Kubernetes, just a lot of sort of really well rounded enterprise solutions. And how can we really, as start leaders with the combination community, RedHat and our GTO team start to drive what the next 10 years should look like?

So we're really excited about that. I think it's it's a little bit of a shift in how RedHat has been perceived and how RedHat and I applaud them for taking the chance on this team. And so basically in this presentation, we're going to cover really three main areas.

I'm going to talk about some surveys and how surveys relate to some of the trends. And then I would talk about the battlefield. Right.

What is what's out there? What are the things that we should be thinking about? How do we learn from talking and having conversations, organization to get to a level of truth, you know, not just saying that the breach was caused by one person.

How do we actually dive A lot deeper. And then the last section is really based on the title of this presentation is really diving into something that we've called automated governance, either automate DevOps automated governance or automated cloud governance. And just, you know, the way we use the terms of a GRC perspective, governance, risk and compliance.

T. these are very overloaded terms. People have different contexts and meaning.

So when we talk about governance here, we're trying to associate how we take the risk profiles that we have institutionalized in our organization and show evidence of how we are able to show the evidence. So from an audit and how do we match back to our governance risk and compliance models that we have institutionalized for organizations? So and again, I know some people call it continuous compliance.

Some people call automated compliance. So for this case, we'll see how we focus in on governance. So to talk about automated governance, I wanted to start with some of the survey data, it's sort of, you know, sometimes when you look at survey data, you can look at it to one or two ways.

One, you can look at it is like, oh, that's just survey data. And, you know, it's reports and stats and all the sort of jokes about stats. Or you can look at it as is it helpful in seeing some particular trends.

So not taking the data literally, but understanding the data. So that's what I'm going to try to do this morning, because I think there is some really interesting surveys that have been done over the last couple of years, specifically around the topic at that suck ups and and something called the software supply chain report. So there's there's two years now.

There's been a survey called the DevOps, the I'm sorry community survey. And, you know, it's about five times the respondents. So it is an echo chamber.

But one of the things that during that survey they were able to identify is really three sort of respondent stages of people who respond to a survey that, let's say about 50 percent call themselves immature in this sort of notional DevSecOps. And then a much smaller subset are in the what we would call mature. So 15 percent.

And I'm going to use some of this and some observations of some of the other data from this report. So the first thing is this was part of that survey where they asked questions about deployment. So when you talk about DevOps and even DevSecOps.

Right. Of course, a big component of this that most people listen out notice is how do you deliver software? How do you do it in terms of your pipeline, your CIC, your software supply chain?

And then we do mostly capture how often you deploy sometimes as an industry over rotate on that information. But sometimes it can be useful in understanding where an organization is and what what sort of the implication is from the data here. If you notice, it says fifty five percent or so in that weekly time frame.

But I dissect this a little bit differently. If you look, I would say here we got sort of less than 20 percent are really interested in the way that we'd like to see high performing or from some other surveys that I'm not going to include here, where you classify high performing organizations that deploy multiple times a day. And so really so it's a little interesting that a majority of the people in this particular survey are out of that realm of daily deploys.

And in fact, only seven percent or sort of every change user they're iterating out of it, sort of very efficient layer. This doesn't mean you're bad or you're good. It's just it's a perspective.

You know, I always love the quote from Mary Poppendieck, which is how long does it take you to get one line of code through your system? Right. And so you can you can take that to sort to DevSecOps.

How long does it take to get a hotfix or a vulnerability correction or patch through your system? Right. So and again, I'll just round say 50% industry is really in sort of a waterfall mode here.

And these are people that were responding in a second survey. Right. So, you know, it's I wouldn't say it's depressing, but it's interesting if we further look, I think this is another really interesting graph of what's going on.

And again, my interpretation of this is a little different from what they do. So if you remember the discussion about mature versus immature, if you look on the left hand side, the gap between mature and immature is shorter, is smaller. Right.

And so that would tell me that that these are the industry. This is sort of table stakes. Do you have a WAF?

Are you doing IDE's intrusion detection or are you doing static analysis, security testing? And so but what's interesting is when you get into the things that are in the box, like container security, dynamic analysis, security testing, software, component analysis, I'm sorry, composition analysis, the gap is much higher between mature and immature. And so this relates to my experience in the field.

I will tell you that most companies would probably identify themselves as immature in sort of a DevSecOps mindset. I mean, some people think if they're just doing vulnerability scanning, they're mature. Right.

But that's sort of nonsense. Most companies that are really sort of evaluating where they're at would probably say they're immature, even though they're doing things like at the perimeter or WAF or they have, you know, open source covenants. And the ones that were considered himself, in my opinion, mature.

Or are the ones that are saying and I take this in relation to the software supply chain or the CIA or the pipeline, are you doing DAST or software component analysis or interactive security testing in the pipeline? And again, notice the gap is really much higher there. So it just tells us a little bit of data about where we are that the truth is most people are not doing sort of more advanced stuff in the software supply chain.

And again, it's a survey. So we take all this with at least a grain of salt. Another interesting data point, which, you know, I hear a lot, I go in an organization and they're reasonably savvy on all these topics.

But they'll say, but, John, you know, the developers think if we do security, it's going to slow them down. You know, or operation this you have this sort of still global mindset in a lot of organizations, the security means that, you know, that it's going to slow down efficiency. And unfortunately, the data from this particular survey year over year reflects that, you know, there is not a big change in in the respondents from twenty eighteen all the way to 2020.

Right. One of the last pieces I think I have in this is that part of the survey was eighty five percent of your code is is sourced from external suppliers. There's a large bank that's publicly on the record, but I'm not going to mention who they are, where the one of the the top fellows in this organization was asked by the CIO.

So this is a bank that has about eighteen thousand Java developers there. Eighty five percent Java based highly a big development organization for in-house development staff. And the this fellow was asked by the CIO how much open source data the answer was.

Ninety nine percent of all the code that they wrote from like eighteen thousand developers, ninety nine percent was open source. And here's the kicker. They we're only using 10 percent of that Ninety nine percent.

So we have we have a lot of interesting challenges, if you will, in this space. So the battle field, one of the things I spend a lot of time and I've done this after the probably the last four or five years is I typically come in your organization at the CIO level. What happens is usually if somebody at influencer level or a chief of staff who go to the CIO and say, you really should listen to this guy, John Willis, I don't know why they say that, only kidding.

But but and then so I get a chance to talk to CIO and CIO say, how would you help us? And I'm like, let me interview your people. So I've been doing this for a few years.

Well, I'll go in. And I've done some banks. I've done two or three, I think three of the top 15 banks in the World.

One Bank. I spent a month, you know, at their location. I interviewed three, 250 people.

And I find there these patterns. And so I call them organizational conversations where I literally just try to talk to the people who do the work and find out what they think about not what management or leadership or some consulting organization told them what they're doing. Just ask the people.

Go talk to one hundred or two hundred people, earn their trust and let them tell you how things work. You'll find more about the organization. And I've I've I've called this the Seven Deadly Sins of DevOps, seven because it's a cool number.

But more importantly, these are common categories that I see over and over. And it's just a way for me to bucket the conversations. And interestingly enough, the reason I made this a funnel is more often than not, when you aggregate all these things that you hear into these categories, it really paints a picture of how effective or what your efficacy is in terms of security and compliance.

And more often than not, my findings is that most large organizations, especially financial institutions, that I have no problem telling the CIO that that sort of security and compliance audits are basically theater. I've done more detailed presentations on this. And so moving on.

And so here's another thing for those of you who have read the Phoenix Project, right. You may or may not know, it was a purposeful rewrite on a book written in the 80s called The Goal by Eliyahu Goldratt has a lot of publications, but one of the things he has is an audio book that he did 20 years later, and he called it Beyond the Goal. And one of the things he did in that book is he talks about how systems thinking is different from own systems thinking or what he says.

He talks about physicist or or non social science thinking. And let me draw this picture. He says that most people, if you ask them in this graph or this picture, which is the more complicated system, most people who aren't like sort of experts in, you know, in complex adaptive systems or system thinking would say that it's system B, of course.

But the people who understand systems and in certainly physicists, which I'm not one, but I've got some friends that are would say, no, no, no, it's system A. And they'll say because the system has more degrees of freedom, System B is actually trying to build a story. It's giving us some directionality and system is forcing us to start from scratch.

And the way I translate this and so I want to tie this back to how I learn, I go into an organization. And how do I figure out the find out exactly how things work and like I said, first stages, I don't talk to the executives, I talk to the people. I call it the edge.

The people who are doing the work fingers on the keyboard. And one of the things you have certainly as a consultant now, I have this luxury. A lot of times I get brought in because I'm touted as somebody famous, which is nonsense.

But but John Willis is here. He wrote to the DevOps handbook. But so I do get some instant credibility.

But in general, a lot of times when I go in, people are very defensive. It's the sort of the the office space. You know, the Bob's right.

Like, don't tell this John guy anything that will get us in trouble. And so what people tend to do when I ask him questions is it's a fine job. And they try to give me this this sort of system B answer like, oh, no, no, everything's fine.

Example. How do you CI/CD, John? It's great.

Now, I don't ask that question because I know I'm going to get, like, these abstract answers. You know, I'm using the fine dog here. Everything's great.

John is really good. I got some warm coffee and I bright colors in my room. But the truth of the matter is, when I'm looking for is the truth and the truth, in most cases there are pockets not always of like it's on fire, it's in flames.

So I really have to try to decouple this notion of what I would call system A thinking. Right. And and so I'll give you a good example of sort of system A versus system B thinking.

So we all have heard of the Equifax breach, right. You know, it's popular. It's you know, if, you know, it was in something called struts 2 it was a parsing module called Djakarta.

If somebody actually understood, they could basically do a curl hit, hit a server, inject some type of command if it was authorized, the kill chain starts. Now, the the system B answer to this question, you know, particularly from the CEO and even the CIO, which was, oh, this is easy. We know what happened.

It was a breach because there was a one person, a five billion dollar market cap. In fact, they lost five billion mark. And I don't remember what their market was really.

It was one person. This is not a system thinking. Right, that that who failed to deploy the patch.

But the truth of the matter is, there's a great document done by the oversight committee and Congress. I mean, if you want to read something fascinating, they went into real detail about all the things that contributed to this breach. And there were so many things far beyond one person not patching the the the CSO reported to the chief legal officer.

So under testimony, when asked how come you didn't notify the CIO of the breach, the answer was, I didn't think about it. Like to know anything about Connollys law. Like, that's a classic.

That person was bounded to the chief legal officer. So they wanted to make sure everything they thought about the CSO write like imagine that, right? Like talk about not thinking like a software company.

And there's just the all the IDS perimeter based had 18 month expired certs, so millions of dollars being wasted on stuff that could have actually seen anomalous behavior but weren't working because there was some oversight and some scripts that didn't work to actually update the search. Right. I mean, just on and on.

And it's the same thing with the Capital One breach, right? Like the the sort of system B the answer was the attacker was carried out by a former Amazon employer after losing 2016 maintain that's nonsense. Was there's no knowledge that somebody who worked for Amazon in 2016.

Helped them with this breach, because you know what it was. Again, this is from external I have no internal knowledge. I have no sort of NDA or anything like that.

This is from research I've done from external is there was a team that was in a hurry that needed to put up their own WAF, which was a general WAF that was part of the the normal prescribed way to do it. They put it up, they took the defaults, they left bypass on and they absolutely took the default on a particular Amazon server and a group that was authorized. And basically, if you know what this is with all that, somebody who is just, you know, running scripts against this woman who actually breached it was just running a script continually against everybody, happened to hit the Amazon metadata server and was able to dump the all the credentials.

And, you know, I don't know, security people say Bob's your uncle, but Bob's your uncle on the kill chain. Right. Because it was a server side to me, became a server side request forgery.

So, again, the point is that you've got to be careful of these abstract answers. You have to sort of dig deeper to find out what the what all the contributing factors are. So this leads me to the next section in this, which is the title Automatic Governance.

So I've been working with Gene Kim, coauthored with him a couple of times, we run something called the DevOps Enterprise Summit. We've been doing that since 2014. And then every year around April, Gene invites about forty five or fifty people to Portland.

And we work on what we call forum papers with six or seven of us to get together. And we work and we do the research and we turn them into publications. And it's just been amazing.

I counted like 30 or 40. Gene says they're 70, but somewhere between 30 or 40 and 70 out there. Just go to IT Revolution in forum papers, amazing, amazing resources.

Three resources that have been related to automated governance and really have gotten me on the track of being maniacally focused on this topic, which was going back to 2015. There was a paper called an Unlikely Union: DevOps and Audit. The first discussion about how you know, why DevOps and audit should actually get along like this 2015 and then 2018 in a paper called Dear Auditor.

And it was actually a GitHub site for this or possibly a GitLab site. But the where not only is the publication, but all the sort of matrix and manifest of all the things, it's basically an apology letter to auditors. And then it's a it's better than that because it's a bunch of detailed actions related to risk that are listed out.

So it's a really well. And then last year, I worked on a project I chaired with with Nike, Capital One, PNC, Marriott and Saber Group, where we worked on this thing called the DevOps Automated Governance. And I'll tell you a little bit about this.

So a lot of it stems back from from a paper that Capital One wrote a while back, 2017, that that sparked my interest in this topic when they wrote an article about creating better pipelines or focusing on the DevOps pipeline. And there was a subsection in that blog article that talked about pipeline design. And what they said is, is that in order to sort of be part of the DevOps flow, maybe also being able to have auto commit or sort of auto deploy, a service group had the evidence at the time that they could evidence these 16.

Opportunities to be created and in the pipeline, they would gate these things like sort of reversing control was their static analysis. You can see the list. And as I was having discussions with people like if you're going to do the gating or enforcement, wouldn't that be a great opportunity at the same time to create evidence we call attestations.

And so that's what really started this whole discussion with a number of people in the industry. And then around that time, Google introduced an open source project called Grafeas, which was an internal tool that Google was using to do attestation management. I've been told it was for the Google Container Registry and it was being open sourced and it hasn't had much use.

It's sort of an underutilized could be an amazing tool. So we decided to get together and really create a project, which was this book I talked about that called the DevOps Automated Governance. Again, there there's a couple other people saying we're going to hear from Microsoft.

So there was a number of people on this project. And so what we did is we sat down and one of the goals was. Could we change subjective attestation of subjective evidence into objective evidence or at decision?

So when I say attestation, I mean evidence, right? Attestation is typically assigned or a signature digital signature. And and so if you think about how most organizations do audit today, basically what they do is it's very subjective.

Somebody creates a change record. They make a proposal for the change. Somebody's evaluated that change.

Typically, they ask for more subjective evidence of the change. And then there's certainly a number of people that might and particularly somebody who might be guarding production who might say, I need this this. Right.

And it's really a sort of a telephone game. Right. It's and then what happens is that all the time somebody looks at an artifact and production or change and they have to follow back this telephone game of subjective conversations.

And by the way, humans writing subjective narratives in short form text describing complex adaptive systems, which are modular systems is sort of folly in the first place. And so could we actually change that model to be digitally signed evidence in some chain of even a link of evidence events, loosely speaking, block chain. But like, it doesn't have to be block chain, but it has that sort of model so that at the end of day, the evidence is a digitally signed list of all the signed evidence.

In other words, you know, the commit X , the was there appearing on a pull request associated with the change rec at the merge the build status, the vulnerability status, just on and on and on. I'll show you some examples. And so you wind up creating is this notion of what we call continuous evidence, right.

Where we are sort of balancing the transparency in the evidence. Excuse me. With the the risk.

Model that you that you've set up, hopefully actually from your risk board enforcement reauthorization. And so one of the things that we set out when we were writing this guide, we said we don't know it was beautiful because it was started out as a whiteboard exercise. Right.

And the answer the question that we were trying to ask is, could we do these three things at the end of this workshop, which is about three days? And actually there was some follow on a couple of like a month and a half of follow on work for the team. But could we shorten the order time from like 30 days to a half a day or in a continuous evidence model to just continuous?

And he hit the button. Look at the status. Could we increase the efficacy of an audit?

So if you think about, you know, sort of especially even pre DevOps or pre micro services, or you could call on cloud nine of our monetization, even that's been hard in large institutions to reconcile, you know, what the audits actually say from the subjective descriptions. I was laughed that, you know, they go through the chain, they talk to everybody. Everybody gets sort of uprooted for a month, a year.

And then at the end of the day, they still ask for screen prints. So could we change like 20 to 30 percent efficacy to 80, 90 percent? And by the way, could we actually get better and including monetization technologies into this fold, which in my experience, a large financial institutions, if you think the disconnect between just standard pre agile or waterfall models is disconnected when people are using sort of cloud need of modernization, working with, you know, 15 or 20, 100 deploys a day, it's just completely disconnected.

Right. And then third is, if we could accomplish all these things, you know, would we have a better answer for the executive team to say, well, here are the reasons why you possibly could reduce centralized cab change advisory authority. So what we did is we sat down and we modeled these boundary points and we weren't trying to create a new model for the pipeline, the one hundredth of one version of a pipeline.

We really were creating these these seven stages, if you will, based on how we thought about how you would create attestations. Again, evidence control points or control points, we actually called common control points, patrol boats. And so, like one of the things in the build stage or development stage, the package stage, and you notice we we actually include dependency and artifact as part of the process.

But we note we realized that they have their own sort of cycle of cadence. How do you how do you fulfill library dependance and artifacts actually can have their own independent cycle. So we broke down what we call common controls, common actors.

And and I won't go through all these. You saw the book. You can get access to it on an IQ revolution.

It's Creative Commons. But if you look at the control, I think when we were all said and done, we had about 70 or 80 control points is evidence, attestation evidence. And it was a collaborative I don't know that one company or one service would use all these.

But since all these different companies were contributing, it was sort of a kitchen sink. But like control points for at the source, which would like peer review, a percentage unit test, courage, clean dependancy scan for sensitive information control points for build would be mutable, build up to approved dependancy store, making sure you have that you have sort of tokenized and you basically have digitally signed connectivity So you can't somebody can't man in the middle, unit testing, linting, security status, dependency license check, security check, making sure that the artifacts or libraries are from approved external sources, library quality check age and so go on and on. And like I said, I'm not going to go to all these and just pick out some at the packets.

The other thing we realize you have to have a package these days. Everything's not almost everything, but everything we care about is either a container image, a jar file or war file. Right.

And so we realize that actually from a control point perspective, really merits its own stage. So and so you look at digitally signed images, you know, sort of secondary vulnerability scan as part of the configuration. There's some you know, a library is a library is a library.

But there are some things about that, especially in sort of docker files. Right. There are some interesting byproducts we have build and run and those kind of things to metadata, artifacts stage.

You get the point proj stage on and on and on and on. And, you know, and so I wanted to show the sort of GitLab model because it works perfectly for this. Right.

So the whole sort of the one thing I think most people like about GitLab. Myself and, you know, I'm not giving a presentation shrilling GitLab right, that people don't invite me to do that. But I did want to make a call out to one of the things I like about GitLab.

And I think a lot of their customers like is it's all inclusive, right? That's one of their models. They don't use the batteries included mantra that Docker used.

But to me, it is that thing that, like, I can use this thing out of the box, but if I need to plug in something specific, you know, like if I want to use another CI/CD or I want to use another build tool. But this model works really well. And one of the clients that we've been working with is using the GitLab model, fully automated attestations.

So they have all the ingredients in place to sort of build on top of this. And so it really is a good fit for this automated governance model. And here again, I'm not going to go through each one of these.

But but but if you look at, like, the different sort of we tried to map out in a document like, you know, what would the attestation be? What would the particular source be? Where would the Web hook come from?

So obviously a source control or unit coverage test or a dependency. And then we went through each wanted to build a proved dependency unit, testing percentage vulnerabilities, getting on and on. Right.

Again, you'll have a copy of this presentation along with a link to the guide. In the end, what we did is we actually we kind of got near the end of it where we're running out of time, but we created a simple macro service, Java get client that went into Kubernetes and we store all the data and graphics. We use something called critics to sort of validate this simple use case.

Later on after we did this, we realized we needed something like COFCO for guaranteed delivery. And in a couple more points I want to make is a couple of clients that I've been working with. We're going to create a second version of this project, X, with most of the original team and really focus on policy, because once we had this model in place, we found that you can do some really interesting things.

And so one of the companies has really gotten really far. What we call policy is code. So once you have that automated attestation model in place, you can then actually start working with in this case, they start working with the policy people.

The bank sort of service delivery team will actually work, in other words, to get involved in this model, which is enforcement and and evidence. You go to the policy team. They work out, in this case, a YAML file, a human readable YAML file.

That is an interface that now. So there's no middleman. You don't have the policy people going to the infrastructure, people that go to the automation people.

You're literally having the policy. People write the human illumine readable, defined policy that interacts directly with the automated policy and automated governance model. It's it's really fascinating.

And here's just an example of one of the clients we've worked with is integrating that with OPA and Rego as part of the Kubernetes open policy itself. Like so if you imagine the Yamal is the interface and then you we create sort of automatically generated Rego files is the implementation that manages the Kubernetes admin controller and all the great things that OPA does. Right.

So really cool stuff here is sort of a model. It's a very simplistic model of using a sort of Kafka as a wrapper and then having this engine that does enforcement and sort of gating enforcement in evidence, storing the data Grafeas is there's been some contributions of adding a MySQL back with the Grafaes. And then one last but not least another project that was a spin off of the DevOps automated governance, which was a couple of people that were working on some large cloud groups and a working board called ONUG the Open Network User Group, really large group that works with really large telcos banks.

They contacted me about seeing if we can do some follow on work. And so we worked on this project in the beginning this year. And you can see here it's it's it's FedEx.

It's the the Kaiser Permanente, Cigna. And then some of the people that had worked on that was VP of Engineering for Goldman. And then so a really large group, another large group of and here is a little bit different spin.

It was it's similar. But what we were looking for is could we get the cloud providers to create attestations back to the consumers. And again, we actually were so successful that there was a write up in the Wall Street Journal on this, and this is also another Creative Commons document, the the automated governance document.

And just to go real quickly here, there were three objectives for this project. Basically, could we get a unified discussion? And one of the things we were really interested in is we weren't naive enough to say, even though I think if I was trying to calculate the I the this is a rough draft of the the buying power of the original team that wrote this book was probably in the you know, the it spend it was well past 20 billion, maybe twenty five billion.

And but even with that, we weren't naive enough to say we're writing this paper for the cloud providers, but we wrote this paper for is the community. If we could get the community to agree, then we up to maybe one hundred company. Now we're talking a trillion in buying power assets.

Right. And so the goal was to get the community to buy this, like these three things. Every cloud provider has a boot sequence from somewhere from electricity to hypervisor.

Typically, cloud providers do not document changes to that. Sometimes the changes to those sequence can have effects on scaled tennet resources using that cloud provider. So and we had some examples from the people on a project that said we've seen this.

It might change the posture, it might open up adversary opportunities, it could create network routing issues. Typically not, but could so could we ask the all the providers to give us a uniformed signature, not tell us anything about the boot sequence? That's, you know, that's proprietary, it's IP.

But could we get a signature in an event format that we could use, that we could test to see if the boot sequence is change and then we could know the first order? Primitive, any sort of incident analysis and resolution is what change? Right.

So so we have some examples we wrote about that and the idea was have that in a uniform from all the providers. Second, which was all cloud providers see all ingress requests from a tenant. Right.

All those API calls, typically the way people sort of scrape and manage that is they look at logs, they look at all these things to find out what usage patterns have happened. And specifically, we're looking for server side request forgeries or overrun usage or somebody put the wrong loop in and created like a million of these when it's only been about 10. And each cloud provider has a different set of logs.

And you have to constantly keep up with the brutal nature of sort of logs and scraping. And so could we ask the providers to provide sort of mirroring function, like think about an event gateway, like a serverless event gateway, so that everything they see gets spit back to the tenet. And so now you can base and can we get that in a I know this is crazy talk, but like I'll tell you in a minute, the second version, there's a couple of cloud providers that are actually going to be involved in it.

So we got their attention. So could we actually sit on it and then Gateway and then look for things that might show us that there's a service side request for or somebody is trying to attack us with the wrong metadata by building a crypto minor and then last but not least, asking the providers to give us a singular view of their. So each one has their own security center framework.

A, could we get sort of a unified view from all the providers? And then two, could they be more central to the security professional because they The security professionals have to do some sort of second and third or an investigation to interpret how that relates to a to sort of maybe, you know, real security or cyber problem. Right.

So anyway, I'm going through this really quick, if you're really interested. You read the guides. Here are some of the graphics here.

And then last but not least, just like the example of a policy is code, we tried to play a little bit in this case. Gherkin So we created pseudocode loosely based on Gherkin to see if we could at least theoretically write some sort of governances code to test out our theory about the boot sequence, or this is one for an example of the boot sequence. And again, I'll let you sort of read the document if you if you're so inclined.

And then then also this is a pseudo code for sort of a crypto mining fingerprint based on that Ingress model, the Gateway Ingress model also. Anyway, thank you so much. I notice a lot of material.

I went pretty fast. com or @botchagalupe. If you want to talk more about any of these projects, please.

I'm always interested in talking to people and I hope you're really enjoying the conference. Thank you so much.