Navigating the Future of AI with Responsible Governance – DevOps Unbound EP46

Hi, everyone. Welcome to another episode of DevOps Unbound. You know, uh, we were reminded by our producers right before going live on the show that the very first DevOps Unbound was in August of 2020.

So, we have been doing this now, going on four years, two months. We we're into our fifth year of DevOps Unbound. And that to me is just mind boggling.

You know, it was, I don't want to take too much time away from what we want to talk about today, but DevOps of B was originally the brainchild of, at the time, the CEO of Tricentis, my friend Sandeep Johari, and myself. And we brought Mitchell in very early, and at the time it was the Tricentis, CMO, Brent, and I forgot Brent's last name. And we, we concepted out this idea of doing a biweekly video podcast series that would explore all aspects of DevOps.

And even though Tricentis, and this was point of years ago, right? Tricentis obviously very focused on continuous testing and testing. The Tricentis folks thought it was very important that we explore the full spectrum of DevOps.

And over that time, we certainly have, and not only that, but we've stayed current as New forces. And, you know, new technologies came into the DevOps space, right? We're gonna be talking about one of them today, ai, but, you know, I don't want to say Outer Boys or Pats on the back, but man, four plus years riding.

Congratulations to our tech strong team and our tricentis partners who co-produced this. A special, special shout out, Jody, Ashley, and Ly nor who never get on camera, and I'm not gonna force them to get on camera now, but these two gals, you know, week in, week out, month in, months out, they, they pick the, the topics, they source the guests, they get the abstracts done. They, they make the trains run on time here.

So shout out to Jody and Lanier for all, all of their work over this time. Let us now, though jump into today's show. Today we're gonna be talking about AI governance, very timely topic and DevOps testing and everything else.

Uh, before we jump into it though, let me introduce you to our amazing panel. First of all, she's a frequent guest on Textron event. She's a friend of Techron, a matter of fact, she's in the tech drug.

Actually, two of our folks here are in the Textron gang, but she is the CEO of Deploy hub. And, um, also an open source board member extraordinaire. Tracy Reagan.

Hey, Tracy. How are you? I'm doing great, Alan.

How are you today? It's great to be on this conversation. You know, I'm always, you know, ragged about ai, so I'm glad that you invited me to this one on this call.

I hope that we really kind dig into the technical standards, because that's the area of interest I have, because if we have technical standards and potentially we can start building those standards into our DevOps platform, and I say that because I know that we, you know, we're, we need to build security into DevOps platforms, and we have this on our tail building some sort of transparency and accountability into the DevOps pipeline for, uh, for ai. So a huge new area, and it's great to be on this call Joining Chay and I from Tricentis. He's, he's been on a few times before, over the last four and a half years.

Martin Klaus. Hey, Martin, how are you? Hey, Alan.

I'm doing well. Thanks for having me on the call again, appreciate it. Uh, and it was great to see you in person a few months ago.

Um, yes. So I am responsible for, uh, customer engineering at Cent. I've had multiple roles including product marketing, product management, and one of the things I get to do in my, in my role is to actually work with customers who are, um, defining their own AI strategy.

And so, I, I'm involved in a lot of conversations where customers are trying to figure out how, how do I introduce AI in my organization? How do I make sure it's safe, it's secure that the data stays in our data center and doesn't get leaked, uh, out on the internet. Um, and also how do we test AI technologies, uh, so that they're safe to be used in our environment?

And so that's a really fascinating, uh, conversation to have with customers of different vertical industries and different segments, because they each have slightly different requirements, but they're also together all looking for the same thing. So I'm looking forward to the conversation today and talk about regulations and governance. Fantastic.

Thank you, Martin. Um, next, she's also a Text Strong gang member and frequent text strong, uh, event guest, our own Hope Lynch. Hey, hope, how are you?

Hi, Alan. Uh, so happy to be on today. And funny enough, Tracy and I had a conversation recently where we started digging into, uh, AI and some of, uh, what we saw as the issue.

So I'm so happy to be here today to talk about governance because it's, it's a critical topic. It's great to be able to elaborate on it for a lot of people. Absolutely.

Thanks, hope, and it's great to have you here. Thank you. Next, well, actually last, but certainly not least, he's my co-host all for every single DevOps Unbound episode we've ever done.

He's also the CPO here Ad Techstrong, as well as VP analysts for DevOps at the Futurum Group. Mitch Ashley. Hey, Mitchell, how are you?

I'm doing real Well. It's a, it's an interesting time to address this. Dora just came out with their report and kind of put the pin on the AI donkey saying that's causing us to be not as efficient, which I don't believe that's really happening.

But, you know, so getting into governance, we are trying to figure out ai, so it's a good time to be talking about governance, technical standards, regulatory, how do we do this, right? So, so folks, look, it's been, I think about two years now, right? Since chat GPT, just about two years since chat.

GPT burst on the scene, I think it was November or two years ago. And, um, and it's sucked the oxygen out of the air of every conversation we've had on tech sids just about it seems we're always discussing it, and we've seen the gamut of this is gonna be the greatest gift to mankind ever, right? I'm reminded of that old movie where they have the, the book to serve how to, how to serve, or how to serve man.

Mm-Hmm. And it's a recipe, um, verse versus, you know, the, we must put the brakes on AI and stop it, because it'll be the death of us all. AI will fight this imperfect something out of a Star Trek ger kind of episode, right?

And, and seek to destroy humankind. Uh, I'm a firm believer that progress stops for nomad. And, and I think that's certainly been the case with ai.

It has moved ahead, full speed ahead, damn the torpedoes. However, in typical fashion, we have seen some governments, many in the US unfortunately, but some governments have started to try to put some governance, and that's what governments do. And governance, uh, you know, around the use of ai and some private industry, public consortiums are trying to establish rules of, you know, what's ethical, what's right, what's not right?

Where does it tread on humans and human rights? Where does it help? You know, what, what's the right, what's, what's wrong?

And right here to do. Obviously, some things are clearly wrong, right? I mean, using it to, for mal, you know, my living mal, I always mispronounce that word.

Using it for bad purposes, right? Is never a good thing. These deep fakes.

And we're seeing it around election season now, you know, prevalent news. No one thinks that's the right use for it, but there are right and wrong. Um, Tracy Hope, I know you guys, both guys both have strong feelings on this hope.

I'm gonna let you go first, if you don't mind. Okay. What do you think?

I think, uh, as far as governance, one of one, one thing that comes to mind, as you were mentioning, um, you know, the eu, they had the EU AI Act so that they can have comprehensive enforceable standards. There is work, I think, also happening in Canada. Australia, they're sort of looking to see what's happening, uh, in the eu, but in the US I think it's gonna take a little longer.

But I do also think if there is a company that can say that they are taking these steps to have critical oversight governance, so that, um, you know, it's not a black box, maybe it's explainable, they can ensure that there's no bias. Uh, there'll be a little bit of a first mover advantage there. I know that, uh, some financial organizations, I think Capital One is one that is taking steps in that direction, but there is a long, long way to go, uh, to get AI governance across industries and to be something that I think is pretty commonplace.

I thoughts, I, I wanna point out that there, there is work being done by the US government as well. We have the Yes, true, true. We had the Algorithmic and Accountability Act passed last year.

Hmm, that's Right. And they, and they kind of address many of the same things that the, you know, the eus trying to address, which is this idea of accountability, really. Mm-Hmm.

Um, you know, uh, particularly around what, and all of these, all of this governance is really just around high risk AI applications. Mm-Hmm. You know, surveillance, um, hopefully medical and warfare.

Mm-Hmm. But we, the, the US is, MA is making some progress, but I do feel like a lot of the governance has been turned over to the, um, European Union, uh, ever since the GDPR we're still, I know they're doing data stuff, so we don't have to, but we, but the US government is still, they're looking at it that that bill was passed. And, you know, it says, Hey, you've gotta make sure that you're developing, uh, fairness and, and, you know, minimizing bias and promoting transparency and mitigating any kind of discriminatory, uh, discriminatory outcomes.

Mm-Hmm. Um, you know, all of this, when I think about it, I, it's kind of morbid, but I think about the opening scene of robocall, right? In Detroit, And recently there was this insane article that came across in, um, tech on text, uh, crunch about Silicon Valley having a discussion if AI weapons should be allowed to kill people.

Oh, wow. Okay. You know, it's like, duh.

No. Well, that is, I think, think that's the first law of robotics, right? Yes.

Mm-Hmm. Yeah. asmas last asmas, lot of robotics.

Mm-Hmm. And that's what that scene went from. The robocop was right?

The robot goes and, you know, slaughters the entire board. Mm-Hmm. So, I, you know, so I'm sorry.

We have why we have, why we have governance, um, and it's only around these, these high risk systems. I feel like we have a long way to go to start trickling it down to all systems that are being written and being able to take these technical standards that's like NIST is working on, and apply them through the DevOps pipeline to start actually working to make sure that they are transparent and that some of these tools now can be applied, uh, to the process. Mm-Hmm.

Opening of, uh, of Terminator one too, by the way, crushing the skulls of humans, but terminate robots. Um, sorry, Martin, I was just gonna jump in and, and say to me, it's fascinating when something comes along that's akin to security, it's akin to data that gets this governance, and how do we do it properly so that doesn't escape from the lab or escape from the application data doesn't leak out, um, or in this case, do harm. I mean, you know, other software could do harm too.

Algorithms could as well. You can argue whether social media does that, but AI is really jumped to the top of the list of what are we gonna do, but do we have to do to, to secure this? And also make sure it's not used for nefarious purposes.

So, Yeah, sorry. I think it was interesting that, that you see folks like Sam Alman or even Elon Musk, uh, even asking governments to step in and help create some rules, guardrails, because to see the potential of where this technology can go, I'm not sure we're quite at the turning level yet, or robocop, what have you, but because it's going to be an evolution, uh, but the evolution is happening really, really fast. And I think we're already a little bit on the back footing, uh, with, uh, some regulations as it relates to use cases outside of business.

Uh, for example, you know, uh, kids in school using AI systems to do the homework for them, uh, which, you know, now puts education systems at, you know, the back footing in terms of like, how do we deal with the situation when we require essays to be written as part of like entrance, examin entrance requirements for universities. And now anybody can just generate an essay in the voice of Ryan Gosling, of whoever, uh, you wanted to in imitate and, and sprinkle in some grammar mistakes and punctuation to make it appear as if it was original authentic work. And so, that's just one example of where we're thinking completely new on chart territory because, uh, we don't know yet know how this technology can be used and applied in some cases of thinking, especially in the business context and the kinds of customers that I'm talking to.

Um, the, what I'm hearing is that there's definitely a need for transparency. And I think that's one thing that we could sort of check out very easily, because companies are asking for where is the data gonna be? What is being processed?

What happens in transmission? What about encryption? Um, who wants did the results?

Who wants to, you know, the models, what models are you using? And then how do I customize it and make it my own? Because I do not want my data to get leaked out on, onto internet at all.

But it's a very broad field, and I think the way, um, for example, the European Union and or what the House White House has also been proposing is a good one to say. Let's think about this from a, uh, point of view of, of risk. What is unacceptable risk?

As you mentioned, Teresa's surveillance, um, or, uh, protected groups, uh, in, in society and other things that, uh, or medical, uh, applications. There may be areas of unacceptable risk where, you know, we, we need to actually have some laws in place to, uh, to control those things. And then you work your way down in terms of high risk, low risk value use cases where, you know, uh, it's, it's a less of an issue, but this is a extremely complicated, important topic that affects all aspects of, of life.

I, I, I don't necessarily disagree, but you are talking to someone who's from the generation of no, no calculators allowed in the school test, right? I, how many did, did you, are you allowed to bring calculators into your test? No, no, no, no.

But kids today, they bring scientific calculators in. They no longer have to worry about doing those equations and figuring pie and all of that. It's all there for 'em.

I think we're gonna come to the same thing. I, I think, and call me radical, but I, I think when we get to new technologies like this, our, our part of our makeup is to think, go slow. Go slow.

This could have repercussions we haven't thought through. And we know, quite frankly, that that never works, people, right? If you outlaw guns only, guns only outlaws have guns, people are, they're still gonna be a segment of the market that's gonna go as fast as they can.

Uh, I think almost these governance breaks, if you will, these governors on the use of these technologies is to give our society a chance to catch up, a chance to get acclimated, a chance to get comfortable with what this, what these technologies can do for us. And I think if we recognize that, we could look at them in a whole new light. But I also think it begs another question of how do you test for AI governance?

Right? I mean, Lauren, I'll throw it at you. You're, you're about zenes, you're the worldwide leaders.

How do you test for AI governance? How, how would you, you know, who's, who's minding the, who's watching the watchers here? Yeah, I don't think necessarily that it's possible to test for governance when you need to be able to, uh, it's more about transparency and really sort of, you know, like in the case of security, right?

Like companies are used to, you know, expose and report and, and, uh, sort of showcase, uh, what sort of, uh, security policies and governance they are implementing the products. And, uh, and we have standards around that. If you look at SOC two, and if you look at many different encryption standards, um, I, I think on the security side, we have gone of, uh, have come a long way already.

I think that could be analogous to AI as well, to some extent. Um, but the thing that I see with our customers that, that we speak with is that like, how do we effectively test an AI system? That it gives you reliable results, um, uh, in addition to how it works at, you know, uh, is it safe to use, right?

So is is the outcome useful and applicable, uh, and safe to use as well as it technology safe to use as well? That's where I think we're in also new territory, because if, um, AI models are passed Turing test then, and it's a knowledge system, an expert system that, you know, uh, can involve with time, then it's going to be much more difficult to, you know, come up with prompts or tests and parameters to say, yes, today the answer I got back was acceptable. But what about next week, next month when the N system has evolved and now it has been enriched with new information or with new, uh, you know, adaptations or waitings learnings that will, will now yield a different result.

And I think that's a challenge with testing that it's no law at point in time activity. You have to almost like have a, an ongoing monitoring system in place. Um, but you also don't know what you don't know and what you should be testing for that could be exposed through some sort of loophole.

And I think that's what some companies are finding out the hard way. Like if you look at some of the, you know, Canada examples, for example, where it gave wrong responses, now there's a lawsuit and so forth. Um, and that's a challenge I think with testing ai.

What we do internally is we're trying to sort of adopt this monitoring model, uh, where we are going, uh, on a regular basis, benchmark test, validate, um, and have sort of a, a red team approach as well to say like, how can we expose, um, the models in a way that, uh, the results we're getting are no longer in line with what we expect it to do? Um, unless there's a new way to, uh, figure out, you know, testing of not the testing models, uh, I think, you know, that's going to be the approach that a lot of people will, will need to take as well. Yeah.

A couple of things I would like to, uh, insert here. For anyone who's listening and is trying to, I guess, wrap their heads around, how would I start to do these things? There are tools out there, and you still have to know what you're doing, but you could look at, um, uh, Q flow, ML flow, um, they help, they can help streamline building, deploying, managing your machine learning models, actually at scale, um, open source tooling.

So, uh, it definitely, you know, there's always a learning curve for these things, but if you're looking for some tools, uh, that can help you get started and figure out what you should do, um, those are two pretty good ones I think that folks can use. Then I wanna add too, on this topic, um, there's, there's, there's two levels here. We have testing and we have compliance.

So if we look at what we've done through, you know, over the last five years in security, we have done everything from adding signatures to, um, repo scans to things like open SSF scorecard that looks and determines how safe software potentially, um, could be. I really believe that these types of tools will also be created for looking in at how compliant AI software is. Now the problem with compliance is that the data is often fragmented, and maybe you can look at a GI repo for one particular component to see, uh, how, how compliant that component is.

But it is a first step. Compliance is a, it will be, it is important today in security in software, and it will be important in securing and making sure AI software is safe. So while testing is important, understanding the compliance levels of the code that's coming across the supply chain is not gonna change.

We're just gonna have different types of tooling to make sure that the, you know, that it's, that the model is fair. For example, how do we scan for that? Not sure.

I think IBM has something called like IBM 360 that, that does some kind of fairness check. So as, uh, you know, hope pointed, pointed out, we do have tools out there, and that's why the technical standards become more interesting, because if we can define what those technical standards are, we can also define the compliance levels that we need to achieve. And while we have seen a lot of these governance docs, um, from NIST to the eu, uh, talk about some of the basic pieces, we still haven't seen a really clear roadmap for what is compliant.

And we haven't done that for, for code in general. So maybe we'll get there. We're trying, but we have to look at the compliance question and how do we track and report compliance.

So the companies who are writing software and consuming these, uh, large language models have a way to judge how safe they are. You know, in some ways this is, you know, there's compliance against standards. I you have to have something to test against, right?

Which I think is lar largely what we're saying, and there's two forms of it. One is IEE and nist and, and, and regulatory types of definitions of what those standards are, what what you're testing against. Oftentimes though, compliance isn't testing against someone else's standard, it's testing against your own standard.

So a lot of compliance frameworks are all about what is your policy for, for this part of security. Like if you go through a web trust compliance process, it's all about here's what our policies are to protect data, to secure this process to the handling of customer information, whatever it might be. It doesn't tell you what this process should be.

You have to define your own. Then the compliance is, we have validated that we, we have systems and processes in place to ensure that we follow those processes. And if we are in fact following them, I, my, I have a sense that this is kind of where we're going.

There'll be some things that will help guide us, um, and getting some insights too to what we should be testing against. I, I almost think we're living in this world because it's so wide open of what you could do with AI that organizations are gonna have to publish what their, kinda like their privacy policy is or what their, their, uh, uh, online community behavior policies are. Same thing for around ai, and then do they in fact meet those?

Because otherwise the requirements could be so vast, I'm not sure you could really test against everything and really know whether is this a safe system? Is this a safe ai? Yeah, I think Mitch, that's a great point, and sorry, um, You go Martin, I was gonna add one more thing.

What I'm seeing already is that, um, this is not a simple question to answer already, and it's going to get even harder to answer because, um, it's going to be a more of a blended approach as well. Like right now, I think oftentimes it's pretty obvious when you're interacting with an AI chat bot on somebody's website, because that's an easy way to have some sort of customer facing interaction. But where, uh, it's much harder to see where AI was used in process is when you have like a composite blended service where AI tools are going to be part of the outcome.

And so you can see this in art, you can see this in, in, in creative, um, uh, disciplines. Uh, you can see this even like in, you know, from a business standpoint where, you know, or, you know, we were just joking about the, the after 40 days topic was, you know, run through an to say like, how, how can we clean this up and bring it down to 120 words, or something like that, right? So in those situations where AI is used as a tool, uh, uh, as part of a larger workflow, then, you know, how can you know that level of transparency, you know, be sort of transported to the, to the end user as well?

Like, how do I know that a piece of artwork that I'm purchasing as an example, I'm not a a a dealer at all. I'm just bringing a theoretical example. How do I know this is original work?

Um, or it was used or AI used as a tool in the creation of, of the artwork or in an email or whatever, what have you. I, I think that's where right now the world is moving where yeah, services are more, uh, an assistant role, an agent role to basically assist with the creation of, of new work in a much more productive way. Um, and then the, the, the notion of compliance and governance and transparency is going to be even harder to, uh, to say like, do we need to put a label on anything that, you know, has, has been touched or involved in some sort of AI tool in any creation of it?

So just to correct my, uh, the name of that tool, it's a open source tool, it's called AI Fairness 360, and it is open source. I just Googled it. Um, and it's or ML models, so something you can put in your DevOps pipeline, right?

There you go. That's, you know, there aren't any examples where things have really slowed down, right? I mean, you think about adoption of the internet, adoption of the cloud, social media, um, I mean, the only things I can think recently around AI are Microsoft copilot recall where we got ahead of our, so they got ahead of themselves and someone thought it was a good idea to snapshot your screen every, every second, but not securing of that information, the market reacted.

The other is Apple's reaction to the EU AI Act, which is, uh, we don't, basically, I took it as we don't understand your reg regulations well enough to know whether we could follow that. So we're just not gonna bring our AI to the U until we, we feel that we can meet that compliance. It wasn't that we don't like your standards, like we just don't know what it is.

Other than that, there's very few places where some, a regulation has stopped ai, uh, in, in its tracks. And, and I think one of the things we're gonna need is internal. Why dunno, this is judgment call, right?

And so much of it is we're gonna need the internal board. Uh, here's how we're using ai. We're, we're thinking about doing this with it.

Are we delivering on the promise to our customers of safety of, uh, protecting their data transparency? We said we're gonna be transparent. Well, are we being transparent enough?

There's so much of a judgment call to this that you almost have to have some kind of an internal mechanism to say, no, maybe it's not a gate review that everything has to go through. We can use some tools, like you're talking about Tracy, to do some of those things, but it's, it's like, here's our stated policy of how we're gonna use AI and, and what we're gonna do and not do, and the line's not always clear. So how do we help clarify when we need to make those calls?

I have a more fundamental question, which is, what is the purpose of all this regulation and governance? It's to build trust, isn't it? To build trust in AI and, and its use where I think the, the best way to build trust in AI is to use it and see for yourself what, what's real and not, maybe not for so good.

Um, would fundamentally do having these regulations allow you to trust AI more, right? I mean, Martin do. The, the fact that the EU has this AI act, it's not even, I mean, it's been passed, but I don't think it actually goes into effect effect until next year.

Um, but by having that act, you say, oh, AI's a little more trustworthy, a little safer to use. Now I have more trust in it, because if it doesn't, why are we doing this? Yeah, I think the, that's a good point because the, the, the what, what builds trust is obviously, uh, transparency, uh, is one way to say, look, you know, there are regulations out there that requires out there, here's what we are doing in order to make you feel comfortable that the technology that we're delivering, the products and services that we're offering are safe to use the environment.

And so, for example, uh, just be super practical for a second. At tricentis, we've actually established an AI trust center that you can look up on our website where we publish, uh, our data privacy, our, the, the kinds of technology that we're using, our security, uh, that we're using. Uh, and we're also publishing the, the, the guiding principles that we use now in internal development, uh, to basically give customers that want to know, uh, a place to go to find out how do we, how do, what do we do internally and, and, and what do we do?

And these are things that we are going to, uh, update and, uh, keep current on an ongoing basis as new regulations come out. Um, I think there will still be an opportunity for like standards like ISO and others to, uh, create more formal, uh, certifications as well that, uh, vendors like us can, can sort of, um, uh, can achieve and publish it as well, just like Tracy, as you mentioned on the security side, where we have a lot of requirements already that, uh, companies can, uh, adhere to. And then also be publishing that and say, here's what we're doing and if you have more questions, let us know because we wanna work with you to understand what specific requirements do you have that we can, uh, support as well.

And then it's about sort of, you know, showing and demonstrating and, um, and putting, um, you know, the proof into the pudding, if you will. I think this, this question of trust is pretty, is really important. Um, I, I, you know, I even in text on gang, I've said many times, I don't trust a autonomous driving call.

Yeah, I, so trust is an issue, but part of that is awareness, right? Um, if I have, if, you know, when I was driving a Tesla, I could have put it in auto, you know, kind of an autopilot, but I never did. But I was aware that, that I was making that decision.

Part of the EU act is says that, you know, if you're a kind of at a minimal risk, at minimum, you need to indicate that this is, this is AI generated. So we know in the United States right now, there's, you know, public service announcements going out about the potential of robocalls or AI generated, um, robocall that says go vote at some other location. The person that's listening to that doesn't know that that's an AI generated, uh, phone call.

So, you know, the awareness of the fact that you're consuming AI data, is it super important? Anything that comes across that's been generated by ai, it should have some kind of a stamp on it. Some kind of a, you know, a watermark that, A hologram or something.

Yeah, yeah. Something like the doctor in Star Chek Voyager or something. Right?

Exactly. Exactly like that. Um, because then we can, then we are aware that we are looking at something that's been, uh, that, that is actually AI and not human.

And that's important. It's very important, actually. One of, uh, one of the things that comes to mind when you say, um, do I get a sense of confidence knowing that these rules of regulations exist?

It's almost like here in Charlotte, North Carolina, let me tell you, speed limit can be 70, it can be 55, it can be whatever it wants, right? But here in Charlotte, 80 miles an hour, 90 miles an hour, pretty much everybody is doing it. And you very rarely see anyone, you know, hold over.

So is the speed limit actually meaningful if there is no enforcement? Right? I think one of the things that will give confidence is, uh, and, and not that you necessarily want to see governments going after, you know, everyone, but key stories about, uh, enforcement and how it actually has benefited people and, and made a difference, right?

Having the rules is great, but understanding, um, in, in common scenarios and common use cases, real world things that have an impact, um, I think that is a big confidence builder when people can see it and connect to it in that way. So I will just rule of thumb, so I could save someone a speeding ticket here. Yeah.

Um, in, in Florida it's a 70 mile an hour speed limit. And the rule of thumb is they won't pull you over up to 80. Anything over 80 you're subject to get pulled over.

Don't take that to the bank, and please don't say Alex in North Carolina, they often say five miles. Uh, but there, there, there are no, there, there are no, but, But you know what, bringing it back to ai, this is a perfect example where if you have autonomous driving, one of Tracy Reagan's favorite things Mm-Hmm. And you tell, and you tell that AI program, Hey, not to exceed 75, you don't ever have to worry about it because it's not the human who's gonna go as fast as they can.

Mm-Hmm. If the AI is told 75, the AI's going to go 75, assuming the AI behaves as intended, and we have trust in it. And that's the perfect example, right?

Um, And if there's a bug and it goes 80 and you get a ticket, whose fault is that? This is true too. And are the tickets now automated?

Because the card, you know, You're In the line. Well, We're all Worrying about a scenario that's gonna go away. If you remember back to the future, national Brown says, where we're going, We don't need roads.

Exactly. Exactly. We don't have the, but this gilt gigawatts, whatever.

Go ahead, Chay. This Brings me to another topic around governance. There has to be industry standards.

Uh, you know, financial is gonna be different than, you know, traffic control is gonna be different from, um, welfare or warfare or, uh, or, uh, I don't know, surveillance. Every, I think every industry is gonna have to look at it. I think the Food and Drug Administration has done some work in healthcare, AI and healthcare defining standards.

But what will be, you know, so the question is what's this? What are gonna be the standards for the industry itself, right? The AI industry itself.

So I keep going back to compliance. If you define standards and industry specific ones, uh, and then we start figuring out a way to measure the compliance of those standards. I think we're making some progress, but bad actors are gonna do things without pursuing any standards as hope indicated.

How do you enforce this? You know, I have a 25 mile an hour si sign on my dirt road. Do I, you think anybody's gonna ever give me a ticket on that?

No. 'cause some, some neighbor put it up there and I fly by it at 40 miles hour, More than 10. But, but, you know, some neighbor put it up, Some neighbor put it up, right?

But it's, so it's kind of like, you know, it's a neighbor putting it up, the eus putting up this sign that says you go 25 miles an hour. Um, you know, do we honor that? And I think that most of us will try when it comes to this topic, but I do think industry standards are gonna be, uh, are gonna be as important if not more important than higher level, you know, federal level government standards on these topics.

Because very different When it, when it comes to governance. Now we see this right there in different governance issues for the finan, you know, what we call the highly regulated industries, financial and healthcare and, and government work and stuff like that. And, and we've adapted to that.

And as an industry, we, you know, people comply with those different vertical standards. And they're not all industry standards. Some of them are government.

Um, I, I would imagine we'll see the same thing with ai. I just, I just, you know, I have that hologram issue where ethical people will act ethically, but unethical people will almost certainly act unethically and they may not make their hologram have the, the, the, the watermark or the, or the, or the standard, and I guess maybe this is true with a lot of governance, is the good people do their best to do good, and sometimes negligently or inadvertently, they, they may miss a compliance or governed standard, but AI has the ability or the potential, AI has the potential for people who are not ethical to really abuse the system. And I don't know if having AI acts in place, you know, we have to go to criminal stuff.

I, I, I don't know what the right answer is, But, you know, I think even the krembles are gonna have their own governance standards Really With their organizations, right? It may not be a governmental or federal standard, but these criminal organizations, some of them are very large, right? They're, they're gonna have their own standards and it, it may Like La Cosa, Nostrom kind of, you know, you Yes.

Only go out with your wife on Saturday night, not the, not the girlfriend or whatever, right? Right. You know, Sometimes lack of a, a federal, well, a, a federal standard, I mean, that's where we are now.

And states are already enacting their own laws around ai. Colorado has a law, I don't remember the name of it, but it's basically says if you're, if you're working with ai, that is high risk, you have to, to, um, there's some transparency requirements and you have to have some type of review of what you're doing to make sure that it isn't endangering people. Something like that.

But that, that's Colorado. Who knows what, you know, Wyoming's gonna be California Or Idaho. Yeah.

So, so we're, I mean, we live in a global world, but if every state has a different policy, different law, that's a complex web to try to build products and use ai, I mean, We see, Alan, your point, your point is, is spot on, you know, this is the whole process of governance is really around, um, uh, kind of a democratic system between the EU and between all the states and the us. Uh, everybody has to act, uh, in a gentlemanly way and, uh, agree to those standards and comply to it. And, and that's what drives it.

And if, if we don't, um, we choose not to, um, there's a very little accountability or it's very hard to even find out that you're not ap uh, complying to those standards. So governance is hard. It really is, uh, especially when you're trying to define governance through a, a democratic process where everybody makes their decisions across these countries and are relying on everybody to, um, be honest, and not everybody is.

Guys, we're over time. I gotta be honest with you. I apologize, but it was an interesting conversation.

I wanted to let it go a bit. I think we could all agree at this. I think there's still a book to be written, or at least a few chapters in how AI governance is gonna take shape here and what its effects going to be in a global marketplace.

And it'll be up, uh, you know, the ethical people, as we said, act athlete, plea, the tricentis of the world will try to give people a sense of, Hey, we could test for this, or we can, you know, given transparency, we can show you that, you know, to, you know, be in the transparency that it's in compliance or what have you, test for it. But a lot of it's good, I think is the book is still yet to be written. So we're trying is I, I guess the, the right thing, right, is we're trying and, and this is still evolving.

Anyway, Martin, Tracy, oh, thank you guys for coming on our DevOps Unbound episode today. Mitchell, thank you as always for co-hosting. Thank you Tricentis, for partnering with us for these four plus years.

Now. We look forward to many more reminder for those folks out there. This what you just watched, for a prerecorded version of DevOps Bound every two or three versions.

We do a live audience, and you get to ask the questions, and you get to make the comments, and you get to participate in this discussion. And we love having you. So stay tuned for our next live round table of DevOps Unbound.

But until then, this is Alan Hummel for Techstrong Group. Have a great day, everyone. We'll see you soon.