Generative AI for DevSecOps – Techstrong Con 2023

Hey everybody. This is the panel should be very exciting our Tech strong and we are basically going to talk about generative Ai and deaf secaps or really anything but devops have circops. Whatever makes you pleasure.

We've got a really great panel today. I'm John Willis. I'm also known as Patrick glue.

Pretty easy to find on the internet. I just show not so probably show. Some what we got Tracy, why don't we start with you want you to introduce yourself?

Sure. Hello there Tracy Ben and I go by Trace software architect engineer a couple of decades doing that spend a lot of time focusing most recently helping the federal government and DOD, but I did plenty of tours with commercial my background like I said architect and engineer but I it's about digital transformation helping people understand different ways that they can apply the technology but also the cognitive aspects of it. How do we understand?

How do we change our culture or augment with these with these new capabilities? So really excited to chat with y'all today? Yeah, Tracy, I've heard you speak and you know, we've bumped into each other, but you've got a great radio voice.

Well, I'll have to tell you a story about recording voicemail systems when I was right out of college and how creepy how creepy it can be so I promise Dr. Alex Lawrence, I won't call you doctor Lawrence anymore. But I'll call you for hours from here.

But Dr. Knox Lawrence, would you introduce yourself? Yeah, after I got my PhD I made my kids call me Dr.

Lawrence flight 10 weeks and then I was good. So yeah, thanks for thanks for having me. It's fun to be here.

Yeah, I'm as you mentioned. I'm a professor at Weber State University in Utah, but that's not really I would say how I identify spent 25 years as an entrepreneur. It's still own an invest in an advise a handful of companies in the tech space and many of those are using generative Ai and chat GPT specifically on the front end.

So more of the consumer side is this application side, but I'm also I found out I'm an early adopter of teaching it in the classroom at the University. We've been doing this since since early December of last year, and so the teaching environment with students and it's been an interesting path. So thank For the opportunity to be here with you looking forward to it.

Great. Take a lot of fun talking about what's going on teaching generalize. So we've got a lot of good stuff there and Kevin wants to bring us home.

Hey, so Kevin bocheck from venify. I've spent my time much of it focusing on machine identity management how to app Services Bots all authenticate authorized with each other and I focus my time most of the day when I hear thinking about technology ecosystems and developer Community which generative AI, you know, really democratizes the opportunity of who is a developer or not. So super excited for us to talk about this.

All right. So Tracy, you know so, you know, you've had a long background software delivery stlc all that. What does this mean?

What is this sort of generative AI or gpt3 all the stuff? What does it mean to you what you think it means to your customers and Order? Boy, that's like a really really big question.

So it's a disruptor. It's it's an yet another disruptor if we track back before the rise of chat GPT before it became a household name. We'd already been looking at low-code no code we had been looking at ways a heck back in 2007.

I was hired to build Frameworks and platforms digital platforms for delete Consulting. So we've been trying to do things faster smarter for a while, but we've been hyper focused on the developer and what I think GPT does because it is so available to anybody is it has us all start to look across the breadth and the depth of the value stream right and begin to look at other ways that we can make it more efficient. Now there are lots of danger when it comes to this because an authoritative voice and sometimes you can get some really bad things out of it, but in general if we're open to exploring it together and understanding how we can Do that.

It can be a wonderful productivity boost. It really can be. so Kevin 1900 kind of jump with you because you know, you you had said during when we were talking before the session that you really been using copilot and so a couple things I I've got a couple of developers that have been working with and one of the things they really like so co-pilot was great.

Right and I want you in your words explain why how it's disruptive or what you think about it but hearing that the thing that the chat GPT is bringing is the error codes. So people are telling me that they can type in like hey this happened and they're getting phenomenal information. That would be much more.

So I think that combo sounds awful. Right, right. Yeah.

So I mean, I mean generative AI is is about improving the capabilities not only have I think the professional developers but of those who have never actually written code to and that can be understanding errors understanding what code is you know is does or what should I be writing now as professional Developers? you know a solution like GitHub copilot is you know, it's kind of like your wingman. I mean always the developers best buddy was control C control V.

I mean that's been it because it allows you to go faster and yeah co-pilot which is, you know, related model down the line of things like gpt3. Yeah, it allows you to get suggested code. On demand I can give it a prompt.

I want it to write some type of ansible code. I've never written anything else, but it will give me actually, you know, some working ansible code and so for both the professional developer, but then the Casual developer it's powerful but then even just being able to go then to something like Chachi PT or like in bing now, I've got the preview version of Bing which it's chechi PT right there with my searches. I can ask it to write code and that really opens up.

I think the the potential for how we can drive speed and what we would call devops, but I'm not sure what that might be anymore in this new world. But of course though it does bring some risks because Trace said it's very authoritative anything that writes code for you is pretty authoritative, but I'll say it's oftentimes either really lousy or or wrong. That could introduce mm-hmm a whole bunch of actually not good, but but will introduce a whole bunch of new ones whether it's chance GPT or something else.

They're just models and models are only as good as the data that's underneath it. So there's an entire growing area of study probably needs to grow faster about model poisoning. You're right a developer who doesn't know a language and you say I want you to in the use functional rights and use my regular functional speech.

And say I need a piece of code that does this it's being generated for them in a helpful way. We hope however as we become more dependent upon it there are some risks along the way right while it can be fantastic as a pair programming a simulated pair programmer for me. Am I going to learn less?

Am I going to become more dependent upon it am I going to understand what happens when the code doesn't work? I can tell you that. I've used GPT to generate some code.

Sometimes it works. Sometimes it doesn't and if I don't have that inherent understanding so that I love it love it as an educational tool. I like it as a productivity helper.

I'm concerned about the dependency if people don't have the basics right? We are going to have to be very specific about continual upskilling not enough to just give that casual or social developer on GPT or any other tool. It's yeah, how do you teach the trades around that?

Yeah, and obviously for the audience a professional developers, we're not going away and and it does then yeah the the risk around that democratization which is in a devops world. I can ask, you know, give me an ansible code to automate Enough Five load balancer with the TLs key and certificate it definitely will give me it now. Is it right or wrong?

It might be an also actually, you know, I can then execute that it's something that I think the audience here may not have considered as the risk the risk might be. Yeah that we're getting some poisoning or that you know now becomes malicious, but I also think we have to risk is now that we've got all these more developers that we might not have been thinking about now that are dangerous and things like Powershell. It's a whole new, you know set of risks because this question though.

Um, So you say you know as professional developers, you're not going away. I was at plural site recently, you know, very popular. Obviously you everybody here knows and we were talking pretty deeply about this concept of like I related it to the population declines in Asia.

So you've got this this model that shows that you know over time. There's gonna be a problem where done the people are being born to replace people that are dying right? And so here's my my thought my question if if these entry level one coming out of school developers aren't really needed which I'm telling you from the business perspective.

A lot of companies are going hmm. I don't know if we need to hire those people. How do you how do you how do you move your way up the chain to become how are there enough coming into the system like the population question so that they can get the learnings move and become professional.

I think the world changes right just one, you know, there's this notion of Chairman's Paradox, right? Which means basically it is a sort of paradox. We think that something's going to replace and actually sort of expands the you know, we've seen this and everything like, you know, we like even automation tracing me and you go back right automation.

Everybody was scared to death automation was going to be put everybody out of work and what happened we all had higher level meta Frameworks to deal with so I think that that I was sort of until the day that we're running the foundation Trilogy up to that point. I think we're okay on that. You know, I think it's just the jobs changed the but the other thing I wanted to go back to like there's sort of a nothing changes, you know, poisoned models or toxic, you know, I mean the before the you know co-pilot now chat GPT basically way most people configured or found out how to code if they didn't have to do something is they Google and found stack over Trace that's like overflow, right?

And so no different. Yeah, there were some of the worst breaches now, we're copy and pasting permission right you so I think it goes back. the one thing I learned heavily, you know, I thought When I friend of mine interest is to call Jasper was one of the first commercial implementations of open AI CPT 3 and I said, oh my goodness.

This thing's gonna write blog articles for me. This is gonna be great and I just started seeing like all this sort of like nonsensey stuff and what I've learned and what I think we have to do is educate people to realize that like, if you said to somebody today, would you just copy and paste something into production from stock? Overflow most intelligent people would say no.

No, I would never do that well, but then treat gpt3 the same way right like do your diligence, but it can it can enhance your code. It can enhance your research for me. It's like you had mentioned perplexity Tracy.

I love that tool it. It allows me to do research while I'm on the phone in a call with somebody. It's just it's incredible.

That they that I was just gonna say the challenge and both the opportunities the volume of the number of developers. Now at least those who are danger is with code increases in ways that we may not yet as cybersecurity professionals to have controls where we might have controls but we may not just have Implement because you're right. Yeah, I could always copy down, you know malicious code but it's now though the the person in the business that might now be copying that malicious code wouldn't have been the person before they're not professional developers.

And so they actually wouldn't know the difference we have to be careful about this is where we get into these these casts of developers. So you got this this is when do you become a professional developer? Is it?

Yeah for your degree and computer science. I can argue that I know some who are not developers. So maybe it's not that is it that they have a certificate from a boot camp maybe so I think it's it's much more difficult for us to Time and who's a developer if I think back to low-code no code as I said earlier, I was brought in to delete a decade and a half ago to help build Frameworks and help jumpstart things to generate code the low-code no code explosion that's happening right now.

It's been around for a while. It's not new but it's getting much better. I see what we're talking about today as another step on that journey to better democratize.

We're always going to have to have people understand what's happening. That's what I think is with the biggest Gap is going to be we're going to be able to have different developers of different types. We're going to need to continuously train them, right?

This is no longer you learn Java and for 25 years you do job and you never learn anything else. We have to get comfortable being uncomfortable and but it's I'm worried about the the middle tier. One of the things I'm hearing a lot about is using GPT or some of the assisted Technologies to take the place of a tech lead so I can submit my code.

It will be scanned for me. I will be told what the code quality is essentially doing a code review. There's missing educational moments.

There's a missing discussion of the trade-offs. So that human element. I don't think goes away but it will in times when people are having financial difficulties with their companies.

I think you're gonna see some gnashing of teeth where you're missing the middle not as much the developers, but kind of those those guides those tech leads in the middle. It will create risks. I mean the reason why we have of course people still in business.

I mean most recent events in the financial system is because it creates accountability. I mean ultimately you can make really bad decisions, but then there are people that are accountable and that's the the thing in I know Alex has a lot of experience with elect, you know, electric vehicles. I mean still the idea of a self-driving car.

Is it still you have a human there who is accountable for the decision-making of a machine and I think that's yes, there will be some bad decisions about the priorities of who you need not and there will be some impact of that probably vulnerabilities or otherwise created because yeah, you didn't have that layer and that will that will take some mistakes. Unfortunately, I think start to even out right we haven't been I mean like like I was think about judging the future and And forgetting the past like we've got a lot of complex infrastructure that has been built and designed by really bad decisions that we deal with every day. So and again back to things never change.

We just have to dab and I love what you said Tracy like to we need to sort of unlearn I think that's gonna be really important. Actually. I'm sure you have a lot to say about all this.

Yeah. There's I mean, I love listening though, too. I mean a couple things popped into my head.

One of them is you know, I think we're really talking about where things are today. You know, it doesn't do this it does this poorly I get this, you know, this this result that's confident, but Incorrect and you know for me coming primarily from you know, entrepreneurship and and business development and seeing all these companies, you know, I just I just would say look at where the money is going. I mean Microsoft just announced they put several hundred million dollars into tens of thousands of Nvidia Graphics chips to build a super computer just to support open AI so so when you think about that you think about The minds of Microsoft and that kind of money and then you multiply that by all these other I mean, I think you're gonna see a trillion dollars of venture capital that's going to go into into versions of this over the next couple years with the smartest Minds in the world are arguably and so then I say, what do you think that how do you think it does with the things we're talking about a year from now or two years from now and and I would argue that it's going to be orders of magnitude better.

And so so do our concerns go down when that's the case the second thing that and I'll probably should pause. I'll just say one more thing because I want to hear when you guys think you know, I I aspire to you know, all the things that that you mentioned about, you know concerns about learning Tracy and concerns about skipping over stuff and things like that. But I also know that the reality is a lot of businesses don't care.

And and they see a way to maximize profits. They see a way to shrink headcount. They see a way whether it's whether they're fully accurate or not, and I'm seeing them and being involved in board meetings and talking to Founders.

They're already shrinking and considering shrinking teams. And this is not a doomsday thing. I'm just saying that that there's gonna be a lot of companies that do that and I think there's gonna be a place in the not too distant future where I've kind of coined it as this made in America stamp where you're gonna have a subset of folks that want to do business with companies and are proudly declaring as a company that they don't replace, you know humans for parts of portions of the process.

I think that's going to be a smaller. I think it's gonna be like made in America where you're not going to have these massive groups of people that only buy from companies there and only say we only produce here, but but anyway, those are just some questions I'd have for the group and and really curious to know if what your opinions are of those those ideas. I don't think our concerns go away.

I think they change. It's the wonderful Consulting phrase. We always say it depends right context is everything.

So if we look at the context and we understand the trades, I agree with you that they're going to be some business and they should go directly for it just like Cloud first businesses or leveraging a SAS a full-on and being able to create features create goodness create value as quickly as possible. So I believe that that is going to happen with my defense hat on thinking about things. I'm constantly wondering how could a nefarious actor take advantage of this.

So that's because of my frame of reference that I'm balancing those things out. I don't think the concerns go away they change there's so much goodness that comes from applying AI to the full sdlc. It's not only for the developer and generating the code for the developer.

Right? If I look at automated test generation and how much smarter that can be if I can look at how I can understand. A profile of the data that's being leveraged by the systems so that I can better generate the test data for anything that's happening pre-production.

Right and testing envelope cases that normally are forgotten. There's just so much goodness. I mean, I look across the stlc and I see in my mind.

I have an infographic that I haven't tried that essentially shows it everywhere because I believe it can help with most everything it is going to come down to applying it appropriately understanding the trades and some businesses are not going to have the funding to understand all that you're gonna see some that do fantastically well and we're gonna be telling some of those dumpster fire stories as well in a couple years or a couple of months, right? What do you think Kevin? Yeah, I mean this is going to so I like Alex's question because we have to think, you know a few years ahead.

I mean the the applications of course in business are going to be applied across decision making not just developers but in finance in Operations, I mean they've already started to be and they will of course will create new sets of problems too. That will be able to be solved faster easier. And yeah businesses will be able to prioritize where they they want to spend.

The one thing though. Of course out of all these models is that if you only rely on the same model to get your answers, then everyone's going to start to look alike, which means that no one really should be in business and no one should be investing. so that you know for me too is where you know, it's not going to be just one model, but there will be many and as well.

What's the what's the value in business really going ahead because it's the data by which you can train. Models on I think this is a concern whether you're a software developer and I'm thinking about trying to improve my you know, software development like cycle or if I'm in finance and I'm trying to optimize around pricing. Who has the data gets the best models presumably and presumably as well the large companies the large businesses that have the most data may get the best models.

So new startups trying to break in. It may actually be more difficult. I'm and that's something to the Future that that you know, I'm concerned about but knowing all things maybe they're new ways that AI actually generative models will create data that actually can be used better for training than real data.

I don't know yet. But that's one concern though. Is that that yeah who owns the date and gets to train it.

We're already there. Right? I mean original deal with Microsoft was to close source to model.

Now. There's all these discussions about other models, but like where I mean, you know deep mind. I mean, you know, I mean, here's the thing Google's been trying to do this all the way back to Alpha go right and they beat best go player, but the way they didn't figure out how to commoditize it right like I mean it, you know, if you try to take a tensorflow course, right?

It's just it's grueling. It's you know, I mean it for the non-pure data scientists. It's just painful, you know, so, you know today I in some ways I think Microsoft Google Google you know what Google did with with the browser right?

Like they just got so far ahead that it was nothing else existed. I hope I'm dead wrong on this but there's a chance to GPT. Is you know becomes this sort of that standard it becomes the Google of AI?

And then in that case to your point Kevin the models already own, you know, I mean, I mean, they're adding sort of these, you know, like you can train the model but like it's not really they open source model. I mean gp2 it was an open model gp3 party agreement was to the first billion dollar investment 2019 was one that they'd run on his or pretty clever move and two that they close source to model and the model can only be driven by an API now I've got dark GPT. I've got a couple of tools and you know sort of whole feedback mechanism, but that isn't Giving me the source of the model.

To be able to construct and now fingers crossed some of these rival ones that are coming out. But where were they before GPT? And you know and then like when you know, my aunts start using chat GPT who never worked an it read romance novels pretty much all their life are now telling me how awesome these things are for the things that they're doing.

I'm like, I don't know, you know, I don't the only thing though of course is the data that these are being trained and it's not what's most I would say in many ways most valuable because what makes you know businesses competitive today is the day to that's not public. It's what we used to call and management, you know, asymmetric information and but Yeah, that's the thing is these big you know larger companies have more of that data. That's not private and then the ability to train on those and I get that but right now if we're talking about GPT, right that model was closed source.

So the only thing coming in here is whatever Microsoft decides into it specifically for challenge GPT. You're right in that ecosystem. Can we take agpt, right?

So if there's there's the model and then there's the implementation of the model hosted and made available at post training but the algorithms for that are not our non proprietary. 0 or gt35. I don't think four is, you know, the open AI has said that's a ways out but the large language models do exist and to Kevin's Point like this is kind of a balancing piece.

They're going to be those who have enough money who are going to be able to stand up and Leverage The Science the algorithms that others have stood up and they are going to be able to train it right they're going to be able to pay to have that model trained with quality data. So you're going to see those with enough money to make that happen. You're also going to see an entire huge part of the population that is leveraging what's available for free or for premium or SS?

So, I think we're going to see some bifurcation from that perspective. We're going to see those who are using chat GPT, or maybe they pay the what is it now 20 bucks a month. You have chat GPT Pro.

I think you're gonna continue to see that and you'll see folks like Nick Chalan who was the chief software officer for the Air Force now retired? Who has created something called ask Sage and it is being trained with governments open open available data, but with government models government documents government data, non nothing as far as I know that is Top Secret in nature, but he's curating that specific to writing Contracting language for example, understanding terms and conditions and those types of things. Same algorithms.

He didn't invent the algorithms. He saw a really fantastic opportunity got with whoever he needed for for his hosting provider and started that run-up and I think the one thing is it relates to spec to devops and deaf secops as we're moving to a cloud first world. The the question also comes are we giving our Cloud providers a whole bunch of data and operations that they get to of course learn from and then, you know, maybe sell back to us from train models around devops.

That's a that's a question. I know that's that's been been coming up because Google Microsoft Amazon. Yes, they may not know down to the compute that I'm running necessarily but they know a lot of what we are are running and how we're doing it.

Well, we've already given that data up right? I mean, oh I like I'm Amazon knows everything about every yes, but I don't know if the audience has or the executives might have been aware of what they're giving that anywhere. It's a reason why I won't yeah Walmart for years decided they weren't gonna run on on Amazon There Was You Know Target for years, you know, I mean, I think they'll give an up now, but for years a lot of those were trying to use alternative clouds just because they knew as much as Bezos would say that they don't look on the covers.

I mean they, you know, all of your Logistics all that stuff could be themed. I think it goes back to the other thing I think is like I still think I'm pessimistic in one way pessimistic in that like I hope you're right Tracy these I mean, there are a lot of smart people. There's no reason why a bunch of people who are incredibly smarter than however be can take these and do these things at scale the way you My fear is that you know, and maybe try just my paranoia which I hope I'm hoping you're right that that that Microsoft has googled Google Google and then the only other thing for the data Kevin I think is there's some really the oxide people, you know, James Hamilton and and then like they're really coming up with sort of coming out at a Facebook, you know open data center stuff.

It's just incredible. I just saw Brian Cantrell gave a presentation in La scale and you know of what they're building there, you know for be able to run your own like massive compute like the way Google like commodity, you know, I'm a bank and you're redefining switches redefine the way Iraq looks so that may play into this game where like to to Alex's point is that there's a trillion investment may see some large non-cloud infrastructure being built to solve these problems Alex Brian follow the money. I mean at the end of the day, The Internet Is for p*** and pictures of kitty cats right and being driven by Gaming so heck yeah follow the money I think you see some interesting way talking about you know You know Innovations in white people are going to use it.

I think you see some really highly specialized very narrow uses. They're going to be marketed. So for example, I've seen some stuff in the legal space where they're like, it's really important that we be correct.

It's really important that we be accurate and so they're they're saying hey, that's all we're developing for. That's all we're going to be using these models for and and I think you know the argument community that means they're gonna get better faster at it. They're gonna create a market for it lawyers are gonna be willing to pay for it.

And that's one of the that's one of the industries I'd be very worried about that that you know, other than people in court, although did you see is it pay it now that that got approval to have ai represent their offended in court for a traffic ticket and then all the law firms threaten to sue them to high hill and they so they pulled it back. Yeah. Well Alex, I'd be interested in your thoughts on not Beyond financial and looking at Medical beyond the legal medical industry right now.

There's tremendous. They're already websites that insurance. Companies that large providers have and they tell your doctors you're going to use this in throughout the process of diagnosing.

I can see with my rose colored glasses on I can see a beautiful world where I'm helping those doctors providing more information to them. With my nefarious actor hat on. I also think about he who owns the model right?

It's like he who owns the data he who you know is in power writes history. It would be interesting to understand your perspective on you know, what's gonna happen in the medical industry from your Vantage Point as an entrepreneur with applying these kinds of Technologies. Yeah.

I mean, we I think everybody saw just recently that the breast cancer diagnosis that AI I don't know if you saw that just in the last like few days it predicted some breast cancer diagnosis before any other method and they put in front of the panel of doctors and nobody nobody caught it and and I did so it's interesting you to ask that I did an interview recently and they said, you know, what a perfect world. If you can see anything that comes out of this what would be the best place the best space the best, you know, and I said the medical field I mean for me that is that is where I think we can all benefit and and I The Optimist in me John. That that you know, it'll be a lot like cancer research and other things where everybody is willing to share what they're learning and what they're doing because they want to they want to defeat cancer collectively and so I'm I'm hopeful that and I'm all very very confident because there are hundreds and hundreds of millions of dollars that have investments in the medical space and AI just in the last three weeks that I've read that yeah Innovation there quickly that okay.

Wonderful. I heard the chief data scientists of nivity like five years talk about a pathology, you know, an AI driven pathology detection. Yeah, then it and I get I hate to be a paranoid but then I get scared about what's going on with some of the drugs and the captures of BIOS.

So imagine the first time somebody patents. Yeah, AI discovery of a pathology and patents it you know, so anyway, I mean it's like everything else. I think we've talked about there's a lot of positives and there's a lot of things we need to keep our our guard up anybody want we're getting closer everybody wanted to talk about or you know final thoughts.

And then how do people reach you Tracy you want to gosh final thoughts Lots of them, I guess the most probably the most important thing is to be open to it, but pragmatic, that's the and don't jump into feet without looking, but don't be afraid right. So, I don't want people to be at either end of the spectrum. I want them to kind of walk the middle line with this if folks are interested in reaching out and having a conversation.

You can reach me through a LinkedIn or you can hit me directly Trace Tracy at Tracy Ben and Dot Tech and I be glad to to connect with anybody and we can carry on this conversation. Great Alex. Yeah, I mean I something that Kevin said earlier I thought was Going to be really interesting in the very near term.

So as you see, you know, almost immediate adoption by things like Gmail where it's going to be predictive and respond and stuff what happens when two email accounts are just talking to each other and you're you know, what what decisions get made what progress get made you just all the sudden have a meeting you didn't know about or did you just approve a budget item and stuff? And so I'm I'm actually a little more right side of the line that I'm talking to people in this community and saying I would I would just dive into the deep end quickly and there's an opportunity to get ahead There's an opportunity to distance yourselves in terms of expertise and there's really an opportunity to you know, do some things that we don't I'm really excited for two years from now. I think it's good this conversation is going to be completely different in in two years.

I wish I knew how it would be different. I think we need to set up a date now for a year from now and for two years from now, like I think we should we should meet up in a year and then in two years and have this same conversation. Will be dramatically different.

Yeah, that would be that would be really cool. And I you know same thing. I'm on LinkedIn.

That's probably the easiest way to find me. I think my LinkedIn name is actually Alex Lawrence AI so that's that's you know, my my name was taken so I tried to do something quick. So yeah, I'd love to continue the conversation.

Kevin bring us home. Well and Alice's point, you know, we've already been living in that world. We already today have machines talking machines making decisions about transactions or diagnosis is already or what's being paid or not.

So we're already there. I am actually really really excited about the future because when you look at children, when you look at how they interact with generative models, they get excited and they see things that we don't so that's the future. There's all we're we've got skatomas blind spots that we can't see and I think it's only going to create new business opportunity.

I would say though your data it's the currency and that's your opportunity in business because that's what you're that's what's really going to differentiate you and as you use these capabilities and then second it's a democratized world around developers. They're more developers. So you're cyber security teams.

Just make sure that controls you have about code just don't think about the professional develop. first but think about Joe or Sally who is never touched code before but now can run Powershell scripts. We've got the capability.

So just think about that, but it's exciting. I just heard you both say that you wanted to be pragmatic but jump into the deep end like hey, you're driving home my point. John I'm gonna assume that you're gonna leave us with the last Golden Nugget.

Oh, I don't know. I mean, I think it was great panel. I think it was a good combination of a bunch of different thoughts and Yeah, I think you know I think you know supposedly gpt4 is dropping.

I don't know who is this week or next week. Is it this week? Yes would even that's gonna look like right.

So yeah, you know, I mean, I agree video Yeah, yeah. No, so oh, yeah, that's right the video. I want one to create cartoons.

I want my GPT that got what image see the comic book. That was created. Just yeah, really really cool.

Oh, you look good. That's what we're in Iowa I want to do is write everything in cartoons, you know, my books not just kidding but accepted their John challenge. No, it's it's been a fantastic tool for research.

I I you know, that is pessimistic as I sound I am optimistic that these tools used right will raise the IQ of humans, and I think that's ultimately going to be a great thing. So, all right everybody. Well really enjoyed this job great pound.

So, thank you. Thanks guys, make sure it's on your calendars next year. I'm gonna hold you all accountable now we're connected.

That's good to me. Hey, thanks guys.