96. Don’t Give Your AI Agent the Keys to Your Business – Tech Field Day Podcast

The bleeding edge of AI is a agentic AI and a agentic AI that can learn new skills, grab new skills off the internet, maybe install malware off the internet as a new skill. Join me on the Tech Field Day podcast as we find out why that might not be a good thing. Welcome to the Tech Field Day podcast, where we bring together a group of it technical experts to discuss a single idea about some key concept in the industry.

This podcast features a variety of perspectives from members of the Tech Field Day delegate community, and it's often recorded in association. One of one of our events, tech Field Day is a part of the Futureum Group, and this podcast is also published on our sister company Site Techstrong tv. On this episode, nope, shouldn't hand your AI agents the keys to your whole business or your whole life.

But before we get into the discussion, let's meet who's on the panel today. It's going on, everyone Super excited to be here. Gerard Kalina.

Uh, I'm the founder of Tech House five seven oh. I am also a network and security engineer for Aqueduct Technologies. You could find me at g Kalina on Twitter, LinkedIn, TikTok, and wherever books are sold.

Welcome everybody. Also happy to be here. Uh, name is Oliver Cybersecurity executive former Deputy CSO Ian Life.

Um, you can find me also on LinkedIn, my primary pages, as well as other chat forms across the IT and I security spaces. And I'm Alistair Cook. I'm an event lead here at Tick Field Day, and also one of the co-hosts of the Tick Field Day rundown.

And one of the topics we covered recently on the rundown was Malt Bot or clawed bot, or Claw Bot, or whatever the heck it's called now, which was an awesome AI assistant that could autonomously go out and learn new skills and be helpful in all parts of your life. Only It became very unhelpful when those skills actually were spyware of various sorts, various types of malware ended up on the repository of skills and very quickly people decided that maybe they didn't want random software taking control of their entire life. And for me, this sits in a, in a wider context around governance of the pace of governance of AI and ag agentic ai as we try and match the pace of possible innovation by applying the things that are being built for AG agentic ai and how the heck do we make sure that these new technologies don't end up being a real business risk business danger for us.

Um, the Malt Bot one particularly showed personal risk, but it could absolutely have translated into a, a very large business risk if you've been using these, uh, interesting skills that malt bot would acquire in your own business. This really does seem like a, a sort of big red flag around how we're approaching using Agent agentic AI and possibly a, a lot of immaturity in our use. I'm gonna just jump right in, don't get me wrong.

And I've been like, I've been just sitting on this for a while, so I'm super again, excited to be here. Talk about it. It's two parts to this.

Number one, I think Claude bot, MBOT, whatever you want to call it, whatever name or new adaptation it will be in the next few weeks, months, or years. I think it's a great tool. I think like everything else AI needs to have, it's such a powerful tool and it can either be leveraged in the appropriate way or not if it gets into wrong hands.

The solution itself I think is really cool, right? And it sounds when you say very broadly, like, Hey, you could install this app or install this program and it starts ingesting all of your data, emails, text, whatever the case may be. And it will start helping you in life and make your life easy, the things you're doing, scheduling things, whatever the case may be, that's super cool.

But you need to have security perimeters in place. You need to have governance, and you above all else need to really understand it needs to be fully vetted. And I feel tested because that's just it.

Anytime you have any type of baseline code or any type of just a basic text file configuration, there's no perimeters. There's no security. And as an IT professional, as a security professional doing this for many years and still doing it, it's, it's terrifying because it's super cool, but it's not leveraged and it's not, it's not vetted properly.

And because of that, you're deliberately exposing risk, whether you think you are or not. It goes back to the old, you know, don't click on that email. You could kind of tell what the difference is between a phishing email.

And that's why we have third party companies that run phishing campaigns and do these types of things. 'cause some love them, but some end users just, they make those mistakes that's gonna happen. But something like this is very scary.

This is like next level Avengers level threat where it's not just clicking on an email anymore, you're putting your machine or your server or your applicator, your whatever. You're saying, Hey, run this, which I don't know why you'd run something without really fully understanding or vetting it. Same thing with, hey, this is this command, let's pop it into the command line and run it.

That's a no-no. Why would you do that? Um, so again, the solution itself phenomenal, I think vetted and, and framed and contained properly could be of use just going, you know, freestyle and Wild West in it and, and, and just, Hey, clicking on this and letting it go and letting it talk to all your stuff and then putting it out there publicly, globally.

No, that's, it's terrifying. Uh, it's just, we gotta do it better. That's, that's, that's my 2 cents to start.

It's just gotta be done better. Yeah, and I think it, there's a conversation that I often have not only with organizations, but individuals on the topic of agency. And I think as you begin to look not only at your environment that that, that you work within, there's spaces in which you operate within that could be either on prem, on, on your location or even at home.

Hey, there's places in your house that you don't want your kids in because maybe that's things that maybe shouldn't be appropriate. You need to take the same approach with the information that you're storing and your using to connect to places like this beautiful internet that we have. And so I think there's, there's this nice parallel of making life simplistic, making it more manageable, and then actually introducing new risks that probably doesn't, I think there's this, and we talk about the FOMO side of things with, with, with ai, and I think there definitely is an element of wanting to be part of the crowd and not wanting to be the last one, you know, to hop on the ship.

That being said, I think there's also an element of risk that also comes into the equation. We've had this thing with open source and somewhere between the first iteration of the Mac and now open source has all of this stuff become the best thing ever. And I think there's a lot of great things that come from open source, but what I'm finding is that we're not doing our due diligence.

And in that absence of that governance and the absence of just that moment to ask yourself what exactly I'm gonna give it this thing agency two, we're having critical mistakes that cost us, you know, so much time that I think it just allows us, we need a minute just to kind of reset and understand what are we actually trying to solve with some of the conversations. And, and just to piggyback off that real quick, 'cause you made a great point, Andre, like from what I even read about this not too long ago, there was like 30,000 instances that were just like, Hey, we're just blowing open the doors and here's all of your data. And here, now, from what I also understand, there was two ways to approach this.

Like the LLMs would find open source based projects or just a ton of different, metaphorically speaking worm holes to start pulling this data in. Now, it's one thing if we're, you know, Joe Schmo and hey, you have, you know, basic emails like your kids' photos, like personal stuff, that's one thing. But what about like, high level executive CISOs, people who are like, Hey, go ahead and start talking all my stuff.

What if there's PDFs, blueprints, you know, NDA classified documents? Like that's when it starts getting even more terrifying. And it goes back to, it doesn't matter where you're at, if you're, you know, just a dad, multiple kids, or you're, you know, a senior vice president running a high level operation or a small medium, very large enterprise business, you're just telling it to talk to your stuff, pull in that data.

There's a ton of, of, of, of, of, of, what's the word I'm looking for? There's a ton of, uh, there's just a ton of open polls and points and vulnerability areas of weakness to just penetrate and take this data. So again, it's just, it's, we gotta be better, right?

Like I, you know, there's gotta be some type of governance or some type of, uh, you know, high level group of technical analysts, specialists who can really leverage. Like, and I think that should be for all ai, I think instead of just having the tool, right? Like it started with chat GPT and we're having fun, we're making images, but now it's evolved and it's grown into something so much more.

So we need to have a better way to control it. That's, that's gotta happen. And I think this is, this is not new.

Let, let's be very clear. When docker containers were new, you'd pull something from Docker hub with no idea what was within it. And the resolution wasn't that the end customer went and validated every single container that they actually pulled.

It was that governance was built into the container repositories. And so there'd be a knowledge of who was releasing this content. Is this trustworthy content or is this, um, less trustworthy?

And, you know, there's a continuum in trust and there's also a continuum in business risk. And it's that crossover point in finding where your business fits for this use case and this level of risk. But there was no governance at all in the skills that mbot could acquire.

And worse than that, it was just a description of how to acquire is what was being stored. And so it was a lot of them, it was just go and pull this script from this website and pipe it into your script interpreter and just randomly install software. And although you could tell tbo, don't do that for me, let me go and do the governance front and check on those things, that's not what customers ended up doing.

And in the same way that as they're using things like, uh, coding assistance, uh, some of these coding assistance, there's, there's literally a switch of yolo. Uh, you know, I'm prepared to take all of the risks there are in order for you to take all the work off me. Um, that kind of both the education of people to not take all of those risks on, particularly inside a business context, but also the need for the governance to actually take away the possibility of those risks occur.

And apart, I can see for the, uh, the chap who developed the Claude bot, whatever we're calling it, uh, is towards actually offering a managed service of a governed set of skills that are absolutely certain and not, or not documented. This is the information that we'll wanna access. This is the, uh, components that are being deployed out.

This is the risk that you're taking when you're using it. Having that explicit governance, I can absolutely then see the stage where this is a product that I use in enterprise organizations where you wrap a lot of governance around it, right? Because as you say, Gerard, when it's just my, me at home with my personal emails, there's, there's a risk, there's a risk to me around particularly identity theft.

But when it's the, uh, the the sort of core of my large enterprise business, uh, particularly if I'm an executive, then yeah, there's a much greater, uh, risk of damage for these things. And so there's more value in that governance. There's One thing I wanted to add to that.

Again, just, just learning about this, and I don't remember. So with the, with the Claude bottom mold, but with the installer, and there was a, a GitHub repository there was that, I wish I remember the, the exact file, and they, normally I do, and he says this, so I apologize, but it was like GitHub, whatever, whatever was built into it to, how do I put it, was almost not like full governance, but when you do run the installer and or if you do pull this directly from GitHub, it would block specific file types and paths using like hash values, so it wouldn't just pull all your stuff. And I don't know who or what somebody removed it outta the repository, so that's why it gives you like, just full open access to just get whatever you want.

There was like almost an essence, like a safeguard built into it. Um, and, and it's like somebody took it out. It's like, why?

You know, like that's a whole other question for a whole other, but why, because like me, the the actual executable, or the installer, I'm not sure, but the GitHub had a, had a safeguard there, so it's like, hey, this is automatically gonna run when you start pulling from the repo and it's gonna let you pull like pictures this, that, the third, but it's not gonna touch, you know, whatever. But somebody took, took it out and they're just like, no, it's just gotta let it have access to everything. Like, so I know, while it's not full blown governance, but it was some type of, you know, like preventer in place and somebody just said, nah, we don't need it.

I I that I'm interested in, and I'm very curious as to why that was taken out. Yeah. That I, I, I think the biggest thing when we think about this space that we, in that we're in right now, and I think also you brought it up before, I mean, when you were doing dock containers, you know, way back when we first started this, there was a, the, the barrier entry was pretty high.

You had to be somewhat technical. You had to know kind of how to, how to operate within the space. And I think what we're seeing right now in AI is the bar is really low.

They make it fairly easy for you to get in, to get going to, to, to obviously to to be, you know, really viable in the space very quickly. And I think that's intentional, right? Because we need adoption for AI for a multitude of different reasons.

Ask any vendor, right? AI is sprinkled on it in some way, shape, or form. And so I think driving that adoption with the broader community allows the vendors to have a better place in society, which allows 'em to sell more product, which is a good thing.

And there's a good and bad with that, but I think at the same time, we, we have to remember that threat actors are also using the same avenues to further their, to further their ventures as well. And so I think the governance piece is big, but I think now, if we're going to allow this level of access, allow this level of, um, what I would say in control into environments as users, generally speaking, we've got to up the bar for understanding what security, and I know oftentimes we call the the team of no, right? But I think collectively, this is the space that we're going to operate in.

We at least have to be maybe the, the team of, maybe I have a question versus just downloading. I think that's that thing that's gotta switch now with us as we begin to connect all these currently disconnected systems into our business processes and applications To finish up. Just test, you know, like it's pretty common anytime, like when docker containers were first released, right?

Like it's a pretty exciting way to transport apps information and data. But again, like everything else, the attackers and nine times outta 10 get ahold of it, first start ripping it apart, and they start figuring out how the hell can we exploit this and how can we take advantage? And then they've already got a few, you know, steps ahead.

I just feel like, not saying that our teams aren't currently doing this and you know, the countries, but I just feel like maybe if we can just somehow some way get a better track record of getting ahead of it, right? Like, this tool is great, but let's really kind of spend a few days, spend a few weeks just ripping it apart, looking at it from both sides, looking how to actually, you know, pen test it, looking to see if we could break it, where are the vulnerability points, where is this that before releasing it to the general public, but it's like you guys said, it's just, it's getting released, everybody's clicking it, and then they're just, Hey, you know, like it's the wild west and we're just doing all these things and we're not really thinking long term and what those consequences could be. I, I think there's a combined thing in there that you've, you've both hit.

One is ease of access to it. So, uh, being able to very quickly deploy just a sort of skeleton and say, go and do things for me, that's a desirable thing. That's something that is, is really positive in what we're seeing.

I think that's, um, just ease of access and that a normal person, not, probably not your, you know, my father-in-law at this stage, but you know, somebody who, who is not an IT specialist is getting their hands on these things quickly because they're well publicized and they're easy to get into. And I think that's where it starts to fall down, right? Because if it was any one of the three of us deploying this thing, we'd be looking at it fairly closely, would be deploying it inside an isolated environment for our initial testing.

We'd be giving it limited access to a very strictly controlled set of data. Even if we did then let it go wild with installing its own skills, we would be much slower to, to trust it with everything. But I think all of the hype that we've seen with, uh, chat GPT and all of the, uh, generative AI tools, generating images, generating songs, we think as, as normal consumers think that AI is the solution to ev every problem.

And that AI is very trustworthy. We from the inside of the industry are quite the opposite of being concerned about it, about this, this whole idea of agent AI taking autonomous action. This is one of the things that I saw at AI infrastructure fields, that whenever people were talking about agents, they often were talking about guidance for a human who's gonna take action rather than fully autonomous action.

And we definitely had conversations about building trust before you'd ever wanna have autonomous action. Yet typical end user doesn't care about that. They wants, they wants the action real fast.

Fire fire on all cylinders and like, it's just, it's just like beating the dead horse. It's just, it's, it sucks because again, it is so cool. And I understand, and that's the cool thing.

Now, 2026 is AI and leveraging agent AI and all these different solutions. It's baked into everything at every turn and corner you're going. But again, it goes back to the fundamentals, like working on the help desk.

Like, it doesn't matter how far I can go back in my career, like don't click on this, don't just, you know what I mean, having it in, like you said, also having it in a controlled environment. How can you mess around with something if you don't understand it? That's all, that's all it is.

And the human element of it, it's like we say it with love, but the layer eight issue, right? Like, oh, we're just gonna click on it and, oh, what's this? Lets just throw it on here, let's throw it on our main data, let's throw it on our, you know, our dc It's like, no.

Like, no, but you know, so is, so is the cool kid, which is ai. So, Yeah, and I think, listen, that a lot of this is driven by social media influencers and, and I, and I make the designation because they're not first and foremost thinking about the things that we're talking about right now, right? You can go and watch any video, any, any podcast on how to do the next cool thing, right?

The reality is that human in the loop is not sexy, right? It's kind of, it actually, it doesn't, it doesn't make for clicks, right? But if I can automate this process that kicks off five different things, starts my email and, and, and, and orders me Uber, right?

That's the next best thing that is that wow factor that we're seeking or question the challenges of what's happening behind the wow factor. It's the uhoh factor, right? And so when we find out those things and have to have those conversations or understanding, I think we're having some of those challenges.

And then it's, you know, I I I go back and forth and it's, you know, a common thing, common people, you know, even with your kids these days, right? The idea of, of the work to get the outcome, right? It's, it's too much effort.

I just want it to be there. I just want it to happen like it should, right? And that's kind of the space that we're operating in, and that's this kind of demon that we're dealing with.

And so I think there's this nice, um, common ground that we can gain as professionals to say, Hey, there, there's a subset of things that I think makes sense for us to automate, but as I'm talking with any person, any, any organization, my question is, well, and you've heard before, what are we trying to solve for? Right? And I think most oftentimes it's the absence of that, that outcome, the absence of that objective that allows us to have a minute to reassess and go through the process.

But when you're just clicking and you're just adding for the purpose of doing it, there's no guardrails. There's very little incentive to think about the risk, the outcomes, the other things. And that I think is where the, the, the gotcha is for us in this space.

So I wanna bring up a completely sort of different perspective on this, in that this is the first really big consumer, you know, influencer led story that has brought up AI security. Now within the industry, we've talked about AI security, looking after you data, those kinds of things. But again, somebody walking down the street, listening to the news may have heard some of this, may have heard the, the hy about malt bot then that the terror at what the skills were doing.

And I think this is leading us to a, a really good beginning conversation to have those conversations that we know need to happen around how do you manage this? How do you do this securely? Who am I gonna trust as I'm starting to use just foundationally trust is underneath all of this, who can I trust to do these automated things for me?

I think that's a, it's really good that we've got this very public failure that is leading us to have, having that mature conversation that I think all of the providers who are doing enterprise work have already been having, uh, hopefully this is driving other conversations always. Are you all seeing other conversations that are outside of the industry, outside of the insiders talking to one another about security and ai? I mean, right now it's, it's always been, especially like within my organization, right?

Like that's still a hot topic because it's always gonna be the way, it's the way of the future. You know, like we are finding ways to better, you know, not only assist our clients and further bettering and heightening like the level of security and services we offer, implementing that across all their data and platforms and such. But yes, you know, how can we leverage tools and ai?

Like, that's why I love that we have like a dedicated little like dev dev team to just work on that. You know, if I get the chance, I get to dabble in it a little, but it's really amazing to see, like we're making forward progress on it. But again, it's not one of those things, we're just trying to slam a solution.

You know, we're not trying to, we're not trying to slam, you know, the square, you know, wooden thing into the circle peg and make it work. Like it's not gonna work. Like we have to properly vet it, you know?

And it's just like with, with that solution and everything else, it gives us time to really hash it out, vet it, see where it works, how does it fit the, the model and more so how does it fit the business needs? Like what's the goal, what's the outcome? Like you said, right, Andrea?

Like, like, what, what are we trying to achieve? Because there has to be a thing or multiple things that you're trying to achieve when leveraging this. And I think, I think, I don't wanna say half, maybe a little more, a little less.

Like some people see it that way, be it, you know, technical professionals, non-technical professionals. I think we're just still at that, that precipice of like, oh, this is cool. Like, I don't care.

I just, like you said, if it could turn out my coffee baker start my car and do this, like, that's awesome. But because they're not security professionals, and I'm not saying you need to be a 20 year it vet, like you just need to kind of take a second and like stop breathing. Like, okay, this is cool, but what's happening on the other side?

And I think people are just like, I want my life to be easy. I want my day to day to be, I want how I, you know, leverage these solutions easy and that's it. All of the other problems are gonna just filter their way out.

And you know, fortunately that's what me and Andrew, that's what we're here for, you know, because we're the ones who are gonna be like, Hey, I had this problem, how come no one told me about this? And it's just like, you know, back to square one. So I, I think it's really cool, like where we're seeing it, but I still think there's that, that that halfway or so point of people who are like really trying to be smart, you know, both technical and not technical about how we're vetting, um, AI and how we're, you know, using, leveraging it in our solutions day to day.

And then I just think there's the other half that's just like, nah, I don't care. You know, they wanna be the cool kids in school and they're gonna use it, and that's that. And, you know, they'll worry about the problems later on.

So Yeah, I really try to simplify it even more, right? Because we, the AI means a lot of different things to a lot of different people because there, there's different flavors of it. I, I try to take it back even simpler to talk about just the data because this is, it's, it's less of an AI problem and more of a data problem, right?

And so if we can have a conversation about what you feel is important to you, what you feel is critical to you, and what are you okay with it just getting out there if it happens to get out, if I can get them to, to take that mindset of things saying, this is absolutely cannot get to the internet. I don't care if this gets here, this is okay, regardless of what happens with the model, we can have some level of governance over what goes out. But just having them take a step back to say what's critical, what, what's deemed to know and what 100% can never get to the internet.

It allows them to have that frame of reference. Because I think trying to explain AI to the common user above and beyond, Hey, you can turn a thermostat off or it's gonna be kind of an exercise of utility, but if I can say, Hey, listen, your daughter's social security card number, graduation date, et cetera in this information of data, are you good with that being out that resonates? And then I can get on things important and I can can say, maybe this piece is important, but maybe you should keep these things kind of at bay.

And so trying to bring them along slowly, we're never going to be able to catch up. And I was, I love the way you said, right? If we get them to maybe test, if we can maybe get them to do the app test, having that same conversation for 20 years, and Lord knows if we're not gonna do it, they're definitely not gonna do it.

So I'm not even gonna start that conversation. But what I will say is, you, we, we don't air our dirty laundry in public for a reason, right? Okay.

Let's take that same approach to using AI and, and actually making the benefits of that work for us. And if you can do that in a way that's consistent or at least okay with you, right? I think we have a great place to start.

I believe as we get further along and we build more into this and we find more commercially available systems, and I say commercially available because the support comes with that, the reputation comes with that, and traditionally speaking, then we build the controls as part of the process. 'cause as consumers, we demand that of the vendors. And I think we can find a better way to kind of, you know, I think marry the two.

But until then, it is definitely, um, you know, you get what you pay for. And I, that's what I always say about open source, get what you pay for. So know that going into the battle.

Yeah, I think o overall this is highlighted for us that, uh, consumers of these tools want something to be simple. They don't want to have to think too much. Uh, there should be safe and secure defaults rather than wide open defaults.

And it should be hard to turn those off. Uh, there should be governance somewhere in these platforms. There has to be for these things to be usable long term.

But the reality is that, uh, consumers in particular, and, and some parts of enterprise are gonna want things before they're ready. They're gonna want things immediately because that drives business differentiation. Um, the risk is always balancing that business differentiation, that business benefit against the business risk if things go completely sideways and all of our data goes floating out all over the internet, both personal data and company data.

Now, we as usual could spend a long time talking about this. And if, uh, you are a delegated attending a Tech Field Day event, you will, uh, recognize these kinds of conversations as the ones that we have when we're not actually in the presentations at Tick Field Day events. Um, but I've gotta thank you as our, our listeners of the Tick Field Day podcast for joining us today, uh, on this, this episode.

Uh, but before we go, if you wanna catch up with you and continue this conversation, where can they find you all? Like I said, you could find me everyone at G Kalina on Twitter, LinkedIn. I'm on TikTok.

I make short long form content. You name it, I do it, uh, wherever books are sold, but I'm always around, especially on social media. So just reach out, say hello, blessed to be a fellow tech, tech field day delegate.

So I've done a few events, hopefully some more coming up soon. We'll see. And, uh, but yeah, love to continue this conversation anywhere.

So follow me across all the platforms. Uh, probably, probably not as much on tech field. I mean, not as much on social media.

I, I I tend to do more on LinkedIn, getting more involved with Tech Field. So I need you on LinkedIn. Um, hopefully again, getting more involved with tech field things, but also commonly speaking in the community.

So most times when you find cybersecurity, you'll find me somewhere in the conversation. And of course, I'm Alistair Cook, you can find me on LinkedIn Is Hours to Cook. Uh, you can find me across all the various types of social media on the fragmented spaces, uh, and you can find me on various Tech Field Day and TUM Group properties around.

Uh, so thank you so much for listening to this episode of Thet Field Day podcast. You enjoyed this discussion and would like to have, listen to some more of these discussions. Please subscribe on YouTube or your favorite podcast applications.

Say don't miss a single episode. Give us a rating as well, preferably a positive one and a nice review that helps other people find these great conversations. This podcast was brought to you by Tech Field Day, the home of IT experts from across the enterprise and a part of the for events, more episode, head to Tech podcast, or view us on tech.

Thanks, listening.