The Prompt Injection Peril and Why AI Agents Are Your Network’s Newest Vulnerability

Hey everyone. Welcome back here to Techstrong tv. You know, I'm really happy to have my next guest on.

I, I got to let you guys in on a little secret. We actually were gonna interview and meet yesterday, and we had to abort the interview because of, uh, air I, air warning Shelter warning that a meet, uh, uh, received. And so we had to stop the thing.

And it, I'll be honest, it's not the first time that's happened to me. This is, you know, uh, I, I interview a lot of people in Israel and, and when the Hezbollah and the Hamas and everything going on over the last two years, it's happened. Interestingly though, amid I, right after the interview, I couldn't do with you, I did another interview with a friend of mine who happened to be on vacation in Puerto Val to Mexico, and there they had cartel members setting cars on fire and, you know, and them, you know, making all kinds of things.

I just, I'm glad that I'm sitting here at my desk. I don't have to deal with all this, but thank you for coming on. It's great to have you.

Um, thank you for having me. My pleasure. Amit, Amit, the Men io is a company we've covered here at Techstrong for a long time, since I started it.

Um, you are the field CTO there, but give people a sense, kind of, of your journey, how you came to be the field, CTO, maybe your background skillset, and then we'll talk more about MEND and some other things going on. Uh, sure. Uh, so hi everyone.

Uh, my name is Amit, people Kta. I come from a background of, uh, cyber security and reverse engineering. And, uh, in my previous company, uh, we built a bootstrap business around the container scanning for security.

So we scan technology called container images to like find whether, uh, vulnerability is exploitable or not, and actually meant bought our company. And that's how I joined mend. And if you don't know, MEND is an application security company that grew recently also to be an AI security company.

Uh, and since then I joined mend, I became the field TO, which means, uh, I'm lucky to speak with for the largest, uh, customers and to help them, uh, with their product and their security issues, and to get more value and also to learn from them how we can improve what we're doing here and to make better security that is tailored to their needs. Uh, and I get to to okay with you, Alan, which is, uh, amazing as well. Thank you.

And my pleasure. And that, and that, you know what, so Field CTO is a, it's fairly new. Maybe in the last five to seven years, you're starting to see a lot of people with, not a lot, but you'll see that as a, as a, a title for people.

It, because back to, like when I was starting companies, right? CTOs came in two flavor. There was the VP of engineering, CTO, who kinda ran the development team, and then there was more of a market focused CTO who really spoke to a lot of the customers and became sort of the, the, the antenna, the, the input from the market that can then got translated back into how this becomes product, right?

And how this becomes service and, and added that grew. I think that's the field CTO heritage, right? From from that line of, of it.

Um, exactly. You mentioned Men IO is a, is a, uh, application security company, and now of course an AI security company. Everybody is AI today one way or another, right?

But you know, there's, there's AI washing where we just put AI in front of things, and then there's really working in ai. And, and that brings us to what I wanted to talk about with you today, which is Mend Reached recently launched a system prompt hardening feature service. It's not a standalone product or is, is it?

No. So it, it, it's part of a much larger product, which is security for any AI application. Mm-hmm.

And so I, I'll tell you a bit about what we spend a lot of our time in the recent years now in men. So we realized that because of AI being such a big trend, any, a company almost build their own AI products to sell to their customers, whether it's a chatbot or AI behind the scene or desktop. And there are new security concerns around it, and you wanna build these AI systems to be secure compliant.

And we try to give you all the tools to make it easy for you to secure, uh, your own AI tools that you sell to your customers so they can trust it. And, and we just released what we call system from Ting, which is a, basically any a chat bot or LLM that we use today have prompt behind the scene that instruct AI how to behave and what to do. And like it can have like security issues or you, you, like, for example, you don't want to expose sensitive data there because people can leak that prompt by talking to the chat bot, or it wants to help the chat bott to understand they have to understand what it can and cannot do security wise as well.

So we, we help companies do it automatically because when you have tons of developers, it's hard to control what they do. And that's what men is about to help secure code. Love it, love it.

Now, when we talk about hardening, uh, hardening the, the, the prompts and, and all of this, and, you know, bringing risk management is the best way I could describe it. Bringing risk management. Part of the problem is, I saw an interesting survey.

60% of workers like knowledge workers, right? People on computers are being given access to ai. I thought that number was low.

I bet you 80% of people are probably messing with ai. But of that 60% that have access to ai, 60% are not using it regularly, which means the majority of the people, and they logged in once or twice who said, okay, this is cool. You know, they may be, I, you know, did an experiment and then what about their business?

Right? I is that number, that number's gotta change, I assume, right? People got to.

But how I, I guess the question I'm asking is how important is it to harden our prompts Now, how hard, how important is it to bring governance to this whole, because it's kind of you, you don't wanna kill the baby, right? Or throw the baby out with the bathwater. You want to encourage the people to use it.

Definitely. You know, where's the balance? So you definitely don't want to stop innovation and you want to help people and help enterprises to adapt more.

Uh, and actually a lot of the reason that large enterprises are afraid to incorporate certain tools is because they're afraid of the security implications of it. And we see ourselves a bit as enablers. Now, you're correct that today a lot of people still don't heavily based on ai.

Uh, and, but it doesn't matter for hackers because hackers are going for the low hanging foods. Yeah. So even if you have like, only 1% of the organization that is vulnerable, hackers will go to that 1%, and that's how they'll breach into the organization.

And from there, they'll go and start stealing information. So you wanna take the, the weakest point in the organization and help protect it. And, and today it's starting to become a ai, especially with risks such as indirect prompt injection, eh, which, like, I, I don't know.

I think it's something that a lot of people have heard of the concept of prompt injection, where you just convince the LLM the AI to do something. It shouldn't, but then it becomes more complex when it's indirect. When I ask for something trivial, the AI goes on the internet and reads a website to do it, and then this website convinced my chat bot to do something it shouldn't, and to leak information.

And this is a super complex issue that we're gonna, we're dealing with right now. And we're gonna continue to deal with as agents do more and more stuff for us. So definitely important.

Definitely we don't wanna stop innovation and, and just be an enabler for it. Let's talk, you know, most people, they think of a prompt look, A lot of people talk their prompts and uh, they don't even type, you know, they talk to their ai and so they don't see it visually, other people type, but they just see what they type and they see what the AI types back. What they don't realize is the hidden instructions, the silent stuff that's going on in the background.

When you hit enter in your prompt, can you talk to us a little bit about those hidden instructions, if you will? Sure. So any chatbot that we use today comes with a few layers of text, basically prompt that instructed how to behave.

So it doesn't really have a sense of self, but for example, when you use chat g PT at the beginning, there's a huge chunk of text selling the AI model. You are the GPT, this is the time today. You should behave like this.

You can do X, Y, and Z. Here is what the user ask for. Then they paste you text that you message, and here are the previous messages, what is your response?

So there's a lot of text going into AI model, and it's fairly interesting. All these prompts are online. So you can actually Google it and find online all the prompts of different share bots and see what kind of information they see about you.

And that's what makes certain models behave differently than others. Also, they have like different levels of intelligence, but also the prompt, right? Uh, and not only that, there's also smaller prompts hidden.

For example, uh, we let today a lot of agents to search the Google for us, search your web forward. So they get a tool, uh, that they can execute a search with certain keywords. And these tools come with instructions, how to use it.

It's more text that goes into the model. Now, the more and more text we have there, uh, there's the risk that the model won't behave like we want it to behave. Uh, and if we, especially as we move from just question answering to doing tasks, right?

So if I, I give my agent ability to delete files, to send an email, uh, uh, to, I don't know, delete my database or change data in production, uh, these dangers become more and more like, uh, complicated. Maybe it reads a tweet online that convinces it to delete my organization, database. Database.

So, eh, that's, that's a bit the, uh, like the background for prompts and why the fact that the agents just get text as an input is complicating things. Yeah. Agreed.

Agreed. Um, look, I'm not going to ask you to say if sir, which prompts are more secure than others? 'cause we'll wind up, you know, we are not, we're not gonna say Claude's better than Gemini or Gemini's, better than OpenAI or what have you.

I'm sure they all have their unique weaknesses and strengths. They're all trying. Like, I don't think any of the AI vendors raise their hand and say, I don't care about security.

They all try. But, you know, in the, in the, in the race for progress and the race to go forward and the race to get users, unfortunately, security isn't always top of mind with them. And that's why you need this type of, of service here for men to, to help with this.

Um, you know, I mean, got a little time. So we're going through our own thing here in Textron. I think I bought three Mac Minis this week for the team, right?

Amazing. That's 'cause that's, that's the new thing right now. So now we're running AI locally, we're running AI on the edge, if you will.

And who figured, everyone said Apple missed. Apple missed the ai, they didn't have a Chama uh, a chat. They didn't have a LLM.

Well, turns out they're the hardware of choice for, for AI, maybe, right? But, you know, so we're putting them on standalone machines. We're trying to, you know, as much as we can isolate them just so we start with almost a zero trust, right?

And then turn on certain things as needed. Google is one of the things. 'cause we gave each agent a, an email address, if you will, and, you know, access to Google.

Um, it, and we we're, and the funny thing is we asked, It's a huge, it's a huge security issue, by the way. Oh, I know. So, yeah, you know what, so what we're doing, I don't want to tip too much online here, but we actually, each agent runs on a dedicated mini, it runs in a vm, not on the mini itself.

We, we install it in a VM on top of that, like another abstract. We have a zero trust that we give it nothing, right? And then we slowly build up what it can do and you know, where it can load and so forth.

But I will tell you, and and I, I've got 30 years in security, right? I'm not just some guy who does interviews. I, I started security companies.

The, it's hard for me because what it can do is, is great. I mean, we're, we're using it to starting automating video editing, posting the videos, you know, uh, editing our articles, uh, uh, newsletters, all of these things. But, but the threat security wise is real too.

And so I don't know if we, you know, and how are we finding out what's best security practices? We ask the ai, we ask the AI exactly what's the best thing to do here. So it's like, I don't know if I, you know what I'm, I'm trusting the AI to tell me how to secure the ai.

It doesn't make sense. What do you, what do you, what's your advice? So I, I like to quote, uh, OpenAI's chief Information Security Officer.

I don't remember the exact wording, but basically, uh, he said on Twitter, that prompt, uh, injection is an unsolved frontier issue with LLMs. And we don't really have, uh, aromatic solution. So you, you have to have multi-layer approach to that.

Uh, and it, it got, it, it ranges from, if you use OpenAI, they train the models to be less susceptible to prompts, malicious prompts and hijack. You add tools like guard rails that try to listen to all the conversation and all the things that the AI is doing and drop messages that are suspicious or like executions of tools that are suspicious. You have prompt Harding that helps you align your model in the chatbot that you are building with your values.

You explain to it what it shouldn't do, what it should do. You make sure not to put sensitive data where it shouldn't, you shouldn't have. So you have a, you need to have multi-layer approach in the end.

It's a bit similar to a person. Let's say you hire an a HR person and they, they see all the salaries of people in the, in the company. How do you know that they don't tell you to someone from outside the company?

How do you know they don't do something bad? So you, you trust it. They know it's illegal and you can hold them accountable because you can take them to court to jail.

Like, so, but you can't really do these things with AI agents. And, and, and, and I think that's the foundational issue, right? We want to give them abilities like humans have so they can replace human tasks, but we can't really hold them accountable.

And that's why we have all these security measures trying to like, make it less and less and less probable that they'll do something that they shouldn't have. But the moment you give them the ability, uh, to do so, you should assume someone can convince the chat bot to do it. And I think that that's the, that's the biggest security issue today.

Should they, should they build my agent hr, uh, or should I not build it? Should it be the limited version of it? Um, exciting times.

It is an exciting time and we're doing the best we can. I'll keep you fo posted. Listen, this new, this new, uh, offering from med, it's available now.

Yes, it's available now. Uh, and take a look at the website. Uh, it's super important, especially if you are building your own chatbots to your customers.

And like, security is something that we all need to do and have to do to keep the data of like, of our users safe. So if that's your situation, I encourage you looking at manned, like, and see what we are doing, eh, and even if not men, right? Look for open source.

Try to protect your ai. Be cautious, eh? Yes.

That's my of course, a biased recommendation, I think's. Good advice. No, no, I think that's excellent, Amit.

Be well, be safe. Talk to us more and, uh, good luck with you and to all our friends at Mend. Thank you.

And thank you Anna. Thank you. It was great.

It was excellent. Amit Chita Field, CTO at men. Here.

Go check out their new, uh, system, prompt hardening solution. It it, you know, in today's world, it's something we need. We're gonna take a break here on Text Trunk Gang.

We'll be back in a minute.