Prompt Injection Attacks in LLMs with Snyk’s Elliot Ward

This is Techstrong tv. Hi everyone. Welcome back here to Techstrong tv.

I am happy to have as my next guest, Elliot Ward. Elliot is a senior security researcher at Snyk. Most of you probably know Snyk, but if not, we'll tell you in just a second.

But first, let's say hello here to Elliot. Hey Elliot, welcome to Tech Drunk tv. Hey Adam, thanks for having me.

Pleasure to have you on, man. So, Elliot, before we, I tell people we're gonna tell 'em about sny and let's, we'll tell her about what you do at your job here at sny of Senior Security Researcher. But before we even get to that, let's tell 'em a little bit about Elliot.

Yeah, sounds great. So, yeah, I mean, my name's Elliot and, uh, I'm, I'm from the uk, um, but I'm currently, uh, living in Switzerland. I've been here, uh, just over seven years now.

Um, and kind of, yeah, my background, so always knew that I wanted to get into security from a, a pretty young age and followed this through school, university, kind of studying security. And then, yeah, after university got my first job in penetration testing. Um, and then before that I also had some internships as a developer.

So I've also seen it from the other side. Um, and yeah, seeing how, uh, security can be a challenge for companies. Um, but then, yeah, this also helped in kind of the, the pen testing side as I know how applications are built and, uh, gave me a good eye to kind of understand Yeah, how to attack them.

Um, so I was, yeah, a pen tester, kind of a, in the consulting world for about six or seven years. Um, and yeah, this was based in London, but then, yeah, we did some work for Swiss Banks, and then yeah, the opportunity came up to relocate to, to Zurich, which is, uh, yeah, amazing here. Um, and then, yeah, after a little while I was kind of done with consulting and wanted to move on to some other challenges.

So yeah, became an internal security engineer for a couple of years. And in this, I was kind of building out the application security program, uh, a company here in Switzerland. And this was kind of focused on mostly application security, because this is kind of, yeah, my background and, uh, yeah, what I can, what I know well.

Um, so this was kind of doing secure code review, building out kind of what things really need to focus on building out the security champions program, uh, giving secure coding training to the, the engineers, and then kind of all of the other things that encompass, uh, a typical AppSec role. Um, and then obviously on the side, I mean, I'm a hacker, uh, so I've always been doing kind of research and, uh, when I was in the internal position, I didn't really have the chance to kind of be doing more kind of larger research projects. This is like I did with, uh, consulting.

I mean, many consulting firms, they allow you to do kind of like 20, 25% of your time on research projects, which is really nice. Um, and yeah, saw the position with, um, as a security researcher and was like, yeah, actually that's what I wanna be doing for the foreseeable future. Yeah, I've been here now for just over two and a half years.

Um, and yeah, we have a great team here. Um, and yeah, we do some awesome work. Very cool.

So look, as I mentioned, most our audience knows Snyk, one of the leaders in DevSecOps and providing security tools for the development environments and security in general. Uh, they may not be familiar with sleeks mission around research, so security research. Why don't you, if you don't mind, Elliot, give us some, we could zero in on that.

Yeah, absolutely. So I mean, as you mentioned, I mean, sleek, I mean, our mission is to allow kind of development teams to kind of build fast and stay secure, um, and provide all of the tools that they need to have a secure SDLC. Um, and one of the things that we kind of strive for is to also push out, uh, first party research.

So instead of also just relying on kind of vulnerabilities that other people are finding, we have a, a research team here where we focus on finding kind of high impact vulnerabilities within, uh, kind of opens things that have an impact against the developer community or open source or kind of new attack techniques. So there may be some new emerging technology, uh, but it may not have been researched from a security standpoint so much. So yeah, we are there to kind of look at this within the, the security labs team at snyk.

Um, so, so, So the, I'm not sure if our audience is gonna, you know, see the nuances here. So there's the security labs at the, they have several mis missions, one of which is, is kind of like, let's call pure security research. Is that fair to say?

Uh, exactly. So I mean, like our full-time jobs, like we are not, uh, kind of helping out with the products as such. We are doing kind of zero day research, uh, finding vulnerabilities in various open source frameworks, uh, looking for kind of new ways to attack new technologies or existing technologies that maybe have been overlooked in the past.

So we're trying to kind of just be a push forward, uh, the state of security, uh, in terms of yeah, what what's affecting the current developer landscape. Excellent. com is the main website for SNY and all that It does, there is a, a web, several website for the security labs team, is that correct?

Exactly. So if you go to security labs dot sny, do io, this is, uh, our dedicated portal. When here you'll find kind of our mission from the security labs team and also all of our latest research.

Excellent. All right. We got that out of the way.

And, um, let's now jump into what kind of our topic of discussion today is Elliot, and that is basically, uh, injection attacks in the LLMs and how, you know, these are being done, right. And I, I, I think a lot of our folks out here probably wondered, right? Is there a way from just using the, the chat GPT interface or any, you know, chat bot interface to actually, you know, do something malicious break something as many of us in security, you know, like to do just, just to test it.

Um, but you know, you guys have done some research and saw some and, and seen some things publish this study. Why don't you tell us about it, Elliot? Yeah, absolutely.

So I mean, kind of the, the other lens, I mean, it's fascinating because I mean, it's a new technology and it allows you to do things not just by writing code, but the, the instructions are just human language. So we kind of have the human language, which is the prompt, and then we'll pass in some data from the user. Um, and then this will be, it will execute the, the prompt based on that information.

So that kind of allows the, the attack surface to change significantly from what we've seen in the past. Um, and I mean, it's quite similar to kind of secret injection back in the day where, uh, people were concatenating the user input directly into, uh, the SQL query and kind of losing the separation of what is code and, uh, what is user input. Um, whereas now kind of where we are back into that realm of mixing the, the prompt and the, the user input, and, uh, it, it's really interesting because it allows, because of how LLMs work, it allows the actual natural language to become an exploit vector.

So we can kind of try to think of ways to manipulate the, the underlying prompt, um, to try and get it to do certain other things. So this may be kind of the typical way that we've kind of seen this up to now. I mean, you'll have some prompt that says, do this, and then the attacker may, like, the application may put the attacker's name or some other piece of information about the attacker into this, uh, yeah, prompt and it will do something.

And the attacker may put their name as ignore all previous instructions and do this instead. Um, and then when the LLM is actually looking at this information, it may decide that the user's part of the message is more important and focus on that task instead of, uh, what the developer actually intended. Um, so this is really interesting.

Um, and then there's like kind of some other things that you can do. So maybe instead of being less explicit by saying like, ignore previous instructions, you can do things like, once that task is complete, now do this. Um, and there's many different ways you can kind of fool these LLMs.

Um, so it's, yeah, very interesting. Uh, It, it really, it is. So for our non-security folks out here, developers, cloud native folks, how, how prevalent, like, like how much should they be worrying about this, let's put it that way, or is this just something that the big AI model people need to be worrying about?

So I mean, a, any companies that are doing something with AI based on some user input, they ought to be concerned about this. And even if you're not doing this now, the likelihood is that you'll be doing this at some point soon. I mean, AI is taking over, and if you're not gonna leverage ai, then it's quite likely that you're gonna be overtaken by those who are, who are using ai.

Um, so yeah, there, there is a genuine kind of concern there. Um, I'm sorry. Yeah, sorry.

Uh, forgot my train of thought there. Um, It's okay. It'll come to you.

Wait till you're my age and you see how it really works. But, um, let me, let me ask you another question. As, as organizations start using their own LLMs, custom LLMs SL you know, small language modules and stuff like that, there's this problem then kind of become exasperated exasperated where they, they're, you know, this is your data now that, that's at stake here and and your functionality that's at stake.

At stake. Yeah, no, absolutely. Um, so I mean, there's many different components there that can kind of go wrong.

I mean, like, I think a lot of the times when we've seen LLMs being used, and particularly with like kind of the prompt injection attacks, it's been very kind of in an isolated scenario where, right, like it is being showed as like a kind of a toy example. Like where here is a secret, don't tell the secret to the user, and then the user needs to try and extract that. Um, whereas kind of in a real world scenario, like the LLM needs to have access to some other company data maybe.

So there'll be a vector database where you're storing information about your customers or internal knowledge bases. And then when you kind of get these kind of wider systems that are like, it's a complete system that has an LLM component at the center of it. Um, and then when you get a prompt injection in these kind of scenarios, it's uh, a lot more powerful and a lot more can go wrong because you can potentially then coerce the LLM to leak information from the knowledge base or leak information about other customers instead of, uh, just yourself.

Got it. I, I guess my next question then, Elliot, is what are we supposed to do about it? So this is a, a really challenging problem that we have, uh, with them and kind of intrinsically by design.

Um, it's a pre-compiled kind of black box. So we, we send in the input and we get the output out, and we don't currently have the mechanisms there that we do, for example, with databases where we can parameterize the queries and the database engine will never treat the user input as the database query. Like we don't have that with, with other lms.

So it, it is quite challenging to solve. Um, so we can kind of try to kind of filter out or tell the LLMs, uh, like do not, like anything that comes from this point, uh, is from the user and do not trust it. Um, any, if they try to lead you to do something else, do not listen and do this, uh, and just reject the request.

And I mean, those things can work, but again, then it comes down to it's just the kind of a cat and mouse thing and whoever's argument is more persuasive is going to win. Um, so that, that's a challenge. But then we have, uh, other solutions.

So we have kind of AI guardrails, um, and there's a great company called Lara, um, and they are producing, um, they're, they're basically a file for LLMs. Um, and basically anything that goes into and out of the LLM, uh, is gonna be screened by Lara, um, before going, uh, well before you kind of process this and pass it to some other component of your application. And, uh, this allows you to get some, uh, much tighter control over, uh, these attacks that we're, we're talking about.

I, I, I sort, I was thinking, you know, along the lines of how waft web application firewalls kind of really helped the fight against, you know, overflow and, and SQL injection stuff. Would we have some sort of AI chat box firewall that, that could help that here? Exactly.

In, in terms of what's going on, there are Basically a WAV for your LLM components. Very cool. Now one has answer what about API traffic going into LLMs, right?

Yep. 'cause that's been sort of the weak point and wax. Yeah, so I mean, this is also somewhere where things like lacera can help.

So basically they provide like an SDK, so inside your application, you, you take your, the user input from the API and then before passing it to the LLM, you run this via like error and it will try to check like, oh, is there a pumped injection in this input? Um, and it will kind of give you a, like a score back saying, yes, no, we think there's a high probability of this or not, and then you can make the decision to invoke the LLM and then the same with the response. So it could be that maybe it's missed on the way in, um, but then the LLM decides to accidentally leak some, uh, proprietary information in the response.

And then, uh, the these guards can also help their to prevent data leakage as well. Sure. io and we can keep it up there.

Elliot, thank you so much for coming on text on TV man, and telling us all about that. It's, I think a lot of us suspected that this is a possibility we didn't want to, but I mean, it it, when you think about it, why not? Right?

You know, if if there's an exploit to be exploited Yeah, no, Absolutely. No one wants to waste a good crisis, right? So we'll, we'll continue to watch this develop.

I think as the ai, the whole AI usage thing continues to develop. Yeah, absolutely. And I mean, in a, in a couple of weeks, uh, we'll be releasing, uh, a blog post, which is going over some, some of the research that we've done as a, a joint project with lacera.

Um, and we've been focusing on, uh, LLM agent systems and these are basically systems that allow you to bring in all of these components and connect, uh, build a complete autonomous AI system. Um, and we found some significant issues in, um, yeah, these systems, which, uh, gonna be quite interesting to read about. So that's something very Cool.

Be watchful. We keep us posted on that. We'll keep a lookout for it.

Elliot, thanks for coming on. Do come back and join us. Enjoy your research.

It's good to, you know, if you're doing something you love and you never work a day in your life, so keep doing what you're doing, man. It seems it suits you. Absolutely.

Uh, thanks Alan. Alright, Elliot Ward, senior security researcher at Sny s Sneak Security Labs here on Techron tv. We're gonna take a break on Techron, we'll be back in a little bit.