Securing Healthcare Data in the Age of Shadow AI – Insights from Wolters Kluwer’s Alex Tyrrell

Hello, and welcome to the latest edition of the Techstrong AI Leadership Insights series. Today we're with Alex Terrell, who is Senior vice president and CTO for the health division at Walters. And, and we're having a little chat here about, well, what is going on with shadow AI in the healthcare space because they, there's a lot of sensitive data at work here.

Alex, welcome to the show. Well, thanks Mike for having me. And, uh, yeah, really, really interesting topic.

I'm excited to have a quick discussion about it. Right. Shadow, it has always been with us and always been something will plague.

But, um, it seems like things might get worse with shadow AI because a lot of these tools are just being exposed to sensitive data, and now we have AI agents that can access and find data, and we don't have a whole lot of policies and guardrails in place. So, um, how big an issue has this become? Yeah, so, so just for a little bit of context, right?

So this idea of like shadow AI or shadow it, I think you mentioned it, is, is basically that this concept that, um, you know, folks are going around the standard of controls in the workplace, so going around sourcing and procurement, legal, potentially security, uh, and provisioning, you know, software and services that they may find useful but aren't necessarily vetted by the organization, right? And that, that creates a number of risks. And I think, obviously in a healthcare setting, the first one that comes to mind is sort of this really, really sensitive private data, PHI that's, that's in a healthcare setting.

And, and it's just, you know, really, really dangerous to think about, you know, have really critical sensitive data mixing with, you know, technologies that might just be on the web and are really not been vetted and really not really fully understood. Right. So that, that's kind of, that's kind of the, the basic concern.

Yeah. All right. I think part of the issue here with shadow AI is though that the threat is not immediately obvious.

I think a lot end users are just taking data, putting it into the LLM, they're not reading the licensing agreement and checking the box that says, don't use this for training. And it, so it might not be months before we see an update to an LLM where suddenly there's sensitive data in there. Um, is that part of our issue is that people just don't understand the scope of the issue.

I think, yeah. So as these tools, and I think, you know, it's been, you know, a little over two years since, like, for example, these LLMs and Gen AI sort of became very, very, um, sort of ubiquitous, right? And I think a lot of folks have used the technology and, you know, there's a lot of value and benefit to the technology in many instances.

But now you've got a sort of that conundrum of like, you know, what you use in your personal life or what you found valuable may not be appropriate in a workplace setting, especially where there's highly sensitive data, especially when you're in a regulated healthcare setting. And I think it's difficult for people to kind of disengage. And I, I, I don't think it's a nefarious act.

It really is just this sense of like, it's so common, it's so ubiquitous. Like how do you know where that line is? Right?

And that's when you have to start thinking about creating kind of like zones of trust in a healthcare setting, uh, where it may be okay to use a certain technology, but as you get so sort of closer to like bedside or where you get really close to the sensitive data, you really have to have firm controls there, right? And it is complex. Hmm.

And as we go forward here, I cannot help but wonder if the bad guys are getting smarter about teasing that information outta these LLMs. 'cause as far as I can tell, they're starting to use like prompt injections and all kinds of techniques that they've developed to tease out sensitive data. No matter what the guardrails are doing.

It's, it's really, it's sort of an arms race. So, uh, as a CTO, right, I have a one, one focus on, you know, providing absolute, you know, the, the best possible value to our customers and, you know, really, you know, developing technologies with AI that are trusted and verified, and that's really, really exciting. But over here we have threat actors and a sort of this rise in emergence and a new sort of, like you alluded to it, sort of a new sort of vector of the ability of these LLMs to sort of escalate and, and make the job potentially easier for these threat actors to get access to sensitive data, right?

So there's constant balance, and I think really the key thing is like, make sure that you're using trusted, uh, information for trusted sources that have been through a sourcing and procurement process that be gone through a due diligence and vetting process that meet the compliance, uh, requirements of the healthcare setting they're being used in, and really having sort of a structured and a formalized approach to governance and compliance within the organization itself. And that's really, really sort of key, right? Mm-hmm.

Do the guardrails need to be better? And I'm asking this question because a lot of the LMS are, let's say, programmed to be, um, exceedingly helpful, right? And that occasionally leads them to hallucinate and do all kinds of interesting things.

But it seems like when someone's trying to en run the guardrails, they, um, maybe put up a first wave of resistance, but then after that they're just kinda like, well, sure, we'll help you with that, this, that, and the other thing. And it's kind of like having a guard dog that when the robbers show up, show 'em where the keys are, so, right, Right. They're just taking treats.

Right? So I think I do think it, it really is interesting because it, again, when we start to use these technologies, we see that they have value in their daily lives. It does tend, I think, potentially to cause people to let their guard down about what you might put in there, right?

So when you think about personal identified information or sensitive information, people may think, well, you know, what's the harm, right? But you realize like the bad guys can connect the dots, right? And they're looking for pieces of information that might be here, might be there.

If they can get access to this, it can be a real threat to the individual. Right? Now, of course, I think, well, geez, what's, what's the threat of like somebody knowing, you know, something about my, my cholesterol, that that's not really the end game.

The end game of these threat actors is to empty your bank account. And that is deeply profound, right? And not that it's not profound to get access to personal healthcare information, but I really, really think that, you know, it's both raising awareness so people understand to be very careful how you interact with these technologies, particularly these LLMs, what you share, what you put into them, thinking about how it might come back to get you.

And of course, as technologists and IT organizations, we really, really have to be focused on not just the guardrails to make sure that the LLMs don't hallucinate, but also to make sure they actually cannot be exploited with court of these jailbreaks or these prompt injections. And that sort of falls more and more under the sort of the traditional approaches of methodologies we take to cybersecurity. We're developing really good plans to deal with that threat, but it is a collective effort, right?

Like everybody needs to be aware that this can be a big risk and use caution. Mm-hmm. Um, is there something that security folks should be doing that kinda, uh, maybe to bolster the guardrails or strengthen the infrastructure or something?

And can they do that in a way that's subtle? Because I think they're all very tired of being the perceived bad guy walking around saying, you shouldn't use that, Right? Like, sort of the culture of no.

Right? I think, and, and I face this in my own organization, right? It's like, well, well, can I do this?

But not to that it really gets overwhelming. That's probably not the best approach to create a list of things you can do and a list of things you ought not to do. Although it would be nice.

Um, what, what, what I think I is potentially helpful and even something we've done on our organization is people are gonna gravitate to those tools because they can add value. So set up trusted zones, right? So rather than having people run in every direction, picking this tool or picking that tool, look at where you can add value, where you think a solution like this can add value for, for, uh, folks in your organization.

Begin to slowly introduce it so people know there's a place to go, a trusted place to go that really has as many as we can proper security and guardrails and kind of reduce the spread, right? Because we talk about things like attack surfaces, right? Mike, every time you open up a new one of these, it's a potential way in for the bad guys.

And creating these sort of trusted zones and making sure the organization knows that for particular tasks where it's deemed, um, of an acceptable risk. There's a technology there for you that has gone through sourcing, procurement, legal compliance, it should work for the task. And that gives people, I think, that sort of place to go and prevents them from just using an old tool off the shelf.

Mm-hmm. So ultimately I need to get everybody to kind of standardize on some sort of set of tools that have been validated. Um, that seems a little harder to do then actually say, so, so what's your Best, it can be a challenge.

I mean, I mean, Mike, uh, you think about like a situation where you're using a tool every day and suddenly there's some Gena capability that just got turned on, right? And now what are you supposed to do, right. Are, are you allowed to use it?

Are you not? I mean, that's where you really get into like making sure you're working, you know, you with trusted partners, um, where you can really understand the terms of use, you can really understand the security controls and measures and level of compliance around sensitive data that they're meeting and adhering to. Uh, and also just kind of reinforcing the fact that like you really, you know, it's, it's a continual process, but kind of creating that outreach, that education that you really gotta be careful about these technologies consistently within the workforce, right?

So that combination of giving people trusted places to go continually to focus on the education, these are really sort of the, the, the, the really the two levers that really have a lot of benefit. Do you think we might be able to use AI to help enforce the guardrails a little bit? And I'm wondering, you know, will we wind up with like, uh, AI agents that are policing the other AI agents and we'll kind of set this kinda, you know, layered framework in place.

Yeah. All sudden, Mike, it's, we're all outta loop, I think, you know, so we focus a really a lot on sort of expert in loop. Like let, let's look at a particular, um, place where the shadow, it can become a problem.

There's great information out there, right? And these LLMs have brought vast amounts of information to, to, to, to patients, to clinicians. And you know, if you've ever used any of these tools, you can see that there's, there's can be good information coming through there, but also potentially information that is, you know, unclear or incorrect potential for hallucination.

You may not know the audit, the provenance and sources of this information. And I think that's an opportunity to really focus down and say, listen, we can bring tools into the workplace where the information is trusted, verified, auditable, you know, the provenance, you can see the gen AI tool, you know, potentially giving answers and insights and guidance and recommendation fast, which is good cuts down on, um, the burnout, the administrative burden. But you can, you don't have to forego that ability to verify that you have trusted information that, you know, basically you may not get if you're just using sort of these tools that are on, on the internet.

Right. Mike, does that kind of help answer the question? Sure, it does.

Um, what's that one thing you see people doing today that kind of makes you shake your head a little bit and go, folks, we need to be a little bit smarter than that? That's a really good question because, uh, this, this is personally a professional, right? I think there's such excitement around what the technology can do.

Just be very careful when you think about, you know, you want to get, you know, an answers quickly and, and it's really neat and it's a cool technology. It can maybe help you with, you know, doing things at work, um, there, you know, make sure you're using tools that are, that, that your organization has said are okay, right? And when you're using tools on the internet, what can be helpful?

Like I think about, I'm not gonna put my LinkedIn profile in there, right? You know, Mike, what we talk about, we use this term, but kinda like what happens in Vegas stays in Vegas. These LLMs have this same sort of, uh, characteristic where they're so large, the information is so vast, they're so expensive.

The minute you accidentally tip your hat and share something in there, it's gonna be very difficult to ever get it back out, Mike. And the way the models evolve when you get generation after generation, that information could be in that fossil record for virtually ever, right? Mm-hmm.

And gen AI has really caused kind of a conundrum where maybe in the past you could, you know, extract information that you really shouldn't have shared. That's almost impossible now with these models, right? Right.

And yet the first thing many people are doing is putting their resume into the llm and you as does up make it better. And by the way, here's a link to my Facebook profile. Yeah, yeah.

And I think, you know, creation, creation of content, you know, who owns that creation of content? It gets subtle and nuanced and detailed. You know, make sure that you understand the terms of use, uh, if you're using it to help you, you know, do something professional, right?

Make sure you actually own that. Um, which is another sort of consideration here. Yeah.

Mm-hmm. Do we need to certify the LLMs for use cases? And, you know, we already have a whole boat boatload of healthcare regulations, right?

I think that they would apply here, but, um, you know, what is the appropriate level of regulation here without necessarily requiring act of Congress? Right? Right.

I mean, so mean, obviously there, there's pending, um, regulation that's coming out in, in, in my position in my organization. We look at it, you know, very specifically like the EU AI Act. Um, I live in Colorado.

Colorado has an AI act. So you're starting to see these things develop, um, and I think it's really important to focus on, on them, um, not just sort of individually like, well, somebody in the organization does privacy or compliance, like make sure everybody has an equal footing. So you really have that line of sight on how this can change, right?

Could be very, very impactful. And you need to be prepared to understand, you know, where is the line? Do, do you need to be HIPAA compliant?

Are you dealing with PHI or is that not necessarily part of this workflow or the solution? And these are questions you need to be prepared to ask and answer quickly. And you have to be pretty concise about where you wanna play and what it's gonna take in order to like maintain compliance.

If you do feel the need that you want to use these technologies in certain use cases, you don't wanna be caught off guard. The other thing I think, you know, you need to be a little bit careful about, and correct me if I'm wrong, but suddenly country of origin for the LLM matters, right? I think the open source LLMs, and a lot of these, um, you know, we saw something really interesting in the news cycle about this model called Deep Seek, right?

So I don't know what to name too many of these brands, but they did something fascinating where they built this very, very powerful model. Um, and, you know, models like that, you know, where you don't really understand necessarily the full provenance, they could become problematic. Um, we work with a number of partners, uh, to make sure that we're getting access to, um, you know, models that are trusted, verified.

We know the, we know the provenance, we know exactly what we're getting. We know that we're deploying in trusted zones. We, we can know the policy around data retention and what's collected and what's not.

It's really, really important to focus on that. 'cause again, the idea, these, these models wanna learn. And if you're not careful, um, if you're using a particular kinda shadow ai, I forget the quote, but like, if you're not sure who's paying for this product, you're the product and you are potentially feeding these models through your interactions with what will be the next generation.

And that's a realization. I think people need to understand that, uh, they're, they're potentially harvesting interactions to make the next version of that model. And we're very specifically a Walter Cork focused on making sure that does not happen.

All right, folks, I heard it here. Hey, these LLMs and these agents, especially when it comes to healthcare, they keep a secret about as well as a 5-year-old. So be careful out there.

Alex, thanks for being on the show. Yeah, yeah. Wonderful.

Thank you, Mike. It was great. And thank you all for watching the latest episode of the Techstrong AI Leadership Series.

I'm your host, Mike Zern. You can find this episode and others on our website. We invite you to check all those out.

Until then, we'll see you next time.