4. Security Audits Cause More Harm Than Good – Tech Field Day Podcast

It Audits are a fact of life. We are always going to have to make sure that we're doing what we say we're doing, but sometimes it feels like we're not getting it done. And that's why it audits lead to more harm than good.

Welcome to the Tech Field Day podcast, where each time we meet, we bring together a group of IT experts and practitioners in their field to discuss or debate a topic or a premise of relevance to those in Enterprise it. This podcast is brought to you by the Tech Field Day Event series. Before we introduce the premise for today's episode, I'd like to take a moment for our guests to introduce themselves, starting with Ben.

Hi, my name is Ben Story. com. Hi, my name's Sky Few Yate.

I'm with Netsmart Technologies, and you can find me on LinkedIn, sky dash f Uh, Taryn, Bryson and I do stuff with things and other stuff, And I am Tom Hollingsworth part of Tech Field Day, as well as the Future Home Group. And let's jump into the premise for today's episode. If you're in security, you've had to deal with an audit.

We all ha someone has come in with a checkbox or a list of things that they need to check off, and we need to make sure that you're meeting all of the policies that are in place. And nobody likes these, right? We've, we've had to deal with them so many times.

We just roll our eyes when we hear the term auditor. And odds are good that whatever they're gonna find is probably something either irrelevant or so difficult to fix that it's not even worth the time. So the premise for this episode is that audits cause more problems than they solve.

You're all security people. You've all had to deal with audits. Do you like 'em?

Love 'em. I detected a hint of sarcasm in your voice sky. So, so what is it about a security audit that just grinds your gears?

You know, it, it's, it's the ability to kind of understand the environment a little bit better. Um, it, there's, there's good and bad that come with that right there. There's the ability to better understand my footprint and what's kind of out in my environment.

Um, but there's some drawbacks that come with that because there's things where we may reveal that there's, uh, non-production gear, um, that are coming up with security vulnerabilities that we may not necessarily care about, that may not have, um, production impact. Um, and, and that kind of, you know, distracts a lot from kind of the priorities of day-to-day for the actual production stack. Yeah, I, I, I mean, I'd contextualize audit though, um, into maybe two, you know, at the macro level, maybe two types of audits.

'cause we've got the audits that are foisted upon you by insurance companies or whatever, um, where they actually do come in with prescribed, um, you know, things that you need to meet. But a lot of audits, you are actually audited to the policies that you have created. And so you're basically auditing, you're, you're being audited to see if you're actually following your own rules.

So that audit, uh, you know, if you're not doing that, you just, you know, that's an unforced error on your part. Taryn, I'm, I'm gonna have to ask, um, you're saying that in certain organizations there are people who don't follow the rules? I've heard.

I mean, you know, I've never been at that organization, So, so, so you're saying like the CEO could, could maybe have a different policy than the organizational standard, and I don't have to follow that. And if you flag that in an audit, well, he's the CEO he gets away with. That Could happen.

So in one way, um, like the external audit stuff we can talk about in just a minute, but let's focus on the internal audit stuff now. You're basically saying, we are following the policies that we ourselves put in place. We are, you know, eating our own dog food, for lack of a better term.

Um, you know, that, that to me sounds critical, like, especially because you want consistency across the organization. Mm-hmm. Yet we have to do these things constantly, because there's always an exception to the rule somewhere, right?

I mean, ideally not, but yeah. I mean, there's, there's always gonna be a reason why something, you know, in the policy isn't being followed, or, and, and, you know, in a lot of cases what it is, is you've actually changed the policy, you know, maybe up here or, or, you know, as far as your runbooks and things like that, but they haven't been updated. Mm-hmm.

And, and so you're kind of catching you, you know, you're catching mistakes if, if you will. It's, it's not so much that there's a exception because that person's special. Although that could happen.

It's, it's one of my old favorite things from the networking space. There's the desired state of truth, and then there's the actual state of truth. And if there's a gap, you need to know that.

Yep. Yep. Well, and, and I think, um, those exceptions, when they do exist, it's important to know what they are and have them documented.

Because you, you can't just go by tribal knowledge and know that, oh yeah, we, we let the CEO slide on that because of X, Y, Z because the next guy that comes into that position won't know that. So let's, let's document it, say that the business accepted that risk. Here's what we're doing to mitigate the additional risk, if anything, and why it was approved or who approved it, you know?

Mm-hmm. Take that onus off of the IT guy and put it onto the business that, okay, yep. Business has said, yep, we're gonna make this exception.

And it's documented so that when that external audit shows up, they don't take, take your policy and go, you're not, you're not following your own policy. Mm-hmm. Mm-hmm.

Yeah. I mean, it's, it's, you know, I spend a lot of time crafting policies and getting buy-in from, from other, you know, folks around the organization. So the audit, I, I find, you know, the occasional, I will say that the occasional audit I find valuable because it does, you know, point out deficiencies and gaps in, in process and things where, you know, something should be followed and somebody's just simply not doing it, or, or it's, it's falling through the cracks.

So that's, you know, I find value in that. Um, you know, definitely, uh, some of the external audits that get, you know, like I said, foisted on you from different quarters, maybe those aren't quite as fun. Let's, let's just, just jump into those then, because one of the things that we've seen a lot is external parties, especially if they work with a third party vendor or they're some kind of a regulatory body, they have certain criteria that they think that you should meet, because that just makes everything better, which is how we end up with 14 character passwords.

That must include five different complexity requirements, and you must change them every 30 days. And as we've learned recently, even according to places like nist, the US federal government, you should not do that because it actually causes more harm than good with, you know, to rotating your credentials too frequently just means that people are gonna have to write them down anyway. And so these external things that are designed to make everything better, because they're creating a baseline for security, actually end up causing more problems in places where the rules are misapplied.

You know, have we ever experienced that where some other organization was like, suddenly, oh, you can't have a guest wireless network because it could potentially impact your PCI compliance or something like that? Well, I think a lot of the external, um, audits turn into check mark vests, and it's a, it's a binary thing. It's either yes or no, there's no allowance for a gray area, but you know, things like the, uh, password, uh, complexity that you're talking about, uh, you know, it's somebody at some point decided that was a great idea, and so it got put on all the checklists and to comply.

We still have to do that. Now, best practices now from the standards bodies is, that's not the way to do it if you, if you have MFA en enabled, so, but we can't go down that path because the people that write the checklists never go back and update that. Checklists are always additive.

They're never correct refactored. Mm-hmm. Well, and, and to that point, there's times where you may, uh, have a password complexity requirement, but then there's technology changes in the industry such as passwordless entry into your accounts, and you're never really going back and retroactively updating that document.

Um, and so while yes, it's your policy, how do we take account into the new, uh, industry best practices of, well, maybe passwordless is better, um, and maybe we want to start using that, but who's thinking about, oh, we need to go update that policy. Yeah. And it's some of the big established, um, frameworks, uh, that get updated the slowest.

So like iso, ISO 27,001 for, I mean, how many years was that not touched? Right. And, and it just re I think in the last year it just got updated.

Um, so it's current now, like 20 23, 20 24, something like that, that, um, but what, where you do find, uh, a non additive sort of policy that, that keeps up with the times is something we were talking about earlier, which is cyber insurance. And the reason for that is there's a lot of money on the line. So they have, you know, they have a reason basically to, uh, you know, to keep, uh, up with the times and actually say, okay, this is the new best practice.

They don't, they're not gonna just have a big checkbox, you know, fest like you said, because, um, you know, it, it, it doesn't serve their purpose, you know? But yeah, some of the big frameworks and some of like socks, uh, you know, stuff like that. I mean, yeah, those are, you know, those are, I don't, I don't wanna say worthless, but they're, They're security or compliance designed by a committee.

Yeah. And the committee has to vote on what they think works the best. And unfortunately, we've seen over a number of years, the clean room implementation of any standard or regulatory compliance issue never survives what happens in reality.

Think about something like, um, you know, my favorite one is the smoker's door. It's like, if I have to constantly buzz in and out of, of security, you know, that's a hassle for me. So a lot of places, like, especially back when smoking was more in vogue, there was a door that you could just kind of go out and prop open a little bit and, you know, you'd go in and out and you didn't have to badge in and out.

It was much easier. And certain penetration testers would look for those doors and wait for somebody to pop out, and then they'd have a cigarette pack in their hands and they just slip in the door. And then they were inside the company and they bypassed all the physical security controls.

Yep. And auditors right or wrong, didn't even think about that as an entry point violation. 'cause everyone had one.

Well, and I mean, good auditors, you know, can bring value, right? Even with bad frameworks, you know, a good auditor can can get through it, but a bad auditor, uh, I had one years and years and years ago, and it was a Sarbanes Oxley, um, audit. And we got, you know, we were obviously dealing with the IT portion of it, and we had our backup software and it, it, I forget what it was, but it put up all these, uh, you know, red xs whenever anything failed.

And so she was gonna fail us on, on this audit because of all the red Xs. And I said, well, you know, this is, these, these are minor files that're not being backed up. So in other words, when you back up, you know, whatever, a hundred thousand files and two fail, you get a red X on the, you know, we just had the reporting threshold, uh, set pretty high.

She's like, well, no, I can't pass you on that. So I said, hold on. And I made a couple changes right in front of her to the thresholds.

Everything went green, green checkbox. She's like, oh yeah. She's like, okay, I can pass you now.

So in your mind, what I'm a bad, that's a bad auditor. In your mind, what is the difference between a good and a bad auditor? What, what, what makes them a better auditor?

Well, I think knowledge, first of all. I mean, I think, you know, if you train somebody to be an auditor and they don't truly understand what they're auditing, then you get somebody who's going by the rule book and, and has no flexibility, no ability to understand, you know, nuance or context or anything like that. So, um, somebody who's been in the industry, you know, as a practitioner and has transitioned into a role like that, I think that's, you know, where you tend to get your better auditors 'cause they actually understand what they're doing.

Mm-hmm. Uh, any other thoughts in your mind? What makes a good auditor or a bad auditor?

I, I, I think knowledge is the key. Mm-hmm. Um, even if they're not a practitioner, they actually understand what they're looking at versus just, you know, comparing, does this match this?

Mm, maybe. Okay. Yes.

Mm-hmm. Um, Yeah. And somebody that's actually able to take in the nuances of the business and, uh, maybe if they're given a sheet where it's all right, well, these are the assets that we have enumerated.

Uh, maybe they don't take that at face value and they actually look and they're like, oh, well you have x, y, Z router that wasn't on that list. Well, that doesn't meet your standards at all because it's first not even in your inventory system that you gave me. So, I'll, I'll dig a little deeper in here.

The, because of, uh, other stories that I've heard, um, what about an auditor who believes that no company is 100% compliant and they must find something wrong? I mean, not necessarily in it, but I've heard stories from the Occupational Safety and Health Administration that if they can't find a violation, they pull out a tape measure and start measuring railing heights because somebody in here has made a mistake and we've got to find it because nobody can be a hundred percent compliant. Do those kinds of auditors end up causing problems?

Because now it's not a matter of whether or not you are meeting the criteria that you've set forth for your regulatory body or your own internal policies, and now you're just, you've got a person looking for problems. Well, I, yeah, I think that can, I mean, I haven't had anything that extreme happen, but, um, you know, when you, I mean, the thing is, when you get audited, obviously you've got a period of time after the initial findings are presented to remediate and they come back and they, you know, check to see if your remediations are successful. So, you know, if they're finding, you know, little ticky tack things that, you know, really don't need to be fixed or they should be low priority, but you have to drop everything else in order to fix those things, to satisfy the, you know, whatever, um, you know, audit you're going through.

I mean, yeah, that's, that does more harm than good. Certainly. Uh, that's 'cause that's a waste of everybody's time.

Um, but I, you know, I, I tend to agree with that premise, though. You know, I think it would be very difficult for any organization to be a hundred percent compliant with everything constantly. That's, that's pretty impressive.

If they are, I've never worked for an organization like that. So maybe the problem to me, But maybe the, the issue is, is that there we have to categorize things properly. So these are major risks, these are minor risks.

But as we're learning, uh, through a lot of security companies, minor risks can come back to bite us later because of, uh, newly discovered information. So, like, for example, up until about five years ago, having, uh, A-W-S-A-P-I keys and clear text files in your organization wasn't a huge deal. But now that people are actively looking for that information, it's considered PII or, you know, secret information.

And if it's unstructured data, who knows what I might have out there? How many copies could have been made? You may be violating policy without even realizing it because your developer saved, you know, uh, API key dash V one dash v two dash final dash final, don't edit, uh, to some random corner of a, a sand somewhere.

And the only reason you know about it is because you now have a $10,000 AWS bill because somebody got ahold of it and has been using it. So, you know, can we unwittingly find ourselves in violation of policy? Because the, the growth of data and the ability to create these security risks has changed exponentially from the good old days when everything required a physical security key or things like that.

Yeah, I mean, I think, you know, that's, that's absolutely a possibility. I think that if people don't keep up with, with technology, um, you know, so, you know, for example, uh, GitHub, uh, most people use GitHub now nowadays for, you know, storing all manner of things, not just, you know, dev software, but, uh, you know, whatever. So GitHub has GitHub security and, you know, I'm not advocating for any particular solution.

There's others out there, but it'll actually scan for, you know, pass keys and, and, and things stored in, in GitHub that shouldn't be there. It'll, it'll scan for, you know, APIs that are written in such a way that somebody could exploit them. There, there are other tools certainly out there that, that do the same thing, maybe even better.

Um, but if you're not using those, if you're kind of the old school person going, well, I've got a firewall and we've got, uh, we've got, you know, antivirus on our end user compute devices, and that's good. Well, you know, you're, you're, you're certainly in a, you know, world of heard if, uh, if anybody, you know, comes after you, Let's talk about those kinds of solutions because we're seeing a lot of them hit the market, uh, data loss prevention that won't allow you to send an email with a nine digit number that includes two dashes, or, you know, a system that's constantly looking for driver's license numbers in photos that are stored on a, um, on a system somewhere. Could those tools eliminate the need for audits because they're scanning in real time, they're constantly ensuring compliance.

Do I really need somebody to come back and check now? Yeah, I think so. I mean, I think those tools, I mean, automated tools are great, but you know, they can screw up, you know, because you still have to set up the automated tool, and if you set it up wrong, um, you know, it just allows you to, like, the old thing with, with automation on the networks, right, is it allows you to fail faster.

Uh, you know, more completely more, yeah. More completely. Exactly.

Yeah. And, and something that I've seen too is, is when you build those automation, uh, tools and, and initiatives to be able to check those things, uh, part of the problem is you may not be able to take into account every edge case. And so there's times where an auditor would be able to say, oh, well, if you're gonna meet this condition, then that fails your audit.

Uh, but there's times where maybe a tool might not know how to do that, or something in the en environment has evolved to where it may not understand it. That's fair. I think the, the problem is, is that we get back into that nuance question of, well, in this specific scenario, this particular condition is a failure.

The auditor has to know what combination of factors can cause that to happen. Like we saw in a field day presentation during security field day, um, a particular condition with a VM could say, oh, well this is exposed publicly, so it's a security risk, except every bit of data on that VM is encrypted. So even if it's exposed publicly, it's a low priority, because even if people can get onto it, they can't read it.

So, you know, are we, are we needing to create more context around the situations that we're in to help people understand that we're still meeting the risk profile that we've created? And in doing so, how do we do it so that it doesn't look like we're trying to get off on technicalities from these audits? Yeah.

It makes me wonder as we go forward more and more with the audits and whatnot, and we, technology just keeps evolving so much. If we almost need to get to the point where we're sort of like, uh, home inspectors, you know, the home inspector says, well, I think this is wrong, but you should have a licensed electrician check this, or a licensed plumber check this, the, these auditing companies, and some of them do, they have people that the auditor can reach out to and say, Hey, can you help me review this? I think this is a finding, but, and you know, so we're gonna have to have more experts helping the expert, if that makes any sense.

Yeah. Our, uh, a past company I worked at, our internal audit team, uh, that ran, uh, they, they mostly were there to do SOX audits, um, before we had our external account, uh, accounting team do it. Um, they would come in and, uh, they always brought with them, uh, a fairly senior level IT person, whether it was, you know, somebody with a network background, somebody with a systems background, but basically somebody who was in the industry and, and wasn't an auditor by trade, they would come along almost like when a sales rep comes out and brings an engineer with you, with them, you know, it's like, here's the, here's the person we can talk to for that context.

But Right. I haven't really seen that outside of, outside of Sarbanes Oxley. And, and outside of like internal audit teams at larger companies, um, most of the audits that I've dealt with in recent years have been, you know, just, just one auditor.

Or in some cases they actually just send out a form and you, you know, some, an officer of the company has to attest that these are all, you know, uh, accurate that we're what we're telling you, uh, and they call it good. So, um, I don't know how much value there is there, but Yeah, I was gonna say the forms are the one, those are the type of audits that really scare me because you get people just going, yep, we got that. We got that.

They don't even understand the concept that they're saying that they're attesting to, which I can't fault the average CIO or CEO for not knowing every little nuance of technology, but like you said, they're attesting to it. They're putting their, their name on the line, uh, in some cases with liability. Oh, yeah.

Legal. Yes. Legally Binding.

Yep. So, as people who have had to survive audits, um, what's one piece of advice that you can give to people who may be finding themselves facing their audit for the first time to help make it a positive experience for everybody? Maybe the process itself isn't positive, but we want to have a positive outcome from the audit.

How can we make that happen? Well, I always tell, I always tell my my team, if you talk to auditors, you know, always, you know, tell the truth. Absolutely.

Um, 'cause you know, it'll get us into a lot of trouble, uh, otherwise, but don't volunteer more than you've been asked. And the example I use is if somebody says, do you know what time it is? The proper answer is yes or no.

Most people, what do they do? When you tell you the time, what time it is, they tell you the time or, you know, do you know the time and say yes, and they tell it to you? That's giving more information than what's asked.

And that's what I, you know, tell my team because auditors, they'll ask a question and then somebody will start blabbing about all this, and they're like, oh, that's interesting. You know, and then, and you've just expanded the scope of the audit. So It's the courtroom answer, it's answer the question that was asked.

Yep. Sky. Yeah.

I, I think the biggest thing that I can say is assume positive intent in the audit. Right? At the end of the day, we're not trying to, uh, just discover everything just to shut down the company.

That's not the ultimate goal of it. It's, it's really to improve, uh, your environment and make sure that you're doing what you're saying you're supposed to be doing. Um, and just be, be open to kind of what those results are.

That way you're, you know, doing the best job that you can for your environment. Yeah, I think, I think positive intent is a good way of putting it and being open to the, uh, I don't know if suggestions is the right word, or the finding, just being open to the findings because it's an iterative process. Security is constant.

Uh, the day that you stopped trying to update your security is the day you probably got breached, um, because you stopped changing. Um, I wish that the standards bodies and the auditor forms or whatever they're using kept up, but, you know, it is what it is. Well, and I, I wish our policies kept up.

Ultimately, an audit is the second half of the maximum trust but verify. We need to make sure that we're doing what we say we're doing, whether it is our own policies or somebody else's external policies. There's a lot of nuance, there's a lot of understanding, there's a lot of knowledge that goes into that, but you have to be sure that everything is working the way that it's supposed to.

And yes, a big list of check boxes that need to be fixed equates to a lot of extra work, equates to a lot of soul searching on the behalf of you and the people who craft the policy. But at the end of the day, it's better to have to uncheck those boxes now than to have to answer those hard questions in a legal proceeding or the news. And nobody wants that.

That'll do it for this episode of The Tech Field Day podcast. I want to thank everyone out there for tuning in. Um, if our guests would be so kind, just to tell us where can we find more about the things that you do online?

Ben? com or on X Twitter, whatever you wanna call it, at N Twk eight zero. Yep.

And I'm Sky Fugate on LinkedIn. com. Yeah.

And I'm on LinkedIn, uh, as well. Uh, Taryn Bryson, uh, easily found, uh, on most of the major social media platforms and even some of the minor ones under either some clown or just some clown. And I have a blog that I am rekindling.

net. com/podcast for the latest episode. You can also subscribe to our YouTube channel and to our podcast in your favorite pod catcher of Choice, just search for Tech Field Day podcast.

We'll be back with another episode next week. So stay tuned. Until then, take care.