Automated Patching with Adaptiva’s Deepak Kumar

This is Textron tv. Hey guys, thanks for the throw. We're here with Deepak Kumar, who is CEO of NF tva, and we're talking about how to approach patching in this post CrowdStrike debacle era.

I mean, uh, things are hopefully all back to normal at this point, but there are lessons to be learned. Deepak, welcome to show, Uh, Mike, nice to meet you, and thanks. We have been kind of, flirting is a word I would use with automated patching for a long time.

Now, some people don't like it. Some people do like it. Other people are apply it in some places, and other people are like over my dead body.

Um, walk me through this a little bit from, uh, in the week of this whole thing that went down involving CrowdStrike and Microsoft and all these other places. What should we be thinking about in terms of how we patch systems? How we update them?

Did, do we need to rethink it? So, you know, 20 years ago when I was designing Microsoft's first enterprise patching solution, it was a very simple word. You know, you just patched windows and office and some Adobe apps, and you were good.

Well, things have changed now, last year for each working day, about a hundred vulnerabilities were disclosed. 75% of them were exploited within three weeks by, uh, bad actors. And these are well-funded organizations, often that by America's adversary nations.

So, so, uh, 70% of companies, we, we, we have spoken with believe that human teams cannot keep up with this workload. And this comes to about 20,000 pat availabilities disclosed last year. So I think manual patching is not an option if you want to maintain security and, you know, an adequate posture in the face of, uh, the relentlessness of the attacks that, uh, any, any company with intellectual property faces today.

So I think manual patching is not an option, even though it may be a reflect reaction to the incidents that have taken place recently. I agree with you in the sense that there are far too many vulnerabilities that are being reported. Um, but a a lot of organizations I talk to, there's a, their reactions run the gamut, and the first thing is they're afraid to upgrade something for fear of breaking it.

And I can't help but wonder if, we've reached a point now where the, the fear of not patching something might be greater than the fear of breaking it, because the, the cost of and the economics of the breaches are changing. Yeah. Uh, so, you know, the reflex reaction, uh, you know, it doesn't point, uh, doesn't point in the right direction.

The, a more sophisticated approach is needed where now, keep in mind, most bad batches are very particular. They will not affect all machines. They may, they may affect some parts of your environment.

They may not impact your environment at all. So the only, the only reasonable way to find a bad patch that affects your environment is to deploy. Now, obviously, you shouldn't deploy it on critical machines first, so the right way, you know, a more nuanced, sophisticated way of patching is to deploy very aggressively, but in the noncritical parts of your environment, and then automate the process so that when, when these a hundred patches get released every day, you're not relying on human decision making.

You know, we, uh, we have to rely on AI and autonomous patching to make consistent decisions on which patches need to be deployed first, where they need to be deployed, and what parts of your infrastructure are critical, and what parts are non, non-critical. So a sophisticated automated approach to patching is really statistically the safest way to patch Mm-Hmm. Shouldn't I have some sort of testing environment for the patch?

And so I can at least see what's gonna happen even before I roll it out to the noncritical machines? Um, and the land of DevOps shall hear the phrase canary sometimes. Um, you know, is, is that part of our thinking here?

We just need to kinda think it through better. Well, you know, for, you know, these are very complex times for enterprises. There isn't one canary that's possible, right?

So, so you have to have a broad spectrum of, uh, of devices, uh, in different parts, in different roles and different personas, uh, that need to be targeted in, in waves. And, uh, and, you know, then you need, you need to have automated processes, which can detect problems, uh, in the first wave, and then a human approval and, and then a and the next wave, and then a human approval. So you have to automate a consistent, repeatable process that addresses successive waves of deployment, starting in the less critical parts of your infrastructure, and then moving on to the more critical parts where, uh, uh, you, you wanna make sure that the patch has, uh, been deployed in other parts of the environment.

First, Not all patches are created equal. Um, in the case of CrowdStrike, there was at the kernel level, and I don't think that happens very often when then a lot of folks are saying, well, now we can't have agents that touch the kernel, so we're gonna go agentless. Um, then other folks say, well, agentless is, doesn't have the level of control that we need.

What's your sense of, um, you know, do we need to kind of, um, create some level of ranking of patches so that we can apply some more freely than others? You know, agentless patching is very dangerous in the sense that the process that is performing agentless patching has to run in a security context that has permissions across all your devices. Uh, that's a very, uh, slippery slope.

Whereas, you know, the agent typically that that is running on a device has permissions only on that device. So in case of a compromise, uh, your entire enterprise is not compromised. So it's very important, uh, to recognize the security risks associated with agentless deployment.

Ultimately. When is your best advice to organizations about how to structure themselves for this? I think part of the issue is it's not so much the technology, it just feels to me it's a people and a process thing sometimes, because, uh, what we did see is when things went wrong, nobody seemed to have a really great incident management plan to respond.

Yeah, I think, I think it exposed, um, a couple of things, right? First of all, people didn't have control. Um, these days, uh, a sophisticated patching software is available, which provides you control.

You can pause it instantly, and when you see things going wrong, in case in, you know, in, in case of the CrowdStrike incident, CrowdStrike responded within the first 90 minutes, it became known. And if people had tools that could prevent further patching in real time, they would, they would've much less exposure. Unfortunately, not many companies had those capabilities.

So those lack of those capabilities has been exposed. Uh, and then, uh, even fewer companies had automated recovery, uh, capabilities. So I think that is really, uh, in the fix.

You know, you will always have bad patches. Bad patches are released all the time. You know, this one caught the news cycle, but there are a lot of bad patches released, uh, that we see every month, and a lot of them have clo uh, caused, you know, last one, there was another patch from Microsoft actually that was causing a lot of blue screens.

Uh, it didn't just ha uh, it went into the news cycle, but it was, it was out there and, you know, it was rebooting machines, um, in a loop. So, uh, the solution is not to be afraid of patching, but you have to do it correctly. You have to do it, uh, with sophisticated tools, with realtime capabilities, with controls in place, and a recovery mechanism in place where you can recover from a, from a bad patch, uh, that is going up.

Aren't we too dependent upon antiquated systems? And I asked this question because in one case, it's clearly, um, they're running a, an older version of Windows, but in another case, somebody was running even older versions of Windows and they didn't go down 'cause they weren't impacted. So, um, the question becomes how current do we really need to be when it comes to patching or 'cause, uh, to me it seems like there is just a lot of older versions of things out there, and it's not quite clear to me that they're as easy to patch as some of the newer things.

Yeah. You know, when you look at the larger and more complex environments, the, the amount of data needed, um, to be served is, uh, is just phenomenal, right? So, so there was a Microsoft patch, uh, which was pretty extreme, um, a couple of years back.

It's called UUP. Each device needs 10 gigabytes of data in order to deploy that patch. If you have hundreds of thousands of computers, that's petabytes of data.

So the amount of infrastructure needed just doesn't exist. So there are software, uh, solutions available, um, which can appear to be transfers where, uh, you, you can, uh, actually keep systems updated, uh, uh, in, in real time, essentially, uh, without impacting, uh, your production systems. So my advice would be the first step is to modernize the system that modernizes your environment.

If you're using an antiquated patching system, you're not gonna get results to modernize the rest of your environment. And, uh, you know, running, uh, running these, um, uh, these antiquated systems, which are no longer being patched by Microsoft, uh, with customer data on it, and, you know, in any regulated industries, um, uh, I think that would be an excusable. Um, if you live by the sword, you die by the sword, as it were.

Um, can AI help us here? It seems as, uh, listen, so you describe this way, have a level of complexity that's beyond what humans are capable of keeping track of. So that's usually a call for AI immediately.

But, um, do we need algorithms to come in here and help us sort what to patch when? Absolutely. And, you know, uh, this is some simple tools are available, right?

In the sense vulnerability are already measured in, in the impact. And if you had a more nuanced patching system, which integrated with your vulnerability management systems, you could actually patch more, uh, more exposed systems first. So if you are correctly prioritizing systems where the trade offs are better initially, where you're, you're, you're neutralizing more risk on less critical systems, and that's your starting point, and that becomes your canary, and then you progress towards lesser ROI and, and, and your most critical systems get patched in the end.

If this methodology had been followed, uh, by some of the airlines, uh, uh, that are expressing so much unhappiness now, uh, I think their, uh, their outcome might have been quite different. We also hear people talking about the concept of virtual patches, and is that something that we should have in our arsenal as well, where, um, you know, it's not quite a physical patch, but it'll, you know, get you over the hump for a a period of time at least. Yep.

So, you know, I worked on the Windows team at Microsoft for many years. You know, um, uh, I would, I would deploy an actual batch. Um, the, uh, some, some of the vulnerabilities, um, are very severe and, uh, uh, increasingly, uh, uh, the adversaries, uh, you know, the consequences of compromise, uh, have multiplied.

We have seen numerous, uh, uh, cases in the very recent past where, you know, the entire automotive industry was brought to and sneeze for, uh, for weeks and weeks and weeks. You know, billions of dollars in damage was cost. Uh, so the consequences, I mean, this is not something you should risk.

Um, I think in the adversaries, adversaries are out there and, uh, nothing good comes from it, right? So, uh, I would, I would actually patch, uh, machines, uh, take care of all exploited vulnerabilities, um, in, in certainly less than three weeks because, uh, uh, you're, you're exposed and, uh, a lot of, uh, you know, the, the game has changed. This is not the fat boy in the, um, uh, in the dorm anymore.

You know, these are teams with hr, with security, you know, funding. They have own, they have their own data centers, right? There's a lot of trained individuals.

Uh, there's whole commerce around, uh, sale and purchase of undisclosed vulnerabilities. Uh, uh, it's a very sophisticated, uh, uh, criminal syndicate now that, uh, we are all facing and, um, uh, preventive security, uh, is really, uh, the best thing you can do to protect yourself. Mm-Hmm.

You quoted some numbers up top, and I'm finding a challenge to get some consensus around, well, just what percentage of vulnerabilities are actually being exploited? Some people say it's only three to 5% of them, and, uh, the bad guys are just going after those 'cause they're easy to find and they don't want to do a lot of extra work. And then other folks are saying, to your point, every time there is a new vulnerability discovered, you know, somebody tries to exploit this thing in a matter of days and sometimes hours.

Um, and it's not clear to any people understand, well, just what is the level of risk and how do I manage that? Yep. So, you know, my recommendation is, uh, you know, you, even though, you know, I make software that allows you to copy bomb all vulnerabilities, uh, it's not tactical to, uh, to patch every vulnerability immediately.

So, uh, the, the more, the most nuanced solution is to integrate your patching solution with your vulnerability solution of choice patch based on a matrix of, uh, the exposure level in the criticality of the system. So if you are patching, uh, in a sophisticated nuance algorithm that works for your company where you're patching in the first wave, you're patching less critical systems for the most extreme vulnerabilities, and then you progress, uh, towards s lesser ROI where you're, you're addressing less severe vulnerabilities on more critical machines. So that is really, uh, now of course, you know, these algorithms are impossible to implement manually.

'cause you know, Mike, think, think of it this way. There's a hundred patches released today. Can a human or a team of humans go research each patch and make a decision which to deploy where?

Well, theoretically, there might come a day when I could take the patch and run it up through Gen ai and it'll at least tell me what it's about. Yeah. But, you know, a piece of software can produce those results with far greater consistency.

Uh, it's not gonna make a mistake. It's, it is less likely to make a mistake than a human. Uh, software is not lazy.

It's not stupid, it doesn't get mad at the employer. It just, uh, you know, it just continues to work. It's very fast.

It, it can do repetitive tasks much better than humans can, which is why, you know, after all these years I've reached conclusion humans should do what the best or do best right. Defining strategy and process and software should do the rest. So, as you look at all of this, and you, and you've been around a while, what is your best advice to IT leaders about having to have this conversation?

Because I feel like sometimes, you know, there's so much fear of the patch that, um, it's become somewhat unreasonable to your point, but, but it's not clear to me anybody knows how to get started. Yeah, I, I, I think, you know, uh, the, the biggest thing is, uh, to, uh, to be objective. Uh, you know, these incidents, um, uh, ca a new cycle, uh, uh, you know, if you, if you look at the statistical, uh, sort of percentage of machines affected, right?

Uh, uh, it wasn't that high. Uh, a lot of organizations that were well prepared actually occurred, uh, fairly quickly. So, uh, so I would do a couple of things.

I would, I would absolutely prioritize my recovery capabilities. Um, it's not just a bad patch that could cause, uh, a disaster. Um, there, there are many other situations, you know, um, natural disasters, network outages, right?

Business continuity, um, uh, is needed for other reasons, uh, beyond just bad patches. There are, there are a lot of, um, uh, unforeseen, uh, circumstances that you're be prepared for, but they're also patching, you know, having a sophisticated process, investing in the right tools, um, and using AI and autonomous, uh, especially the, the modern, so autonomous tools, uh, instead of relying so much on human effort, I think these are, uh, these are the lessons for IT leaders. All right?

And folks, you heard it here. There are a lot of scary things out there, but the more we understand about them, the less terrifying they become. Hey, Deepak, thanks for being on the show.

Thank you so much, man. All right. And back to you guys in the studio.