Security Innovations at Veeam

I'm Rick Vanover from Veeam, and we are gonna talk about our second, second segment here at Security Field Day 13, uh, security Innovations at Veeam. And this really sits in the specific wheelhouse of our capabilities. Uh, I'm Rick Vanover, Emily Tes, And this is a good one.

So, Emily, walk us through kind of a timeline. You know, Veeam's had backup product since 2008. Yes.

But security's been a first party conversation here, but walk us through this history a little bit. Absolutely. So we've had this extended roadmap with a lot of really significant accomplishments that we've been doing or pioneering, right?

Started with instant virtual machine recovery, um, which was being able to take a backup and actually, or essentially run that, um, in the background so that way you could get that temporary machine up and running as quick or as close to, um, as, uh, operational as possible. But then we started to take a, a different look into different ways that we can help organizations. So I would say within this aspect of 2022 to 2025, and where we see now, this has been the bulk of the largest innovations that Veeam has had to offer or has made, uh, that offering.

1, which had a ton of new innovations around threat center. So being able to have a dashboard that's gonna highlight any potential threats within an organization, state protected environment. Um, looking into AI built in malware detection.

So being able to scan in line during the backup and identify any types of malicious activity or any types of encrypted data, or if we start to see large changes in compressions or algorithms, things of that nature, and being able to alert to that while that backup is actually happening and being processed. And then also looking into some additional options to do some proactive threat hunting. Now going into 2024, we extended onto that and we made our biggest acquisition, which was with Cowork and that incident response firm that specializes in helping with organizations that have been impacted from ransomware incidents.

They are the people that you call when you have to start negotiating specifically on behalf of a organization to a threat actor. Um, but one of the other pieces that they hold true is being proactive. So sitting down with customers and taking them through what does it truly look like to be in an active incident, right?

Do you, everybody says, you know, we have a no pay policy, but then mm-hmm certain applications, certain data leakage that could possibly happen or certain, uh, uh, customers that could be affected. Then all of a sudden the board changes their mind and they say, well, maybe we should be looking at, you know, if we do pay or, or what does this say from a legal standpoint? So they actually forced those conversations to happen.

One of my organizations I I actually worked with, they said it took them about a year to decide in what terms or what time would they actually decide to pay and what would have to be impacted, right? So if you take a year for a very large organization to determine that, I doubt people are gonna be able to make that decision on the fly when they need it the most. And then going into 2025, I mean, we've extensively kind of did this uplevel of not just security inter integrations with our ecosystem, with our partnerships, but then also funneling in any of that pertinent information into, um, what do we see inside of our data protection logs.

And then actually funneling that into, um, the security analyst hands. So, so when we talk to, um, cyber defenders or lead cyber defense, um, uh, people within their security organization, we wanna understand what information, what events are actually gonna matter to you most, right? Last thing we wanna do is send you 500 plus different events and now you're having to go through this entire alert fatigue and figure out, okay, which ones are gonna matter to me?

No, let's figure out which ones are pertinent that are gonna give you good examples as to what could be a potential threat. And then make that a little bit more easier for you to, to determine and flow into your own work tools. So those have been some of the biggest things.

3, we had a year full of security features and enhancements. So we started off with threat hunter. So signature based scans.

So this essentially is Veeam actually working with our own AV provided detection tool. So essentially we can actually go in, scan the backup once it's already been taken, and then be able to identify if there's anything, any potential threat in there, um, and be able to alert that to both the backup operator as well as the security admin. The AI based inline detection, that's the data stream that's scanning to detect, um, for any type of anomalies that's happening in line.

So again, just two different ways in which we're gonna be able to provide scanning both in line as well as after the fact indicators of compromise. We started to look at what are the types of tools that known, uh, utilize in order to compromise information. How are they moving within an organization?

Um, so indicators, our compromise is gonna actually help us identify those known toolkits and then bring those up for those organizations to see. The benefit here is you have that historical viewpoint. If it wasn't in the backup from two days ago and now it's in the backup today, did somebody install TeamViewer on that active directory machine?

Or was that somebody that did that maliciously, right? So being able to identify those types of items. And then we can also look at it from indexing as well as suspicious file activity analysis.

So again, just diving in deeper into the actual machines that we're protecting themselves. And then we're gonna show some of the security and compliance analyzers. So essentially looking at best practices around the protection of what we are, um, protecting in the overall environment, and then some additional items around role-based security and putting a spotlight on ransomware.

So with that, we'll change it over to, Yeah, first demo Rick for a demo. Yeah. So I'm gonna demo really the first three of these kind of in quick form.

Um, I love a good demo and let's, let's get into it. So I am gonna do this live. I do have a backup that is a recording.

But let's first start off with what we're looking at here. This is Veeam data platform and I want to draw your attention to a job that I made here called, um, XFD 13 Malware on Demand. Now, I've been playing with this technology for a while and you might say, well, why would I want my data protection solution to do this type of protection?

The good news is that we have plenty of examples. We can give you very diluted summaries, but we have plenty of examples of Veeam detecting anomalies, including exfiltration when the primary security process has not detected it. Now in the interest of, uh, magic of television, I ran this job literally, as you can see right here, four hours ago.

So I did a backup of a system that I have intentionally created something that will make this detection trigger. So I'm gonna go over to that and right away I can, yep. Sorry for interrupting.

Yeah, no problem. Jack. Yeah, with Uh, paradigm technic Cut.

Yeah. What we're looking at here, this is the Veeam Console, or what are We actually Veeam Console. Thank you.

Yeah. Um, everything I'm gonna show you is Veeam only. Some of the stuff Emily will show you is ecosystem.

Perfect. So thi this is that first bucket of integration, uh, uh, innovation, sorry, and this is all Veeam technology. Perfect.

That I'm, thank you. Yeah, no, good, good question because we're gonna try to jump through all of it, but we're gonna start with the VMO technology and, and good call there. So I did this backup right before, and you'll see that I have nine events.

Now, any of you that are really good will also notice that, uh, several of them are quite dated. I actually like to leave the log of these other potential anomalies in there for good reason, because some of 'em in the second column you'll see is actually marked as infected, which is a positive confirmation of a problem, whereas others are suspicious. Okay?

And I'll get to what that means here in a second, and I'll kind of show you right here that in that far right you can see whoop, double zoom, not uh, you can see that there was some known extensions, some encrypted data, which is a behavior. There was an antivirus scan that a signature matches what that means, right? So three very distinct pieces of logic that were detected on these image-based backups that we're doing.

And I think that's really cool. You get some, um, our tech support team likes to say it's a very highly calibrated smoke detector. Okay?

Um, when I go into what are we doing, how are we getting to this, I'm gonna go into the settings of this inline detection and I want to draw your attention to this one right here. It's a little slider that just predicates how aggressive I'm going to be on this scan. You know, a lot of times for show and tell that I do often, I'll put it on the extreme side to kind of catch anything.

Uh, I will say there's sometimes some false positives, but I think any security practice, that's a normal. Okay? Now that being said, uh, Emily's gonna talk about the incident, API as well as some of the notifications that go with this later on.

But the technology we're using here is really, it's just a checkbox in the end. We've made this like machine learning powered technology to take a look at this data and look at behaviors like files that contain onion links or a whole bunch of data that wasn't encrypted yesterday, but is today or worse, a whole bunch of data that's gone that used to be there. All kinds of things are there.

You look at specific files. Ah, yes. Yep.

So you recognize files, formats, yes. And I'm glad you asked Lars from Norway, because I wanna draw everyone's attention to, and I really love showing this, uh, right here. This, we have a knowledge base article, Veeam KB 45 14.

And in this, we actually, we actually published a list of files that we're looking for. Now, of course, the threat actors are looking at this too, but this thing changes probably two, three times a month. We try to take these learnings from the Cove ware acquisition.

So what can we see in these backups that'll help feed that? Now it's super important to note if I'm a threat actor, okay, go after this file. Can't do, it's digitally signed.

So we've protected the manipulation. But what I want to do, Lars, is show you what this file is. Now, I pulled up the actual XML file.

Um, I remember when this was launched, it was like version 40 or something like that. We're at version 1 0 3 we right now. So it is constantly updated.

But I wanna put a, a test to the room. Gimme the name of the most bizarre ransomware or the stream, the most bizarre ransomware you've ever heard of. I guarantee you it's in here.

Anybody got one that they can think of? Lemme just make one up. You.

Oh, come on. Made a check. I'm sure it's exists.

Yeah, you might, you never know, but like, um, so chances are we've seen it. If you think about the threat intelligence that the cove wear by Veeam Incident Response Firm has, it's in here. Um, I'll give, uh, one that's pretty nasty nowadays.

Akira, I don't know if anybody's heard of that one. So like for example, if we see dot akira files on disc and do ako and things like that, those are things to look for, right? com, the cove wear by Veeam folks, um, highlight for everyone.

Just go to cove wear and click on our quarterly reports. They do a fantastic job of highlighting just what we're seeing. Not, I don't want the image, um, highlighting what we're seeing in these, um, individual, uh, their, our own incident response.

It's not the whole market, it's just the ones that we do incident response for. And it's important to note that we require that this intel is shared with law enforcement. That's one of the terms of our engagement for this.

But like, if we look at q uh, 1, 20 25, this was the top, uh, several couple of them tied in terms of percentage of Kira I just know is up there. But let's take, uh, Quillin. That one is a new one for me, and it's new in the top variance, QI something.

So it's like they're all in here. And what's really interesting, uh, well maybe not that one, but uh, maybe it has a different file that might be its name, I'm just guessing. But, uh, what I will highlight about this, uh, this logic here is that when, um, we think about having cove ware in the Veeam stack, what has been really interesting is we've learned different, um, threat behaviors.

So she mentioned, um, team viewer for example. Now team viewer isn't bad, but if it's until it is, well come on. Until it is, right.

Exactly. But what I wanna highlight here is we're looking for indicators of compromise. There's a whole new section in here that says, Hey, does team viewers show up?

It wasn't there yesterday, but it's there today. We have debrief from customers where the threat actors are getting ready to exfiltrate and this file zilla some of these other things. These are the, the tooling that they'll use.

And here we go. This is the, this is mapping exactly to Mitre here. It's command and control TA double 11.

So that's just an example of these things we're looking at. Lars, on the file system, there's over 5,000 entries on this XML file. Yeah, we got, Are the content of the files, uh, it do only look at by the extension?

Or do you also look at, uh, proper file names and look at the, Um, it looks for both. The indicators of compromise will go for full file name, the ransomware behavior, the suspicious file will be the extension. The inline detection will look for the contents.

So you have inline detection for Yes. Types of files or specific files. Ah, so it is currently on Windows file systems.

Our upcoming release, we'll bring it it to other file systems. But this particular inline detection is for the con, the file detection, which is the next tab right here, is the one that will basically read that uh, XML file I was showing with those two different things. So the file activity really is for the suspicious files, and then we can even map to specific attacks if we see these disc behaviors file.

And we're hitting it from all directions. Is our logic here. 'cause as a backup provider, we have a very interesting perspective.

And once the data gets under management by Veeam, it gets really interesting because the threats can't jump around and rename themselves and, uh, hide in memory and stuff like that when we're talking about a static image based backup on disc, which is a perfect segue to a capability that we have now called threat hunter. So, um, what I'm looking at here are many, but not all of my image based backup or all backups that are in the cloud or on disk with Veeam. And you'll see a couple of 'em have this red little icon that's actually the malware detection.

Those eight or nine they've been hit. Now I'm actually not worried about that. Two reasons.

Um, I have purposely put some simulated things in here for that. I haven't gone and downloaded these actual threats because, uh, let's just say, uh, my Veeam security team would get on me. But the second thing I'll say is that I'm also really confident because we have, I'm storing these on what I call double play immutability.

So I'm not gonna get into the 3, 2, 1 rule. But a good rule of thumb with backups is that more than one copy, both of them are immutable. One's on-prem, on a unified object storage, one's in the cloud, two totally different control planes by the numbers.

This gives me a very high confidence of data recovery if I need it. What's even better, and this is a bold claim I wanna throw to this team here at Veeam. We've been in market with backups since 2008, starting in 2006 before backup.

We've had immutability since 2019. I think when, um, we implemented that with object lock just for AWS now there's like 60 50 different, uh, immutable targets with all that runtime. We do not have a record of a customer unable to do a data recovery when their backups are in an immutable target and they have their own encryption password.

Both of those conditions need to be true. We, we can do smoking hole recovery where all you have is a bucket, but they need their encryption password that they set this. It's not like a media server or a catalog like a string.

And I recommend to people watching at home put that information also in your password manager, your delineates and things like that, your access keys and your passwords. Maybe a security folk here would shoot me down for that advice. Mm-hmm.

But when you go to an absolute recovery situation, those are the things you didn't think you needed is your encryption password and your bucket key. You know, your service account password, but not those things. But anyways, what I can do with that, the last thing to kind of close the loop on this, uh, Lars, is I can do this threat hunter and I'm gonna launch this.

I won't let it, um, finish, but I just wanna highlight that there's this logic here to find the last clean restore point. If I have these, maybe there's a hit, maybe there's a problem. This will crawl through all the data under management by Veeam.

And this is a signature scan. I realize signatures are not the end all be all, but it's an important data point. And we can do that with threat Hunter or if an organization wants, if they have, uh, invested in their own like EDR platform, there's like six or so others that we integrate with that they could call and we'll just do this and we could even put a date range in if we wanted to.

But I'm just gonna, um, launch the threat hunter scan and it'll just go And, uh, with that I'm gonna stop the share. Um, I'll let Emily prep the share and just any questions on that demo? I know I kind of went quick, but that showed how we're looking at the files, we're looking at the contents a couple different ways we can get a hit.

We also can launch in some scans to, um, try to, you know, get a signature match if there's any, um, if there was any signature that was detected and I ran through that, I know kind of quickly, but that is as basically 20 20, 23 through. Now I, I I do have one question. Yeah.

Um, and it's, it's a question that I ask all the backup vendors. I feel like in my experience, uh, the enemy of the backup vendor is time to restore. Hmm.

How long does it take to move data from the cloud back to on-prem? Or if you have on Yeah, So great question, Tony, But, but my other, but, but my question is about like, like when you're doing these scans of the data in the backups, I have Terabytes and petabytes. Yeah.

How long is it does it take for me to start a scan and get result? Um, you know, because if I'm in the midst of an incident, I need as quick results as possible and I can't be waiting a week. Yeah.

For you to, I've set out a, an Excel file, So it's not an Excel file, but what I, what I will say is that the threat hunter scan is about two to four times as long as a full backup. Yeah. Okay.

Okay. Now that's just something to get expectations to. But one thing I can tell you, my, and that's what I see in my lab and my lab is Franken sand.

Sure, it's all hand-me-downs from the alliance partners, but I, I joke on that. But the reality is Veeam is super agnostic. You know, if you want to build a, um, all flash or non rotational storage system, um, our support team likes to say veeam's a lot like a bicycle.

Your experience is 100% predicated on the terrain in which you use it. So if you have high latency storage, you these things are not gonna go well. Lemme go back to the high speed recovery technique.

Veeam was first in market with instant recovery. And still to this day, I, I, I need to bring this thing up, but the beat, the ghost dev challenge was done. We were able to move 144 gigabytes a second in backup and restores within 90% of that speed.

What that tells us is that there's no software limitation on these throughputs. Okay? We don't make the storage now we do with our Veeam data Cloud vault, but we still actually recommend having something on-prem for that first copy for data center, use cases, cloud, different story.

But chances are you can, you can do any throughput discussion you want to do with Veeam. 'cause we, by being software, we provide the flexibility when that's a priority to a client, we can do what they want And let's take one step further back, right? If a customer or a client's in an incident, we have multiple other ways in which we're gonna go and try to find what's happening, right?

So another thing Rick and I didn't talk about are Yara rules. Veeam supports YA rules. I saw that.

That's we go in there, right? And so if you get hit by a Kira, we know we have to go look for a very specific string And that's quicker. Alright?

Ahead because Go ahead. You know what you're looking for. Exactly.

Security team, give me the YA rule that I need. Let me go ahead and execute that and run that across oh my backups or during any type of restore so that way I can identify if that's something in there versus trying to scan against the 104 different versions or variations that you had in that list. Right?

So that would be the other option as well. It's, it's again from Veeam side of things, we're looking at it as a way to help them during the recovery process. Um, and then again, getting them closer to that meantime to detection past, uh, what their EDR tools have done.

So we're gonna switch to two different topics and then jump into the security ecosystem. So I mentioned Veeam Threat Center. This is just a high level overview of that data platform.

Um, what we're, what's happening inside of the actual, um, uh, data that we're protecting, but then also what's happening from the Veeam side of things. You know, the Veeam backup server, your proxies, your immutable repositories that we are discussing earlier. So this gives anybody just a quick high level overview of like, you know, where are we currently at today?

How we identified anything from a malware detection perspective. Do we have any anomalies from any of our backups that have currently been running? Are we hitting our SLAs, ensuring that we are maintaining compliance, right?

So we get that at a high level overview. This data platform scorecard gives users a good indicators to how they are in overall resilience, right? And overall readiness.

So essentially, are you leveraging an immutable repository? And if you're not, why aren't you? Right?

That should be number one. The very basics that everybody should be doing is ready to an immutable target, um, ensuring that everything is backed up. But then on top of that, are they following all the best practices around security and Harding and any of those additional pieces in here?

So we can actually run what we call the security and compliance analyzer. This is gonna run against the infrastructure of, um, the Veeam environment. Plus it's gonna look at anything from their data protection or their best practices, like things around MFA.

Do we have MFA that's been turned on? Is it being utilized correctly? Um, do we have four eyes authorization set up?

So that way if somebody was to come in here and try to make any type of changes or deletions inside of the actual product itself, can we stop that from happening? So these are gonna be some of the additional checks that we are going to enforce with customers, especially ones that have just upgraded, that haven't gone through, uh, a best practices or anything like that, right? We can actually help them to go through all of this, um, specifics around the health check and ensure that they are implementing and securing their environment the best way possible.

And just to comment, a lot of these are defaults of the platforms we use. Mm-hmm. If we have time, we're gonna get to our futures where a lot of this just goes away.

Yes. Uh, the last thing in here that Rick had mentioned, right, is our incident API. So again, the one thing that we wanted to do is actually focus in on working with our ecosystem providers.

So everybody that we work with in terms of, you know, the, the Sophos, the progress flow months, um, all of these other SOAR platforms that have the ability to be able to call out and run runbooks for those different recovery methods that need to be performed, we wanted to leverage that from an incident API. So essentially what this could do is this gives, uh, the customers an additional opportunity that if their security product or if their tool scans and finds something inside of production, this will automatically run into the Veeam product and this will actually create a quick ADDA band backup for that machine. So let's imagine if this machine is actually experiencing an active encryption attack happening, right?

And those files are being encrypted, we can go ahead and run a quick backup on this so that way we can try to capture as much data as possible before it was encrypted so that way customers have a way that they can work their way back to a clean state. Um, so this incident, API is giving us that bidirectional communication between not just what we see from a security standpoint, but then also into the data protection side. And that was a perfect segue because I knew that was going to happen.

So let me, Anybody got a a question while we reconnect? It's doing its thing. That's what it does.

Any comment? I know we had some great social I saw that by the way. Um, but any, any comments here, Emily, when we're looking at some of the best practice recommendations, specifically on the compliance and security side, are there also like tips for the teams that are actually implementing them of how they can actually then bridge that gap if they are not implemented?

Yes. Yeah. So, so there's tips for how they can implement it, but then we also have a, a nice KB article that'll actually run them through and it'll harden all of those items for them automatically so they can run a script and essentially have, you know, items like, you know, the turning off, uh, old TLS encryption algorithms if that hasn't been done on the Windows server.

Uh, so it's looking at best practices for hardening both Windows and Linux by default, not just the Veeam software itself. So we could actually have them run that script and it'll go through and harden it all for them, or they could take that to their security teams and say, what are the ones that we need to be doing? And so then that way they could go through their best practices and make sure that they're doing it that way too.

The one thing I'll add me, Lou, is that a quick go, quick Google search. Veeam Best practices will take to a specialized subset, um, that's been r and d approved and architect field proven. And in there there's a hardening guide as well as there's professional services to get there too.

But, uh, there's DIY options or there's get it done options too. Yep. Um, that concludes our, uh, innovations at Veeam segment here at, uh, security Field Day 13.

I'm Rick Vanover. Emily Task. All right.

Thanks for watching. And tune into our next segment talking about the ecosystem.