Microsoft Security Introducing Security Copilot Agents

Thanks for having me, everybody. Um, I'm Nick Goodman. I'm a product manager here at Microsoft.

I'm also one of the founding members of the security copilot team, and thrilled to get to share, uh, where we are in our security agents journey with you. Uh, so even before agents, we've seen good results, uh, in customers using security copilot. Uh, for example, uh, security teams doing incident response.

See that that's happening about 30% faster when they're using security copilot. Um, we generally speaking are seeing, uh, security copilot driving faster results, making it easier to do the hard security jobs, um, and allowing us to, to have results that stick. But even with the first generation of generative ai, we still need something that's smarter and faster and more effective.

Um, and I think all of you have seen agents, you've seen ai, uh, evolve past the first generation of chat, GPT and style inspired things into, um, a agentic type capabilities. Um, and that's where our agents come in. Security copilot is now evolved to have AI agents that operate on, on, built on its platform and across the Microsoft Security suite.

And all throughout this, the, the core concepts are it's going to, it's gonna meet you where you are. It's going to, uh, help you do existing security jobs better, um, and all with people still in control of what's happening. And I'm gonna contrast this a bit with automation, not because agents just are an upgrade on automation or replacement for automation, but because a lot of the value that we talk about will sound like automation.

And I think it's pretty important to have the, the right mental model for how security AI agents, uh, and our platform are different than traditional automation. So we think of traditional automation that's gonna exist in your SOAR or, um, any sort of framework. You've got if statements, you've got four loops, potentially you've got functions.

People wrote them, people wrote the logic for them. And if there was logic that a, a human could not capture in code, it didn't get captured, if there is a, uh, fundamentally new data type that a human could reason about but was not captured in the code, you're rewriting that automation. If you're dealing with unforeseen circumstances, you're rewriting that automation.

So automation does many, many things well, especially routine, highly similar tasks, but there are also just many, many things that does not do terribly well. And what security copilots agents do is fundamentally different. They dynamically plan and reason about tasks and execute that plan, changing it as they go.

So very much the way you and I operate, we might think about a task we're gonna do, use our experience and our training to figure out how we would do that task. But then as we're going through it, we might discover new information that changes our mind about how we're gonna approach it. Or we might realize that a thread we were pulling in an investigation didn't lead where we thought it was going.

It's more interesting, so maybe we put more time into it. Security copilot agents do the same sort of reasoning, uh, throughout their execution of their task, which allows them to do significantly more, uh, than we were capable of in the past. So as of, uh, today, we now have 13 agents that are alive and and active.

Um, six written by Microsoft. Um, and, and personally I just love seeing that there's actually one more agent written by somebody outside of Microsoft than inside. And I realize that it's, it is sort of an arbitrary thing.

We easily could have ended up with a seventh, one of ours, or sixth or third party, or eight or third party agents. Um, but what this really is indicative of is our commitment to building with partners and, uh, security copilot being a, an ecosystem and a platform where others beyond just Microsoft are successful and are able to use the AI capabilities we're delivering. So, uh, we've got six different agents, uh, that are out there today from Microsoft.

I'll primarily today talk about those and I'll walk you through what each of them does. Um, but these, uh, agents written by our partners are also really great and really show the breadth of, uh, capabilities possible with ai. So let's start with phishing triage.

So obviously phishing is still a problem in the world. I don't think I'm breaking new ground to say that. And one of the consequences, the amount of phishing is the, uh, there's a button in most email clients that allows you to report phishing incidents.

And, uh, many of our customers have told us that that button is just clicked all the time. People, people click it on marketing emails, they click it on spam emails, they click it on emails from people they don't like, they click it sometimes by accident. And so many security teams are dealing with a deluge of user submitted phishing incidents.

And often it's a case that 95, 90 8% of those are benign positives, say positive 'cause yes, the user clicked the button. There's no false positive involved here. The user did click the button to report that it was phishing, but it wasn't phishing.

And from those teams, what I've heard is it still takes 30 minutes to triage it, even if it's not phishing and the team hates doing it, they know it does not accrue to security value. So the question is, why, why do you keep doing this? Why do you still triage this queue that is full of, of marketing emails and spam emails?

Well, the answer is the last one to 2% is too dangerous not to triage the full queue. This is a perfect use case for ai. Can we have the AI read the content?

Can we have the AI look at the threat intelligence data about the sender, the content in the email itself, if their links, can we have the AI look at those links? Yeah, we can. Absolutely.

And and what's great about that is the AI can do that, that 30 minute investigation, if it's a marketing email, it can tell you it's a marketing email. I'm gonna go ahead and resolve it for you. But if it's actually a phishing attack, well, I've just got the first 30 minutes of my work done for me.

So this is an agent that, uh, primarily is about saving time, but also really about driving focus into the right security, uh, behaviors instead of having to waste time on things that you have to do, but don't really create much security value. So let me show you how it works, uh, here in a second. As we're going, uh, just pay attention to the, the elements of the autonomy.

You'll see, uh, the learning, I'll show you how we do active learning in the system, and I'll show you the dynamic reasoning as well. So here we are in the Microsoft, uh, defender, XDR portal. If I look at an incident queue, obviously I have lots and lots of incidents.

Um, and now you can set up the agent. When you set up the agent, it's like an app in an app store. It tells you what it does, who's published it.

In this case, it's Microsoft tells you information about licenses. Again, in this case, it's in defender, it's the defender phishing agent. Uh, you've already got the license, no worries.

Now this is kind of the cool and interesting part. So when I set up the agent, I set up the identity at bru as it allows me to govern it effectively to audit it effectively. All of the agents you see from Microsoft security copilot are designed for enterprises, uh, so that they fit the, the, the models of deployment, the models of, uh, compliance and security and privacy that all of our customers need.

So, like I said, set up de identity. After that, I can set up the roles. These, I'm sure you hear, these are actually the defender roles, uh, that the agent can take.

If I were showing you something from purview, I would have purview roles that I could apply. And then as we deployed the agent, it starts operating. Now, agents are really different than security copilot in the past, and they're operating asynchronously autonomously on your behalf, 24 7.

So in this case, we're kind of fast forwarding time a bit. This agent is operating, and what we notice is that our incident queue starts to have this agent tag on it showing where the agent has operated. So this is a case where the agent autonomously triaged the ticket may create, come to a verdict, and updated the ticket with that verdict or the incident with that verdict.

Now that tag is on there to make it easy for humans to find it. So part of our principles of, of responsible AI and having humans in the loop. And when I drill into one of these, I see the standard short human readable summary.

So it's been common in security coot to provide human readable summaries generated by AI of what's going on. We still have that, but this is way more complex under the hood than anything that we've released before. And in a moment, I'll show you how, um, uh, and why I say that and like how it, how it gets to that point of handling that complexity.

Um, but once I'm comfortable that the agent is operating, I don't even need to look at the incidence, it's auto resolving. They're still there. I can come back to 'em and and review them on a regular basis, but I don't have to do that.

And if I do that, I'm able to reduce the, the review load by about 95%. Now, let's use an example of a case where AI gets it wrong to both show the learning. And I'm also gonna show off the dynamic reasoning that I was talking about earlier.

So if we look into one of these, uh, examples here, I know something this agent doesn't, which is that this email, which looks like phishing, it's correctly recognized, it looks like phishing. This email is actually from an HR training vendor of my company. Um, and I'll show you why, how we know this here in a second.

But just like bringing a new person into a team, there's business context that the new agent doesn't have. And so, again, I'm gonna show you here in a second how we train the agent on this business context that allows 'em to be more effective. Uh, but first let's drill into the agent.

So again, I'm in defender still within the workflow, uh, that, that I would already use if I'm a SOC analyst. And, uh, here right next to the the attack story, we now have the agent's activity. This tree that you're seeing, this is not a sore playbook, this is not a piece of static code.

This is a dynamically reasoned tree where this agent used the instructions it had from the developer, guardrails from the developer, and it's set up and its dynamic reasoning, ability to pull threads in the investigation and look at different pieces of data. And all I know it's hard to see here. So let me hit a couple of high points and we'll zoom in a little bit.

What this agent actually did here was it looked at threat intelligence. So right here in the middle, you'll see five leaf nodes in our tree, each one of those. Is it actually looking up data from Microsoft's threat intelligence sources?

That could be things like, uh, reputation of, uh, a domain or an IP address. It could be looking at the, uh, are there any binaries on the link? It's in the email, uh, that sort of thing.

Up towards the top, you'll notice three kind of small red rectangles, and I'll zoom in on them here in just a second. Those are places where the agent detected something suspicious. Like I said this, the agent flagged this email as a phishing attack, uh, requiring my review.

Um, and so did actually see suspicious behavior here. So when we look a little closer, what we see is it's the semantic content of the email and, uh, the, the, the, the structure of the email itself that makes it look suspicious. None of the threat intelligence sources I mentioned reputations.

What, what content is on the link in the email. None of that had anything suspicious happening. It was just the content of the email.

'cause back to like, I know I personally know what happened. It was an hr, uh, HR training company, send an email to everybody, say, Hey, click this link, take your training so it looks suspicious. Uh, but the threat intel sources don't say anything.

So it's kind of a mixed verdict. Still requires a human to look at it. Now, when we do this, we've got our verdict.

Um, we've got even a, a screenshot of the email again because this is built into the workflow. So one of 'em, security copilots, the lessons over the last, uh, two years has been build into the workflow rather than trying to move the user to an entirely new workflow. So you can actually see the email during this investigation so I can see what it looks like.

Yes, it does look kind of suspicious. Let skip ahead. But again, the reality is I know that this is not a phishing email because it's from our training vendor.

So like a lot of solutions, I can change the classification, I can fix the mistake. That's great. And like a lot of systems we're gonna learn from this, we're doing a better version of learning than what's been available to prior generations of ai.

And it's all down to this explain why box. So I'm gonna actually explain to the agent that the problem with its reasoning was a piece of context I had that it didn't, this sender is our corporate training vendor. Don't treat this as phishing going forward.

It's the same way you would teach a new employee who is taking on a task, oh, they made a mistake, let's correct a mistake. What's the piece of feedback necessary to make the correct decision the next time? So this right here updates the agent's instructions, it's immediately taking this feedback into account and we will use it going forward for me.

And then at the end of all of this, we have the, oh, sorry, is there question? Yeah. Who sees that explanation?

Like is it humans? Is it for documentation? Does do the models, does it, does co-pilot see it?

Yeah, good question. Um, lemme go back to it here. So, uh, this explanation that you're putting in is fed into security and copilot into the agent itself.

Good. Um, it's also inspectable, like I said, we're big believer in the enterprise and like building software that's designed for the enterprise. So other reasonable questions might be what if the person types in bad feedback?

What if somebody types in malicious feedback? Yeah. Poisoning it, Feedback, all of those things.

Um, and so, uh, this is, this is organizational data. And so there's actually a, a screen in defender review can go and edit the memory of the agent. So if you're an administrator, you can say, ah, I see this piece of feedback looks like it's related to some errors we're seeing, I'm gonna remove it.

I could say, Hey, I made a mistake. Let me go edit the feedback. I also could survey the feedback to see what types of feedback it's getting.

Um, as an administrator, you can also govern who in the organization can provide feedback in the first place. Um, and so you get to control and say maybe, maybe you only want, um, experienced analysts to be providing this sort of feedback. Maybe you don't want to tier one team or a contractor, uh, doing so, so you're in control of it.

Thanks. So Nick, uh, Jack Poller with Paradigm Technica. So why did this agent get it wrong?

Uh, that's a good question. I would claim it didn't get it wrong. So in the same way that a, a human analyst without business context walking in and looking at an email that said, click here to take your training, the button to an external link, it's sent to my entire company on the surface.

If I know if in the absence of any other information, I would say block it, phishing, the business context we have is external to any system is that I have actually paid this company to send these emails to my employees. And so, um, and we find that's fairly common that that sort of thing occurs, that there's some knowledge that the human analyst has that's outside of any of the security systems. And so in this case, we're able to improve upon the result of an AI only system with the human business context.

Alright, that, that makes more sense because I looked at that and I said that should have been marked as phishing, even if it was sent by your external vendor, right? That, that your external vendor sent a poorly crafted email and we should very Poorly crafted email, right? Yes, for sure they did.

Um, uh, absolutely that is relatively common in this space, but the d to let you get really quickly to the ones that are malicious and badly telegraph. Okay. Uh, Nick, I have a question as well.

My name is Belu Meyer, and I'm curious about, I like the idea of being able to resolve the ones that were already looked at by the agent, and I would assume that as the SOC is getting more on board, doing more filtering, they can trust that better. Is there like a best practice recommendation then though, of how often the team should still be going through and double checking that the resolved ones actually were properly vetted? Yeah, that's a, it's a really good question and something that we've, uh, we've heard from our customers in the design and the early preview process.

Um, and there's kind of two questions to ask that is one of them. And let me give you the other one and answer it first, which is, uh, should I do training on it before I trust it? Um, and a lot of the customers I talk to plan on doing training of the agent following this process and actually reviewing every one of them.

Now fall to that is why on earth would I do that? Am I not gonna waste a ton of time? The answer's actually, no.

It sounds like you would, but it's not because the doing the first 30 minutes of the investigation for you already, so you're checking its work on the work you were already doing, and it's much faster to check the work than it is to do it yourself. So pick your time window, whatever, two weeks, four weeks, six weeks, seemed reasonable to evaluate it. And we recommend looking at it like, what if you brought in a new team member?

How long would you look at every result from the new team member? Obviously if they're getting every one of wrong, you're probably prolonging the, the, the period, the training period. But if they start getting 'em all right pretty quickly, you're probably more comfortable moving forward.

So use a use judgment to decide when you've hit the quality bar that you're looking for. So that's the upfront period. And again, I frame the your question, how frequently should I review after the fact?

Back to kind of the same grounding, if you had new team members, how frequently would you do this? Um, what I've heard kind of runs the spectrum. So what I've heard from some customers is, hey, we like to have a process where we review everybody's work, maybe the infrequent review sample mistakes.

That makes a lot of sense. We also have heard from customers that say, 98% of these are false or sorry, uh, benign positives. Anyway, I don't care.

I'm not gonna review 'em once I hit a comfort bar, except potentially maybe I've got a, a reason to go back and review. So, um, that's really what we're hearing from our customers, uh, or the, the ways that larger enterprises are looking at, at the review. Nick, hi, uh, Fernando from, uh, Futurum, what can you share?

I mean, this has been out for a few weeks now. Uh, what can you share about, uh, the reality learnings from rubber meeting the road in that, uh, that surprised you and the project team? Um, let's see.

Learnings from rubber meeting the road and what surprised me, a couple of things. Um, one, the, um, the conditional access agent, which I'll throw to you in a minute, is more popular than we realized it would be. Um, which is actually why I added it to this, uh, demo.

'cause I wanted to make sure you all saw that one too, since it's our most popular one with customers. Um, and we knew it would be, we knew it had value, but it's clicking very quickly. Um, second area, uh, again, we're a few weeks in here is the customers who look at this from the lens of like, like I said, like the, the, what would I do if I were bringing it to the new person are able to adopt faster, um, than ones who are looking at like a traditional SaaS software purchase.

Um, because it requires many of the same motions in your organization, uh, that bringing in new team members takes. Um, and it's a little less like procuring SaaS software. Um, uh, and so that mental mindset shift is helping the, is helping helping customers move faster.

Um, I'm trying to think if we've got any others at this point. Have people been training them at the rate that you would expect them to be trained? We trained them them at the rate we would expect.

Uh, you know, I don't know the answer to that. Um, that's a good question. The there is definitely, uh, definitely customers providing feedback to the agents, um, for, uh, their recommendations.

The two, the two that probably have the most feedback, um, are, are ones or the ones with the most feedback are ones like the phishing agent, where there's a lot of business context missing. That's maybe one thing. One other thing we're seeing is ones where there's a lot of business context in the, in the task already are ones where there's more feedback that's relevant in coming.

In ones where, um, feedback is more subjective, there's less. So a good example would be, uh, our threat intelligence agent. We have had feedback to us, Microsoft, hey, here's some different ways you might consider writing about this.

Here's some formatting things. Um, but a lot of the content generation itself, there's less business context, uh, that's relevant to its task. And so as a result you see less feedback that's relevant that, that that's relevant in the first place.

So more feedback whenever there's a lot of business context that's relevant to the task, less feedback is, is necessary when there's less of it. Perfect. Thank you.

One more, if I may. What's the, uh, any insights on the third party agents as they're being, as they're being used? Um, I said some third party agents so far, nothing I can share.

We're, we're on an open non NDA. Okay. Be cognizant of our partner's confidentiality.

Um, we can definitely follow up under NDA with specific questions and probably even loop in some of those partners. Perfect. Thank you.

Hey Nick. Uh, sky fugate with Netsmart Technologies. Um, I, I understand, uh, for the explain why section.

Are you, um, building a rule set where that agent is following a rule set and putting that into the way that it's scanning those emails? Or is this strictly instructions for the agent where, you know, if you see this domain or it looks like this, this is safe to accept. Um, how are you doing that processing?

Yeah, so this is truly different than, than traditional approaches. Like this is a generative AI first system in play here. And so it's a mixture of code GPT instructions, uh, machine learning, traditional ai.

Um, uh, so I'm, I'm not gonna give you the specific algorithms, but you can kind of think of it like we're mixing all of the relevant techniques and using AI to drive the planning elements and the reasoning elements where it really does something that you couldn't have achieved with the prior generation pieces. And so this feedback here is driving the AI itself and our reasoning process. So it's helping improve the reasoning process of the agent.

Gotcha. So I guess one, one of my follow up questions would be, um, a AI is generative AI is not always the most accurate. I think everyone can probably agree on that.

Um, so if I were to say, you know, if, if there is an email that comes from this tld um, dot biz, for example, um, would it be smart enough to say, um, let me build a RegX pattern that looks for that, um, and rejects it? Or is it going to go through and scan that email as a, as a new rule on that agent to say, do I see this, um, in addition to all the other logic that it's gotten? Yeah, let me, let me put it a little different way.

Like, 'cause I, I don't think we would, I don't think we would want something that says, let's block all of fz. Maybe we do, actually, let's take this for example. Let's take exactly that.

Suppose take X, Y, ZI once worked at a startup with an X, Y, Z domain, and the whole space can be viewed as fishy. So imagine I really, truly did wanna block a top level domain. I could tell it to block the top level domain here and or say I could say, always assume that biz or XYZ is suspicious.

That would influence how it approaches the task. Um, it is not, again, I'm, I'm dancing around an element of your question is I'm not gonna get into some of the technical details of how it works under the hood. Mm-hmm.

But the effect is for it to, um, to learn what you taught it. So you taught it, my organization treats do biz as suspicious. It will recognize the DO B is suspicious going forward.

Got it. Thanks Nick. And it is more, and sorry, and I wanna, it's more, and what I'm getting is it's more powerful than just like, what could I encode with a RegX?

Um, so like you, it's, you're capable of teaching it things the same way you might teach a person. So you might say to a person, you know, this is generally a little sketchy, but it's maybe it's sometimes used here and here and here. Like you can give it more qualitative guidance that you could never capture in traditional code or, or in RegX.

And it's capable of follow that as well. Awesome. Yeah.

Uh, hey Nick, uh, I have a question. It might be kind of taking, taking the tail of what Sky was asking about, but, um, it, it seems like processing email through, uh, through an AI agent, um, is resource intensive. There's a lot of resources, compute resources spent on analyzing what could be a basic email.

So once something, uh, I guess, and again, maybe this isn't something you can get into, I don't know, but once something is flagged as known, malicious or something like that, do we have to continue to spend the same amount of resources looking at every email that goes to every, uh, user in my organization's account? Or are we building, is the agent building a rule set to, to add that to, to be blah? Yeah, that's a great question.

So, um, I'm gonna answer this in, in a couple of ways here. So the task this agent is designed to do is triaging user submitted phishing, which is a big problem for 80 to 90% of the customers that I've spoken to. And in that task, there's no, it's difficult to get a rule set.

You almost probably can't because it's literally humans making a mistake to submit. And so to the task of clearing those from the queue, um, there's probably not a rule that you could write, um, other than to simply process 'em and, and make sure that it's reasoning about them effectively. Now, there is an upstream question, which I think is also a big piece of what you're getting at, like what happens at the email level when the user is clicking that same button, if that is a Microsoft email system that is also triggering an equivalent event to Microsoft.

And our research teams are, uh, doing precisely what you're describing, attempting to build high, high performance, low latency, high quality, uh, rules that could be applied in the actual email systems themselves. And so the idea is generative AI piece is not fast enough to run inline across billions of emails a day to just to resource intensive. So we're applying it to the very time consuming human task of triaging the false or the, the, the benign positives that come from users submitting phishing emails and having the other out of band approach handling the, uh, the super high volume email systems themselves.

Right? Yeah, Nick, that was, that was exactly sort of what I was getting at. It's just too resource intensive for, uh, an entire organization's volume of emails to run through an, uh, a generative AI platform, uh, for analysis in real time.

So that's why I was kind of wondering, you know, once that analysis is made, you know, what's the next step? That was very cool. Thanks.

Yep. Yeah, and it's, um, and, and again, one reason that it works in this instance for this particular task, many security teams are already investing human effort in it. And we could, and the AI components can do it faster at the same quality rate, um, as what it would take a human team.

One of the, uh, SOC manager in Germany last September told me that three and a half of her seven people do this task all day. Um, they had that, they all know it's a waste of time because they would prefer to be focusing their time on the, on the true phishing instance. And actually that's a good segue into, lemme show you this slide here.

As I said, the concept of these agents, they're tasks, they're tasks many companies are already doing. And so like tasks that cost three and a half people on a seven person sock, they're measured. Uh, so we've worked with our design partners to figure out what are common measurements for these tasks and they incorporate those directly into the product.

So in this case it is incident resolution rate and time to triage. So, uh, the, the team says, whatever my time to triage is 20 minutes, it's 30 minutes. Well now I can come in here and see is this agent performing, uh, the task, uh, as quickly as, uh, the, the human staff was, which tell, which really helps me make this like a sound economic decision of is this worthy of the cost, is it worthy in the investment of my time to use this agent for this task?

So any other questions before I move, move on from the phishing triage agent because I'm, I'm gonna hit the other agents and then another demo. Yeah. Hey Nick, just another quick question on that.

'cause I love that there are KPIs on there. Um, is there, uh, either a different panel for this or, or somewhere where it's tracked where we can see KPIs for, um, where I had to go back in and retrain the agent or teach it? Um, because it may have misclassified something Interesting.

Um, we don't have dashboards for it. It is relatively likely. Um, I'm just running through the various ones in my head, the ticket based ones and incident based ones.

You probably could get that data from the system itself. Um, it's an interesting idea. Uh, I had not, uh, we've not built that yet, but I, uh, I'll definitely take that back to our team.

Yeah, Yeah. I, I know that it's useful 'cause we track that KPI on on our team, So. Hmm.

Okay, cool. Thank you. Anybody else have questions on phishing triage before we hit the others?

Have you noticed anything different in relation to multilingual uh, performance? Uh, that's a also a great question. So let's scope multilingual into two buckets.

So the, the languages that the open AI models that we use support natively and the one are the ones they're good at and the ones they're not good at. So, um, English is one of, uh, quite a few languages that the models, uh, understand. Um, we've observed, actually, one of the most surprising things early in the product in the development cycle, this is probably about two years ago, was we noticed that if you interacted with the, the assistive chat mode of security co-pilot in a non-English language, you would get that language back.

And once you did it, it would recognize it and preserve that throughout the session. So we've had pretty good success, um, for the core languages that the open AI models support. Um, the models out the languages outside of that is a different story, uh, where understanding can be weak.

Um, uh, but yeah, the intention of the product is that, like, in that feedback section you can provide in your natural language, in your own language, uh, the, your feedback through the agent. Um, and that extends to the set of, I don't remember the exact number of 'em, but the, the more, I think it's more than 10 languages that open AI's models support well out of the box. Yes.

Yes. So for the next couple of agents, I'm not gonna walk you through the details and when I show you the indirect conditional access agent, you'll see why, um, the screens, the, uh, the, the T LDRs the screens look virtually identical for a lot of the components. And so I'm gonna talk about the business use cases, the security use cases, and then give you one more demo.

So, um, the data loss prevention and insider risk management agents in purview, uh, really fundamentally do a a, a task that seems pretty straightforward, that's immensely valuable for privacy analysts reading and classifying documents, something generative AI is pretty darn good at. And if you think about the use case here, a friend of mine, uh, recently changed jobs. He left one large tech company and went to a different large tech company.

And as we would expect in that circumstance, um, the, the, the privacy software they were using was monitoring his activity after he'd given his notes and he plugged in a USB device into his monitor at work. And from that point on, he had entered into a new risk class in that privacy software that they were using, and it started generating lots and lots of alerts and there were so many alerts that their privacy team was just putting them all into a spreadsheet and sending them to his boss and asking every single day, what did, what did he look at, what did he, uh, since this is public, I won't say his name, but, uh, what do, why was this person looking at this document? You can imagine just the, the workload required to go look at those documents and make a decision about is this benign behavior, reasonable behavior, or is this a problem?

Um, great use case. AI can read the document, it can help you understand what's in the document and whether or not you should care that this employee is opening this document. Um, and it's a task that's almost impossible for human teams to get through.

What we hear from most of our customers is that these queues for document review, they just get longer and longer because people can't get through, uh, more than about half of the inbound alerts in a day. So they're already missing quite a lot. They're already having to prioritize.

And what the AI is doing is allowing them to get the queue down to zero, understand where the highest risk areas are, and focus on cases where if it's insider risk where the employee did actually look at a document or download a document, move it to a USB stick, whatever, where it really was high risk, or if it's a, a data loss alert, uh, the same concept applied there. Lemme pause and see if there are questions. Um, and again, this one, I'll show you all a demo at the end of this.

Um, I would absolutely love this use case. So if you've got conditional access policies in your organization, I would be willing to bet that they're out of date. I don't need to know much more than that.

You're using them so good that you're using them, they're super helpful, but they're out of date. And that's because the pace of business change is so high that the policies drift from the, the state of the business almost immediately. You got new users, new groups, you've got new cloud applications, uh, new deployments, changes in the organization, things contribute to the policy state drifting and no longer matching the, uh, real, uh, state of the organization.

And people, uh, companies already do review this on a regular basis. Often, um, I talk to customers and, and people say, I review it monthly. And if you pause, sometimes you'll hear people say, well, quarterly I review quarterly and every, if you pause some more, sometimes you even hear people say, well, I review 'em once a year, but there's risk.

And so you've got this risk window that's like this big, and once a quarter you deal with this. Imagine if instead the AI agent could just close that risk window, reduce it down to a matter of minutes or hours versus months, it's a massive, massive win for the, the posture of the organization. Um, and there's another element here too, which is, which I think is a major contributing factor to why people don't do this very frequently.

It's really hard to get the policies right, and if you make a mistake, there's often a pretty major business consequence. So that could be, imagine you change your policy and you knock your sales team out of your CRM, now you're losing revenue. Nobody wants that.

And so security teams that optimize these policies often end up spending days working through the what ifs and testing and, and doing this well. The agent can do all of those tasks for us, can do the cognitive load of thinking through, uh, which policies are the most important to adjust the cognitive load of working through the crafting of the policy, uh, and getting it to a point where, uh, you can roll it out. And like I said, I will walk you all through this one in more detail here in a couple minutes.

Any questions on this one before you hit the next one? Um, okay, so vulnerability intelligence, what this is gonna do is gonna read the vulnerability intelligence reports for you look at your device estate, and I really do mean devices here that we're not talking about servers, we're talking about laptops, we're talking about desktops. Um, look at that estate and see where do you have these vulnerabilities and look at available patches.

So if you combine those three, that's the actionable thing that you can do from vulnerability intelligence about, um, endpoints, um, in your company. This does that work and then builds the patching group in Intune so that you can move directly to deployment without having to do the cognitive load of reading and reasoning about these tracking down vendor patches. And it's gonna cover, it's limited only covers, um, uh, windows, um, the, the operating system.

And so it's gonna cover operating system level patches, Microsoft patches, four windows, and third party. So non-Microsoft, uh, patches on Windows. Um, we don't handle Mac, we don't handle server side, uh, Linux, that sort of thing in this, in this release.

Just the, the Windows and endpoint. Hey Nick? Yeah, John Poller here.

Um, Do you have any information on sort of what it's using to do prioritization? Uh, yes. Um, we are looking at the, think of it about like reasoning about the impact and trying to prioritize patches based upon the biggest impact to your organization.

Um, there's a, there's a judgment element in that. Mm-hmm. Uh, uh, we've tried to code into the agent the type of judgment that, uh, we apply, uh, to that equation.

Um, uh, that's also the type of thing where feedback can come into the, into play where you might say, Hey, uh, for example, a good example would be, Hey, uh, this device is a honey pot. I don't care. I don't ever want you to propose a patch to it.

It's not even a real device, just let it go. Um, so that there is still room for business context to float into it, but yeah, we try to prioritize based upon the, the impact. Right.

But when you say impact, does that mean, um, risk of threat, active threats that are, you know, that are being currently being exploited? Uh, sensitivity of the laptop owner, like the CEO might get, uh, higher priority than, you know, the, the reception desk, those types of considerations? Since day, it is primarily focused on, um, a few data sources.

So we have, I think, sorry, I'm trying to make sure I'm walking the right line of what to disclose publicly and what not to, um, there's a lot of information that's obviously super public about our vulnerability mm-hmm. About the impact if breached your, your standard CVSS score. Right, right.

Obviously we're using that. We also have intelligence about, um, which CVEs are being exploited in the wild, um, just from our own visibility of the internet. Uh, so we can use information like that.

Mm-hmm. Um, I don't think we are yet using, um, like crown jewel asset status mm-hmm. On the impact question.

Um, uh, that's obviously something that we might do. Um, it's kind of a, I think it's a trade off. I've, when I've talked to customers in this space, it's like you could look at it and say, I wanna focus on the crown jewel assets because like you said, that's my CEO and I don't want that device breach.

But you could also look at it in from the perspective of like, some of the biggest data breaches in history came from a single piece of software being unpatched on a device that nobody thought was a crown jewel. It just happened to have access to the database and right results. So I'm not sure our customers have given us the signal yet about precisely how they want this, uh, prioritization done.

And that's one of the things we hope to learn in the early preview process and also potentially maybe we see, hey, the right answer to this is actually let the organization drive that decision through feedback to the agent or configuration of the agent. That's, that's actually the answer I was looking for, which is sort of, it's, you know, we're, we're, we're looking for, you know, every, every organization is different, and so how much tunability or how much input can you give to the agent to say, in my organization, this is what I care about and this is what I don't care about. As you go about looking at all of my endpoints, That's, that feels like the right long-term answer.

Mm-hmm. Uh, to, to get to that point. Yeah.

Sorry, Nick, just a question. Are you planning also to, uh, gather also information that is not, uh, uh, as an the client or from, uh, open data such as, for example, sort of detection if an asset is, uh, is a crown jewel despite the customer doesn't recognize it, for example? Uh, I don't know.

I, I, I prioritize vulnerability patching on asset that, uh, has, uh, target the documents on Azure information protection, or they are, uh, sharing internal information between users that could be not recognized the con jewel, but, uh, maybe is, uh, as a, as a, as a, We haven't gotten there yet. Uh, we're early in this journey, like all of these agents are on their first steps, and so we've done something that we think produces pretty good value towards a job. We try to shoot for that like 80% in the 80 20.

Uh, what you're describing to me is a really interesting idea and is the type of thing that could be used to improve the agent over time. Could we, could we incorporate things like you're saying, like the connections between the device and documents and other, other information? Absolutely.

We could do that. Um, and we may, I'm not obviously gonna commit to anything on, on a call, but, uh, uh, it's a very good idea. Uh, you know, the, the big issue of, uh, the big weakness of every company is that, uh, the asset inventory is weak and, uh, and is not up to date.

So, uh, having different, uh, helps may more way of doing the, the results may help. Yeah. And, and this capability is using, uh, information from defender as well, so you don't have to have a, again, it's, it is focused on employee device side devices, um, or like shadow it, but the, the idea with it is, um, if it's got a defender, uh, software running on it, we can see it.

And so we can see what software it's running, um, and look at the, have a pretty good understanding of what the, what the risk on the device is. It's not dependent on asset inventory, it's dependent upon real installed state and our threat intelligence briefing agent, what it's doing. It's kind of the server side equivalent, or at least the first parts of the server side equivalent of the Intune vulnerability patching story.

So it's looking at intelligence about threat actors and, uh, what actors are doing, what cyber criminal groups are doing, and looking to see do those, uh, is there, is there evidence inside of your system that there is a problem? Uh, is there signal that you might be, uh, somebody who would be targeted by these organizations? And so we're looking at things like geography and sector and vulnerability intelligence and the software you're running, um, and the types of software and infrastructure that you run.

And this produces a briefing report covering the cyber threats and the vulnerability intelligence that's relevant to your organization. And it's meant to really empower cyber threat analysts to do more or to, uh, empower organizations that they can't afford to have a cyber threat analyst, um, on their team to have something to, to get them started.