Threat-Informed Pentesting: The First Step to Continuous Assessment (OnDefend)

So tell me, how did this even start? PlexTrac OnDefend Threat informed Pen testing. How did we get here? Yeah, I’ll share kind of my thoughts and then feel free to chime in with yours, but so I started PlexTrac as a mechanism to help security teams win the right battles, understand how to get better at their security posture, and really show the right progress that they’re making over time.

The way I started that was through the realm of automated reporting and doing pen testing myself and hated writing reports. So having a better way to paint that story for not only the testers, but also their customers, or if you’re an internal team, their constituents.

I’ve been on this mantra around continuous assessment, doing continuous based assessment, particularly from a proactive perspective where you can highlight the key things that an attacker actually would do in your environment. And so as that message has continued to be heard and Honed really wanted to find a way to really encourage folks to look at it from the lens of what would a threat actor be able to do in our environment? Right. And being able to communicate some of those known known capabilities within your defense posture. And that’s when I started talking with Ben and Chris over at OnDefend around this concept of threat informed pen testing. Of why this is really important is that it’s an initial approach into having a continuous assessment model and framework so that you can actually show the stories of how you’re making progress against known threat actors and their procedures that exist and how they would execute those in your environment. So that’s kind of like where I am coming from. And then I love your perspective because you’re in the trenches daily, right? Yeah.

Thanks, Dan. For me, what it came from was this idea we go work with customers all the time conducting different pen tests and Red Team engagements and always with the idea of how do we know when a real threat actor comes that we’re going to be ready to stop this. Right. That’s what they’re being asked to tell their board of directors or whoever that we’re ready to safeguard our customers data and all those other things. And it’s so hard to try to figure out, based on what we’re doing in some of these other engagements, to be like, yeah, you’re good against whoever the threat is. When we talked about this, Danny, you were putting this idea forward, a Threat Informed Pen testing. It was like, of course.

Right. We spend a lot of time with customers after a pen test, kind of helping them incrementally get better, understand how their tools work, understand how the teams can use them better and those sorts of things. And it seemed like natural fit to talk about what we’re going to do today. Right. The Threat and Form Pen test, really as a bridge into like, we’re going to continuously test and make sure that we know that things in our environment work the way we want them to, not the way attackers want them to. Right? Yeah, it was really interesting. So, Dan, we’ve known you for a long time now, a big fan of obviously the PlexTrac Tool, and the team has been fun to see you guys grow.

We’ve used PlexTrac for a super long time, and so we’ve always used it as our reporting automation tool and a lot of other things that customers interact with us on. So it was natural for us to also think about when we start talking about Threaten Form Pen testing, how do we make sure people can see it in PlexTrac? So I’m really glad we can do this today. That bridge to continuous assessment that you guys just mentioned right there. So I think we’re going to start off with the most obvious question here. How do you guys define threat Informed Pen testing? Yeah, so I view it as a mechanism to be able to test certain tactics, techniques and procedures. And I think that the Miter framework really provides kind of a baseline for that. But it’s a way for the Red Team and the Blue Team or the Proactive Team and the Responsive Team to be able to work together and like, hey, we identify that these are some of the common threats that would exist for us, whether it’s in our industry or our sector or just our company.

These are the things that we know we need to be able to protect against in some fashion. We need to be able to paint the story of how we defend against this, how we get better, and so that it really takes the approach of working together with the pen testing team, whoever that is, whether it’s internal or external to say, like, this is the scope of the things that we want to test and we want to know how we’re doing on a continuous basis. It really lays that groundwork, right? And so it’s not the notion of we’re going to go hire an external team or we’re just going to let our internal team run wild and be able to do whatever they want to do and there’s a time and a place for that. But that’s not what this is, right? This is really, hey, we want to know how do we do against these things? And as threats emerge, can we build it into a framework to be able to test that in a quick and efficient manner and have that story to be able to tell to our board or to our stakeholders or executive staff, whomever or auditors that we’re making progress? And these are the things that we know we actually can defend against.

Yeah. I’ll take a step further from what Dan has said and say that one of the ideas that I really like about threaten form Pen testing is that it takes what is I think pretty applicable in what we would traditionally call like a purple team space or some of those other ideas of exercises of doing this. And most organizations want to do that, but don’t necessarily have the budget, the coverage or the buy in to do it. But they already do a Pen test. And so if we can kind of sneak this in with them and help it become part of that Pen test package, suddenly they’re getting those same benefits and activities without having to like it’s a whole separate thing that has to get like a budget battle and all that sort of stuff.

If you could go to the next slide, please, I think we can kind of dive into really differentiating a threat informed Pen test versus sort of a regular one. A lot of bullets on this slide, I think, for me. And I’ll let Dan chime in too, because he’s got some awesome thoughts here for us. When we talk about a threat informed Pen test, we want to really tightly focus a series of activities based on things we know threat actors really do. So unlike a traditional Pen test where you come in, you have some starting point and some gold and the attack path is wide open. We want to have it be much more narrow, much more focused. And we are really after the specific techniques and tools and things that those threat actors use as opposed to everything that the Pen tester might know how to do still requires pen tester knowledge we’ll get into in a second.

But it’s really about the idea. Like, I am really worried about this particular threat and I want to make sure that that happened to us, that we have thought through all of the things we need to block, all the things we need to log, all the things we need to build alerts for and that sort of thing.

Because I know I’ve heard many times before. You hear from CISOs, they’re like, hey, this threat, are we going to be good? Are we going to be okay? You’re like, all right, we had our traditional pen test. Yes, you hit it on the head. And that’s actually what I was going to highlight is that there’s a lot of times when you get some of those, like, big what’s popular to name the attacks now, but when those things come out and an executive staff or just somebody’s going to ask like, hey, are we good here? Or what are we doing? When you have this approach to a continuous assessment where you can build in emerging threats, you can say like, yeah, we’ve tested our defenses against those specific techniques for the last two weeks now, and here’s where we’re at, you can start to communicate a better story. And I like what you said, Ben, in terms of helping with some of the justification because back in the day, kind of extending, exposing some of our experience here is like, people would hire pentest firms to be able to help justify security budget in general, right? Because it helped them actually tell the story of like, look, we have some problems security team, I was on a security team once where it was just like pulling teeth to convince people that this was something you really needed to invest in. And so that was what a pen test had been used for. And now you come into the threatened form, pen testing of like, you can use some of that budget or all of that budget to really say, hey, we’re honing what we’re capable of doing.

And then it builds morale. It shows that you’re making progress and actually facilitates the conversation for deeper investment. We did some research a little over a year ago now related to adversary simulation and penetration testing from a purple team perspective, but that the outcomes are the same as this threat informed approach, where it helps them educate everybody in a deeper fashion and helps justify deeper resource investment, which is ultimately what we all would like to see, right? So I guess my question a lot of people are asking, is this a replacement for your traditional pen test? Oh, man, I would love for it to be. Frankly, I think it’s super valuable. But I think the answer probably for a while is no. There’s a lot of folks who rely on the pen test outcome that don’t really aren’t thinking the threat informed landscape that we are yet. So I think if you think about it, a lot of firms, a lot of companies conduct an annual pen test that gets sent off to an audit group of compliance group, to a lot of third parties who have to validate things.

So I think the standard internal external pen test is not going anywhere for a while. And I think that there’s still value in that. I don’t think either Dan or I are saying that there’s no value in a traditional pen test, total value, but the idea that it is preparing you for the very specific threat actor that you may bump into, right? Like if you’re preparing for the Super Bowl and you’re going to play the Chiefs, you’re not going to go ask the Naval Academy and the spread option, whatever, to come play your team, because that’s still football, but it’s not football. Like what you need to play, right? And so that’s what we’re looking at here is you are preparing against a specific threat actor. How do you know that your stuff is going to work the way you think it does when that threat actor shows up? Can you give me some examples of a threat informed pen test? That’s a good question. Go ahead, Dan. Before we go to that, I do want to kind of highlight too kind of answering that question replacement.

I think what’s really important to hone in on here is that it’s taking a more programmatic approach, right? When you’re building a security program, you have all of these areas that you got to focus on, right? But at the heart of it, when you can tell your board that, hey, not only have we invested in these areas of security like firewalls or certain technologies or other frameworks, not only have we invested here, but they’re actually working. You can actually show like, hey, against these specific threats. They’re working and on a continuous basis. And that’s really important. So it doesn’t replace the need for maybe like a more black box type scenario of a pen test or maybe like a deeper scoped or wider scoped, but it really helps you build a program approach to identifying the key gaps. And I think it’s really important to hone in on is that, yeah, it doesn’t replace your traditional pen testing. It helps facilitate a more continuous approach.

And I think it can also be the introduction into a more formal penetration testing program because I think some folks get nervous about like, well, we don’t have the capability, we don’t have the skills. And that’s where you can partner with someone like Ondefend to actually come in and help you in a continuous fashion utilizing some of the budget that you already have. So it’s not a replacement by any means. It’s just an enhancement, in my opinion.

So sorry. Okay, now jumping. What was the next question? I lost. No worries. I was so mesmerized by everything. I was like, it makes sense. And I think that’s what you’re trying to say.

I keep saying to everyone around me, I’m like the motto for this year is do more with the same budget. And it really is, how do you take that existing budget and how do you just get the most? Because cyber attacks are not going away anytime soon. So let’s go a little bit into some examples of threaten Form Pen test. Awesome. Yeah, great question. Let’s get to it, right? Exactly what are we talking about here? So if you could go to the next slide, please. This very busy slide is going to walk us through one example of a threatened Form Pen test.

I believe this threat actor is fin seven. But it doesn’t matter because the basic layout and the flow is based on the information that we have from some different sources. I’ve been seeing the chat pop and ask some questions about where we get some of this and we will definitely talk about that in a few moments. The idea is we’re going to look at the way these threat actors work and we’re going to start where they start and conduct our operations in those same operations.

So this one starts with a spearfish. It could start in multiple ways. One of the ones I really like are some of the threat actors that obtain valid real credentials and connect over the VPN and then are browsing the network like they are. They have real credentials, right? So they’re real users browsing the VPN. And a lot of times organizations are not really instrumented to detect that. But let’s talk about this one. So the idea would be we’d have these stages laid out ahead of time.

We know that there are activities we want to execute in each of these. And our goal is to help the customer figure out what do we have in place that prevents this part of the attack path from working, how hard is it to get around those controls and what telemetry do we get from it? Do we know that something unusual is happening? Right. What other side benefits do we get? So we would have the first one. The whole purpose of the first phase is simply to get a working payload into the customer’s inbox the way a lot of attacks start. Is that way very useful to do sort of like email filter testing? What we’re looking at on this particular slide is an example of when we walk through stuff in red is where the standard checklist part of the walkthrough of the scenario, that standard payload or step or something failed. And what you’re looking at here is this is where the pen tester, who’s conducting the threat informed pen test looks at why that action was prevented and works on either modifying the payload, the approach, the technique, whatever it is, within the scope of the threatened form. Pen test, based on what the threat actor does to see if we can evade that particular control, get around it and get to the end of the objective for each of these phases.

So you can see the goal at the end of the first phase here. Spearfishing is a payload arrives intact in someone’s inbox Dan. I know we’ve talked a lot about different ways that we could start this thing off. I think Spearfishing is probably the most common, but we’ve got a bunch of these. What do you see when you think about the vulnerabilities that folks are trying to track and trim and PlexTrac? Is it stuff like this? Are they worried more about exploitable systems on the Internet? What do you guys see? Yeah, I think for the deeper compromises, this is what you really want to have awareness around. What is the efficacy of those types of attacks because they are so, so easy. Right? I mean, obviously I would say the low hanging fruit is being able to scan your external surface and being able to identify what systems might have known vulnerabilities on it.

That’s going to be the easy vector in.

An easy vector in, but it’s also so much easier and you can get so much deeper through Spearfishing. Right, or some form of Spear phishing campaigns. So, yes, those would be things that are always going to be part of a little bit more of a threat informed approach. Right? Yeah, and you’re right, we just saw just this past week, right, all these problems with vulnerability for VMware ESXi servers. But to be a problem, they kind of have to be available network wise to the attacker, which means they probably are on the Internet. Some of them, and some of them obviously an internal network. So it’s good to know what you have hanging out there on the Internet.

Yes, exactly.

Go ahead. Yeah, here we would move into assuming we were able to get an email message in, we then move into the next phase, the user execution phase, where we’re looking at the payloads and things that happen. Something really interesting happened a few months ago and some folks may have noticed it and some folks may not have, but one of them is that Microsoft changes the way Office documents issue trust ratings to macros and files that contain those macros. Now, that didn’t necessarily affect the enterprise channel right away. You gave you some more opportunities to fine grain tune, turn those controls on if you wanted to. But it affected enough that we saw a shift in the way that malware developers and these sort of ransomware gangs were sending their payloads through different means. Instead of being a good old fashioned macro to Word document, we got all kinds of different things like ISO files, which if you in Windows, just double click on it, it magically mounts that image and all of the files inside of it.

Avoid that mark of the web downloaded from Outlook, type of distinction that we usually use to prevent those things from automatically running. So we’ve incorporated a lot of that type of techniques in the thread of Form Pen test. Because this is based on the thread intel we have, the behavior based threat intel, right. The corpus of data after a breach occurs. This is how we know thread actors work. These are the types of things that we see showing up. So in this case, we have an ISO image that we mount.

The inside is a shortcut file, an LNK file that actually runs some commands. And we see in this example that it initially was blocked by the customer’s EDR tool. So we go refactor that, place that on the system and kind of work our way through and then reexecute that one.

I’ll say this, there are going to be times in threat and Form Pen testing where especially if you’re doing this yourself internally, you’re just not going to be able to get by some of these steps. Right. There are just going to be sometimes where it doesn’t matter what you do because certain controls in place, either your skill set is not enough or your team doesn’t have enough time to figure out a way around it. And that’s great. You should document that as like, hey, listen, we tried a wide variety of these things. We looked up different ways to obfuscate our commands to do all these things. We couldn’t get anything past the EDR, so check that one off for the win.

But that doesn’t mean that the rest of the attack path shouldn’t be tested. So what I would recommend that you do is that you plan on even if you can’t get work, stream number two to finish, you say, okay, documented where we failed all the things we tried. Let’s go to number three and let’s start there. And if the goal in number three was to have this JavaScript payload execute, that moves you onto the more like recon phases and the execution phases, then let’s skip ahead to do that. And you can still put the report like, hey, we couldn’t get phase two to work. But phase three, here’s what we did. Because there’s going to be valuable things you’re going to find at every step of the way through this.

Right. And this really gets back to sort of I think part of the one thing we really loved about Flexrac is their ability to help us kind of think about other ways we can show this. And I think that shows up really nicely in the Run books module. Right. So Dan, if step number two in a runbook a test plan we come up with doesn’t work, that doesn’t mean we skip and skip the whole rest of the test plan. Right now we’re talking. Yeah, exactly.

Because you’re kind of starting to rely on some potentially false assumptions. Right. So I think it’s important to say like, okay, well, we think we’re pretty good in this category, but we have some concerns about where deeper in the stack or deeper in the chain of the kill chain that we would have. And so kind of addressing some of the questions that are coming up, which we’re going to get to as many of these as you can either in line or at the end. But one of them is like, just keep in mind that this is the general approach, right. And then you would actually want to be testing like a specific threat actor. And so you can also do that in piecemeal.

And that’s where something like PlexTrac really can help, is that you can have different test plans for how deep you want to go with that test plan, whether it’s the whole kill chain or maybe we want to go very deep in a specific tactic. But that is definitely also a good place to get started too, if you’re nervous about, hey, this is overwhelming, I don’t know how you get from start to finish. Okay, well, maybe just test a few procedures using a test plan and maybe an automated tool like Blind Spot to say like, hey, we just want to make sure because if we can allow for credential compromise, we want to just fix that first and something like that before we have to know all the other things. And so I think it’s important that you can break the US down as well as full scope as needed.

Yeah, that’s a great point, Dan, and I think that is the other idea behind these phases is that you’re right, this is a lot to do all at once. It might be super valuable to just spend a day or a couple of work sessions just going through the recon phase, right, and you just go through that and you replay those things over and over to understand if there’s anything there from a detection perspective that is useful to you and your team. Say this definitively looks like somebody who doesn’t know anything about our environment, trying to figure it out versus normal admin traffic on the environment, right? Yeah. And when we talk about kind of from a continuous approach that’s the mindset that you want to have is like, hey, this is not like, hey, we got to get all this done over the next week or month. It’s like, we’re going to do it in bits and chunks, we’re going to do bits and pieces and you can even build on it. Right. So when I was a security director, we started a similar program where like, hey, we just feel like we’ve got some gaps in lateral movement.

So we tested some of those procedures every two weeks and then we would have the blue team kind of evaluate the results and be able to identify where they could fix. And then we kept testing it so it kind of built that muscle that was kind of the foray into a continuous assessment but then being able to actually tie it back to specific threat actors and emerging threats. That’s another question that we’re getting like how do you get informed on these things? And I think that’s important is that you can use threat intelligence feeds that exist open source and it’s another way to use it. It’s not just a matter of IOC and things like that, but also being able to just bring that in to specific test plans, working with working with groups like OnDefend and Blind Spot where they’re bringing that stuff in all the time as well I think is really important. Ben, can you elaborate a little bit on that threat intelligence, what you need and where you even get it? I know about a million people have asked that question. What do you know that we don’t know? This is good. Yeah, so let’s definitely talk about that because that is important.

Okay, so when Dan and I are talking about the word threat intelligence, we do not mean a list of domains, IP addresses, file names and file hashes of previous attacks. That may be useful in a very narrow sense. In other regards, what we are talking about is more behavior based activities that we’ve seen. So some really good sources that I recommend that are free for everybody that you may already know of. One of our very favorites if you go to the next slide please, is a group called the DFIR Report. They do phenomenal work in doing reversing the malware and the breach events that occur and bringing people up to speed on how these access brokers work, how these ransomware as a service operators work, the ways that different pieces of components of software where they’ll build their own tools. Sometimes they’ll use other off the shelf or open source tools and they go through all of it and they have really detailed breakdowns of timelines of every action that occurred.

What occurred when? Like very, very technical, very detailed. And it’s all tagged in mitre attack. It’s very, very helpful. And so I would highly recommend you a lot of what they do, they produce free. We are one of the folks that sponsor them and we just love the work that they do. So a big shout out to that team.

Another one that we’ll highlight here is the Mitratech team themselves. They conduct I don’t think it’s quarterly, right, Dan? They do a different cadence. They’ll have evaluations of commercial products and those when they build those evaluations, they are nice enough to open source everything that they do. So they have a number of very detailed walkthroughs where they either indicate the tool that they use that is existing in open source or if they created it themselves, they provide that open source tooling for you step by step. Literally every command they ran on this thing that they. Were using to test it. So if you want to know this is how the sandworm works or something like that, you can find that data in there.

They release a lot of it. We find sort of sporadic a lot of other folks in the industry who are involved in this, whether it’s an EDR vendor or other sort of like anti malware vendors will do good reports on kind of as they come across them some different ways that ransomware gangs operate. Some of the MDR service providers that we know really well will do a lot of this depending on they’ll kind of talk about how they foiled a new ransomware strain right, that was coming through and hit one of their customers and how they prevented it from going. And they’ll provide a lot of that data too. And they’ll usually be pretty public about it. Not necessarily access to the malware itself, which is of, I guess, some value, but more of what did it do in what order so that you can understand, like those things still work in the environment. And I would say that there’s actually a pretty good shelf life on a lot of this stuff, Dan.

I mean, I know everybody’s looking for the cutting edge stuff, but the reality is and when we go in a lot of environments while the cutting edge stuff is hard to talk about, a lot of the old stuff still works, right? Like good old PowerShell is still pretty frequently successful in a lot of places.

Yeah, I guess I would say it’s easy to get caught up in like, well, am I getting the latest information? Which isn’t a good question, but can you still prevent and detect the things that we know about and have known about for almost 15 years now? I mean, some of these techniques go back that far. Even within Miter attack, they’re tracking. And this is the nice I guess one nice thing about being able to now share this information is that we have a collective set of data that exists. And so being able to test not only the known knowns, but as things do emerge, there is a nice community that we have in terms of being able to share, like, here’s what we’re seeing, here’s what the attackers are doing that you can go pull. Someone’s going to write a blog post, you can pull that in and be ready to test it as well. So I agree the shelf life is very long and it’s important to keep that in mind that this is a mechanism for getting you to get started as well as start the process of getting into a programmatic approach. So you’ll start to figure out where your good sources are in addition to the partners that you utilize that can help along the way.

Yeah, what? Well said, Dan. Something else that we keep a lot of eye on is as new cool open source projects come up that are things that, from a pen tester perspective, we’re interested in. We also recognize that if it’s useful for us, it’s useful probably for actual attackers. And so that’s worth investigating too. Right? So if we see this really cool technique that allows us to obtain Windows credentials without interacting with LSAs or doing some kind of cool new way of getting a handle on LSAs, that is different than the standard that really gets our attention. And then we start thinking about if we’re going to use this on an engagement and we are successful, what are we going to tell the customer to make us not successful? Right? Because the whole point of us doing this work is so that the next time it doesn’t work on that customer, right, they detect it, they block it, whatever. So if we’re not doing that, then we’re not really helping.

Right? And so that’s a big part of it. So as you’re you see those cool tools that come out, you’re like, oh, that’s awesome. How does that work and would that work here? Those are also useful things. I know sometimes it’s hard to put open source tooling on a corporate environment. I don’t know if you have any thoughts on that or not. Dan. I know sometimes that’s a case by case, but that’s some of what we try to help with as we build other tools on top of this to make this more continuous.

But the funny thing to me is that a lot of these firms will want not allow the people in house to run open source tooling. But if you hire a third party pen test, I can promise you there’s an extreme amount of open source tooling happening on the network. Right? Yeah. There’s no doubt you even see it in some of these threat reports, too, where it’s like, hey, we identified a cobalt strike beacon or mimi cats being ran. I mean, like some really handy tools for the Pen tester side, but they’re also the attacker as well. So yeah, so being able to detect those is important. Yeah, I mean, you guys can go on for days, but I got to wrap you up because I want to kind of talk about how do we make good, informed pen testing just part of your continuous program.

How do we even do that? Where do you start? That’s a great question. I think the simplest answer is Dan said it earlier, is let’s chunk some things up that are interesting to us to understand how our security tools would handle it and let’s try those out. Run books with Plexrac is a great way to both build a test plan and go through that engagement process of like, we’re going to try this today on this computer. Under these circumstances, what happens? Right? What was the outcome? Did we block it? Did we just get a log on it like we found it when we were in there or what? How long did it take? All those different dimensions of that.

Moving on from that, you’re going to want to get to the phase where you can kind of spread that out a little bit more if you’re going to move on to being able to sample more systems in the environment. Right, because Run books gives you the ability to both track and report on it, but you still are in a position where how do you kind of scale that out? But before we move on to that, Dan, I know that part of your intention of Runbooks is to keep track of this. So you have the baseline of where you start, where you’re going, but also really flow through remediation and things. Right? Right. Yeah. That was one of the biggest pain points that the reason I started PlexTrac was like I hated delivering pen test reports and then coming back later and finding that nothing had been resolved. And there’s a variety of reasons for that, but still stuff was not getting done, having more visibility into what are the actual problems and what are the priorities and who’s working on these and are we making progress? Right.

So being able to tell that story is an important piece of the puzzle. And then by having that in a test plan where you can do it over and over again, you can actually collect analytics on showing progress.

I think that’s an important piece of the program aspect. Sometimes there’s not like an automated way to just track progress. Right. You just got to do it. You got to go in and make the notes of who’s working on it. But it’s important information that can be very valuable as well to show here’s the things that are actually working versus not. I think also in terms of how you can get started, I mean, we’ve got it on this side.

You may not have all the skill set internally, but you can hire an external firm or deploy an automated resource like OnDefendant and Blind Spot to be able to do some of this testing for you. Right. And within Plex tract. It comes with some open source tech TTPs drawn from like Atomic Red Team and Red Canaries. Atomic Red Team resources. Miter Attack is also a very good one. And then utilizing a partner that can come in and you can have the framework set up of like, hey Ben, we’re not really interested in like a full scope pen test.

We really want to narrow in on this tactic or this technique. Can you help us automate as much as we can around this or show us how we can be testing this and utilizing your expertise to really see where the bigger gaps are? And I think that’s one thing I think Ben and I are continuing to hammer home is that this is a continuous mindset. It’s a programmatic approach and trust me, the results are demonstrable. Right. And it’s the best way to make demonstrable progress in your security posture. Yeah. Well said, Dan.

I think you’re right that even if you check something in January and it works great, that does not mean in March, it will still be working great. Right. All these things are changing around these computers. These things themselves are changing, getting updates. You have no idea when it’s setting. A default setting suddenly changes on you important to make sure that the end result is what you want to do by doing this type of functional testing, right? Yes, exactly. And that’s what the attackers are doing.

While we call them like Apt 41 or Fin Seven or Conti, they’re still going to be adapting their techniques as new tools are able to detect them. So you want to make sure, like okay, well, here was the procedure that we ran three months ago, like you said, or even two weeks ago, but we’re going to run that one and this new one that came. Out, right? Because then now we can identify, hey, we’re still good on the one that’s been known for a while, and now we want to make sure we’re able to test the ones that have come out recently. Yeah, Dan, I think you really hit that home. You don’t have to look at it just as an add on to your Pen test. It can be a standalone exercise throughout the year, and then it just naturally leaves that continuous assessment of Blind Spot and then Plex track, add on runbooks. Tell me about that.

And then I think we should just show them at a certain point. Yeah, absolutely. I mean, we’re excited to show so it’s like what we want to highlight is that we have a great offering together with On Defense built an amazing tool in and of itself, attack platform with Blind Spot, and then it integrates deeply into Plex track for the reporting and tracking and analytics pieces. So that’s what we want to show you. We’re also just trying to highlight that it’s not just a plug for the services and platforms that we provide, but it’s really the mentality that we’re trying to also bring to the table and that we’re both here as partners for everybody right.

Or using us. So, yeah, I think we’re excited to show it off, for sure. Yeah. Great. Let’s do it. All right. This is my job.

So while it’s pulling this up, the first thing that we’re going to demo is what it looks like going from a completed Blind Spot campaign where we’ve run an automated simulation on an endpoint. And now we want to get this data into PlexTrac so we can go from here. So we’ve got this very handy uploaded PlexTrac button. If you use Plexrac or you use our instance with Plexrac, we can do that. We’re going to go from here into our PlexTrac. We see. We have the built in purple test team plan, but no run books yet.

I really like how in Plex Rack you can break apart and you understand the different tactics or the kill chain phases. You can understand the different procedures that are inside of these because techniques are really just labels. Procedures are what you care about. Okay, so we’ve just sent pause it for a second. Pause it for a second. They got sentence I paused, I paused. I felt it.

What I’m sure is ben, where is Blind Spot getting these tactics and techniques? Yeah, that’s a great question.

We collect them from a lot of the sources I mentioned earlier from the Deeper Report from different open source, sort of like community places like what the Mitra attack folks have put together. We are partnered with a number of both incident Response firms and security service providers who use Blind Spot as part of their assurance for their customers. And so they were able to feedback this type of intel to us about how these different threat actors work. And so what we do is we recreate that activity in our own payloads, right? So our payloads have the steps, whether it’s command line doing stuff with the Windows or Linux APIs or whatever they are, the operating system, I mean, bringing in third party tools, whether they’re obfuscated or not, those things we replicate exactly those types of activities. We do it in a safe way though, so you can run it in production without worrying about downtime and then we bring it back. And our job is to help you start figuring out like, okay, we’ve identified gaps, let’s get this over into Plex Rack so we can track it there and really flow this through to remediation. So this particular one I think we got off of an Incident Response partner of ours combined with some open source intel we had on this particular threat actor.

They do a lot of fun things with downloading commodity PowerShell, malware and other things that normally get detected sometimes maybe. And it’s really important for us to know where the controls exist in the environment when we do this. This campaign, as you can see here, is run. Stuff in green is stuff that was blocked on the endpoint. Stuff in yellow was simply logged. The one lone blue item there was alerted and we have a bunch of red where there was no telemetry, no indication from the security tools that they observed the attack activity we were running.

Yeah. Anything else? We should go over here, Dan, I think, before we move on. Yeah, I think that was good. I just want to highlight that you have an automated capability to not only bring some of the data in from other sources, but also your own expertise through your assessments where you’re conducting this. And now you’ve identified the gaps and that’s when you send it over to PlexTrac to help on the reporting and the remediation side. That’s right, yeah. So we have a development team, obviously, that builds the tool, the functionality.

We have a content team of Pen testers and tradecraft engineers who all they do is think of different ways to run malware on customer systems with the goal of making it better for harder for real threat actors to do that. Right. So we’re always adding new stuff, either from cool stuff we’re seeing on our own Pen test, cool stuff we’re getting from partners in the community and things. So we’re always adding new stuff to it. So at this point, we’re ready to kick it over to Plex Rack, which we just did. So if you want to play the video. Lauren I think we’re going to go over there and you’ll notice before we only had the one test plan and now we’re going to load up into run books, too.

And we’re going to go load look at the test plans here. We’re going to see that Apt 41 purple team showed up and now we actually have two test plans. Can you pause it? Lauren one thing that’s really important is that the test plan is literally the test plan, right. Without it, we have no expectation of what’s going to happen in this scenario, what we’re looking for and those sorts of things. So, Blind Spot, when we do the integration, we look first if what we’re running, the simulation we’re running doesn’t exist as a test plan of Plex Trek, we go ahead and build it. We map it all out so you get to use all the tags and everything that the folks at PlexTrac have put together. So you get all the visuals, which is great because we want this to go as fast as possible into you getting this resolved and not just be like a science project for the next three weeks.

Right. Go ahead and play it. Lauren so all the procedures are in here. Everything is in. You can see it was imported from Blind Spot up there in the description. And we’re going to go back over to the runbook here and we will look at the engagement itself. And what I want you to see is all the stuff there, all the scoring that we did on the previous side.

On blind spot, we flow over into PlexTracs. So the fact that it was logged if there was no evidence, if it was blocked. When you open up one of these items, we highlight and we bring in a lot of data that we generate from our automation as well. The procedure, the timestamp of when things occurred, all that stuff is there and it matches exactly. If you were to go look in here in the Blind Spot interface, we’ll go find the same. This is the WMI Exec PS one file that they downloaded and ran. Super commodity, I would say, like not indicative of a very advanced attack, but something that actually works from time to time.

So here’s the same one you can see. We flow all the same data over into Plex tract. And now it’s here for you to use in your reporting and in your statistics building. Inside of Plex tract, it is bi directional. So if you sync it once from Blind Spot, and then you make some changes in PlexTrac, we’ll flow that back into Blind Spot. So wherever you’re working, whatever you’re doing and collaborating, that data stays in sync across both platforms.

Yeah. And that’s what’s exciting is the one thing that I’ll highlight is that and I’m going to show some of the Plex tract side is now that you have this test plan, you can retest some of those same procedures or if you wanted to dive deeper. But also, really the power of Plex tract is in the reporting and tracking. I will demo. I’ll pull up our video here.

Are you seeing that I’ll hit play? Yeah, we can see it. So you can see here that we’re in the Run books and the same test plan that Blindspot had provided. And you can see we’ve executed this maybe a couple of times. And what I wanted to highlight so I’ll go back just briefly because I skipped over it quickly.

Say you have a completed test plan, or you’ve even made a little bit of progress. Within the Run books, you can use this button and hit Submit Engagement, which actually means, hey, we’re done with this engagement, and we want to actually get to the report so that we can then identify the findings that came out of it, what worked, what didn’t, and really have the tracking and analytics power of Plex tract.

We had submitted this report already, so now we can just hit the report button, and it goes to the actual read out of the report. We added this narrative just to show that you can have some other commentary that you might want to show detailing what you did and how, and that you can add as many narratives as you want. We have a content database that has narratives tied to it as well, so you can bring in as much content as you need or generate your own. And then you’ll also notice here and I’m going to pause it for a second you’ll also notice here that from Run Books and also from Blind Spot, it pulls in, like, what worked and what didn’t. But then within Run Books, you actually want to specify, well, is that okay? Right? Because you may have ran like, who am I? And that may not necessarily be a vulnerability or a finding, but it’s something that happened. And you can still document that, hey, it worked, but for the procedures that you actually want to specify, like, no, this is a severe risk, that’s the power of PlexTrac Run books is that you can specify, like, this is an actual finding. And so these are the procedures from that test plan that we identified as true findings.

And you can specify them as critical. If you’re not familiar with the Plexrac interface, these are all the risks that then you can actually start to show the tracking of the status. So you’ll see here, it brought in all the other data from the Run book itself. We also have the procedures. And I think this is another important piece of just the power of reporting through PlexTrac as well, is that we always like to paint a picture. Typically we tend to stay focused or almost myopic on the findings and the risks. But it is important to notice that, and I’m going to pause it here for a second.

It is important to notice, like, these are the things that got tested and had potentially a different outcome, right? Whether it was blocked. So you’ll notice here that we’ve got the outcomes here as successful or no evidence, and then whether or not it was included as a finding in the report. And I think that’s really important because over time you start to identify, hey, this procedure initially was included as a finding and now it’s not because we’re making progress on how we’re detecting it. Right. And then one kind of final highlight is you can always export this into a document based report within Plex tract. It can be in your own custom template. We have an analytics module that has a lot of power, but you can also see this is even just the default template that comes with the platform for delivering to those auditors.

Or if you had a different reporting structure where you couldn’t use the platform to do board reporting or executive reporting, you could utilize a document template that’s some of the back end of PlexTracs for that tracking and remediation. And we have a whole way to collaborate on those findings. And obviously OnDefend, you’re very deeply integrated with us as well, which is just a cool integration that we’ve got. And I think if I had to think of the critical step as much fun as the front end Red Team stuff is right condensing the reporting and the reaction time is the stuff that we find that when that happens. These things go better because you spend more time doing the fun stuff and less time writing the report and figure out, how am I going to make this matter to somebody who doesn’t know anything about security? Right? So that’s really critical. If you’re going to make this adopt this as a real, true continuous test, that’s something you both thrive on is how easy it is to read your reports, because I’ve looked at them both and communications background and it’s like, yeah, this makes sense, right? Exactly. I think probably those of us that have been in security a long time, you start to recognize how important being able to communicate risk effectively really is.

An important part of anybody’s job in security. Whether you’re the most elite hacker or not, you have to be able to communicate why that’s a problem. And that’s really why we’re here to help, is that we have automated tooling to help identify gaps. We have systems to help clarify your priorities and see who’s working on things and then ultimately have analytics to show whether you’re making progress. And at the end of the day, that’s how we really help you win. I think one other thing to highlight is that you don’t have to have all the deep skill sets, right? You don’t have to have a true internal Pen test team to be able to do this. You can utilize a partnership with on, defend or blind spot.

You can use open source resources to do some of this on your own and just get started. But it truly is a better together kind of story. So we’re getting close to the ten minute mark, so I want to get some questions in there. I really like this question from Katie, but how far does the support go after the test? So after threatening form and Pen test if vulnerabilities are found? Oh, great question. In terms of as like an offering threat for Pen test, we would treat that as a regular Pen test in terms of the results and the findings. So if we have the results of a threatened poor Pen test and we walk through with your team, the way that we deliver this report, we have the complete results of what happened. So we can kind of mash it up against what the security tool saw, identify places that we would recommend maybe making changes, and also what I think equally important expectations that you had on things that would happen.

Like, if we show you our test plan, there’s no way that steps working and that stuff works and you didn’t detect it. That might not be as critical to us, but if it’s really important to you, those are things we want to know about. And then we come up with, like, here’s our recommendations on how we do it. And then that’s really where we switch into how do we go back and retest this again and again? We definitely are there to support you. You have full access to the test plan, right? And if you are already a Plexrac customer using Runbooks, you know how to do that. And if not, the team of Flexrac would love to give you a trial of that to see how that works. So we can put our threaded Form Pen test plan in there as separate stage test plans for you.

You can rerun those things and sort of see where that goes. But yeah, this isn’t like we handle the report and we’re instantly gone. We’re definitely there to hang out and make sure that it’s useful information to get to your objectives.

Couldn’t have said it better well said. Well said, sir. I mean, in a way though, it does feel like it’s leading to the idea of purple team exercises. Yeah, for sure. Yeah. Collaboration is always important, right? And whether that’s using an external service like OnDefend and Blind Spot or just having internal collaboration with the people that are the defenders of the organization and the ones responsible for fixing it, that’s really the glue that Plex Tract provides is that mechanism for the reporting, the tracking and the integration with any kind of partner that you’re utilizing. Right.

But definitely the Blind Spot integration is a fantastic way to bring that data in. It shows what you should be fixing from PlexTracs. You can assign it to people and identify who’s working on what. And so it really is meant to be a mechanism for the whole workflow. Right. So this is a question actually from me because as you guys are talking, it just makes sense. But you know, you’re going to have some people at home who are like, it’s just going to cost too much, or I don’t think I’m going to find value.

What do you really say to those people to try to understand this new offering because it is so new? Yeah, I’ll take that one first, I think, because when we think about executing a threat of form Pen test, because we are so focused and so tight about what we’re going to do, it’s not like, I don’t know, give us a month and we’ll see what we can find. We are coming in for like one, two or three days max. We’re going to get through this exercise. We’re going to have a lot of really great results about things that your tools did well and things your tools didn’t. And then that will lead directly into those improvements. Right. So it’s actually less than a traditional Pen test with sort of more open attack path planning.

And it’s really, again, hyper focused on a particular threat actor or set of techniques that you’re worried about. Like the example we talked about. What if a threat actor was suddenly using my VPN? Would I be able to realistically reconstruct the things they did if I found out they’ve been using this VPN for two weeks? I don’t know. Let’s go find out.

Yeah. And I think there’s an extreme amount of value in just having more visibility, right.

Without making a broad statement of like, oh, you can afford it, or it’s going to be affordable for you. I think it’s really more of how valuable is the visibility? And knowing that you’re actually on the right trajectory, then the price becomes a different discussion. So I think what’s important is that you don’t have to in order to even take up this concept, you don’t have to go buy anything. Right. It really is a mindset shift. But then just knowing that there are resources and there are partners out there. That will work with you regardless of how large you are or your goals or budget and all that.

All those things can be accommodated in essence from a sales perspective. Right. But I think what’s important is it’s the mindset and knowing that you have resources available and expertise as well as automated capabilities, which is really nice.

What is your guy’s favorite part about the integration and the partnership? I’ll go first. Yeah.

We developed the Runbooks module under the guise that we want to have a true, better capability for conducting Purple Team engagements and collaborative engagements, being able to bridge the gap between the complexities of the Red Team and the nuances of the Blue Team and helping build more empathy and better collaboration on here’s what we’re doing. Here’s what we saw. And so that was the whole notion of the Runbooks module, but then having a tool like Blind Spot to be able to integrate all of the automated capabilities, it’s a fantastic one two punch right. In terms of being able to do the automated testing, have consistent feedback, and then being able to track and show results. So that’s what I love about it. From our perspective, I obviously love working with Ben and Chris and the team. Right.

Same. And what I’ll say is you mentioned earlier today in the webinar, Dan, about the idea, like, you show up for the Pen test the next year and it’s like all the same problems still exist. Right. I don’t want the information that we create in any of our exercises with a customer going into some silo that a very few people have access to. It’s got to go where all the other security data goes to be resolved and to turn into real improvement. And that’s not something I’m interested in building because PlexTrac has done a much better job of building up than I could ever do. So we put it there.

It’s where all the vulnerability data goes in. It’s where all the other stuff goes so people are already looking in there. It just made perfect sense for us. It goes in there because that’s where security data goes to be tracked and resolved and then scored on. Right. And that’s really how you drive all of that into a single place so you can get real good risk visibility that you would. I mean, we are just a very small part of a bigger QA picture, right.

Quality assurance. We just happen to be worried about post exploitation adversary activity. Right. But we want it to feed into all of that. Yeah. And I guess I’ll clarify kind of what you were saying Ben is like. Yeah, the Runbooks module is just even the blind Spot integration.

From a PlexTrac perspective, that’s a integration. We integrate with all the other scanners. If you are hiring Blind Spot to do like a black box Pen test or somebody else, or your internal team, you’re having lots of different sources of data and we help bring that all together for the risk remediation and the prioritization. So we have a broader approach and we work with partners to use PlexTrac as the portal for that too. I hope that kind of answers one of the questions we had around all these tools for customers to purchase and use and the answer is absolutely yes. But you can also purchase services from OnDefend to do this for you. Right, and I think that’s an important piece from the PlexTrac perspective.

We don’t do pen testing, we just facilitate the mechanism for reporting and tracking. Right? Yeah. Another question, we got this one’s from Steve. While clearly valuable, the concept of tip still does not address dwell time from zero day attacks or doesn’t. Oh well, dwell time is a fun topic because I think the only dwell time that Cecils are happy with is anything that’s like basically zero, which is really hard to do.

The idea of dwell time really is getting back to detectability when the actions occur and then sort of like long lived like command and control detection at some point to consider I guess a compromised host still there. Somebody is either logging in from the outside or they have something on the inside beaconing out. Right. And so the idea that you’re going to detect that, a lot of what we think about when we think about doing this is that ability to detect what represents C two traffic. So we have a whole bunch of scenarios that we just kick up different types of C two and just send little pings back and forth across it to exercise it. But that’s it, it’s just an exercise. And can you detect command and control beacon traffic kind of stuff? I don’t know.

What do you think about that? That’s a good one, Dan.

I want to clarify like dwell time from a zero day, that’s going to be hard for anybody, right? Because it’s a zero day, you don’t know until it’s exposed to the public that let’s take the concept of just any kind of exploit where we’ve gotten access, gained access, and now we’re in. And the dwell time from when we’re in to when something gets detected or alerted. And I would say yes, this concept does address that because within run books and within blind spot you’re getting the timestamps of when these procedures are getting executed, right? And then the blue team can go in and they can identify logs of when they might have first seen some indicator that this activity was going on. And that’s the true notion of how long that dwell time is from execution to detection. I think that’s a really important feature is that you can test with probably a lot more confidence than just a forensic exercise post breach that like, hey, we’re testing this now. And we can say, like, we know when we kicked off the engagement and all these timestamps of when these things actually happened and then how quickly somebody actually identified it so you can start to see in real time well, in a much more proactive perspective of how long it’s taking to get some identification around these procedures.

I think this does actually help that. Yeah. All right, well, we’re hitting close to that two minute mark final. I know it goes so fast that you’re having so much fun. Final takeaways from you guys. What do you really want people to know about this new service and this partnership? Yeah, I’ll go first. I’ll let Dan close it.

I would like to say something. We’ve talked about a lot of this is don’t be afraid. Don’t feel like this has to be this big grand thing. Just pick one thing you’re like. I wonder if we detect this and go try it, right? And then we can work on ramping that program up from there. And we’re here to help. We love doing this kind of work with folks, understanding what the risks are and figuring out what’s the best way for us to help you safely test that.

And we’re really excited to be able to share with our friends at PlexTrac because we think that does the best job of helping you, once that happens, communicate and get it resolved. So dance. Yeah. I think that the biggest thing is that we’re all here to help everybody as much as we can in the journey and the marathon that is improving your security posture. I think it can be overwhelming, and we’re here to help provide clarity to that. Right. Through PlexTrac, you can have a full scope of your vulnerabilities that are getting addressed and bringing that data into light.

So you have this continuous history over time of all the progress that you’re making and then utilizing blind spot on defense to be able to do the testing when you need it and establishing a more continuous framework. And I think that one two punch really does help. But just knowing that we’re here to actually help people get better, right? Yeah. We are selling products and we’re selling services, but at the end of the day, our mission truly is to make the world a safer place. It sounds cliche, but I think the biggest thing is starting to do this in a continuous mindset so that you actually will get better and show your progress.

It has been enlightening from somebody I know. We’re already getting questions of where can we review this material because it’s that good to all those people. Don’t worry. We’re going to go ahead and we’re going to send out the webinar after this, the recording and everything. And if you want to learn more, PlexTrac has plenty of resources. They’re on Youtube.com, Twitter, LinkedIn, Instagram. We have all of the pages right there.

Same with OnDefend. Yeah. Guys, best way to reach out through the website. I’m sure you guys could give your phone number away at this point, there’s plenty of ways to get in touch.

This is fantastic. I’m really excited about the integration with Blind Spot and the partnership that we have with the OnDefend team, and it’s exciting to see how this progresses. All right, well, Dan, Ben, you guys are amazing. Thank you so much for joining us. And everybody who has taken an hour out of your day, thank you so much for joining us. Thanks, everybody.