Measuring Your Offensive Security Maturity: Perfect Your Processes

We’ve kind of broken it down into four kind of different key areas of conversation. I do encourage that if you have any questions, feel free to use the Q and A function. We try to monitor the chat function, but it gets a little bit busy sometimes. So we just really encourage you to throw a question into the Q and A function and if it aligns with something that we can answer in line with the conversation, we will do that. Otherwise we’ll save those questions towards the end.

Basically, today we’re talking around how the importance of mature processes really make an impact in how much progress you will actually make in your security program, especially from an offensive security perspective. So we’re going to talk around how do you develop your strategy and your mission and your goals around your offensive security maturity. Aligning that strategic vision into an actionable plan, not just being aspirational but actually making progress and how do you show that progress and then how do you measure your results executing on the processes consistently and then obviously building in those checkpoints of how do we know that we’re on the right track? Are we making progress? What are the things should we be focused on? So that’s really the goal of the topic for today. And with that jump in Dan and Devid, I’d love to kind of know as you’ve developed your consulting practice and then you’ve also been involved in other practices before this, both from a consulting perspective as well as either big government, the federal space, as well as some of the private sector. What has been your experience as you have built these teams in documenting your strategy and your mission and goals and what role does that play in your overall process? So I will speak from a recent engagement that we worked on, where we worked with a fairly large institution, that they had their own testing team. And we were doing a maturity assessment over that team. And we looked at everything from their own individual capabilities, but then also what their processes looked like, what the outcomes were, the consistency and their reporting risks and how they did that.

But it all kind of starts at the very beginning with what is the goal of this team from an organizational perspective and what are they set out to do? Is there a team charter? Has that been thought through? What is their mission every day when they get out of bed and they execute their operations? What are they after? This organization that we just recently worked with had it really nailed down pretty good. They were unique in the fact that this testing team sat with an internal audit, which is the third line of defense, and they were very clear on their mission, right? Their testing team was very specific to support all of internal audit operations. It wasn’t designed to be like a separate Red Team, for example. So they were very clear on what they were and what they were not trying to be for the organization. And I think that really helped them sort of get buy in and understanding whenever they went out and did the tasks and the operations they were supposed to do.

I think it also helped set expectations upfront with the team internally as well that had to execute those things because they know that their goals need to be aligned to the internal audit plan, for example, in helping be the technical testing arm of the internal audit group and the risks that they’re focused on for the next one, two, three years. And not specifically, you know, being a red team or, you know, per se.

So in, in that instance, right, it was, it was purely baked up, you know, specific for them. I thought they nailed it. But, you know, so whether you’re internal at a company like that or, you know, if you’re at a consulting firm, or I could let even devise talk specific from the government mission perspective and how that looks, but making sure everybody’s on the same page at that top, top level, I think it’s so important.

Yeah, I mean, taking a step back, not even necessarily government, I actually got to have a really cool conversation with one of those companies that do the credit ratings on other companies, like how trustworthy somebody is for a loan, right? So you hear of the United States having AAAA credit rating or stuff like that. This guy was part of one of those organizations that provides that rating, and he was saying the documentation of security teams, cyber security teams, is starting to get factored into that level. Right. So we talk about documenting your strategy, your mission, your goals. It might make sense from a process of making sure that your people have repeatable processes, but it also is starting to make sense for these credit companies. I don’t want to call it credit union because that’s not the right word, right? But these credit companies to determine what is your trustworthiness, right? How can you reliably detect and protect against cyber attacks and threats if you don’t have documentation saying, hey, this It security team, their security, their mission, their goals, this is what we want them to do, this offensive security team, they’re going to be testing those goals. Right.

Without clear and understandable documentation, it’s really hard for an outside entity to understand exactly what is the process you’re going through as a company to ensure that you’re secure from like a non consultative practice, right. From a consultative side of the house, let’s say, ultimately having your strategy and your missions, I kind of put those in the same bucket, right? But having those set out like, you know exactly what you want to accomplish, it gives your team a much better understanding of what do they need to be providing to the client so that they can level up their particular security. Right. Because I’ve said it many times, you don’t know what you don’t know. And if you’re just all over the place with every assessment that you’re doing from a consultative perspective, you’re giving mixed results, and you’re always wanting to give the best of the best assessment as you can. So if you don’t have that laid out in documentation, it’s really hard to replicate. There’s a reason why word of mouth has never really worked over history, and just to piggyback on that from a consultative point of view, right.

I think it’s important. So for us, our mission is to help our clients be better and to level them up.

So in that same vein, we always say, like, if we’re able to pawn a client or get the main admin or get to the flags that we want to get to, we don’t celebrate that, right? We celebrate actually when our tactics, techniques, and procedures are thwarted by our clients defenses, because that means the investments that they are making are starting to pay off.

And that’s why we outline wins and strengths in our pen testing reports. Because our mission is not to get in and be a domain admin and to destroy the client. Our mission is to help our client better defend against these types of tax and show that the investments that they have made are paying off. Right. So that’s our actual mission. And I think there’s a lot of firms out there that they do a victory lap when they’re able to deploy someone or they’re able to get domain admin. Right.

And that’s their mission. That’s their goal. Right. That’s not how it should be.

Yeah.

I think it’s important to recognize this is really a lot of, in many ways, organizational leadership. Right? It’s like, how do you take your organization, whatever it is, and really give them a purpose and a defined set of goals? And really, in the words of Simon Sineck, if you haven’t heard, I would assume a lot of people have, but you start with Why? What problem are we trying to solve? Right? And I think that really gets to the mission of, hey, at the end of the day, it’s not just, look at all the cool new tactics I was able to execute or the cool new hacks, oh, you got it right there. Yeah, there you go. He’s trying.

Great book. If anybody hasn’t read it or heard Simon Sineck speak, he’s a fantastic speaker. But being able to actually say, this is the problem we’re trying to solve, and here’s the actual mission that we have. Right. It’s to make the organization better from a security perspective and actually show progress and be able to communicate effectively. And so I think it really is important for these teams, whether you’re building an internal team or you’re even working on building a consulting practice or trying to mature your consulting practices, like, hey, what are the goals we’re setting out for these customers? And are we communicating those up front to say, like, hey, this is why we’re doing what we’re doing and what our mission is around it? So I think that’s obviously the first step, and then that helps to lay out the framework for what does it take to get there. Right.

And I think from a process perspective, you have the administrative side of the process of, like, how often are we scheduling engagements? What’s the scope of the engagements? All of those things, and those are all operational things versus the actual TTPs that we’re going to execute and how do you do those? So that marrying the operational process with the technical expertise is really important. Yeah, it’s something that we don’t really acknowledge in our field as much as we probably should. Looking at it from an adversarial standpoint.

Most of our criminal counterparts, I guess you can call them, right, the non emulated criminals, the real criminals, they’re just cybersecurity companies with no morals. And something that they do really well is documenting their strategies, documenting their individual teams, goals and missions, what they’re supposed to do. Conti really opened the door to show how bad is it on the other side or how I guess bad is a bad word, terrible descriptive word for this, but how good they are at actually being bad actors. They have clear and understandable metrics goals and strategies that they want to obtain. Right. You have your Red Teamers that are there to actually do the exploitation and the infiltration. You have Malware devs that are writing malware.

You have QA testers. Right. Like, some of those guys have better QA testing of their software than major companies. But their documentation is phenomenal and when it does get leaked, it’s not talked about enough. But it’s something that if they’re doing it really well, we should be paying attention on our side, especially if we’re trying to emulate those criminal activities through offensive security or through a breach in a tax simulation, whatever it may be, we have to take it all in. So if they’re taking the time to stop and write out a TTP for the goals that they want to accomplish for a team, then we probably should be too.

Yeah. Sometimes it’s not like I kind of alluded to before. It’s not the funnest thing. Right. But it’s really vitally important, especially, and I would say especially as we continue to mature as an industry at large. Right. And we start to have like a more structured approach to like, what, you know, it’s not the Wild, Wild West anymore.

And really actually trying to say, like, here’s how we show we’re making progress. If you don’t have that documented, then it becomes hard, the conversations become harder when you’re asking for budget around certain projects or certain initiatives that you want to deploy. But if you can actually have the documentation, like, here’s what we set out to accomplish, here were the goals, here’s what we identified and here’s the resolution that makes a much stronger case for how you can move things along as well as just being ready. And I think also continuity. Right. So if somebody leaves the organization, you have some of that documentation available for continuity’s sake. What were you going to say, Dan? Yeah, I was going to say so much of success in cybersecurity is the absence of something happening.

Right? So when all things are quiet on the Western Front, nothing’s happening, that’s a good thing. But that’s something that’s hard to quantify and hard to show executives, board members, audit committees, et cetera, that like, oh yeah, are cyber is doing well because nothing’s happening? Or were we just lucky enough that nothing has happened yet recently? I think that’s one of the reasons why it’s so important to have a strategy that has an offensive security testing program in the first place that is emulating those criminal activities. So you could actually show in chart and graph like, yeah, here’s all the Miter TTPs. In 2018, we were pretty successful using 70% of these. Okay? Now it’s 2022 and the adversarial success has decreased by this amount across this whole TTP set. And having tools like Plexrack and like other tools to show that adversarial success or not is a great way to illustrate how these programs and investments are paying off. Because of that problem, the fact that success in cybersecurity is the absence of a bad thing happening, it’s hard to show that.

So I think that’s why this is so important. This is why we’re so passionate about it. Survivorship bias is real. You definitely see it every day. You’re like, no, we definitely still need this firewall. You can’t take it out. Yeah, but is it that bad? Yeah, that’s good.

So, yeah, I think there was one question. I’ll just answer a live request. Will these slides be available after the webinar? Absolutely. We’ll post these and you’ll get a copy of the recording as well.

We talked a lot about kind of having an overall strategic vision and the mission and the goals which are important in any kind of organizational structure regardless of what we’re talking about, but then really aligning that to an action plan. And I think this is kind of where that rubber starts to meet the road and being able to document like, here’s the goals.

When I was a security director, we tried to kind of map out, here are the things that we know we want to be able to a validate. We have some gut instinct, right? We have some theses that we want to validate. But then there’s also things that we just don’t know. We don’t know yet. Right? But we’d like to be able to start showing, hey, just because we haven’t heard anything doesn’t mean everything’s great. It just means that our tooling may not be tuned correctly.

We may be getting hacked by a very sophisticated adversary. And that’s why you do if it’s not logged, it didn’t happen, Dan. Just remember that. Yeah, exactly. Ignorance list. Right? But not necessarily.

You talked about, hey, these are the things that we really want to know, and that’s kind of those goals and that strategic plan, but then actually putting it into, here’s how we’re going to get there. And we took an approach of hey, we want to start small, right, and validate. We know if we were to identify some issues here, we would have the resources today to be able to address them. And this was from a holistic security program where we took some team members to actually go execute the tests, and then we would hire external firms to come in and do more holistic tests. But I think what I learned and what I also see through now, working with our customers through PlexTrac, is that being able to communicate effectively really improves the collaboration over with the overall arching goals, right. So customers can either one communicate with their clients. This is the path, and this is the journey we’re taking you on.

These are the things that we’re going to test in a continuous fashion or vice versa. If it’s an enterprise, they can say, like, hey, these are the goals we really have laid out for the year of things we really want to start getting a handle on, and they can communicate that to their external consulting firms to help them in that journey. I’d be curious what your guys’s experience has been in kind of putting the rubber where the rubber meets the road from the strategy to the actual action plan. I think you called it out really right off the bat, right? Just communication. It’s one of the things in the military, the phrase was communicators are the worst at communicating. Right? Like, the guys running the show on it or the radio don’t really have the skills or soft skills to be able to communicate clearly on what their expectation of a strategic vision is.

We’re getting better at it, I think, as a whole, as an industry. Right. For the longest time, like, when I first came in to the commercial world, what, 2018, it was still pretty bad. The last couple of years, it’s gotten a lot better. Like talking with prospective clients or current clients. Hey, what is your hope? What is your goal? What are you trying to prevent? What do you assume is the threat here? If you don’t know what your threat is, like, what kind of e crime actors or nation state threat actors would be going against you? Let’s figure that out. Right.

Understanding the holistic view of what are they trying to protect helps create that strategic vision, which then helps pass that down to either the consultant or to the internal team. And usually leaving that actionable plan, like, how do we get to this point? Leaving that to the team. The actionable team allows them to align to your strategic vision, especially when you’re giving feedback, right. It’s not trying to close it all off at the executive level or at the directorate level. You bring it down to the tactical level. Having them come up with the action plan usually goes back up to, all right, here’s the vision. Does the action plan actually meet that vision? If it doesn’t, then you can return back.

It’s like a nice little feedback loop.

Yeah, it’s like, do the goals of our client, for example, match the plan that we’re going to execute? Or the plan that they’re executing? The strategic vision or the goals of the team are to, let’s say, identify the unknowns identify the unknown vulnerabilities out there versus maybe their strategic vision is to test specific types of technologies, emerging technologies that they might have in their environment. Right. Let’s say they’re working a lot more with containers and cloud and serverless infrastructure.

Is the strategic vision of the testing team aligned with the strategic vision of It is a very important thing. If your testing team, if all of their skill sets reside within the traditional sort of network based hacking approach, they don’t understand cloud, they don’t understand server lists, they don’t understand containers, mobile web apps, et cetera, then there’s not a good alignment there to an actual plan to test the emerging things that matter at that organization.

And we see companies kind of struggle with that. I think it’s hard enough to find the right people with the right communication skills, like the deed said, and put them in the right seats and then having them keep their skills up to a certain level where it’s keeping up with the strategic direction of the company and moving at the same pace and speed.

It’s a lot to keep up on.

And I think you highlighted an interesting point that I certainly experienced and have experienced in the past, too, is that you may have like an overarching goal of your offensive security program, but when you put it into an action plan, it’s probably going to be a little bit different depending on the domain of what you’re testing. Right. So across an enterprise organization, you’re going to have some different skill sets and different maturity of different teams and different technologies. We would always emphasize we were developing software. Right. So it’s like from an application security perspective, you can get into a mode of always you’re basically always in this legacy testing mode where you’re just testing more legacy code because you’re not really getting ahead and trying to implement what the term the industry is kind of calling shifting left, getting sooner in that development life cycle.

On the application security side, you may be dealing with people that might be more sophisticated and you can actually show a little different progress there, as opposed to say, the vulnerable management team or like the It team, where there may just and I’m just calling this out as like a general thing. It’s not always the case, but they may be a little bit further behind the curve. Right. So you have to kind of address your action plan based on where they’re at from their maturity perspective within the team. Right. And then that can yeah, so kind of borrow the thought from the military, right. Separate It into your three tiers, your tactical level, your strategic level, and your operational level.

Right.

Operational being that big picture, the strategic vision and actionable plan from an operational level is still going to be slightly different from the strategic level and that’s going to still be slightly different than the tactical level. But when you take all the tactical level action plans like, hey, we need to secure cloud, we need to secure mobile, whatever it may be, whatever the goal is for that particular team, those get built into the larger organizational goals.

And that’s why this is kind of hard to perfect in a sense. Right. Because it is so many different sections and so many different ideas and visions that get to get wrapped up into a singular kind of plan. From a company perspective, it makes that difficult to kind of corral, I guess is a good way of putting it. From the consultative perspective, it’s nice because we’re able to ask who the technology leaders are and we look at it from more of the strategic level and not necessarily the operational level. As a consultant, we’re injecting from the strategic down, not from operations down. And from a company perspective, being able to take some of that off their book, it allows them to focus down on exactly, all right, how do we take the rest of what the consultants aren’t doing and build that into our vision? Right.

Hopefully that made a little sense. Yeah, I think kind of what it highlights is that you want to be ambitious with you start with the end in mind of like, here’s where we’d love to be at some point, but you’re never going to be able to tackle everything that you want to do in an amount of time. So being realistic of what the plan is going to influence and what those goals. And I think that is where we’re seeing the shift in the paradigm of moving from very longer periods of time in between assessments to a more continuous approach. Right. And I think that helps bite the pieces of that elephant off so that you can actually attain, you can show progress. You’re tackling things one at a time and actually feeling like you’re making good strides towards the end goal, as opposed to, like, hey, we know it can be overwhelming, I guess, is what I’m trying to say.

Right, yeah, 100%. But yeah, at the very basis of it, like making sure that whatever your testing team’s goals are, that they do have the company’s goals and vision and strategy and it direction in mind. It can’t just be, oh, I saw this thing on this webinar and now I’m going to incorporate this into our month long testing plan. It might not be the highest and best use of your time. So folks that can think in terms of risk and help put it in good terms of business risk, I think tend to be the most valuable testers out there, right. Because they could marry the two things together, the technical abilities with what does this mean for the business overall? Yeah, marrying the impact with the actual components of what it took to get there is really important thing. Also to add here, right, is some of the better mature offensive security programs that are internal to companies are those that anticipate the vision of other business units.

Right. There’s a really funny meme of the Ant Man movie where one of the characters is in a car and he’s basically acting as the sizzle and he’s like, oh yeah, marketing dropped a brand new web page without telling anybody. It’s never been tested that got hacked. But we got stock two.

Probably stock two was on the known strategic vision. Right. But the marketing website Drop probably wasn’t known. So being able to corral the other business units or at least understand what their goals are from a strategic or operational viewpoint can help drive an actual plan that can help secure those before they become a problem.

Probably most people you talk to are going to say that have worked outside of consulting are going to say, oh yeah, this has happened at least once. It wouldn’t be the first time I’ve heard a prospective client or a friend of mine who works in industry go, I don’t know how to keep these people in line. They’re just dropping everything left and right. Oh, there’s a new NFT project. For some reason we’re doing this and you’re like, wait, what? So having the anticipation for what your other sections of the business are going to do, especially in the It world, is going to help build an actual plan that can prevent such issues.

I think there’s a conflation not a conflation, but the correlation between being proactive not only in your security testing of like, hey, what things should we try to predict? But also being proactive on the business side and being able to go reach out to the key leaders and say like, hey, we are a partner with you in this journey. And so starting to build those relationships, which is an important aspect of assist though and a security leadership is not just understanding the business risks and the impacts that vulnerabilities and threats play in their organization, but also the relationship capital is really important. Being able to have good conversations with the other leaders, to be able to say like, hey, before these things might happen, we’re here to partner with you to make sure that it has the least amount of impact on the business or we’re not exposing ourselves to something that we don’t need to. Right? Yeah, exactly. Proactive approach on everything. Okay, so we’ll move on because we’re making good progress here. Okay, so you developed strategy, you’ve got an action plan.

Now you’ve actually got to go do it. So now the true rubber is meeting the road in terms of. Like putting this into play, putting an action plan into the hands of the operators.

I think that we talked a lot about people in the last obviously, the whole focus was people in the previous webinar. But this is where you’re now empowering them, right? Not only as a leader of an internal team or as a team that’s developing and maturing.

I always come back to some of the primary goals of leaders are to empower and equip their people to be successful, to do their jobs well, right? And so, from a consulting perspective, they’re trying to empower their clients to be successful on it. And so you’ve got the plans in place. Now you actually have to go find the tooling and the training to make sure that you can execute on the plan, but also any other capabilities that would be required to communicate it effectively, to collaborate, to show the analytics and the progress. Right. What are your thoughts on kind of how you actually start executing on the action plan? If you got a map out, what are the things we need to execute here as part of our plan, and what can we execute with the capabilities we have today versus what capabilities do we need to go and learn and supplement, and how do you supplement it? So I think a good way to do that, right, is if you know the strategic direction, you know sort of what the plan is, you could take a step back and you could do the Miter mapping to TTPs of Adversaries that you want to emulate. You could look across capabilities of the team and say, yeah, we could execute 80% of these or maybe 50% of these TTPs that we’re trying to emulate throughout the year. There’s these specific ones, or I have no idea.

But we know we need to emulate them. We know we need to give these a shot. How do we do that? Right? That’s where, quite frankly, a great business case for firms like us to come in and help our clients with that 30% of TTPs that they don’t know how to execute. We’ll work with them in like a journeyman program to show them how to do it, surf the shoulder, or we could get hands on. They can watch us operate. That’s one way to do it. Finding unique training avenues to learn those tactics, techniques, procedures, setting up pest labs, all of those different things.

So I think a lot of it is knowing what you don’t know first and foremost and whether or not that’s important to the plan or not is crucial to that. And then going out and finding ways to either learn those capabilities in house or supplement it with external teams, consultants, et cetera.

Yeah, something I say almost all the time is like cybersecurity is massive, right? It’s a massive field and people don’t acknowledge how big it is. An offensive security within that, even though it’s a small section of cybersecurity is still massive in its own right. From my own experience, I’m really good at network pen testing and malware development. But the moment I need to get into Web and start getting super in depth in the Web, I’m probably not going to be the best or strongest in that realm. Right. Cloud the same. So understanding exactly where you are as an individual and then how you are as a team is super important.

And being mature, especially in this realm, is acknowledging the fact that you just don’t know at all. There’s not many people out there who can say, hey, I’m really strong in everything and usually those people that are end up working for government somewhere, right.

When you look at training, it’s acknowledging that, hey, I don’t have this, I don’t have this skill. Maybe I should go out and find somewhere that can teach me it or at least give me a baseline in there. And if training isn’t going to fill that gap, then sometimes tooling can, right? Great example is this emergence of breach and attack systems that have been coming out into the market as of lately utilizing some form of AI to try to drive network testing. Not to say that network testing is something that people are predominantly weakened because if you want to take the two largest known skills, it’s probably going to be network and Web. But a web app tester may not know network testing. And if you can supplement that skill with having an application that can take care of that, why not? Investments can go a long way, especially into ensuring that your security is as good as it can be working with the people that you have and not necessarily going well. I wish I had the best candidate in the world.

We already know there’s a skills gap and a job gap inside our career field, so we got to make do with what we have. A lot of times there might just be issues with hey, I’m spending so much time on doing X-Y-Z-I don’t have the time to go learn these new techniques and procedures. There literally isn’t enough time in the day reporting, for example. I spend half of my year just on reporting things and going to meetings and having to present. Maybe there’s a better way to do that, like PlexTrac for example, to make things a little bit more efficient in the reporting process. Or maybe there’s some automation tools.

There’s obviously some pen test automation players that we like and we use to help cut out and take some of those, we’ll call them more rudimentary TTPs out of the equation and make it more efficient so the hands on hackers can focus their time on the more value added activities. So that’s something that we’re trying to do every day ourselves too, right, is to cut down our time that we spend on the nonvalueadded stuff and automate, make it more efficient, spend our time where it matters the most. So that’s really important too, right? To step back and look at all the necessary things that you do and say, can we do it better? Can we create time? Yeah. And I think you guys hit it on the head and truth is, like, catering to your strengths, like, hey, knowing what your team can do and then supplementing that either through automation or additional resources like, hey, we’re going to go hire this firm that really focuses in these areas. And then I think that also really continues to hone in on the importance of documenting these things along the way. So that way you can actually not only get external training for how to do additional tactics or just needing to learn different technologies, but you now have developed an internal methodology, an internal knowledge base, so to speak, of how things work in your environment and be able to train each other. Right.

And I think directly we had this experience where we would do the read out internally where it’s like, here was what the testers did, here’s what the blue team took and kind of tried to adapt how to defend against these things. And everybody got exposure to both sides. Right. And I think that’s really important is that you can continue to train yourselves as you learn and grow and it’s that notion of continuing to give back, right? Yes. And something about training. We have a lot of great courses out there, especially around offensive security, whether it be from like Elearn or offensive security sans wherever it may be. But we don’t always need to focus on things that are marketed towards offensive security.

Right. There’s never going to be that one tool that fits every shoe size.

You’re going to need different tools. And sometimes the best tools to use are the ones that are built in house. So looking at training options that can teach an individual those critical thinking skills or those technology skills like programming languages and whatnot, can go a real long way with making your team effective and efficient, give a team member like a three week course in C and see what they can get to. Granted, three weeks and C is probably not enough, but getting the baseline, some understanding of how to get code execution to do things that they want to do to at least understand where they can go and look that can add to your tooling without having to make an investment into a third party application.

Were you going to say something? Damn, I didn’t want to touch you on no, I could go on more and more there. But yeah, I think it’s sometimes hard enough to find the time to do the right things. I think really taking a look at processes and making them more efficient so you could get more done with the same amount of people is definitely crucial, especially in this inflationary environment where things are tougher nowadays. In the current environment, you got tech layoffs and all those things happening. So being wary of like, look, there’s going to be opportunities to get people training, but most of the time you’re not going to be able to send people away for three weeks, some three week boot camp to learn all these things. So you have to get creative. You got to get creative.

Hey, we can have dreams.

I think the best way to learn is also to be not only learn from the past. So if you’ve already developed a pretty good system where you have a lot of results and testing methods from previous exercises, then you can actually not only train up your team as you bring people in, but make them more efficient as budgets might get tighter or you want to make the current team as efficient as possible. And when we talk about training, being able to have a mentor, whether that’s internal or external, is really important and can go a long way.

So we’ll dive into kind of the last, certainly not least, but this is really the notion of metrics and how are you actually showing progress. And I think this is where kind of everything comes together. And obviously we’re big huge proponents of this at PlexTrac. I mean, this is kind of why we’re here, is to help bring the process together, to not only help document what’s going on and who’s doing what, and eliminate as much friction in that process as possible, but also showing progress and showing what were the key things that were identified and how does that map to where we should be? Right? Long term. And so it’s really important to establish those metrics. And I think you alluded to it before, and I’d love to kind of maybe hear you expand on it a little bit of like celebrating the wins. Right.

Understanding the mission is not just to go hack them and show them how awful they are at not detecting all the techniques that you did or how awesome of a hacker you are. Right, right, exactly. But you come in with stated goals and here’s how you handle our techniques against these goals. Here’s the things that we think you should focus on next. Right. I’d love to kind of hear your perspective on some how you’ve helped clients along that path. Yeah.

So we do TTP tracking with all the techniques that we do, and when we present our findings, we show the TTPs, and we relate them directly back to Miter, and we have the client understand exactly what we executed and how and whether they want us to help recreate that scenario or if they want to redo it on their own. That way, I think it takes away the shroud of mystery of what happened or how we were able to do something.

Then what’s nice is if you start to track against these TTPs for clients. You could sort of start to build a database over time, especially if we’re doing tests quarterly or at some sort of frequency, you can show trends over time to say, okay, against these TTPs, you guys were really good. Your defenses were such that we could not execute successfully across these TTP families.

But here’s where we did have adversarial success, and here’s why. And look, if you make these sorts of changes within your network or within your application, it’s going to knock out this swath of issues that were created here. So I think it’s trying to show those key themes and how we did things and then tracking it against the taxonomies that we have available to us that try to make sense of it all. And if we could keep those taxonomies the same or similar over time, then we could really start to have data, show trends, show investments paying off with adversarial success going down over time. So that’s really what we try to get to, like, the best state possible with our clients is regular assessments tracked by TTPs show progress? Yeah, only a people perspective. Dan and I have seen some really cool databases and tracking systems that various clients and friends of ours have implemented and just having an understanding of where do their individual operators skill lie, right. When they look at the strategic vision or the mission and goals of their team, is that trend going in the right direction or is it going in the wrong direction? Based off of like, hey, I successfully was able to do DNS Tunneling or I executed malware or something of that sense, right? Like, as long as it’s directly relatable to mitre TTP or whatever framework you’re using to track the checkpoints and evaluations become quite easy.

I think the part that tends to be a little bit difficult for most people is like, tying in the tooling and training to evaluations and checkpoints. One thing that we used to do in the military, and I took it over to our team as well, was after somebody goes to a training course, you come back and that Friday you give a presentation on what you learned, right? It sounds kind of like, oh, God, now I got to go give a presentation. But you’re going to not only cement that information in your head because you’re like, okay, let me review exactly what I’ve done. But it also helps out the remainder of the team who maybe didn’t have the time or the opportunity to get to that course in particular and give them at least a framework or a baseline to what they learned so you can build in those checkpoints for training. When it comes down to evaluations, having clear and understandable timelines like scrums from the dev world is always a good thing. You can never have enough feedback is my opinion. Right.

You want to know what you’re doing well, you want to know what you’re doing wrong. And as long as you can talk up and down the leadership ladder to everybody, you can figure out exactly where you are in your vision or your plan or your goal, whatever it may be. Right, so implementing some true timeline, checkpoints and evaluations is super important, especially if you want to have reproducible results. Yeah. And I think obviously we’ll tune our own horn here because we can. But our PlexTrac, our mission is to help teams, especially with our runbooks module here’s, all the things that we’re going to execute in the test. Right.

And then you can track that and you can have those pre built like, hey, we’re going to check in in two weeks, and here’s the stuff we’ve done. Here’s what your team has shown they can or cannot detect and prevent.

And then here’s the risks that we’re seeing and be able to have kind of that real time report generation of the outcomes. One other thing that we also emphasize that is obviously important is not just how did they do as a result of the assessment, but also how are they doing with the outcomes and are they tracking those findings that came out of that to remediation, to then provide that input, feedback back to the beginning of the process where it’s like, hey, we feel like we’ve fixed this. Is it showing up again? Right. And how long did it take us to fix those items? And I think those are all important parts of the metrics that sometimes we lose sight of as security teams because it’s like, oh, well, I get my opinion on the one or two things that I think were really important at the time and then lose sight of all the other aspects of the report. Right. And so having something like PlexTrac where you can track your SLAs towards remediation of these findings, how quickly it takes to get them resolved and things like that are also very important metrics from the operational perspective.

Not just the offensive security team, but I think, something that they should actually be very focused on helping that team drive forward.

Yeah. Like the mean time to remediate and how quickly, whether it’s a volume through a vault scan or more complex issue identified through a Red Team engagement, at the moment that that thing is discovered and identified, the clock starts. Right. And how do you respond to it, how do you react to it? How quickly can your team put together a result that closes, that it’s so important? And to have the tools now, like Plexrac and other tools that are in your stack to help you track those things is immensely important. The last thing you want is to find something that’s serious and not be able to act on it and do nothing with it for 16, 9120 days. Right. That’s the worst case scenario.

Yeah, absolutely.

And so we we have about four minutes left there’s. I’ll put out kind of like, if you have any other questions that you want around kind of the process and strategy and team building and perfecting your processes, by all means shoot them out. Now, there was one question. I thought it’s kind of an interesting thought as we kind of head into 2023. Where are most of the offensive security jobs going to reside as we head into 2023? Obviously, there’s probably a lot on people’s minds, but I’d be curious what your thoughts are there.

Where? In Russia.

No, I didn’t say that. This is off the record right now. It’s all recorded now. That’s a great question. So where will the offensive security jobs be in 2023? I would say hopefully there’s more of them, but there is more to life than offensive security jobs across all cybersecurity where teams need help. But I think it’s going to be more along the lines of the emerging technologies and offensive security related skills in those emerging technologies. So if you are an aspiring Pen tester or wanting to break into the field, I think if you choose a little bit of a niche and hone your skills around something like cloud or serverless or web or mobile, that would be a wise thing to do, right? Pick a niche, be strong in that niche.

I think that’s where the growth will be in offensive security jobs and maybe what will shape up to be potentially a slower year, so to speak.

Yeah, I think we’ll see actually a governmental push towards ICS as an industry, like any sort of industrial industry. So energy, manufacturing, transportation, the systems that control the infrastructure of countries is going to be extremely important as we move into 2023. I mean, the House at the United States just introduced a bill to put a ton of money into research grants for universities, for the energy sector, not necessarily because it’s particularly vulnerable, but because it’s extremely critical. And we’ve seen exactly like how far damage can go when it comes down to physical attacks. And we’ve all acknowledged that ICS is probably way behind the tide of cyber security. So if you find your niche in being able to decrypt XOR over wireshark or Caesar ciphers or whatever they’re implementing now on ICS encoding or encryption, you’ll definitely see a growth in job needs and research in those industries.

Yeah, absolutely. And I think probably the way the industry from an offensive security job proper, I think you’re going to find more of those in the consulting realm. I think that organizations are going to continue to recognize that they need these services, they need this expertise, but don’t always have it in house. And so they’re going to look to service providers like yourselves to be able to supplement this. And so I think there’s going to be more opportunity, as always, regardless of what the economic times say, that there’s going to be always more opportunity on the consulting side, but I think also kind of what we lined out today really is a good path for building that offensive capability in house. Regardless of whether it’s your formal title or not, it’s still an important piece of the puzzle. Right? And so I think that my advice would also be, like, don’t get hung around the axel of having an offensive security job proper.

Take these principles and start to build them into your Blue Team capabilities, where we would actually say, like, hey, this is more of like, building a true Purple Team from the ground up and taking what you know and your gut is around. We should be in a proactive approach, and you can have an offensive security job there without maybe the title. Right? Great.

Well, we are right at the button at the top of the hour. This is a great conversation, I think, obviously very important, and we appreciate everyone’s time. And I guess I didn’t score the Q and A slide, but if you need any other resources, like I said, this is recorded. We’ll be able to share this out. If you need more information about Plexrack and how we can help with your tracking and remediation capabilities, please check us out. If you need some security assessment work or just some mentorship in the security field, definitely reach out to Dan or Devede over at Echelon Cyber. And just again, thank you both for taking time to chat today.

We have our final installment coming up next quarter, so stay tuned for that, where we will talk all around the tools and technologies to really implement a mature offensive security program. So a little teaser for the last, but probably the most intriguing one.

Definitely. Thanks, everybody, and appreciate your time. Thanks again for having us. Stand. This is awesome. Thank you so much. Much.

Great to be here. Thanks, everybody. Yeah, thanks all.