Measuring Your Offensive Security Maturity

How Do You Stack Up?

The cybersecurity industry is full of blinky lights. Organizations are inundated with endless threats and nearly as many solutions, while also facing limited resources and a persistent talent shortage. Everyone is busy, but is it really making a difference? Every program is a work in progress, but how do you know if you are actually becoming more mature? 

Instead of just doing testing and assessments for the sake of doing them, actually becoming more mature is all about defining success for your program, determining what activities will get you there, and measuring if what you’re doing is actually working. 

PlexTrac exists to help offensive teams become more efficient and effective so organizations can become more proactive and, ultimately, more mature. Check out our webinar series with Echelon Risk + Cyber to learn more about leveling up your offensive security game. 

Let’s take a look at the major questions around offensive security maturity. 

What Do I Measure?

One of the most important aspects of creating and growing an offensive security team is determining the team’s purpose in the forms of a charter or mission statement or by defining goals and objectives. Once the team’s purpose is established, milestones and performance indicators can be tailored to accurately measure progress and maturity.

Most agree that you’ll need to consider three areas when thinking about building and maturing an offensive security function, just as you would any business unit: people, processes, and technology. 

People

An offensive security team’s personnel composition and needs will vary based on a number of factors, including the industry, consumer of services (internal business unit or external entity), budget, and previously established goals. Measuring people goes beyond simple headcount; the people who are employed, retained, and trained who comprise the team will need to have technical and soft-skills necessary to achieve the team’s goals. A good pillar for determining success is to evaluate based on the team’s personnel composition and individual, as well as corporate progress, against defined benchmarks and indicators of success.

Processes

Haphazardly executing ad-hoc tasks against nebulous objectives is a recipe for burnout and failure for any team. Offensive security teams are no exception. Along a similar vein of establishing the team’s charter is establishing documented and repeatable methodologies. These can be high-level, then broken down into sub-methodologies, but the important piece of the puzzle is that they exist, are kept up-to-date, and are followed. Having processes in place allows organizations to begin to grind them smooth and identify areas for efficiency. 

Technology

Tech stacks are an integral part of organizational ecosystems. However, technology is meant to compliment, enhance, and enrich capabilities not stand alone as a measure of success in and of itself. When it comes to offensive security teams, you cannot implement tech in lieu of people and processes. And, you cannot successfully achieve your goals with your personnel and methodologies if they are not properly supported with the tech stack they require. There exists a tension. On the one hand technology is meant to be a force multiplier and drive efficiency and excellence. On the other hand, you need appropriately trained staff who can expertly utilize the technology or the tech investment can be a waste. 

What Measuring Stick Do I Use? 

There are a lot of definitions and opinions on the different approaches and activities of offensive security. What’s important is that you choose something to aim for that aligns with your goals to keep moving your program forward.  PlexTrac offers our take with the Offense Security Maturity Model to define basic to advanced offensive security practices that can help a program become increasingly mature. 

In addition to using a resource like the PlexTrac model above, the factors that are relevant for measuring success and progress vary greatly based on the purpose and needs of the team. 

Some common areas for measurement include the following:

• Technical Prowess and Proficiency

As personnel grow more advanced in their skill sets and capabilities, offensive security teams can provide a more varied array of services. Determining benchmarks and industry accepted technical standards that can be used to gauge “leveling-up” in skillset can be a valuable metric to measure personnel technical enhancement.

• Time 

As the saying goes, you either have time or money, but rarely both. Using time based metrics can also be a powerful tool. More efficient use of time can mean more value for effort. Look at finding ways to gauge time. If your organization is services oriented, time can equal costs-of-goods-sold. In a non-billable sense, saving time can increase productivity.

• Operational Capacity

As your program matures, the team’s ability to take on more work can be used as a Rosetta Stone to determine progress. The definition of “more work” can be determined by volume of work, type of work, or an amalgamation of both. 

• Vulnerability Metrics

Defining what success looks like when referencing findings or vulnerabilities is a task that can be nuanced as well. Some organizations want to count the number of flaws, based on severity, and want to see the number of flaws diminish over time. Others may focus on both individual finding numbers and mean time to close metrics. There are a number of ways to deal with vulnerabilities; however I think most would agree seeing observed flaws being fixed or mitigated is a sign of progress.  

• Detection Capabilities

The teams who are tasked with defending environments should also be enhanced and matured as a byproduct of the maturation of offensive security teams. Blue teams, incident responders, and SOCs should all find that the offensive security testing efforts are sharpening their skills, enriching their data feeds, and informing their detection and telemetry systems. As your offensive team grows, the blue team should be making the red team’s job harder and harder.

How Do I Know When We Are Mature? 

Maturity is about demonstrable progress and is anything but static. Maturity is relative to the goals of your team and organization and should be a constantly moving mark. The real question is less about reaching a particular stage of maturity than it is about being about to make, measure, and communicate progress. 

Goals and More Goals

Success is relative. However, when considering the constantly evolving nature of information security, hardware and software, applications, and technology in general, it’s a safe bet to assume your team will never hit a point where they can’t just stop maturing. Stagnation is never an option in the realm of tech. As the technology landscape, as well as threats and risks change, your offensive security team’s goals will need to grow and change in stride. Determine what is important to the consumers of your services, whether they are internal business units, or companies enlisting your expertise. Find your baseline, then set short-term and long-term goals that are measurable and attainable. Then go set some more.

In the process, make sure you can demonstrate what you are accomplishing to leadership both inside and outside of the security team. Teams able to articulate their growth and the value it brings are better able to secure the resources needed to exponentially accelerate their progress. 

The PlexTrac Solution

The PlexTrac platform can be used as an aid in measuring the maturity and efficacy of offensive security testing efforts and teams. It is designed to provide data on findings’ status, allow teams to collaborate effectively, and provide analytical insights into trends on the data derived from the testing efforts. 

PlexTrac is a force multiplier for offensive security programs. Book a demo to learn how PlexTrac can accelerate your path to maturity. 

Nick PopovichPlexTrac Hacker in ResidenceNick Popovich’s passion is learning and exploring technology ecosystems, and trying to find ways to utilize systems in unexpected ways. His career has focused on adversarial threat simulation, offensive and defensive security, and advanced technical security assessments. Nick’s mission is to help individuals and organizations involved with defensive security operations to have an opportunity to observe the mechanics and methods of the attackers they’re defending against, and to assist in realistically testing those defenses. He’s a lifelong learner and loves finding new ways to get under the hood of systems and networks. He is a father of three and a husband to one.