nCircle The Lens Blog: June 2011 Archives

One of the metrics collected and shared in the Vulnerability Management Benchmark is Average Days Since Last Scan, otherwise known as scan frequency. It's available for free as part of the Basic package. This metric was a fairly surprising one for me to watch develop in the Benchmark community. Over about a decade of working with customers on VM programs, I can't help but have developed my own impression of how often organizations run vulnerability assessment scans in their environments. Most organizations have a monthly scan cadence; some work towards weekly or even daily, and a few outliers work on a quarterly schedule. Based on this anecdotal evidence, I had assumed that average scan frequency would end up being around 30 days. I was definitely wrong. Here's the Average Days Since Last Scan scorecard for the last six weeks:

You can see that the line has been hovering between 53 and 70 days, ending up at 67. I suspect there are two things happening here that have pushed the average up.

The Best Intentions

We all know how our best intentions sometimes don't work out. Well, the same can be true of an organization. It may be that the Vulnerability Management program specifies monthly scans, but delays happen, outages occur, and assets don't get scanned. It may very well be that what we're seeing in this Benchmark metric is the reality of life in the enterprise. A plan for 30 days comes out with an average of 60. It might be that planning for a 15 day turn around would come out closer to 30.

Outside Influence

In the Basic Benchmark package, we're rolling up everything contributed. This data set includes, and doesn't distinguish between, external and internal scans. It may be that we're seeing the influence of an externally mandated cadence being expressed in this metric. PCI requires quarterly scans by an Approved Scanning Vendor , and that distribution of ASV scan schedules may drive a schedule of 'preparatory' scans, which in turn drive the average up (more scans occurring, but less frequently). This trend, however, is really only evident over a longer period of time. Finally, there' s a general push towards more continuous monitoring, driven in part by the Federal government. As this trend continues, we'll likely see more frequent scans across the board.

Of course, you should be wondering how your organization compares to the Benchmark. If you want to find out, join the Vulnerability Management community. Remember, the community isn't just nCircle customers. If you're using a different vulnerability scanning tool, it's still completely free to see your metrics compared to the Benchmark. When you want to drill further into the details, compare and slice in different ways, then the Benchmark Premium package is available as a simple upgrade.

I haven't talked much about the other Benchmark Communities, but they're available as well. It might be interesting to join the Configuration Auditing and Vulnerability Management communities to compare change rate in your environment to trends for average risk, but that's a completely different blog post.

Remember, keep in touch with the VM Benchmark on Twitter: @BenchmarkVM

Until now, the information security community has relied on rumors, conversations and sparse breach reports to develop some kind of consensus on what vulnerability management metrics should look like. The metrics themselves haven't been hard to come by, but how do you know if you're doing well or not? How can an organization truly assess the performance of their vulnerability management program without an industry standard benchmark that provides a comparative metric for the rest of the world?

At the RSA show in February we introduced nCircle Benchmark, giving the security community the first product that actually allows you to compare your metrics to those of your peers. Since then, we've added new Metrics Packs, new capabilities and most importantly hundreds of new community members. Those of you who have already joined a benchmark community know what the combined benchmark results look like. For those of you who haven't joined yet, I'm going to give you a preview of the Vulnerability Management Benchmark right now. This is the last six weeks of average host risk scores for the Benchmark community.

For this Benchmark metric we're using the industry standard Common Vulnerability Scoring System (CVSS) base score. We've calculated the average of the aggregate CVSS Base scores for each contributed asset and trended that data over six weeks. There are hundreds of thousands of assets included in this benchmark, and it's growing with every new community member.

If you're wondering how your risk scores compare to those of the Benchmark, you can join the community and see this scorecard with your data included for comparison for free as part of the Basic package. There is no other way to compare your risk score to your peers, and there's no reason, budget or otherwise, that you shouldn't be able to speak authoritatively to your peers, management, and board about how your Vulnerability Management program is performing. This data is only available from nCircle Benchmark. You don't even have to be an nCircle IP360 user to join and contribute. If you're using Qualysguard or Rapid7, you can still join the Vulnerability Management Benchmark community (and lots of you already have).

The Basic package includes scorecards for average risk score, average scan frequency, vulnerability distribution by platform and vulnerability distribution by severity. You'll see more of the Vulnerability Management Benchmark here in the future, but you can see it for yourself today, including the comparison of your metrics, by joining the community.

Welcome to a world in which you know where you stand.

Don't forget that Benchmark isn't just for Vulnerability Management. There are communities for configuration auditing, endpoint security, anti-virus, identity and access management and more. Each of these has a Basic package that's available free of charge as well. If you like what you see there and want to drill into the details or create your own scorecards, there are Standard and Premium packages too. And you can keep up with the VM Benchmark community on this blog and on twitter @benchmarkVM.