I've ranted on here about Quality a great deal - I'm finally going to post something that shows a little bit about how seriously we take quality here. But first, a bit of history...

This industry was started by a bunch of hackers and admins - as such, the science of vulnerability signature writing came from the science of exploit writing. You either started with an exploit, or you started with a banner. We came from there, too. Our company's founder won Capture the Flag at Defcon 2 years in a row. We were all about exploits and banners at the beginning.

And then, somewhere in early 2001, that all changed. We started thinking about detecting vulnerabilities as a science in itself. We created a decision-tree structure for detecting applications. We invested our brains heavily into "feature-based" and "behavior-based" application and service profiling.

And we stopped looking for exploits and banners. We committed to 100% non-invasiveness, and we started looking for detection signatures. It was often heard at the time: "Banner checks suck!" That's obvious, now, of course.

Of course, when you start doing that, the first question that comes up is: "how do I know that what I'm doing is better than banners and exploits?"

And one of our engineers (Minoo) came up with the concept of what we came to call "Signature Precision" - it's the idea of limiting how far away (by codepath and information reliability) a check is from the ideal.

We take great care to differentiate this metric from the concepts of accuracy and quality: they're different things. It is possible that a rather imprecise check can be quite accurate. Or that a very precise check can have false positives and false negatives.

But our research shows that, over the long term, the higher your precision, the more likely it is that your checks will be better.

As far as I know, we're the only one in vulnerability management who is willing to talk about quality - I'd love it if our competitors were willing to talk openly with us about this. It'd let us serve all of our customers better.

You can find the precision metric whitepaper attached here. Please give it a read, and let me know what you think.