The Project Quant: Report/Survey was released on Monday. Project Quant is a research project to develop a metrics model for measuring the costs and effectiveness of patch management. I hope that my comments here are constructive to the community.

The costs and effectiveness of patch management is just one of many management domains that together make up what we know as Information Technology Infrastructure. I think the report does a good job of calling out the assumptions and it is important for the reader to interpret the findings under these established parameters. I think the output of this model will need to feed into other models in order to get a more complete understanding of the whole.

In the end, my understanding of this entire project comes down to this: a metric for the costs associated with a predefined process. More specifically, the costs of human labor within the context of a process we call Patch Management.

Measuring the costs and effectiveness of a process is not groundbreaking so the value here is specific to the craft of Patch Management. By craft, I mean that a craft is a domain that consists of humans, tools, materials, and techniques. Crafts over the years benefit from a continuous feedback loop where humans change, tools change, materials change, and techniques change; progress then is when this change is an improvement in operational efficiency. This metric model is most valuable when used to analyze the level of operational efficiency gained by a change in humans, tools, materials, or techniques. Vendors who are in the business of decreasing the costs of human labor associated with this patch management process (techniques) should be all over this like flies to you-know-what.

I’d like to see what McAfee (who acquired Citadel), HP (who acquired OPSware), Patchlink who is now Lumension Security, or EMC (who acquired Configuresoft) have to say about this report. These are companies who know their market and I am sure have a lot of knowledge of the processes we call Patch Management.

Is there a difference between metrics that measure operational efficiency and metrics that measure security/risks? Hell yeah! (as they say in Austin TX) I think there are a lot of techniques and processes out there that are highly biased toward operational costs and do not account for risks of being too efficient and operationally lean. I’m not trying to throw operational metrics under the bus, I’m merely putting it in a context where it belongs. A metric model that measures operational costs is a metric model that measures operational costs.

I can see a lot of promise in the work Rich Mogull has done and ultimately, with enough support, it may grow to include many other crafts within the IT operational domain. For those of you old and grumpy like me, you have seen this movie before. Just take a look at the Supply-Chain Council’s SCOR. This is what a social and technological system ultimately has to become in order to account for an eco-system of humans, tools, materials, and techniques. The management of a modern supply chain must take into account both operational efficiencies as well as security & risks, it accounts for the cost of doing as well as the costs of knowing, and it must do all of this across multiple administrative boundaries. It may take a while to get there but we will get there.