Patterns

Forgive me - I may be out of my depth on the details - but two things struck me as I read this.

The first was that mapping is not an act of precision but an act of imprecision - that the link between the representation and the thing itself is necessarily imprecise. So that misunderstandings are possible because the representation maps over to a class or a group of associated "things" and not "the thing itself."

Similarly, it seems to me worth looking for the common ground in the different ways that different communities use the same terms. We can think of all maps, really roughly , as Venn diagrams where a word maps onto some portion of reality and different groups pick out different pieces of that reality and mistake them for the whole of what the word means.

Linda

I don't know that I agree with you completely, however, I think you already know that :)

I'd argue that there are two reasons why the risk and compliance metrics are absolute failures so far.

1) Those are that are in the risk and compliance space choose to use their own metrics because it is something that they can sell. It pays for them to come up with their own methods of measurement, mostly because it becomes easier to sell to others who don't fully understand. Their metrics may be more feasible and/or more efficient and/or more realistic, but it doesn't matter if no one else uses it. No standardization leads to a lack of control, which leads to a lack of understanding, which leads to confusion.

2) The counter argument is also true. The problem with standardization is a lack of context. Even CVSS requires a whole other set of inputs outside of a traditional score to have it make sense in your environment. It's too generic to be useful to anyone except for those who like averages without knowing what the range of possible values or value skew.

In the end, that is the problem with CVSS. It's is too static to be useful, as people long for a metric that is a little more flexible. However, others take that to mean that they can create their own metric and have it be meaningful/useful. Hopefully, someone finds a happy medium.

However, who knows? Look at the financial and insurance industries. They have way more experience in risk then the infosec field does and even their models are continuously proven wrong or incorrect.

Designing a model or metric is great. Believing in it and not attempting to prove it wrong is where the failure lies.

Linda,
The notion of precision that I am addressing has to do with a communities of practice’s required resolution of the world. In some practices, only a few names are needed for the thing named and as practices specialize, many names emerge and are made stable by the processes of the domain.

I do agree and recognize that the utility of the map comes from the fact that it is _NOT_ the territory and thus a purpose-built lens to the domain: one of many maps (models) that earn its keep by remaining relevant to a community of practice and significantly smaller than the territory (the domain being modeled). Frankly, if the map is as big and complex as the territory, it has no value.

This reasoning asserts that all standards (names, maps, models, etc) have consequences. When was the last time you went to fill out of form and could not find a name in the multiple choice that faithfully represented what you were trying to communicate? Under race, I never find the term Hawaiian, or under type of company, I never find ‘information security vendor’, and so on.

The metaphor of a Venn Diagram or of imbrications do well in describing that boundaries are not tidy but messy and there can be value found in the mess. Dimensionally, it is often too simple to be thinking in just 2D, one must think of a network graph where a single node can have many vertices (roles) to other domains. This is more of the form that I am referencing with the word ‘asset’ in my posting. A persona like an executive, an internal auditor, a IT system administrator, a helpdesk agent, all have a different understanding and draw very different inferences from the name ‘asset’ and this is not a bad thing if we accept this heterogeneous viewpoint. The abuse that has taken place so far with metrics is analogous to American scientists forcing Canadian scientists to _understand_ Fahrenheit in order to understand some result set.

Thanks for the comment and helping me clarify my points.

--tk

Your temperature example made me think of metadata layers. Here the thing is specified with some precision as san Francisco, and the attribute as the temperature. Once the object is specified with some level of precision, then we can have layers of metadata representing a different view of the attribute. Temperature can be in units Fahrenheit or Celcius but also be a symbolic layer like hot, cold, etc. Or it can be another layer like average, 1 sd above, 2 sd below, etc.

I think this way of thinking addresses the imprecision question. Here I am not referring to the spatial imprecision of the thing but the precision of the metadata, which would depend on the application using the metadata. You can be as imprecise as you want to be in your view as long as there is comparability. What is nice about this way of thinking is that imprecision introduced by scoring can actually make the scores more comparable from a semantic perspective to the Canadian viewer.

If we really wanted a tighter metric of comparability, then we can score each temperature around the globe with its percentage of deviation from the temperature in Toronto (my favorite Canadian city). The generalization of this kind of thinking to information security is tricky because it in the temporal event space. Buffer overflow tag to some vendor is unauthorized access to another, which is fine as long as there is precision in the object being referred to. On the other hand, attributes of spatial objects like vulnerability of an operating system of a server are easier to deal from a semantic perspective because of the lack ambiguity both for the object and its metadata.