My kids were asking me about laws and I told them there were laws in computing. As always they did not believe me so I had to gather my evidence. Here are a few I took the liberty of summarizing.

Cargill's 90/90 Law: The first 90% of the code accounts for the first 90% of the development time. The remaining 10% of the code accounts for the other 90% of the development time --Tom Cargill

Page's Law: Software get twice as slow every 18 months -- Larry Page

Brooks' Law: Adding manpower to a late software project makes it later -- Fred Brooks "The Mythical Man-Month

Metcalfe's Law: The value of a network grows proportional to its number users squared. -- Robert Metcalfe

Amdahl's Law: Multiple CPU cores are only as fast as the slowest serialized code -- Gene Amdahl

Moore's Second Law: As CPU transistor counts grow geometrically, so does the cost of manufacturing. -- Gordon Moore

Asimov's Three Laws of Robotics: Protect, Obey, and do not Injure Us -- Isaac Asimov

I said that if they could prove any of these wrong, I'd give them 5 bucks. :-)
--tk

The Project Quant: Report/Survey was released on Monday. Project Quant is a research project to develop a metrics model for measuring the costs and effectiveness of patch management. I hope that my comments here are constructive to the community.

The costs and effectiveness of patch management is just one of many management domains that together make up what we know as Information Technology Infrastructure. I think the report does a good job of calling out the assumptions and it is important for the reader to interpret the findings under these established parameters. I think the output of this model will need to feed into other models in order to get a more complete understanding of the whole.

In the end, my understanding of this entire project comes down to this: a metric for the costs associated with a predefined process. More specifically, the costs of human labor within the context of a process we call Patch Management.

Measuring the costs and effectiveness of a process is not groundbreaking so the value here is specific to the craft of Patch Management. By craft, I mean that a craft is a domain that consists of humans, tools, materials, and techniques. Crafts over the years benefit from a continuous feedback loop where humans change, tools change, materials change, and techniques change; progress then is when this change is an improvement in operational efficiency. This metric model is most valuable when used to analyze the level of operational efficiency gained by a change in humans, tools, materials, or techniques. Vendors who are in the business of decreasing the costs of human labor associated with this patch management process (techniques) should be all over this like flies to you-know-what.

I’d like to see what McAfee (who acquired Citadel), HP (who acquired OPSware), Patchlink who is now Lumension Security, or EMC (who acquired Configuresoft) have to say about this report. These are companies who know their market and I am sure have a lot of knowledge of the processes we call Patch Management.

Is there a difference between metrics that measure operational efficiency and metrics that measure security/risks? Hell yeah! (as they say in Austin TX) I think there are a lot of techniques and processes out there that are highly biased toward operational costs and do not account for risks of being too efficient and operationally lean. I’m not trying to throw operational metrics under the bus, I’m merely putting it in a context where it belongs. A metric model that measures operational costs is a metric model that measures operational costs.

I can see a lot of promise in the work Rich Mogull has done and ultimately, with enough support, it may grow to include many other crafts within the IT operational domain. For those of you old and grumpy like me, you have seen this movie before. Just take a look at the Supply-Chain Council’s SCOR. This is what a social and technological system ultimately has to become in order to account for an eco-system of humans, tools, materials, and techniques. The management of a modern supply chain must take into account both operational efficiencies as well as security & risks, it accounts for the cost of doing as well as the costs of knowing, and it must do all of this across multiple administrative boundaries. It may take a while to get there but we will get there.

In my independent study of Gregory Bateson and Alfred Korzybski I truly understood for myself that the name is not the things named or as some would say the map is not the territory.  I call your attention to this manner of thinking because we have a problem with metrics in that the count is not the things counted.  Many metrics for risk and compliance describe beautiful mathematical formulas but only see a limited success because the classification of the things being counted is narrowly understood.  This blog posting makes the assertion that our problem with effective metrics is not one of numbers but one of semantics; not of the counts but of the things counted.

The things being counted must be named, defined, and ultimately understood by a community of practice.  The very act of naming is an act of mapping or classification; it comes with a certain level of precision and consequences. A useful classification standard for one community may be useless for another. To the degree that this mapping or classification is common with others in your community of practice, you achieve a mutual semantic coherence (some call this objectivity but I reject that term).  The durability of a set of metrics is challenged when multiple communities of practices are asked to engage in a common objective for the business.  Such is the case when one proposes a standard terminology and metrics that apply across a large enterprise consisting of multiple communities of practice and diverse personas.  To be useful one must know what these metrics mean and to be able to draw inferences from experience.

A measurement system must be judged on the notion of "usefulness to a community of practice" and this scoping must be made explicit.  The utility is a function of the audience's ability to draw inference from the counts and things counted.  Let me share with you an example I experienced with my Toronto team.  I said to one of my Canadian coworkers "Dude, it was in the 90's in San Francisco today".  A blank face appeared as I saw him think and convert this implicit 90 degrees Fahrenheit to Celsius ((F - 32) x 5/9) because he could not draw an inference from Fahrenheit.  Inferences like it being weather for shorts, no jacket required, that it is odd for San Francisco to have a high of 32 Celsius, that homes in San Francisco don't have AC because it is never that hot and so on and so on.

When you look at the notion of temperature, you can see that the different communities have chosen different standards because of the way they have come to know those units and it is more about the semantics than the mathematics.  This becomes exponentially more difficult when the syntax is the same but the semantics vary.  Take terms like 'asset' or 'platform' and you can fill a page with what it means in certain context with certain communities even within the same enterprise.  Each community of practice has come to know the term 'asset' in very different ways; this person has encoded work and meaning in ways that are different than others.  While mathematics remains important, we must turn our focus to formal ways to share semantics. Only then can we share both the numbers (the count) within their intended context (the things counted); semantics that can only be seen through a keen ethnographic eye that respects heterogeneous sense-making and the diverse viewpoints of an enterprise.

Yesterday (Monday) was all about Metricon 3.5 in San Francisco.  It was a long day beginning at 8am and concluding around 5pm.  The event was at the San Francisco Google office and a special thanks to John Flynn and the Google team for hosting this event.  I can’t even tell you how impressive the lunch buffet was at this place.  If I worked at Google I would be 400 lbs in a few weeks.

The event as you can see for yourself from the link above was broken up into case studies, panels, metric frameworks, measurement of real data, and last but not least modeling R&D.  The material was very high quality and for the most part, there were no surprises.  I took notes and from here on out you will get my humble opinion. 

In the Enterprise Case Studies, it was interesting to hear eBay, Kaiser, and Google speak about their measurement systems.  I have a very sensitive ear toward the community of practice for these systems and while eBay and Kaiser was your traditional start at the top with these measurements, Google was more of a bottom up which is great to see.  The role of the designer of these systems is to put data in terms that the audience can understand, not to dictate the way in which the audience should understand it. This required both a ethnographical evaluation as well as a mathmatical evaluation.

In the Metrics from Real Data, Jeremiah Grossman from Whitehat always has good stuff and it was followed up with Wade Baker from Verizon on their breach investigations.  In the framework section, I found Fred Cohen’s work on legal matters very educational.  This community of practice, judges and layers, have a very well established method to understanding information and it was great to hear the challenges for measurement in that space.  Essentially, a bag of bits is real if and only if it has an intersection with other bags of bits and event that support the claims.  It is like a n-dimensional crossword puzzle where just being correct up and down is not sufficient.  One has to be right across and in some cases many other vectors.

Its about 8am in SF and I begin another crazy day at RSA.  In closing, I want to make an observation about all of these experts who claim to have the ultimate measurement system.  Your challenge is not in the numbers or mathematically consistency.  It is in the semantics and the classifications of the objects within the domain.  The reality is that a large enterprise will have nothing short of 5 very discreet personae who on a good day can’t even agree on what to order for lunch.  Getting them all to come to common terms on the meaning of ‘x’ is much more difficult than getting them to understand that 5 is one more than 4.  This standardization of object within a domain is a prerequisite to measurement and must be addressed before one can impose a metric system across multiple communities of interest.

Research in this area [Star 2009] shows that standards are:

• Nested inside one another
• Distributed unevenly across the socio-culture landscape
• relative to communities of practice; one persons ideal standard can be another's nightmare
• increasingly interwoven in ways that are not always hierarchical
• consequential on the value systems of the community

The measurement is not in the numbers but in the understanding of the numbers. 

—tk

Well, here we are again.  This years RSA show will be interesting given all the changes in the world.

For what it is worth, I’m going to blog as much as I can this week.  Tomorrow, it all begins with Metricon 3.5. This year our host will be Google and the day goes from 8am to 6pm.  Yikes. 

For those of you not familiar with Metricon, it is the product of securitymetrics.org.  While I go to these Metricon events, it is awkward because I’m not on the mailing list.  I have been waiting to get on the securitymetrics mailing list now for 3 years.  I wonder if they still have my subscription request. Oh well.

Tuesday through Friday will be all about RSA mayhem.  If you will be there, stop by the nCircle booth and say hi.

—tk

In Nassim Nicholas Taleb's book "The Black Swan", he explains a type of ingratitude that I think the security professional knows all too well. It goes something like this: Who gets rewarded by society, the person who nearly kills himself trying to avoid a huge problem or the person who corrects a bad situation after it is already in progress? History will show time and time again that it is the latter. He says "Everyone knows that you need more prevention than treatment, but few reward acts of prevention."

The other day, someone asked me "If this DNS Vulnerability was such a big deal, then why did we not see horrible things happen on the Internet?" We as humans find it difficult to value that which we don't know or have not directly experienced. There were many people working their tails off once they were notified of this DNS bug so that the highest level of preventative steps could be taken. I salute those who listened to what Dan had to say and took action.

The administrator that worked over the weekend to remediate an unruly set vulnerabilities will not be rewarded on Monday the same way that he would if problems happened over the weekend and he fixed it before doors opened on Monday. We prioritize our preventative measures on likelihood and impact and that is an entirely different topic for another blog entry.

The same pattern can be seen at the personal level where until you have a bout with death, preventative tasks just don't get the priority they deserve. IMHO, it comes down to an individual being able to experience the bad situation that is to be avoided so that when asked to spend time, energy, or money on the preventative action, the avoidance is self-evident.

If you follow me so far, you would come to a sociological theory of information security that says that in order for your community to understand the value of preventative measures, they must have had to experience that which is trying to be prevented on a personal level. Don't take this like I am trying to make everyone into a communicator of fear, not at all. All I am trying to do is to present the biases that we have as a society so that we can leverage them when it is appropriate to do so and we can avoid them when they get in the way of good decision making.

I'm back from BlackHat 2008 and had a great time. This year, most of the press coverage was on Dan Kaminsky's DNS vulnerability. Dan is smart, clever, and will always go out of his way to recognize other people's good work - gotta love it.

This weakness in DNS has been seen by some as over exaggerated and by other as one of the deadliest the Internet has seen in years. No matter where you stand on the issue, the problem is what this weakness makes feasible and not the weakness itself.

Although the discussion is about DNS, your countermeasures should focus on man-in-the-middle (MiTM) attack scenarios - this is where the game is played. This weakness when exploited makes many MiTM attacks extremely feasible and difficult to detect by the victim at the time of the incident. If the attacker is able to get in the middle of the applications you are using directly (your web browsing, file transfers, etc) or ones that you use indirectly (auto-updating of software packages, automated agents including email MUA/MTA), you better hope there is proper cryptographic methods to protect the data and validate the other-end of the connection. Not only are most applications in bad shape but studies have shown that if you warn a user about this type compromise during their session, they will likely just click-through the warnings because remember, they are busy and need to get their work done. More about this behavior later.

Now before you start blaming the big bad Internet for being so insecure, when did someone say it was ok to start trusting services like DNS anyway? Some of the very first requirements for the Internet was that "the host shall never trust the network, and the network shall never trust the host". The sooner we all stop trusting insecure protocols, the better. I'm not saying stop using them, I'm saying use them but know their limits and be accountable for the risks within your design.

Why do people take shortcuts in their designs, cheat when they don't think they will get caught, and generally pick the "easy" route? Because we are creatures that favor convenience and the Internet and its protocols are dangerously convenient. We like all other living organisms fundamentally are wired to conserve energy. We will always try to find the most efficient path to our goals and in turn do so at some risk. We are quick to understand the benefits of an action but not always quick to evaluate at what future cost.

The Internet and its protocols are dangerously convenient. Can we not design systems that are both convenient and secure? The correct but not so useful answer here is "It Depends". My point in all of this was that these social biases point toward a much more fundamental security issue than any line of code. We must never forget that we are not designing system for arbitrary faults, we must design knowing there is an active opponent out there trying to get at something of ours that has a high utility to them and when they have taken it from us, we still have it.

Call me paranoid, call me what ever you like but if you are going to download software to my system please offer me the chance to review the ingredients before I click OK.  Ultimately, it would be nice to know what I am about to approve don’t you think?

I wonder if I am the only one that feels this way.  Major application and OS’s do a great job at offering this review before a user approves the update but such is not the case in the land of the Xbox 360 game console.  Sure you could argue that console gamer is not going to know a DLL from LSD but nonetheless, offering optional information about what the update is going to do for them is good form.   In Xbox360 land, you get a screen that looks something like this

and it would be great if the X or Y button gave you information on what was about to change on your system.  And while your taking down my feature request wonderful product manager of the xbox360, it would be nice to see the update history of the machine. 

Does the information exist?  Sure it does but you have to really hunt for it and I’m not sure all the updates have made it to the web.  For example, http://blogs.msdn.com/xboxteam/archive/2007/11/30/december-2007-system-update.aspx

http://www.xbox.com/en-US/community/news/2006/1030-novemberupdate-completelist.htm

From a security stand point, it just spooks me out when I approve an update to my system and have no idea what has downloaded or what has been modified.  The number of independent game developers for Xbox360/Xbox-live are taking off and Microsoft has a solid program.  Lets just say that things will start to get very interesting.

—tk

I buy lots of electronics and have been experiencing a trend lately with rebates.  It may be just paranoia on my part but thought I would post this blog entry to see if anyone else is seeing the same pattern.

I bought another LCD monitor and with it was a mail-in rebate for 30.00.  Like all of these, you spend time to gather the required information, sent it in, and after a good 6 weeks time, you get a check.  Done?  Not quite because the “Pay To the Order of” has misspelled my last name.  If this was the first time this happened, it would not be an issue but 3 times in the last 6 months, something seems wrong.

Could it be that there is a strategy out there to raise the cost of accounting on the payee so that they at some point think it is not even worth it to pursue?  I wish we could see the statistics of all the people who go through with the mail-in but because of the run around, end up ultimately not redeeming their rebate. 

This information is not available so all we have to go on are patterns and paranoia.  Is 30 minutes of sitting on hold and filing more paperwork worth $30.00?  At some point, everything come to a cost/benefit decision.

—tk

Anyone who has been going to RSA year after year has seen lots of change.  Changes in the quantity of vendors, changes in the vendor types, changes in the booth personnel, even changes in the swag you get if you sit through a presentation.  I’m so glad we are past that dry spell of just pens and mints, we like t-shirts, USB-drives and remote control helicopter s!  This year was a great show and I’d like to share with you some observations. 

When I first started going to RSA, there were more vendors than there were customers.  It was a huge vendor boondoggle and while the business development people were having a great time, I was looking for customers to speak with and have a great conversation about what they were looking for at the show and what type of problems they were trying to solve. 

This year was great in terms of customers-to-vendor ratio.  We had a great turnout at our booth and I’ve almost lost my voice from non-stop conversations.  What does this change mean for future RSA shows?  I remember one year being at the show and having a customer tell me “You know what TK, this is a show of car parts, and frankly, I need transportation.”.  I’ll never forget this statement and I have a working theory. 

In the early days of the RSA show, the exhibitors sold all kinds of parts that when put together by a skilled craftsmen, created a powerful solution.  Composability was more important than Usability.  As the attendees change to more of a business level buyer persona, consumers that are not security subject matter experts, we move toward deeper solutions where Usability trumps Composability. 

When I hear those words “…this is a show of car parts, and frankly, I need transportation.”, I imagine a trend on the exhibit floor dominated by much more complete solutions.  Product designed for a persona that does not know how to fire up a debugger, does not know how to read a set of ACLs, but knows how to read market results and can use Excel to model any financial system you can imagine.  That might be a little extreme but nonetheless, the customers out number the vendors by a larger and larger margin. 

I predict that RSA next year will have less small highly technical one-trick-pony companies and more multi-product solutions and managed services companies.  To use that great quote, there will be more vendors selling cars and transportation services than there will be vendors selling parts. 

—tk