Hmm, rereading the last entry, I realize that, before I go on, I need to put on the table some of my assumptions about what constitutes effective "continuous monitoring."
While CyberScope is a clear step in the right direction, in my opinion, there's a long journey ahead. Scanning your environment 12 times a year instead of a 'couple' times, and uploading the information to DHS instead of dumping it into a report, is not going to create the fundamental changes in organizational risk posture that have been documented at places like the State Department, Medicare, or USAID. Sure, a couple of agencies might get more heat from OMB if their numbers look bad, but how is that different from getting a bad grade on FISMA in the past and what are the consequences?
In the meantime, I can only emphasize that just measuring with greater frequency isn't enough: It doesn't address the foundational challenge of risk reduction which is that, in order to be effective, you need the active cooperation of system owners who have very demanding IT jobs and for whom IT security is an afterthought, on a good day. Everyone who touches the network, and their managers up the chain, needs to be invested in securing the organization. How do you accomplish that? Perhaps John Gilligan (former CIO of Air Force) describes the solution best in a recent NextGov.com article, "[by establishing] a means of tying together the machinery for a holistic view of security status department-wide."
Using technology as a means of providing a holistic view department-wide for all IT stakeholders is key. USAID, State Department, and the Center for Medicare/Medicaid Services (CMS) each got there by opening up the results of their frequent scans to the broader community, along with a letter grading approach. Immediate, accurate, intuitive feedback (who doesn't understand letter grades?) coupled with all the detail needed to fix the worst problems has proven to be very powerful, so powerful that Microsoft, GE, J.P.Morgan Chase, RSA and the like are exploring successful models. Ambassadors (at State) or mission directors (at USAID) were not historically concerned with IT security. However, when these agency CIOs starting issuing monthly letter grades down at the level of the embassies and missions, these high level executives took a sudden interest in their performance.
It's also critical that what stakeholders see is actionable. All three of these organizations (State, USAID and CMS) provide the data in formats that make it intuitively obvious - even for security newbies - to understand exactly which actions would have the most benefit for their component of the organization. It's whatever action would improve their score the most. System owners and their managers can see - at the same time security teams and auditors do - exactly what risks need to be addressed in each of their environments. AND, each environment is graded, using highly intuitive scoring systems that help system owners know exactly which weaknesses are having the worst impact on their grades. As John Streufert (CISO at State Department) will tell you, the heart of the system is the scoring: carefully designed to focus effort and attention on the worst problems. (More details on his approach in the Wall Street Journal this week).
USAID, State Department, and CMS have all experienced dramatic reductions in overall risk. Compare their approach to the CyberScope methodology of flowing a monthly collection of aggregate numbers from the Agency CIO's office into an online portal. In this scenario, perhaps the most serious offenders will be identified and instructed to improve, but the feedback loop is still very long, not very directive, and not yet engaging enough of the right people.
Next week I'll get back to the ADDITIONAL benefits of continuous monitoring.
