NetSec Letter #33, 23 February 2004
Measuring Security

Fred Avolio, Avolio Consulting, Inc., http://www.avolio.com/

When it comes to security, many of us find ourselves in the situation of having to ask the question, "How are we doing?" Sometimes we ask because it is our job to ask. Sometimes it is in preparation for asking permission to purchase a new security device. Too often we measure how we're doing in this way.

  1. Start with a well-thought-out standard. Well, a standard anyway. It is our security policy, perhaps, or our security architecture. For example, "get a firewall and configure it this way and that."
  2. Then we backslide. It is never part of our plan. (At least I hope not ... it is a strange organization indeed whose plan says "Two months into this, screw up badly.") We change our configuration; we add connections or procedures without running them through the security process; etc.
  3. We evaluate what we've done by realizing, "Well, we've not gotten broken into, so I guess everything is fine."

What is wrong with that analysis? It is the same problem as that which was hinted at years ago, by my friend and security-meister Bill Cheswick. He would mention near the beginning of a talk, "We've never had any undetected security breaches." It sounds right until it seeps into your brain that undetected security breaches are by definition UNDETECTED. We need a metric ... or at least a different one.

Some think it is not possible to measure things like security or risk. Or, if it is possible it is really, really difficult. They forget that in the security realm we deal in gray areas, not absolutes. We live with educated compromises. As I tell students we are not just securing, rather we are securing something. Security is easy. Allowing us to carry on business in a secure fashion is the challenge.

As I was thinking about this, two things came to mind. A few years ago, Dr. Peter Tippett of TruSecure Corporation (disclosure: I consult for them) formulated a risk equation,

Risk = Threat x Vulnerability x EventCost.

He also discussed the power of combining synergistic security controls. (Check out the paper "Keep it Simple" at ugly URL http://www.trusecure.com/cgi-bin/download.cgi?ESCD=W0073&file=doc611.pdf.) We can measure risk. We can measure effectiveness of controls. It just takes thought and a bit of analysis.

The second thing that came to mind was a discussion I had with Adam Joseph of Sequation (http://www.sequation.com/), who is building a business on getting an answer to the question "How are we doing?" Sequation (I don't work for them) provides a report card for enterprise networks taking a multidimensional approach to analyzing different metrics from different security controls. Quoting a Washington Post article from December 1, 2003, "Sequation's software does not protect computer networks, but rather analyzes data on the effectiveness of existing security applications, including firewalls, intrusion detection systems and anti-virus software. That information is then packaged and reported to personnel who watch over the system."

Is it possible to do? Sure it is. Is it very, very difficult? Not usually. It does take thought and planning and a handful of bright people periodically addressing the problem. Will you get it perfect? Never. But. we're not after perfect or even the best. As Voltaire said (according to the Internet, which is never wrong), "The best is the enemy of the good." And as I said (or was it Tippett?), "Good enough is often very good."

It is always better than the metric, "Well, we've not gotten broken into, so I guess everything is fine."

Promotions, Self and Otherwise

I want to remind you of my weblog, to which I post more frequently than I post this newsletter. I also cover more topic areas (e-mail, security, theology, and the catch-all "misc"). http://www.avolio.com/weblog/ gets you to the main page, http://www.avolio.com/weblog/security/ gets you to all security items, etc. If you have an RSS reader, plug in http://www.avolio.com/weblog/index.rss to get the whole site and http://www.avolio.com/weblog/security/index.rss for just security. An index of recent security postings ( http://www.avolio.com/weblog/security/index.index):

Secure Security Products? Quick -- What was the first commercial firewall product with an announced serious (as in, one could "get root") security vulnerability...

Secure Coding? Of Course. Andy Briney, in his February Information Security Magazine column, called "Secure Coding...

Getting Rid of the Last Click for Secure E-mail It is well past the "live" date, but through the magic of electronic media and the Internet, you can catch Jon Callas' webcast on "The Dawn of Pervasive Encryption" at http://www...

Save your sanity -- Backup that PC! As computer disks have gotten larger, we, their users, store more and more data on them...

Worse than the Real World My good friend and some-time colleague, Kevin Shivers works in information security on the front lines...

The Dilution of Truth on the Internet "Not only is all human knowledge on USENET, it's typed in every two weeks...

What do we think firewalls do? (Fred Rants) Do firewalls just filter on IP packet header information...

The Institute for Applied Network Security I spent an interesting and unique 2 days this week with some fascinating people in the computer security field...

Last year I wrote two columns on the topic of "The Microsoft Factor" ( http://www.avolio.com/columns/29-TheMicrosoftFactor.html, http://www.avolio.com/columns/30-TheMicrosoftFactor2.html.) I also blogged about experiences with "A Linux Desktop" (http://www.avolio.com/weblog/misc/linuxdesktop.html). Someone pointed to another person's experience with debunking his preconceived notions about both UNIX and Windows. See http://lists.netsys.com/pipermail/full-disclosure/2004-January/016063.html