FISMA Security Metrics and Gaming the System

Sometimes it can be mind boggling why some agencies are getting A's and some are getting F's for their annual FISMA score. Last April, GCN questioned the validity of many FISMA scores. In a recent post, Joel on Software discusses how trivial it is to game metrics in knowledge organizations. What we clearly need are better ways to measure improvements in a way that makes gaming the system more difficult. Unfortunately, the FISMA scores are more likely to reflect management's attention to improving FISMA scores rather than improve security. For example, completing certification and accreditations improves one's FISMA score regardless how many findings turned up if the DAA accepts the risks and grants an ATO. In fact, the more findings there are, the more an agency can appear to be doing in its quarterly POA&M reporting. When your network is swiss cheese, it's a lot easier to claim that you've plugged holes. Why should agencies get credit for having lots of low hanging fruit?