Information security has long suffered a poor reputation for being an expensive overhead with no real business benefit.   Of course, we security practitioners are acutely aware of the value of the solutions that they recommend and implement (aren’t we?).   We know that we are protecting the critical information assets of our business and enabling decision makers to venture safely into new ways of working.   But the question remains – how can we demonstrate the true value of our preferred information security controls to business stakeholders?   Conversely, from the perspective of the business stakeholders, how can the effectiveness of the security functions be measured and monitored to enable more informed management and better targeted investment?   This blog talks a little about the effective use of security metrics to aid both security practitioners and the businesses that they protect to judge the performance of their security solutions.

The first step towards implementing a successful suite of security metrics is to understand the nature of security metrics.  The Oxford Dictionary of English defines the word “metrics” to mean:

 “a set of figures or statistics that measure results.”  

We can therefore view security metrics as referring to:

“a set of figures or statistics that measure the effectiveness of a security capability or control.”

These definitions show that there is a difference between a measurement and a metric.   A metric usually consists of a combination (or series) of measurements which together provide a discrete status of the behaviour or control being monitored.   These metrics can then be monitored over a period of time to show whether performance is improving, deteriorating or remaining stable over time.

A well-designed security metrics programme offers a number of benefits, including:

  • providing senior management with visibility of the outcomes from existing investment in security; justifying further investment where necessary
  • providing advance notice of required changes to security controls via trends in security metrics that indicate negative impacts on the business
  • enabling cross-charging for security services via meaningful Operating Level Agreements (OLAs)
  • providing worthwhile accurate input to compliance and audit exercises
  • identifying unnecessary security controls that either hamper business effectiveness or cost too much for the protection that they offer
  • monitoring defined risks to the business – particularly relevant to supporting innovative new ways of working, such as working ‘in the cloud’.

Metrics programmes must be designed with specific stakeholders, goals and budgets in mind if the benefits described above are to be realised.  The audience for the numbers of scans detected by the firewalls at an organisation is likely to be limited.  The audience for whether the organisation’s losses due to payment card fraud are increasing or decreasing is likely to be substantially larger and certainly more influential. 

Organisations evolve; new opportunities arise, acquisitions and divestments happen, strategies change.  Metrics designed to cater for the business needs of 18 months ago may no longer be appropriate today or tomorrow.   Metrics must be maintained to ensure that the metrics solution is able to evolve alongside the organisation.

Now, implementation of security metrics is not without risk. One issue that can adversely affect a metrics programme is that stubborn cause of many security misfortunes – human beings.  For example, how would an organisation make use of a metric measuring the number of reported security breaches in specific business areas? Should a manager be rewarded for reporting breaches (and perhaps inadvertently encourage more breaches)?  Or should the manager be punished for suffering the breaches in the first place? In the latter case, the individual will almost certainly become more reluctant to report security issues – definitely not a desired outcome.  Beware the law of unintended consequences!

Another potential pitfall is the production and dissemination of metrics information that is of no relevance to the organisation; it can be very tempting to collect information simply because it can be collected rather than because it should be collected.  This is a symptom of a wider problem of failing to scope the metrics programme adequately but the consequences can be catastrophic in terms of:

  • loss of stakeholder engagement
  • losing sight of the valuable information amongst the noise
  • increased cost and performance impact

I don’t come across many effective security metrics programmes.   I see a lot of SIEM implementations, a lot of standard reports being delivered (if not read) and some very fancy (patronising?) dashboards.      The reasoning behind the lack of effective programmes can be attributed to a number of factors, e.g. they can be seen as an unnecessary expense, organisations may be engaging only in compliance-style security rather than risk-managed security or, and it pains me to say this, we may be more interested in playing with the shiny shiny security technologies rather than in judging their effectiveness.

However, I believe that a well-designed, business-focussed and tightly scoped security metrics programme can be an invaluable mechanism for fostering better understanding of the value of the security controls in place at an organisation.  This enables our clients to identify those controls which either offer real business value (e.g. fraud prevention) or operational excellence (e.g. spam filters) and to distinguish these controls from those which are failing or redundant.  You can start to make a business case for the metrics programme through the retirement of tools or processes that demonstrably no longer add any value to the business.

Furthermore, metrics can be used as a management tool to incentivise both the security function, and the business as a whole through performance management processes, to improve the security regime within the organisation.

There is little alternative to the use of security metrics for those organisations that wish to manage their security risks in an objective, empirical manner.   Without visibility of the effectiveness of their controls they are forced to default to blind trust in the capabilities of their staff and implemented security products.  Some organisations will be fortunate enough to maintain highly competent security functions, highly disciplined staff and top notch security products.   Do you know if your organisation is so fortunate?