In my first article, I outlined the challenge of semantics and interpretation of information presented to managers in 3 different situations. In this article, I will give examples of how information quality differs between various industries because of the challenge of agreeing the semantics of information.
However, before I start, I must mention a very interesting book called “The Black Swan” by Nassim Nicholas Taleb, an ex-quant trader who writes on how “Black Swan” events always take us by surprise. His thesis is that, as individuals, and as a society, we forget how that it is the random, unforeseen evens that shape our world. He gives many examples, but the 9/11 attacks in New York are a very good one where, post the event, we rationalise that we expected the event and that given access to a little bit more information we would have been able to prevent it. This is relevant to my work because in business, the information to make decisions is typically there, if only we could find it.
My points are that:
- With better knowledge, not of the data and information we have, but of what the data actually means, we would all be making better decisions;
- To achieve this better knowledge, the interacting parties need to have similar (or balanced) desires in order to want to arrive at a semantic understanding of their shared data; and
- This balanced need does not always happen in mature markets but having said that, certain events push mature markets into wanting to achieve that balance. Where this balance is missing, the ability to understand the meaning of information will remain challenging.
I would like to offer further details in the situations I described in my first article and show that if there is a balance in the need to share and understand information, then the next step is to agree the format and the semantics of the information to be shared.
In my first article, I suggested, in a homogenous retail environment, the wrinkles of information management had been smoothed out over many years to a level where errors were significantly less than in my other examples. The provenance of information was easy to establish, so the information could be trusted.
How come the retail example seems so much better than the other two? Is it due to the maturity of the retail market compared to the public sector? Surely the need for control in government has been around much longer than the level of control I suggest. Is the rate of change in retail less than in government?
Let’s look back at the history of Information Management in retail. The stakeholder map for retail is typically very complex; retailers of all sizes (supermarket chains to corner shops) all have to take goods from a plethora of suppliers. The larger retailers have driven the need for easy ways to talk up and down the chain - standards were essential and, more importantly, there was a will to standardise.
All parts of the retail chain understood the value of reaching agreement to the basic concepts of retailing such as EAN (Barcodes), sell by dates, invoicing terms etc. Standards, such as ARTS, go further but have not had, perhaps, the same impact as the basic things.
In financial services, the same pressure to standardise is in everyone’s interests up and down the value chain. SWIFT, and the FIX standard (and its successor FIXML) were created to avoid the chaos of manual trading and manual reconciliation. It would not be possible for the global markets to work at the speed they do if these standards were not in place.
Both the retailing and financial services worlds rely on all parties having a common understanding of what the information means so that a number in a FIXML trade is known to be a buy not a sell. You may say this is obvious, and yes it is, but it shows why getting to this level of understanding is beneficial. If only other industries could say the same.
So, if retail and financial services are in such good shape, how do other places fare?
In my first article, I alluded to the challenge of Information Management in local government. The example I gave was central government asking local government for information on school absenteeism where families move between councils/schools and when figures are demanded per term and the information is collected per annum.
The parallel (or, more precisely, divergence) to my last example is based on need. In retail or financial services, each party has a need to standardise. The better the understanding of the environment, the smoother things work. I believe local government does not have a balanced need when compared to central government, so neither party will be able to agree how to communicate effectively.
This links back to the “Black Swans” book I referred to at the start of this article which referred to unusual events that are so far out of the ordinary, they take us by surprise. The Victoria Climbie case is a good example of a “Black Swan” event; we had all the information we needed except not in a form that could be easily understood.
The link back to my main point is that, after the event central government initiates a change that affects all local governments. Examples of this are i) incidents (such as Climbie or Soham); ii) Policy (school performance) iii) fiscal (reducing social security fraud) which typically require local government to perform a new task and report on in centrally. Therefore, the new task has not been standardised and the variation in the data/standards are compounded (e.g. the double counting in my first blog entry) to a point where the error rate starts to reduce our confidence in the data.
The question here is: Is the need balanced between central and local government as it is between banks in a financial transaction. Local government knows the demands from central government are likely to be targeted at the short term and another Black Swan is likely to change the focus and create new and different demands. As a local government manager, why waste effort on agreeing standards when the targets are always moving. Somehow, the needs of both sides of the transaction need to be equalised to make it work. Government departments have tried for many years to create this balance by trying to be clear what they want from local government. However, the resulting standards are typically so bland as to be near worthless. These standards indicate, for example local government should collect details of school absenteeism without saying whether it should be collected monthly, termly or annually and often don’t agree what it will be used for (because ultimately they may not know). Getting agreement to this is very difficult because stakeholders are likely to resist costly change.
Criminal justice and in particular Policing, where I am currently working, has its own challenges which are best illustrated using an example.
The murder of two young girls in Soham was a Black Swan event. Unforeseen, except we had, at one time, all the information to stop it. A number of police forces had information about Ian Huntley and yet there were no robust processes and standards in place to maintain and share that information.
The Police Service has a structure based on forces being autonomous units with, as one would expect, strong reasons to focus on their own patch. Managing their own information, and sharing it with other forces was seen, until Soham, as a back room activity done by the IT department.
Linking back to the rest of this article, what was happening was a demonstration of unbalanced need whereby the current operational challenges were taking primacy over managing what should be the lifeblood of the police service, information.
The Soham “Black Swan” event highlighted the importance of balancing the need and since then the Police Service has realised the benefit of this. This is as simple as the budget holders realising the benefit of spending money on better information management to avoid future Black Swans (and to avoid being the one who didn’t put their house in order) which is what the information management staff have been crying out for forever. Various initiatives are happening at force level to standardise how to manage information about criminal and suspects. At a National level, large programmes of work to share force information more effectively are underway.
Once this need had been balanced, and following the retail and financial examples, the police service has realised that standards for information are essential. It is not possible to have 40 different ways of defining the meaning of a date.
To this end, the police service have developed standards, similar to those in retail and financial services which are currently being rolled out to help with that challenge of information sharing.
I believe that the retail and financial services markets, even though they have their challenges, have shown that, with a balance need between parties, having agreed ways to define information is a good thing. The balanced need reduces the possibility of Black Swan events. However, with the credit crunch, perhaps the financial sector wishes it had fully understood the risk of underlying instruments.... another example of poorly understood data!!
Local Government seems a harder nut to crack. We do not have the ability to second guess the next Black Swan and its subsequent demand from central government. This indicates that having standards to share information will be very difficult to achieve, so to work in local government information management, perhaps means focusing on managing the imbalance rather than trying to create the balance.
In the police service, the balancing of the need is beginning to happen and the subsequent thoughts on how to agree what and how much to share information has started. There must be opportunities to achieve the same in the wider criminal justice arena.
So, in summary, I believe we will only achieve greater confidence in our information when we have balanced the need to share and use the information and, once we have achieved this, to agree the means by which we understand what we are sharing.
My subsequent articles will dwell on how we need to capitalise on a balanced need, and turn that in to a real deliverable by building and agreeing the means to share information.