Lean IT: Data driven steering of IT Organizations

In new (agile) as well as hybrid (bimodal) IT Organizations we’re facing a lot of common misconceptions about organizational steering: “We are agile, we don’t need steering metrics” is just one of them, but by far the most common.

Especially agile collaboration methods – such as ‘scrum’ – are designed to measure constantly the performance of a team in order to be able to adjust the team structure or the workload if the performance changes unfavorably. Other methods such as ‘lean startup’ go one step further – they critically measure each facet of the target product, if the performance is not following an initial hypothesis, the product will be altered or pivoted.

“So why should we work less accurate in organizational steering than at the base where the actual work is carried out?”

In short: There is no rational reason.

With the broad emergence and availability of easy to implement AI technologies, comprehensive data collections on all our activities in work and private life, it was never easier than today to implement intelligent and data driven systems. The main problem is the image – KPI systems facing negative connotations for managers and employees as they intent to ‘measure everything, stating nothing’. Examining the KPI systems implemented in IT companies in the 90s or later showing strong focus on the measuring process itself or input factors rather than on organizational output factors. Further, everything identified as measurable was highlighted as ‘key performance indicator’, without any judgement on the statistical significance or significance in terms of steering impact on the organization.

Today, we don’t need 50+ key performance indicators. We need a few, telling us where we need to adjust the organization and how to adjust it – to modify the organizational output. IT organizations are today highly leveraged through external suppliers, various levels of on- and offshore sub-suppliers, non-comparable levels  of system customizing and business-adjusted individual system landscapes. Focusing on popular, allegedly comparable, input factors such as total IT costs, total headcount split into service, development etc., gives zero indication on the effectiveness and the efficiency of the IT Organization, and results mainly in perplexity about the next actions. Just imagine, your Java developer costs 20% more than the ‘industry benchmark’ – but is the benchmark developer also a full-stack developer and saves you 6-digits in license costs through efficient use of cloud infrastructure?

“Ok, same product, different packaging?”

In short: Same ingredients, different concoct, superior outcome.

In output oriented metrics KPIs are mainly not as easy comparable as input KPIs, as they are individually fitted to the organization with the aim to increase the efficiency and steerability of the analyzed organizations.

THE DATA DRIVEN STEERING COOKBOOK

Step  one – determination of the main functional, personal and technical issues driving the organization.

Therefore, we search for high-level assumptions such as “We have a problem with the time-to-market of our projects”, then identify the relevant factors for the time-to-market, such as labor availability, technological expertise, infrastructure provisioning etc.  We cluster the main assumptions in form of measurable metrics, such as ‘internal resource utilization’, ‘external resource utilization’
, and split them down if needed on factor level – e.g. ‘sickness absence’ or ‘employee fluctuation’. Statistical models especially in time-series modelling are then applied at the existing data of factors to identify correlations, statistical significance and if needed other statistical metrics. One outcome might be that an increase of absence leaves leads with a time-shift of 3-6 months to an increase in employee fluctuation. This is usually followed by projects running completely out of budget and another 3-6 months later by a rapidly declining customer satisfaction.

Step two – implement intelligence.

As we see in the example, the input factors are tangled closely to each other – the complication is to build a model for a small set of key metrics considering the input factors to measure the organizational output and generate indications for management interventions. Conducting a dependency analysis on >50 factors let us identify usually around 10-25 factors per organization which are crucial for the efficiency and output of the organization. As an organization has several, often contradicting, goals, we categorize the key factors according to their relevance for the achievement of an individual goal.

Per each goal, we using these factors to build up forecasting metrics based on the gathered data and the past correlations, we can use statistical models (e.g. ordinary least squares,  ARIMA, etc.), or artificial intelligence based models such as k-nearest neighbor, random forest, etc. – depending on the set of data. Additional data pre-processing steps might be needed in order to reach the expected level of data consistency.

Step three – set up the data driven management cockpit.

After the definition of the measuring metrics and the calculation of the as-is KPIs and its forecast, the gathered information should be displayed to the management. We recommend building a management dashboard with 5-10 high level KPIs and a drill-down possibility wherever necessary. An economical use of drill-downs is recommended. The recipient on CxO-level should see immediately the status and where his action is required – we suggest an appropriate use of graphics with clear color-based indications.

The underlying technology stack is depending on the IT infrastructure and the existence of in-house tools and competencies. Out of our previous experience we can recommend Python (with Pandas, tensorflow, scikit-learn, etc.)  as open source software for the data science and predictive part, full data pipelines from processing to proceeding with large amount of data could be implemented with Knime. The deployment is recommended as containers in a cloud environment. Due to the easy configuration and implementation, we use usually Microsoft PowerBI as out-of-the-box frontend solution.

Organizational and customers benefit

The major benefit is not only to understand the cause-effect relationship of several organizational factors, but also the gain in advanced steering competence of the organizations CxO-level. Data driven steering enables:

• the CIO to allocate more resources in software systems before they crash – based on the prediction of incidents and predictions on the usage
• the CFO to adjust the financial planning before projects are running completely out of budget
• the board to prevent fluctuation

All displayed factors lead to a higher and more professional level of customer-centric service provisioning through the IT organization.