In the blog “BI &” I’m discussing Business Intelligence (BI) and adjoining topics and trends.

In this blog I use the car industry as an example in order to point out why testing of BI is crucial for your business. 

Imagine you are the CIO of a renown car manufacturer and you are about to launch a new car model on the market. Based on reports you received in the last couple of weeks, you are fully confident this car is going to be a top seller;

  • The design and built of the car is top notch since you are familiar with the car designers and specialists working for you

  •  The company has an impeccable product line in place which guarantees a delivery time of 10 weeks

  • Market research confirms there’s a huge demand for this car

So far so good.

This morning, however, you’ve had a meeting with your stakeholders and they expect this car to reconfirm the customers brand perception as a leader in safety, quality and reliability.

Furthermore, they stated that it’s absolutely essential to maintain these standards, as they guarantee substantial revenue over the next couple of years.

No pressure! But how do you live up to the expectations of your stakeholders and customers?

You know last year things did not work out quite well for Tesla (which you secretly admire for its innovative and cool design); this car was recalled by the end of November 2015 due to safety reasons.

Instead you rather follow in the footsteps of Volvo, which states in its vision that by 2020 no one should be killed or seriously injured in a new Volvo car.

So, in order to ensure that this new model meets the required standards, you need to be sure that each car successfully completes various assembly line checks plus its final road test.  

Some doubts are creeping into your mind:  

How can you be sure the quality of the car remains intact throughout the development and assembly process?

What if, for example, the car’s design had to be tweaked a little due to product issues in the build phase?

What guarantee do you have that the redesigned and newly build component will integrate perfectly with the car?

What gives you the guarantee that after redesigning the car, it still meets the set standards?

In search for an answer you meet up with an engineer who explains the design and development process of the most complex parts of the car; the software, hardware and mechanical parts. He also mentioned that in the last year, the project team had set several goals in order to obtain better quality and shorten delivery times of the newly built components:

  1. Decrease total developing and manufacturing time.
  2. Ascertain continuous and on time delivery of quality products by various internal and external parties.
  3. Assure tests are performed in an efficient and effective way.

In order to achieve these goals, the project team has implemented the following critical success factors in the software development process:

  • Agile methods have been introduced in various parts of the manufacturing process e.g. during the requirements gathering, design- and product development phase and the testing of functionality such as driver assistance, vehicle dynamics control (1). This allows to design, build and test the software and hardware of each new component in short iterative cycles. Component and integration testing confirm each part is compatible with the prototype of the latest model of the car and consistency checks will guarantee that the functionality of the prototype as a whole remains intact.

  • An agile way of working has improved teamwork on team level and on project level; this is important as communication and work relations are two crucial  motivators for timely delivery of high quality output and building trust within the project team.

  •  Standardizing the testing helped introducing an incremental test approach. This makes the test process transparent and comprehensible and guarantees that the most important tests are always performed. Standardizing testing also enables swift knowledge transfer within the team, which makes it easier to delegate test activities within the team or to introduce new team members.

  • The Automation of unit testing and especially regression testing has speeded up the software development process and has made continuous releasing of prototypes feasible.

Summirized: standardizing and automating testing have reduced the chances of finding software issues in the latter part of the manufacturing process, plus saved time and development costs.  

The engineer confirms that although implementation of this agile process did require extra investment, time and effort, it surely has paid off in many ways. Test results obtained from various stages of the manufacturing process have confirmed safety, quality and reliability standards have been met. The biggest result, however, was the fact that the overall time to market has improved significantly.   

 6 months later the car is launched and it has been a huge success. Sales figures have exceeded expectations and you have no doubt this is one of the best cars produced in the history of the company.

Driving to work you receive a phone call with the message that the stakeholders have decided to recall the car because of a major design flaw. It seemed that due  to the fact that the start/stop and radio button where located in the same area, drivers were able to accidently turn the car engine off instead of operating the radio.

A research showed out that one of the main reasons was lack of usability testing. Furthermore this also proved that the project team was as strong as its weakest link; the absence of customer feedback meant they weren’t able to recognise poor product design.

What should have been a huge succes has turned into a failure which most certainly has damaged the brand image of the company.

Realising you have to take full responsibility for this failure you decide to call it quits

In this blog I used a car as an example; when you replace the words car and component for e.g. a data warehouse solution or data-centric application, you get a clear picture of why testing your Business Intelligence (BI) Solution is directly related to trust and brand image.

The reason why I wrote the fictional story above is to address one the most important issues in BI Projects; how to get and keep the business involved throughout a project.

For many reasons business fail to grasp why this is crucial or aren’t willing or capable to do so. Failing to do so means a company is willing to spend a few hundred thousands of euros on a BI Solution that upon delivery might not meet the business expectations.

In my opinion this outcome can be averted. A solid business requirement definition phase combined with effective test approach and Test Framework in place, will enable the business to start working in close collaborate with the BI (test) team.  


Documentation studied by me / references:

1. Master of Science Thesis in Software Engineering and Management Bharat Bhushan Konka

2. Extending Agile Practices in Automotive MDE, Ulf Eliasson and Håkan Burden


Thanks for your support: Jorgen Heizenberg, Jurjan Koenen, Remco Swart, Daniel Ghebrial