BI & testing; Why you need to stop looking for an experienced BI test analyst

In this blog I’m going to explain why you need to stop looking for an experienced Business Intelligence (BI) test analyst.

Why do I bring this up?  

In the last couple of years I’ve browsed through many job descriptions posted on the internet. What caught my attention was that the BI test analyst often needs to have a good knowledge of BI, BI tooling, plus 5 years of work experience or more.  Additionally the BI test analyst also needs to have good social and communication skills.

Whilst reading this, it seems to me that organisations are looking for a jack (or Jill) of all trades; somebody who can do everything. 

So what does this mean? Here’s an example of what a BI Test analyst needs to (be able) to do:  

  1. Testing the ETL, Data warehouse, Data mart and reports / cubes
  2. Performance testing
  3. Automate testing
  4. Set up and maintain test environment & test data
  5. User acceptance testing
  6. Coach / lead a test team
  7. Coordinating testing

Personally, I think you should alternate your hiring strategy in finding the right person for the right job.  

Let me explain:

In my career as a BI developer, Test coordinator and test analyst, I’ve often had to perform a quick analysis of the BI product and then provide a fitting test approach on “how to  test a data warehouse or data mart solution”

In order to do so, I always use the same strategy; break down the complexity of the BI product by asking plenty of questions and then, once I’ve got a good grasp of the BI architecture, how it’s designed and build, set up a test approach and test it.   

For me the success of designing, building, testing and maintaining a data warehouse or data mart requires a team effort; it’s the quality of the team as a whole which determines the quality of a BI product. Additionally a test approach determines what (not) to test and by whom. Again, input for this test approach is provided by the whole project team and not solely by the one BI test analyst whose name is Jack or Jill. 

So, supposedly you want to alter your strategy, where to start?

Firstly, setup a test approach which will help you realise the goal of shorter delivery times and improving the quality of the BI products. This approach focuses on the most essential checks: quality, completeness, correctness, plausibility, robustness and performance. These checks can either cover the complete data warehouse chain or start off with the testing of ETL and then the rest of the chain.

Secondly, start to automate testing, by:  

  1. Standardise testing of the BI product; discuss and share the knowledge “what / why / how / what’s required”. Standardising testing will enable you to delegate test activities within the project team
  2. Build up an regression test set
  3. Set up a Development, Acceptance, Test and Production street with uniform source (sub-) datasets. This is a “must do” because this will shorten your iteration test cycles and will reaffirm test results throughout the chain.  
  4. Automate the running of your regression test set in a separate test environment.

Thirdly, lower the bar and start looking for technical talents in the following order:

  1. A test automation specialist who can automate testing
  2. (If needed) a test analyst who can manage test environments and data
  3.  A BI test analyst who supports user acceptance testing and performs manual tests to look for data quality issues and anomalies.

Make sure they’ve got the following basic knowledge and/or competencies to learn on the job and become an expert in their field. Here’s my list of most essential knowledge / skills to ask for:

– Programming: SQL (must) plus affinity for programming,   

– (Gain) Knowledge of specific ETL / Data warehouse Test automation tooling

– BI Basics: Data warehousing Concepts & Design and architecture.    

– Database design: Entity Relationship Modelling, Data modelling,

And do keep in mind:

Data warehouse tooling provided by specific vendor can be mastered by training on the job. Make sure the tester or test team has full support and commitment from the project team and stakeholders.

Doesn’t this cost a lot of time and money? 

Yes, it’s a true fact that you need to invest time and money in the initial test & automation phase.

But once this job has been completed you will reap the benefits; it’ll reduce time and effort needed to complete regression tests plus the quality of your BI product & information is guaranteed. However, automation of testing doesn’t mean that you can skip the manual testing; exploratory testing is still needed for functional testing and to make sure you find data anomalies & data quality issues.

So what’s your hiring strategy? Are you ready to rethink your strategy?