In a data-driven organization, everybody needs to be a bit of data scientist and data engineer. The best insights are created in close proximity to the business and to do that that, data must be discovered, prepared, analyzed, and visualized right there. But real skills are rare; and secure, high-quality access to the right data is far from a given. AI and automation fuel a new category of easy-to-use, augmenting tools that provide high productivity to a much wider range of people. It offloads the pressure on central delivery while democratizing access to data and algorithms. Data for all, right on.
What
- Within any Technology Business, insights need to be adapted in almost real-time with a “DatOps” delivery approach – the DevOps-inspired, integrated way of continuously providing data-driven solutions to the business.
- This needs to be driven by an automated, AI-augmented, factory-style data pipeline to ingest, select, transform and prepare data – making the right data available with the minimum of specialized (and increasingly scarce) human intervention.
- Creating insights from data – whether as business intelligence, analytics, algorithms or AI – also benefits from automation and AI-augmentation, putting data-driven power in the hands of more users, even if they lack the deeply specialized skills typically needed. Conversational interfaces may be used to further ease the process.
- Automated Machine Learning (“AutoML”) allows non-experts to make use of machine learning models and techniques without being an expert in the field, with some providers even claiming to provide “driverless AI”. The most promising insights that are therefore created in the work field, may then align with data science and engineering experts for validation and scaling.
Use
- A consumer products company created “data science on demand”, enabling the business to work with data experts on specific challenges to rapidly have the first proof of solutions – then production versions – reaping early business benefits.
- A financial institution turned its data assimilation into a highly industrialized, automated, managed service, making new data and insights available in a matter of days rather than months.
- A US airline supplied business users with intuitive self-service data tools, creating much more data exploration, innovation, and a true “self-service movement”.
- A bank’s marketing department identified a surprising wealth management segment using AutoML, whilst their businesspeople built algorithmic models that reduced loan defaults in microfinance by 5%.
Impact
- Cost effective production of BI and analytics results, reducing manual effort and increasing quality.
- Speedier availability of new insights for the business.
- Better access from the business to more relevant data from various internal and external sources.
- Increasing cultural and practical awareness on the business side of the potential for turning data into insights, algorithms and AI.
- A true fusion of the business and IT sides – crucial for a Technology Business.
- Automation and AI-augmentation also frees up time for specialized data scientists and data engineers to work more on their actual models and business results.
Tech
- Continuous, agile delivery: Jenkins, Bamboo, Git, Subversion, Puppet
- Data pipeline technologies: Alteryx data science and analytics platform, Informatica Data Engineering, Talend Data Fabric, AWS Data Pipe Line, Trifacta Data Wrangling, Microsoft Azure Synapse Analytics
- Self-service BI, analytics and AI tools: Tableau, IBM Cognos Analytics, Microsoft Power BI, Qlik QlikView, SAS Visual Analytics, Dataiku Collaborative analytics platform, Saagie collaborative DataOps
- AutoML: AutoML on DataBricks, DataRobot AutoML, Google Cloud AutoML, BigML, H2O driverless AI, Microsoft AutoML research