Two great events in two weeks, Informatica World and Strata. Coincides with two events of our own, the release of our study about the Big Data Payoff and our announcement of the Leap Data Transformation Framework. So a few words on each.
We surveyed 200+ Execs across the US and Europe to understand how successful, or not, their Big Data (Analytics) projects have been so far. I think it’s fair to say that most are ‘still on the journey’ to realising the goals they had when they started. This resonates with me and I’m sure a few others; there has been an awful lot of hype about Big Data but I think, OK hope, we are moving past that. About a third of the respondents suggested they were already seeing value from their Big Data programmes and if you listen to the companies presenting at IW16 and Strata16 you can start to see some proper examples, indeed my team has some great examples themselves that have generated real value for our clients.
What I find interesting about the study are the challenges that the respondents recorded. There are some obvious ones you’d expect, like budget, but for me there are two key challenges that once overcome will, in my opinion, improve the adoption and the success rate of Big Data projects. They are Integration and skills. I know from current experience, that finding the talent on the job market is tough (I have open job spots if you are interested!). Many organisations are competing for the same resources, but these Linux gurus, software engineers, Hadoop & Spark experts etc are a seemingly rare commodity…so it’s no wonder Big Data projects have challenges. What we need is a way to tap into the arguably more abundant skills market of SQL and ETL engineers and the way to do that is to use technology that takes some of the pain associated with big data tech away, such as with an Informatica (…of course not the only fruit). Not only does this have the benefit of giving us access to a wider skill base, it also provides us with integration technology, that my colleague Steve Jones referred to as ‘making integration dull’…and dull is good.
Onto Capgemini’s Leap Data Transformation Framework. We launched this at Strata16 alongside Cloudera. First of all, just want to say how proud I am to have some of my team work on this and get it to where it is today. One of my team, Andrea Capodicasa, had the vision and fortitude to drive this through (perhaps that means he gets the title ‘Father of Leap’?!) and I’m glad he did. Whilst partly borne out of necessity, the framework packages a lot of the creativity we used with one of our largest customers in moving them to a modern data architecture. It encompasses discovery and estimating tools to figure out how much of your legacy estate is needed, it includes probabilistic matching modules to figure out overlap to suggest target optimisation and it even provides a testing framework to speed up the process of migration. Why not read more about this Cloudera Certified solution from Andrea himself here.