Moving Big Data delivery from the West Coast to the East Coast—part 1

Publish date:

Over the past few years we have started to see a paradigm shift in the capabilities that enterprises are using to deliver Big Data projects.

If we go back in the recent past to 2014, we would see very highly skilled software engineers deliver data pipelines using a variety of low-level tools.

As these projects have matured a number of other open source tools (I have used the term West Coast for this) have appeared onto the market in order to bring some enterprise stability and management over the data pipelines. These would cater for relatively mature enterprise capabilities like data pipeline execution, monitoring and reruns.

Over the past 30 years there has been a stable enterprise capability (I like the term East Coast for this) in this area with products such as BMC Control-M and IBM Tivoli. What is not well known is that these enterprise products also provide Big Data pipeline execution, monitoring, and rerunning capabilities.

Utilizing enterprise capabilities that companies already have in their application stack will reduce the reliance on new open source products or home-grown capabilities, increasing reuse of current license agreements and also de-risking Big Data projects.

Related Posts

big data

Organizations need to give unstructured data its rightful place if they want to get value out of data

Paul van der Linden
Date icon August 28, 2018

How can organizations derive more value out of data by reorganizing the unstructured data?

big data

BigTech firms take measured steps toward the insurance sector

Shane Cassidy
Date icon August 23, 2018

BigTech firms – large, multinational technology corporations, such as Amazon, Alphabet/Google,...

AI and analytics

Spotlight on Capgemini NA @ Informatica World 2018 | May 21–24 in Las Vegas

Jackson, Dusty
Date icon July 10, 2018

Spotlight on Capgemini NA @INFA World 2018 with key representation from Dusty Jackson, Scott...

cookies.

By continuing to navigate on this website, you accept the use of cookies.

For more information and to change the setting of cookies on your computer, please read our Privacy Policy.

Close

Close cookie information