The Rise of the Data Scientist

Within the past two decades, data science has progressed from a relatively unknown term, to one of the most sought after disciplines in the job market; by the time Hal Varian, Google’s Chief Economist, declared that the “sexy job” of the future would belong to the statisticians with the ability to analyse big data (in 2009), data science had emerged as an essential, and highly sought after tool within the corporate framework. The data scientist is now seen as the key to unlocking the treasure chest that is unstructured, big data.

Companies collect inordinate volumes of data in an ever-more connected global economy, where technology is facilitating the accumulation of information regarding individuals, and data is readily available from other sources that give an even richer picture into peoples lives. But, the hidden gems within this big data remain hidden unless the talents of a select group of individuals can unearth them. Hal Varian himself said that this “essentially free and ubiquitous data” needs data scientists to “extract value from it”. A relatively scarce supply of people with these skills exist, and that has created a significant gap in demand for the data scientist in today’s job market. It is little wonder then, that ‘Data Mining’ was ranked as LinkedIn’s “hottest skill of 2014”.

There is often a reference to the ever-widening “talent gap” between the demand for big data experts, and those in the labour market who can offer such skills to employers. McKinsey report that by 2018, the US economy will face a shortage of 140,000 to 190,000 people who have the skills to perform “deep analytics” of data; furthermore, the US will lack “1.5million managers and analysts with the know-how to use the analysis of big data to make effective decisions”. This shortage is beginning to visibly manifest itself in several indicators.

How do you become a data scientist?

The number of graduate, post-graduate, and short courses designed to train students to become data scientists has risen markedly, as seen in the Chart below, in response to this gap in the supply of data scientists and the demand for such skills. Prominent Universities across the world now offer data science-specific degrees; Harvard Extension School alone offers four courses aimed at creating data scientists to fill the “talent gap” in the economy.

The chart above shows the number of jobs posted on, a jobs-posting website, that relate to employers looking for ‘data scientists’; this demonstrates how recent trends in data science job postings show remarkable growth in the labour market. The discipline has risen to the fore in the last five years, and is set to continue to grow, as the wealth of data being accrued continues to expand. Data science continues to be the hot topic within the field of business intelligence and analytics, and this is reflected in the continual adoption of big data capabilities within large companies; 14% of large employers (100+ employees) in the UK had big data capabilities in 2012; by 2017, this is set to rise to 29%. It is clear, that data science is becoming an increasingly integral element of data analytics in the corporate realm, making data scientists hot property within the labour market.

How good are these jobs for data scientists?

This remarkable demand for data scientists has created an attractive job market for those who wish to pursue data science as a career path, as illustrated in the Chart below. The wages paid to data scientists reflects the value that employers now acknowledge these big data analysts are generate. The median wage for a data scientist is $98,000 globally, with a significantly higher average in the US of $144,000; this has grown significantly over time, and is set to continue to grow as the scarcity of data scientists drives their wages up.

What about the companies that support data scientists?

It’s not just the numbers of data scientists who are increasing. The rise in the prominence of the data scientist is also reflected in the growth of companies offering specialist software which facilitate the process of data mining, analysis, and visualisation. Companies such as SPSS and SAS have undergone tremendous growth in recent years, profiting from the fact that they provide some of the key tools Data Scientists need to unlock the insights hidden in big data. Between 2012 and 2017, the number of big data users is set to increase from 233,000 to 644,000. Furthermore, a report by Transparency Market Research suggests that the global market value of Hadoop (an open-source analysis software popular with data scientists) is set to rise from $1.5bn in 2012 to $20.9bn in 2018.

So, if you’re interested in pursuing data science as a career path, a host of influential figures and convincing indicators paint a compelling picture of your career prospects. Data scientists have come from a past of relative obscurity; but with big data now being amassed universally, the skills of data scientists are more sought after than ever before, and this trend looks set to continue. All of this points to a prosperous and invigorating career ahead of those who are passionate about delving into the sea of unstructured, big data to seek out those hidden pearls of insight few are capable of finding.