A scarce resource

McKinsey, in this study – http://www.mckinsey.com/business-functions/business-technology/our-insights/big-data-the-next-frontier-for-innovation – foresees data science jobs in the United States to exceed 490,000 by 2018. However, only 200,000 data scientists are projected to be available by then… Globally, by the same time this demand is projected to exceed supply by more than 50 percent.

At the same time, 99% of CEO’s indicate that big data analytics is important to their strategies (KPMG study). Beyond big data analytics, the rise of predictive analytics (PdA) creates the need for very advanced data scientists able to model complex concepts. Not surprising then, that Harvard Business Review describes data science as “The sexiest job of the 21st century “(https://hbr.org/2012/10/data-scientist-the-sexiest-job-of-the-21st-century/).

Data scientists’ job consists of amassing vast amounts of data, sorting through it and, eventually, making sense of it. In order to do this, they don’t just use old BI (Business Intelligence) tools and techniques but rather rely on the latest statistical technologies, such as neural nets, deep learning, Bayesian statistics, etc. Data science is more than just science and its practitioners are sometimes referred to as “part analyst, part artist.”  To be proficient at data science requires a combination of talent, education, creativity and perseverance. It also requires kills in various domains such as math, analytics, statistics, computer science, etc. And to make sense of all that data, also some level of domain expertise. So, even though the numbers mentioned above list quite a shortage, the problem may be even worse in some areas which require specific domain expertise or at least a thorough understanding of the problems one tries to solve.

Facing this conundrum, one is offered a variety of solutions, such as: run and hope it goes away, throw lots of money at it in order to acquire the (rare) resource or come up with a smarter solution. The latter is exactly what Predikto has done through automating much of the data scientists’ job. Pure automation will not get us very far but smart automation with advanced machine learning does! And so it happened. This way, Predikto can afford to tackle predictive analytics challenges with a (much) smaller team and focus on the really important matters. Indeed, while it’s good, very good even, to deliver any kind of predictions (many projects relying on data scientists fail – for a variety of reasons), it’s even more important to be able to correctly interpret these forecasts. Only then can they be turned into actions. Without which they’re no more than a science project.

Faced with the scarcity of arguably one of the most important resources in our industry (computing power is another – and it’s not scarce), Predikto therefore chose to find ways to lower its reliance on that scarce resource. And achieving that successfully, makes us unique!