Big Data has a Big Problem

One of the keys to success in Big Data is having the skills to make Big Data projects successful. While some of these skills are ‘soft skills’, the core requirement is to know what we are doing with complex, highly interrelated and fast moving technology. I recently read a Cap Gemini report on How Successful Companies Make Big Data Operational 

The report itself made for interesting reading but also highlighted one big problem –


Global organizational spending on Big Data exceeded $31 billion in 2013, and is predicted to reach $114 billion in 2018.


Almost 300% growth in 5 years. Now – think about the critical issues this creates – not enough Data Scientists, Admins, Developers, NoSQL experts (plus many other critical skill sets) are being trained right now to come anywhere close to satisfying this demand.  As long ago as 2011 McKinsey estimated that there will a 1.5 million person hole in the US workforce alone of managers and analysts capable of using Big Data to make effective decisions.

So – as others have pointed out, if you are not Facebook/Google/Linkedin etc what can you do. One interesting set of results lies in some research conducted at Microsoft earlier this year. The full report is here and a great summary is at the Register  The study effectively shows that Big Data – the technology – in Microsoft is not failing but – equally – the results are not yet packaged in a way that helps the consumers of the information to use it.

This is the critical point – when we talk to clients we discuss the details of the technology we believe will meet their needs but there is absolutely no substitute for experience. And experience in the customer is critical. For this reason – whenever a client asks for help on Big Data we always tell them – get started, get started right now and start building your own institutional knowledge. That way you will not get trapped in a skills shortage dead-end.

What do you think? Is the skills shortage going to stifle the adoption of Big Data? For an interesting slant on this question take a look at this Venturebeat article from Cameron Sim at Crewspark.

If you have additional questions, get in touch with us!

2 + 4 =


Corporate Head Quarters

2205 152nd Avenue NE
Redmond, WA 98052

+1 (425) 605 1289

Latin America

(Mexico, Colombia & Chile)

Mexico City

Córdoba 42 Int. 807, Roma Norte, Cuauhtémoc, 06700, Mexico City

+52 (55) 5255 1329

United Kingdom


85 Great Portland Street, First Floor, London, W1W 7LT

+44 2030 971584



77 Camden Street Lower, Dublin, D02 XE80, Ireland

+353 71 915 9710

Search Guard is a trademark of floragunn GmbH, registered in the U.S. and in other countries. Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries. OpenSearch is licensed under Apache 2.0. All other trademark holders rights are reserved.