Spark / Cloudera One Platform initiative – important? Maybe…probably

Last week Cloudera announced the One Platform initiative  http://vision.cloudera.com/one-platform/ . This is an interesting development –  we know that Spark replaced Hadoop as the most popular Apache project and that Cloudera first backed SPark publicly over a year ago.  (Hadoop still has more committers though (https://projects.apache.org/projects.html?number) .

In making the announcement Cloudera also confirmed the gaps that exist in Spark to-day, somewhat similar to lists I have shared here :-

  • Security
  • Scale
  • Management
  • Streaming

We know that Cloudera intends to replace Map-Reduce with Spark. The only way this can happen is for Spark to  scale. Our experience with Spark is that it does work and has a lot of advantages in use cases like Machine Learning, way faster than Hive, alerting and aggregating in streaming analysis, joining multiple data sources in memory. But the one thing Spark does not do well is scale. Compared to the scale advantages that Mapreduce provides Spark is a long way behind.

But most interesting is the fact that Enterprises want to have a single data lake/warehouse/repository/whatever you want to call it. Until Spark can scale then all those pesky structured data systems that have been built by customers using Oracle and all their competitors will continue to be used and deployed. Therefore – if Cloudera succeed – and I think they will – the hollowing out of the traditional database vendors is going to really take off.

What do you think? Is the One Platform initiative just marketing hype? Will Cloudera succeed in replacing Oracle and their like? Let us know and check out our new website at www.exceleratesystems.com.

If you have additional questions, get in touch with us!

9 + 6 =

USA

Corporate Head Quarters

2205 152nd Avenue NE
Redmond, WA 98052
USA

+1 (425) 605 1289

Latin America

(Mexico, Colombia & Chile)

Mexico City

Córdoba 42 Int. 807, Roma Norte, Cuauhtémoc, 06700, Mexico City

+52 (55) 5255 1329

United Kingdom

London

85 Great Portland Street, First Floor, London, W1W 7LT

+44 2030 971584

Ireland

Sligo

77 Camden Street Lower, Dublin, D02 XE80, Ireland

+353 71 915 9710

Search Guard is a trademark of floragunn GmbH, registered in the U.S. and in other countries. Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries. OpenSearch is licensed under Apache 2.0. All other trademark holders rights are reserved.