Build Newsletter: The Rise Of Real-time Data At Strata + Hadoop

April 7, 2016 Gregory Chase


sfeatured-buildStrata + Hadoop World has long showcased some of the best thinking in software and data, and last week the biggest topic was most definitely real-time, streaming data platforms, like Apache Spark™.

It is not really a surprise this was the dominant topic, as it was predicted to be a major focus leading up to the event. However, when you dig into the details there were some interesting points.

To start, Wikibon released their Worldwide Big Data Market Forecast at the event, which projects big data growth from $18.2B to $92.1B at a 14.5% CAGR. They also published the first-ever forecast on Apache Spark, saying the market will be 16% of the overall big data market by 2022 at $11.5B, signaling this will be one of the major topics in big data for some time. Datanami agrees that the real-time access that Spark provides is leading this big data boom. For the underlying use cases driving this growth, Wikibon cites three—maturing data lakes, self-tuning systems, and intelligent systems of engagement. We also suspect the rise of IoT as a differentiator for consumer goods and industrial equipment, could pull this number even higher.

ESG also highlighted the rise of Spark and machine learning at Strata. They pointed out the shifting markets for data warehousing and cloud computing, explaining that IT leaders are more regularly evaluating a data warehouse replacement by Hadoop, and 35-45% are considering public cloud based big data and data warehouse solutions (not on premise). This is a powerful indicator that Hadoop growth predictions are realistic, as it shows the appetite for on-demand, ubiquitous data is on the rise.

TechTarget also reinforced this prediction that streaming analytics are coming to the big data forefront as their open source roots and real-time capabilities have finally right-sized the economics enough to open up a variety of compelling use cases.

This right use cases was also the single biggest theme for us going into the event, with our own Ian Andrews, VP Products, focused his Strata keynote, “Delivering Information in Context” along with an interview on TheCube.

The point? While we believe BI won’t die, it will live on, in a more integrated, useful way, lingering underneath the covers, and creating more meaningful engagements and dramatically more intelligent systems.

For example, consider Uber—when you order a car, you don’t need monthly reports on the average arrival time. Instead, you need a small little button within the customer experience tells you one really important thing, “The car can be here in 7 minutes.“

That is the power of real-time.


About the Author

Greg Chase is an enterprise software business leader more than 20 years experience in business development, marketing, sales, and engineering with software companies. Most recently Greg has been focused on building the community and ecosystem around Pivotal Greenplum and Pivotal Cloud Foundry as part of the Global Ecosystem Team at Pivotal. His goal is to to help create powerful solutions for Pivotal’s customers, and drive business for Pivotal’s partners. Greg is also a wine maker, dog lover, community volunteer, and social entrepreneur.

20 Examples of ROI and Results with Big Data
20 Examples of ROI and Results with Big Data

In this post, we share a recent research on the business outcomes of big data efforts. First, key statistic...

Diversity at the Cloud Foundry Summit
Diversity at the Cloud Foundry Summit

Last week at the inaugural Cloud Foundry Summit, now run by the new Cloud Foundry Foundation, we had a uniq...