Pivotal Big Data Suite: Open. Agile. Cloud-Ready.

February 17, 2015 Michael Cucchi

Pivotal announced today groundbreaking product enhancements to Pivotal Big Data Suite, the world’s first open sourced enterprise data portfolio. Through significant global customer adoption of Pivotal Big Data Suite we’ve worked with customers across every industry to understand where they need the most value out of their data; such as enabling advanced data science and machine learning through SQL on Apache Hadoop® and distributed in-memory processing of scale-out transactional applications. Pivotal will be delivering a completely open source approach and we’ve added new powerful entitlements for increasing data agility and rapid application scaling capabilities.

Pivotal Big Data Suite, will include new entitlements for Pivotal Cloud Foundry, and open source components for Applications at Scale.

Pivotal Big Data Suite, will include new entitlements for Pivotal Cloud Foundry, and open source components for Applications at Scale.

Open Source—The Pivotal Way

In 2014, Pivotal Big Data Suite simplified and combined a set of market-leading proprietary technologies built to work with our Apache-compliant Hadoop® Pivotal HD. While our customers loved the the capabilities and gobbled up the Pivotal Big Data Suite’s flexibility, they were clear that their buying preferences were changing. Customers these days are seeking to incorporate open source technology as much as possible, especially in the area of IT infrastructure.

So, in hearing our customers requests, and given Pivotal’s record-breaking success with the Pivotal Cloud Foundry business, we have decided to embrace a totally open source approach for Pivotal Big Data Suite. This includes incorporating additional open source technologies as entitlements to our popular flex licensing capabilities. We plan to release all components of our powerful data portfolio to open source, and using our experience with the Cloud Foundry Foundation to help create a more stable, enterprise-friendly core for Hadoop.

Releasing long-time proprietary products as open source might seem to be as easy as simply posting code to Github, but there’s quite a bit involved. This has been in the works for some time however, Pivotal is actively championing each of these technologies through the path to open source, selecting licenses, protecting intellectual property, and establishing governance.

Our detailed plans are still being finalized, but we plan to begin release and incubation of Pivotal GemFire, Pivotal HAWQ, and Pivotal Greenplum Database in a quarterly cadence. We’re closing in now on the structure of ownership of GemFire, Greenplum Database, and HAWQ code to the most appropriate entity for working with the big data community. As we get closer to actual release milestones, we will continue to update on the progress of each product.

Moving Beyond “Big” to Agile Data

HDFS and the Apache Hadoop® ecosystem have always been a core component of Pivotal Big Data Suite to provide flexibility for handling data stores of any size, type, and speed. But today, what customers have been asking for is a more agile way to wield their data. Our customers are looking to Pivotal to transition their data infrastructures in the same way we have their application development, engineering, and IT operations methods and platforms.

For Pivotal, this means a next generation data infrastructure that seamlessly provides the enterprise with the data they need, where they need it, and with the right tools for the job. We feel particularly armed to solve this challenge as we continue to integrate our world leading platform as a service and big data technologies. And here, with this new version of Pivotal Big Data Suite, we have focused on increasing options with support for bare metal commodity hardware, appliance-based delivery, virtualized instances, and now public, private, and hybrid cloud support. In addition, we now include an instance of Pivotal Cloud Foundry, and the ability to allocate cores of Pivotal Big Data Suite to Pivotal Cloud Foundry data services utilizing Pivotal Big Data Suite components—all in one easy to utilize and flexible enterprise subscription.

Adding More Heft for Data Pipelines, Analytic Applications, and Enterprise Applications at Scale

Pivotal GemFire, with it’s premium scale-out NoSQL support has always been the leading in-memory at-scale processing option of Pivotal Big Data Suite. By customer demand, we have added several new additional open source capabilities to round out support for high scale data-centric applications for both analytical and customer-facing use cases.

These new entitlements include support for:

  • RabbitMQ—high scale, high resiliency messaging for applications
  • Redis—in-memory data structure server and cache
  • Spring XD—versatile scale-out analytics pipeline framework for data ingestion and processing

Supporting Open Data Platform-based Distributions

In 2014, Pivotal Big Data Suite was based solely on Pivotal’s distribution of Hadoop. In 2015, with our enthusiastic support and inclusion into the Open Data Platform (ODP), Pivotal now provides Pivotal Big Data Suite with the option of supporting other Hadoop distributions based on the Open Data Platform Core. This further lends to our customer’s data agility, by allowing any or multiple Apache Hadoop® distribution investments and decisions, removing lock-in, and providing a more stable and predictable Hadoop.

The Open Data Platform is the combined engineering effort of its members to accelerate stabilization of Hadoop® versions. With it, companies can more quickly create tested, enterprise-ready solutions supporting Hadoop®. As a result, Pivotal’s next version of Pivotal HD will also be significantly advancing as a result. This next version will include all of the available Apache projects, including support for Apache® Spark, making Spark’s distributed, in-memory analytics processing available to our customers.

Access Pivotal Big Data Suite Today

While some of these capabilities are coming soon, others are available today. All components are available as community trials and downloads. Download Pivotal Big Data Suite today!

Related Reading:

Editor’s Note: Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

About the Author


VLDB 2015—Pivotal’s Chief Scientist Recaps Keynotes and Papers
VLDB 2015—Pivotal’s Chief Scientist Recaps Keynotes and Papers

In this post, Pivotal’s Chief Scientist, Jignesh Patel, recaps the key talks and papers contributed by Pivo...

Operations Has Plenty To Do In A Cloud-Native Enterprise
Operations Has Plenty To Do In A Cloud-Native Enterprise

In an age when the role of IT Operations is shifting faster than ever before, there's no end of jobs operat...

SpringOne 2022

Register Now