Apache Way at Pivotal

October 8, 2014 Roman Shaposhnik

featured-feathersEver since I joined Pivotal, one of my favorite topics of cocktail conversation has become explaining to folks unfamiliar with the company just how much of a quintessential open source shop we are. Just rattling off the names of the projects and communities we stand behind feels like listing ‘who’s who’ of open source. Being a betting man as I am, I typically introduce Pivotal by making a $1 bet that whoever I’m talking to is likely to have used software we contribute to at some point in their career. Given that this list includes Cloud Foundry, Spring Framework, Groovy, Grails, Redis, RabbitMQ, Tomcat and most recently Apache Hadoop® the only folks I ever lose to are the poor souls still stuck on Windows 95 or mainframes. Paying $1 to them feels less like losing a bet, and more like a random act of kindness.

As a side note, this incredible diversity of Pivotal’s open source portfolio helps our business model a great deal. After all, the days when one could have a sizable stand-alone business built around a single open source project are as distant as dial-up internet access. There’s a lot to be said on what a winning open-source strategy looks like (I may dare defining one in one of my future blog posts). Whatever it is, if you are an enterprise infrastructure vendor in 2014, you can’t survive without having one. The market has clearly spoken in favor of enterprise infrastructure products built around open source projects. With this came a realization that the only sustainable way of doing so was to build long-term relationships with open source communities. This, of course, meant that business now found themselves thinking deep and hard about open source community governance.

It is perhaps ironic, that the single most important detail of how open source communities operate doesn’t get as much air time as licensing or patenting. Somehow there’s a perception that open source communities don’t even need much of governance. They are expected to self-organize into utopian worker collectives that could serve as a living illustration to “from each according to his ability, to each according to his need” principle. Nothing can be further from the truth. Each big, successful open source community is unique and typically very particular about the governance model. Open source communities, and by extension projects that they work on, live and die by how effective their governance model is. A great community can alway fix technology, a poor community is bound to ruin the best technology that is entrusted to it.

Now, remember my earlier point about the sheer size of Pivotal’s open source portfolio? Community governance is where it starts to matter yet again. Because you see, regardless of how much a runaway success Spring is (and standing at 8+ million users nobody would argue it isn’t) the governance model that made it so successful is unique to it. It is true that Spring uses Apache License, but the similarities with the Apache Software Foundation (ASF) governance model ends there.

ASF governance model, also known as “The Apache Way”, is a very unique, extremely well thought out approach to building successful open source communities. Communities that are capable of innovating on a diverse set of projects all living under the Foundation’s umbrella. “The Apache Way” has been covered in great details over the years, its elevator pitch fits in just three words: “Community over code”. No really, that is it—everything else is an implementation detail.

It must be said, that even before I joined Pivotal, the company has had a relationship with the ASF. We have been a constant sponsor of the Foundation and with guys like Mark Thomas and Bill Rowe working for Spring side of Pivotal “The Apache Way” was practiced quite well when it comes to Apache Tomcat and Apache HTTPD. At the same time, the scale at which Datafabrics side of Pivotal has embraced ASF projects around Apache Hadoop® ecosystem required a different level of focus on ASF.


So that was part of my mission when joining Pivotal: become a resident ASF guy for data technologies organization and make sure that in a year’s time Pivotal becomes a company that grows the pool of Apache Committers, not a company that just fights over the existing pool.

This project started with putting a framework in place that enables Pivotal engineers to collaborate on ASF software with minimum distraction (this required a close collaboration with our outstanding legal team). Our next step was to review our product roadmap and based on customer’s feedback start focusing on a project that they needed. This is what our previous announcement around Apache® Ambari was all about.

In the short couple of month we have not only contributed a wide variety of improvements to it, but also helped the project get more stable by working on build and CI automation and suggesting process improvements to innovate even quicker while not losing the sense of project stability. Pivotal’s product has benefited from this relationship, but so did Apache® Ambari: it has become better and more featureful, but most importantly its community has grown stronger and more diverse. In fact, recognizing all the hard work that one of Pivotal’s engineers Jun Aoki has put into the project, the Apache® Ambari PMC has voted Jun in as a committer.

This may be a small step for Jun (and, to be sure, the first one in many other ASF projects that he is going to join), but it actually signifies a giant leap for Pivotal and the kind of role it is assuming in ASF from now on. We’re already delivering on that fundamental vision of making sure that our engineers get fully empowered to lead most impactful open source projects within the Foundation. If you are an engineer dreaming to add the coveted Committer/PMC of Apache X to your resume, join us. We can’t guarantee that you will get that line (after all, ASF recognizes your personal merit and your personal merit alone) but we can guarantee that we will move heaven and earth to empower you to do so.

After all a stronger Apache Software Foundation (ASF) will benefit the entire industry. It is, you see, that proverbial tide that lifts all boats big and small.

Editor’s Note: Apache, Apache Hadoop, Hadoop, and the yellow elephant logo are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.

About the Author


Churn Prediction in Retail Finance and Asset Management (Part 2)
Churn Prediction in Retail Finance and Asset Management (Part 2)

Financial firms collect large volumes of data from all realms of our daily lives. These data assets are use...

All Things Pivotal Podcast – Now Available!
All Things Pivotal Podcast – Now Available!

Today we are launching the All Things Pivotal Podcast. Narrated by Pivotal Field CTO Simon Elisha, who prev...