While the entire CF Summit rocked and included great talks by SAP, Monsanto, and many others, one of the most popular talks at CF Summit was based on the work done by Dr. Mike Maximilien from IBM Cloud Labs and James Bayer, Pivotal’s Cloud Foundry’s Director of Product Management.
The two shared research to date done to compare Cloud Foundry, Microsoft Azure, Google App Engine, Amazon, Heroku, and OpenShift.
Detail included formalized matrices comparing 50 features as well as qualitative feedback on what works well and not so well. The talk started off explaining how the research started with IBM in 2010—their intent was to help make purchase decisions as well as improvements to PaaS solutions.
The 30 minute video (and slide deck) ultimately covers background, limitations to research validity, methodology, results, insights, and potential next steps.
Dr. Mike Maximilien from IBM Cloud Labs and James Bayer, Pivotal’s Cloud Foundry’s Director of Product Management present the PaaS Comparison
PaaS Comparison—Validity and Methodology
Immediately, Dr. Maximilien underscored how important the limitations and methodology were to explain.
While the talk provided a very good baseline, he explained how it was not a scientific comparison. There are still challenges in terms of marketing or documentation claims versus day to day reality of use. As well, the market was very early and no standard measurement existed. Lastly, the popularity of PaaS created a low signal to noise ratio, making it difficult to compare. The two also recognized their bias with Cloud Foundry and hoped this framework could be used by others to do their own analysis or even improve the methodology. Bayer also explained—at the end of the day, the team wanted to measure the quality of experience (QoE) for developers.
The two cover the comparison in detail. Dr. Maximilien highlighted 50 features across eight areas—workloads, tooling, integration services, SLAs, data stores, programming models, management, and miscellaneous. Fundamentally, the key question was, “To what extent does each PaaS support a feature?” The answers included non, basic, strong, or leading.
PaaS Comparison—Quality of Experience
Bayer added that QoE portion was approached from a user experience perspective—they wanted to know what it subjectively felt like to experience a popular PaaS scenario. After some quantitative input, they chose a Java app on a relational database and inquired about the ability to push, scale, and update an app along with things like automatic healing.
The video and slide deck cover each PaaS (Cloud Foundry, Microsoft Azure, Google App Engine, Amazon, Heroku, and OpenShift) in terms of the 50 features. In addition, it covers a more qualitative view of what works well and not so well.
PaaS Comparison—Insights and Lessons Learned
At the end, they shared the following slide with insights and lessons learned:
Insights and Lessons Learned from the Pivotal Cloud Foundry and IBM PaaS Comparison
Learn More:
- See the video and slide deck to learn more or get involved
- Check out the Cloud Foundry or CFSummit Website
- Get Pivotal CF product information, downloads, and documentation
- Research other blog articles on PaaS or Cloud Foundry
About the Author