Kubernetes is one of the hottest technologies around right now. But are people deciding to adopt Kubernetes because it’s the best solution for their applications, or because it’s the cool kid on the block? The evidence suggests many are advocating for it based on a fear of missing out on the latest trend, not because it’s the best solution for the problems they are trying to solve.
How we make decisions
The thought came to me recently after I finished reading the book The Undoing Project (W. W. Norton & Company, 2016) by Michael Lewis. It’s about two Israeli psychologists, Daniel Kahneman and Amos Tversky, and their research into how humans make decisions based on implicit biases in uncertain situations. Kahneman, after the death of Tversky, expanded on their work together in Thinking, Fast and Slow (Farrar, Straus and Giroux, 2011). Their research, which proved that human decision-making isn’t always rational, had led to the development of a two-part system for describing how the brain makes decisions. As Kahneman writes:
-
“System 1 operates automatically and quickly, with little or no effort and no sense of voluntary control,” whereas
-
“System 2 allocates attention to the effortful mental activities that demand it, including complex computations. The operations of System 2 are often associated with the subjective experience of agency, choice, and concentration.”
While many of us would tend to think we use System 2 for decision-making, their research showed that System 1 is usually in control.
The Undoing Project starts out by introducing Daryl Morey, the general manager of the Houston Rockets. Morey is a numbers guy. He has a computer science degree with an emphasis on statistics from Northwestern University, as well as an MBA from the MIT Sloan School of Management. The NBA has the highest average player contract value of any professional sports league, so when it comes to signing players to guaranteed contracts, there is a lot on the line. Morey observed basketball experts at that time making such big decisions based on very subjective assessments of the players. He even found he and his own staff taking that approach, more than once. Upon realizing what was happening, Morey shifted the Rockets’ scouting strategy to a more scientific approach, one that valued hard data over simple observations. In other words, a System 2 approach. The Houston Rockets haven’t had a losing season since.
That got me thinking about technology. How do our implicit biases affect our decision-making process around which technologies to adopt? Is the Gartner hype cycle a result of human decision-making and the errors we make? After all, the hype cycle essentially shows us that when a new technology is introduced, a lot of us get really excited and start trying to use it (Peak of Inflated Expectation). Then reality sets in that it is harder to use or not as effective as we thought (Trough of Disillusionment). As the technology matures, we see it move to the Slope of Enlightenment and eventually, the Plateau of Productivity.
The Gartner hype cycle, courtesy of Wikimedia
This pattern repeats over and over, with almost every new technology. Why?
Most engineers are curious by nature and are continually learning so as to stay on top of the latest trends. They also like cool tech, and what’s cooler right now than Kubernetes? Right now, everybody thinks they need Kubernetes.
Here is a three-year look at Kubernetes on Google Trends:
Moreover, according to Seen by Indeed, over the past four years, “[T]he share of Kubernetes jobs per million grew by 2,141.03% while the share of Kubernetes job searches increased 2,125.66%.”
Which brings me back to my original question around implicit bias and technology. Are the decisions people are making to adopt Kubernetes based on a rational evaluation of its effectiveness, or simply a desire to use the latest and greatest technology that everyone is talking about right now?
For example, in a recent podcast, Tony McCully of The Home Depot recalled how, despite many developers within the organization saying Kubernetes was the platform of the future, when an analysis was done of all of their applications, 80-85 percent were found to be better-suited to a “full-service” PaaS. He also talked about the need to focus on problem-solving—not technology.
Many analysts, among them industry analyst Fintan Ryan, are talking about this as well:
For those looking for the TL;DR - #Kubernetes is a superb choice, and often the best, in a wide range of cases, but for a lot of others it really isn't. Don't start with the technology as the solution, start with the problem and work back from there.
— Fintan Ryan (@fintanr) May 30, 2019
Sometimes Kubernetes is a great choice. T-Mobile, for example, described at both SpringOne Platform and Kubecon the success it’s having with Tanzu Kubernetes Grid Integrated Edition in production.
And while we are seeing a significant uptick in the number of enterprises adopting Kubernetes, according to Forrester Research (Digital Transformation Requires Development Transformation, December 2019), “A majority of software development teams still release fewer than four times a year.” Why? A 2020 Forrester Consulting study commissioned by VMware, Improving Customer Experience And Revenue Starts With the App Portfolio, found that “78 percent of executives said lack of skills with modern development platforms (e.g., containers, Kubernetes, serverless) is an important application improvement for them to overcome.”
It’s all about the developer experience
As you start to modernize existing apps, build new cloud-native apps, and move to a multi-cloud deployment model, abstracting Kubernetes (and other infrastructure) away from the developer is usually a good idea.
As this January 2020 article excerpt from The New Stack argues, having developers deal with infrastructure takes them away from building software that delivers business value:
“Developers have found a mixed bag with Kubernetes. On the one hand, it eases the provisioning and management of complex applications. But it can also quickly throw the developer into a kind of configuration hellscape and—though this may not be the fault of Kubernetes itself—architectural chaos as different IT teams struggle to incorporate into existing operations.”
The article references a podcast interview conducted with analyst Janakiram MSV in which he talks about how one of the challenges he sees with developers using Kubernetes is its lack of a cohesive, “platform”-like interface. As he notes, the virtual machine provides end users with a “seamless, coherent experience” whereas Kubernetes doesn’t yet have a cohesive abstraction that developers can easily understand.
“Kubernetes is still pretty much an infrastructure,” he is quoted as saying. “There is no developer experience or user experience offered to the consumers of Kubernetes.” And ever since Kubernetes transformed into a platform for managing workloads, he notes, the key layer on top of the infrastructure is missing.
What’s really missing?
As VMware Senior Staff Engineer Bryan Liles stated during his 2019 KubeCon keynote, “Kubernetes is not a product...it’s a cloud native platform for building platforms.” In other words, it needs a lot of things on top of it that you are now responsible to provide and manage.
My colleague Richard Seroter, a VP of product marketing, sums these “things” up nicely in a 2019 blog post about Kubernetes as a platform for building platforms:
“The reason you see companies like Microsoft (with Azure Spring Cloud), VMware (with Tanzu Application Service for Kubernetes), and even Google (with Cloud Run powered by Knative) jump into this space more aggressively is because we recognize that Kubernetes is awesome infrastructure that still needs a dev workflow. It needs a container build system. An app deployment API. A marketplace. App-focused management commands. You know, an application platform atop an infrastructure platform.”
And then there is this tweet, from Kelsey Hightower, staff developer advocate for Google Cloud Platform.
So you want to roll your own application platform. All you need is:
— Kelsey Hightower (@kelseyhightower) April 3, 2020
Linux
Docker
Kubernetes
Istio
Prometheus
Fluentd
Grafana
Jaeger
Harbor
Open Policy Agent
Vault
Spinnaker and Jenkins
Oh, almost forgot, you're also going to need servers, people, and glue. Bring lots of glue.
How VMware helps improve the developer experience today
Getting your software to production is critical to the success of your business. Unfortunately, many companies are struggling to get it there successfully. Their developers are burdened with a myriad of technology choices (languages, frameworks, databases, etc.) and bogged down with infrastructure configuration tasks that distract them and limit their productivity.
My colleague Dormain Drewitz, who leads product marketing and content strategy for VMware Tanzu, describes how by abstracting away the underlying infrastructure and standardizing deployment processes, VMware keeps developers productive, especially in a multi-cloud world:
“Standardizing deployment processes helps developers get to production quickly. Our strategy isn't to build all the services, but to let developers seamlessly tap into open-source and cloud provider innovations. Instead of asking developers to make choices everywhere, we focus on supporting a developer experience with the choices that matter, among them:
-
languages and frameworks, in the buildpacks supported by the Tanzu Application Service
-
production-ready open source software, with Tanzu Application Catalogue
-
databases and other building components via the Open Service Broker API, which includes brokers to AWS, Azure, and Google Cloud AI and machine learning services”
VMware, as one of the biggest contributors to Kubernetes, in recognizing a critical opportunity to make it more accessible and developer-friendly at the end of 2019 made the largest acquisition in its history by buying cloud native platform provider Pivotal. The result has been the launch of the VMware Tanzu portfolio, including the beta of Tanzu Application Service for Kubernetes, which makes Tanzu more developer-friendly for both on-premise and public cloud Kubernetes services. We also launched the beta of Tanzu Build Service to automate container creation, management, and governance on Kubernetes. And we released Tanzu Application Catalog, providing developers with a self-service catalog of production-ready, open source building blocks such as application components, databases, and runtimes from the Bitnami collection.
Next steps
As you look to move to the cloud and modernize your application portfolio, consider what implicit biases might lie in your decision process. Focus on the business problem you are trying to solve and look to businesses that have had success in their digital transformation journey for inspiration. VMware has a number of customers that are happy to talk about their lessons learned and what is helping them be successful.
To learn how Tanzu Application Service yields a three-year, 142 percent return on investment from productivity gains, increased revenue, reduced downtime costs, and more, check out this Forrester Total Economic Impact study, which VMware commissioned from Forrester Consulting in December 2019.
Additional resources:
-
Building on Developer Platforms to Deliver the Best Developer Experience
-
Kubernetes is a platform for building platforms. Here's what that means.
About the Author
Follow on Twitter More Content by Scott Kelly
